Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-11-08
14:56:00 mriedem 1. build fails, goes to cell0, send notification. 2. external service consumes notification, purges spot instances based on it's own policy that nova doesn't care about, 3. external service triggers rebuild from cell0 on non-spot instance
14:56:11 sean-k-mooney well it would nice to be able to do premtiable isntance with openstack
14:56:21 mriedem sure, i agree :)
14:56:28 sean-k-mooney i guess i dont mind if its in or out of nova ownership
14:56:31 mriedem otherwise i wouldn't have spent time reviewing it
14:57:26 mriedem but it's also complicated enough that unless a core takes it over, it can't just be a thing thrown over the wall that gets worked on 1 day a week
14:58:01 sean-k-mooney ya, internally it would proably still land on the compute team to manage at redhat
14:58:03 mriedem with all the SIG hoopla and people that want stuff, there is always a laughable amount of effort put into a more than half day per week developer to push things
14:58:39 mriedem i'm not blaming theo, he did what he could and i'm sure he had other priorities downstream, especially since they already had that running in production
14:59:40 sean-k-mooney out of interest this was started around or before placement was a thing right
15:00:06 sean-k-mooney do the premtiable instace actully get modeled in placemnet?
15:00:28 sean-k-mooney im just wondering how it woudl interact with unified limits
15:02:42 sean-k-mooney they proably do have allocation as normal but i dont think they current unified lmits proposal could partion the quotas based on normal instance or premtiable at least at present
15:03:02 sean-k-mooney i think the workaound for that was to have a seperate project for each
15:04:19 mriedem don't know, been talking about spot instances and placement forever
15:04:28 mriedem and spot instances has changed many times over the years
15:04:41 mriedem doesn't really matter to us (nova) because the reaper is external and that's why we want it external
15:04:52 mriedem it's on the external service to define what the policy is
15:05:27 mriedem btw, if red hat were to take this over, at least the rebuild from cell0 part, i think melwitt would be a good person to take that over, since she's at least familiar with the spec
15:06:03 mriedem a lot of that piece is really just stuff that overlaps with the cross-cell work, which is moving data between cells and a buttload of testing
15:06:11 melwitt efried: fyi gibi_ptg's +2 on func test sleep removal now https://review.opendev.org/692252
15:06:31 efried melwitt: ack, should get to that before lunch.
15:07:28 mriedem melwitt: can you rev the bug on that to be bug 1783565?
15:07:29 openstack bug 1783565 in OpenStack Compute (nova) "ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was up."" [Undecided,Invalid] https://launchpad.net/bugs/1783565
15:07:30 mriedem your new one is a dupe
15:08:05 melwitt ah dangit
15:08:24 melwitt I didn't know there was already a bug related to that
15:08:31 melwitt ok let me update
15:08:42 mriedem i think i pointed it out in your functional test that changed from sleep to use the oslo stuff, but sure :)
15:09:07 melwitt you did but it didn't register in my brain somehow
15:09:22 mriedem it was pre-halloween so i only speak in riddles
15:09:45 mriedem take a peek at this patch, it holds the key to the answers you seek
15:09:51 mriedem *smoke bomb*
15:10:02 melwitt lol
15:13:04 melwitt mriedem: hey, that patch is mine, not gibi_ptg's (lp comment) :P
15:14:06 openstackgerrit melanie witt proposed openstack/nova master: Replace time.sleep(10) with service forced_down in tests https://review.opendev.org/692252
15:15:28 dansmith lyarwood: do you know anything about how bcache actually works?
15:16:05 mriedem melwitt: oh heh, i saw you said gibi above so assumed it was his
15:16:19 melwitt I said he was +2 on it
15:16:34 mriedem yeah, sorry
15:16:38 mriedem commented in lp for posterity
15:16:58 melwitt yesss
15:21:17 lyarwood dansmith: nope not bcache or CAS tbh
15:22:35 dansmith lyarwood: well, CAS is six months old and just exists as source to create kernel modules, so I assume we can ignore that
15:22:50 dansmith lyarwood: bcache requires a specially-formatted backing device.. it's not transparent
15:23:35 dansmith lyarwood: meaning brick would have to *convert* your volume before hooking it up under bcache, and then apparently it's difficult to reverse if you attach that volume elsewhere that isn't using bcache
15:23:54 dansmith which to me makes it nearly useless for nova
15:32:42 melwitt seems like the gate is back to being angry again
15:42:15 mriedem efried: i did my professional sleuthing on that arcane NOTE in the resource tracker https://review.opendev.org/#/c/693435/ - let me know if you want me to just address those comments inline or follow up
15:42:21 mriedem tl;dr the note should be removed
15:44:41 efried ack, thanks.
15:48:18 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/693554
15:49:30 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/693554
15:49:30 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Revert "openstack server create" to "nova boot" in nova docs https://review.opendev.org/693240
15:58:44 lyarwood dansmith: sorry calls-- , agreed on CAS, seems like it's the main driver behind Intel looking into this even if it's brand spanking new.
15:58:57 lyarwood dansmith: eww okay so bcache also wouldn't work
15:59:10 dansmith lyarwood: yes, another "sell more hardware" feature for nova for sure:/
16:00:01 dansmith lyarwood: yeah, so I think the only way to do this reasonably is with CAS, which is crazy new and unproven, would require compiling kernel modules from source, etc
16:00:05 mriedem when is the last time zvm or powervm ci actually reported and worked? is it time for a warning log on startup of those drivers?
16:00:22 lyarwood dansmith: isn't dm-cache also an option here?
16:00:28 dansmith lyarwood: I love their FAQ page which says "Why do filesystem benchmark tools report that there is disk corruption? Oh don't worry about those silly errors!"
16:00:38 lyarwood L O L
16:00:39 dansmith lyarwood: not AFAIK, see my comments
16:01:07 dansmith lyarwood: dm-cache is static partitioning only as far as I know.. i.e. one cache device per block device to be cached
16:02:10 lyarwood dansmith: ack
16:02:32 mriedem oh i see a recent zvm ci job, took 2 hours, failed
16:03:33 dansmith mriedem: no (real) changes to either of them in 2019 either, as far as I can tell. I bet that doesn't mean "they just work and have happy customers" either :)
16:03:53 efried mriedem: +2 but prefer fix-here
16:04:18 dansmith the most recent change to zvm was "add get console support" which was part of the original set of work, IIRC, from feb 2018
16:04:25 mriedem huh zvm ci passed on an earlier patch set in the same change, the latter ps ci failure was due to ssh failing on some guests
16:04:32 mriedem i'm surprised they even enable ssh validatoin in their tempest runs
16:04:46 mriedem yeah zvm in tree is mostly not a usable driver
16:04:53 mriedem create, get console, destroy
16:04:58 dansmith yeah
16:05:21 dansmith I believe I called that as abandonware before we merged :)
16:05:47 mriedem efried: i can do it in there, it just means rebasing and pushing the other 3 changes on top as well
16:05:57 mriedem which likely means i'll be doing rechecks
16:06:01 efried oh
16:06:10 efried then I guess fup it
16:06:24 efried unless it waits too long to get another +2
16:06:50 mriedem yeah if i have to resolve a merge conflict obviously i'd fix there, but i'll fup for now
16:07:04 mriedem dansmith: you just still have s390x ptsd from your early days
16:07:27 mriedem remember, they were doing virtualization since before it was hip
16:07:43 dansmith no, I have ptsd from all of ibm's drivers
16:07:46 melwitt mriedem: we had to ask them to enable ssh validation bc otherwise their config drive stuff could not be tested
16:07:54 mriedem melwitt: ah that's right
16:08:18 mriedem the powervm driver in tree can at least do some things, but i haven't seen ci working for it in years
16:09:13 mriedem looks like maintenance of the out of tree driver has basically been farmed out https://github.com/openstack/nova-powervm/commits/master
16:14:21 openstackgerrit Merged openstack/nova master: Replace time.sleep(10) with service forced_down in tests https://review.opendev.org/692252
16:16:50 openstackgerrit Matt Riedemann proposed openstack/nova master: FUP for Ib62ac0b692eb92a2ed364ec9f486ded05def39ad https://review.opendev.org/693556
16:20:50 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Use long_rpc_timeout in reserve_block_device_name https://review.opendev.org/693537
16:22:06 openstackgerrit Matt Riedemann proposed openstack/nova master: Convert nova-live-migration job to zuulv3 https://review.opendev.org/693364
16:39:38 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove service_uuids_online_data_migration https://review.opendev.org/693421
16:51:53 sean-k-mooney mriedem: i filed https://bugzilla.redhat.com/show_bug.cgi?id=1761864 to track adding support for it at somepoint but its in our backlog and not prioritised yet.
16:51:53 openstack bugzilla.redhat.com bug 1761864 in openstack-nova "[RFE] Preemtable Instances" [High,New] - Assigned to smooney
16:52:12 sean-k-mooney actully that should not have an assignee
16:53:02 sean-k-mooney dansmith: bcache is pretty stable
16:53:09 sean-k-mooney dansmith: i have been using it for years
16:53:26 dansmith sean-k-mooney: didn't say it wasn't
16:54:24 sean-k-mooney true

Earlier   Later