Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-11-08
14:38:47 sean-k-mooney just move the bits and you can leave devstack do it
14:38:53 mriedem i realize we don't have any troubleshooting docs for that
14:39:19 sean-k-mooney mriedem: i had not seen that but ill take a look.
14:39:28 sean-k-mooney it does not really work that well
14:39:37 sean-k-mooney there are still some gaps i think
14:40:07 mriedem i'm pretty sure you identified several gaps at the denver ptg after rocky which are still open
14:40:07 sean-k-mooney it shoudl work betwwen thigns like ml2/odl and ml2/ovs
14:40:21 sean-k-mooney ya i think we adressed 1 or two
14:40:23 mriedem e.. https://review.opendev.org/#/c/594527/
14:41:04 sean-k-mooney ya that was how we found that
14:41:38 sean-k-mooney if your going between linux bridge and ovs there as also an issue where we were not passing hte bridge name
14:41:53 sean-k-mooney and a few other minor things
14:46:23 dansmith lyarwood: have we not moved to using the qemu-integrated iscsi driver yet? I was thinking we had
14:47:01 lyarwood dansmith: nope, I actually nuked the code for that a while ago as nothing was using it.
14:47:11 dansmith huh, okay
14:47:35 lyarwood dansmith: the original implentation was pretty bad, I've got a note to look at reintroducing it if there was a need somewhere.
14:48:06 lyarwood https://review.opendev.org/#/c/668750/
14:48:10 lyarwood ^ was the removal btw
14:48:10 dansmith I thought we were supposed to be moving towards it because it's vastly better and simpler than attaching those to the host
14:49:01 dansmith ah recent
14:49:28 lyarwood it has been a long 4 months downstream ^_^
14:49:44 lyarwood dansmith: what prompted your question btw?
14:50:03 dansmith lyarwood: this volume cache spec that I don't like
14:50:23 mriedem i got tim bell's attention https://review.opendev.org/#/c/566473/
14:50:25 lyarwood ah
14:54:25 sean-k-mooney mriedem: it would be nice to see teh pending state become a thing and the reaper service either become a nova deliverable in its own repo or be broght in tree
14:54:45 mriedem double wrong
14:54:56 mriedem first, you have to do what i said in there and land the rebuilding from cell0 piece
14:54:56 sean-k-mooney no?
14:55:03 mriedem ignore pending for now
14:55:16 mriedem second, there was agreement that the reaper thing would be external to nova so we didn't need to own it
14:55:20 mriedem that's what the notifications are for
14:55:26 sean-k-mooney oh i taught that was already landed
14:55:54 sean-k-mooney oh ok
14:56:00 mriedem 1. build fails, goes to cell0, send notification. 2. external service consumes notification, purges spot instances based on it's own policy that nova doesn't care about, 3. external service triggers rebuild from cell0 on non-spot instance
14:56:11 sean-k-mooney well it would nice to be able to do premtiable isntance with openstack
14:56:21 mriedem sure, i agree :)
14:56:28 sean-k-mooney i guess i dont mind if its in or out of nova ownership
14:56:31 mriedem otherwise i wouldn't have spent time reviewing it
14:57:26 mriedem but it's also complicated enough that unless a core takes it over, it can't just be a thing thrown over the wall that gets worked on 1 day a week
14:58:01 sean-k-mooney ya, internally it would proably still land on the compute team to manage at redhat
14:58:03 mriedem with all the SIG hoopla and people that want stuff, there is always a laughable amount of effort put into a more than half day per week developer to push things
14:58:39 mriedem i'm not blaming theo, he did what he could and i'm sure he had other priorities downstream, especially since they already had that running in production
14:59:40 sean-k-mooney out of interest this was started around or before placement was a thing right
15:00:06 sean-k-mooney do the premtiable instace actully get modeled in placemnet?
15:00:28 sean-k-mooney im just wondering how it woudl interact with unified limits
15:02:42 sean-k-mooney they proably do have allocation as normal but i dont think they current unified lmits proposal could partion the quotas based on normal instance or premtiable at least at present
15:03:02 sean-k-mooney i think the workaound for that was to have a seperate project for each
15:04:19 mriedem don't know, been talking about spot instances and placement forever
15:04:28 mriedem and spot instances has changed many times over the years
15:04:41 mriedem doesn't really matter to us (nova) because the reaper is external and that's why we want it external
15:04:52 mriedem it's on the external service to define what the policy is
15:05:27 mriedem btw, if red hat were to take this over, at least the rebuild from cell0 part, i think melwitt would be a good person to take that over, since she's at least familiar with the spec
15:06:03 mriedem a lot of that piece is really just stuff that overlaps with the cross-cell work, which is moving data between cells and a buttload of testing
15:06:11 melwitt efried: fyi gibi_ptg's +2 on func test sleep removal now https://review.opendev.org/692252
15:06:31 efried melwitt: ack, should get to that before lunch.
15:07:28 mriedem melwitt: can you rev the bug on that to be bug 1783565?
15:07:29 openstack bug 1783565 in OpenStack Compute (nova) "ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was up."" [Undecided,Invalid] https://launchpad.net/bugs/1783565
15:07:30 mriedem your new one is a dupe
15:08:05 melwitt ah dangit
15:08:24 melwitt I didn't know there was already a bug related to that
15:08:31 melwitt ok let me update
15:08:42 mriedem i think i pointed it out in your functional test that changed from sleep to use the oslo stuff, but sure :)
15:09:07 melwitt you did but it didn't register in my brain somehow
15:09:22 mriedem it was pre-halloween so i only speak in riddles
15:09:45 mriedem take a peek at this patch, it holds the key to the answers you seek
15:09:51 mriedem *smoke bomb*
15:10:02 melwitt lol
15:13:04 melwitt mriedem: hey, that patch is mine, not gibi_ptg's (lp comment) :P
15:14:06 openstackgerrit melanie witt proposed openstack/nova master: Replace time.sleep(10) with service forced_down in tests https://review.opendev.org/692252
15:15:28 dansmith lyarwood: do you know anything about how bcache actually works?
15:16:05 mriedem melwitt: oh heh, i saw you said gibi above so assumed it was his
15:16:19 melwitt I said he was +2 on it
15:16:34 mriedem yeah, sorry
15:16:38 mriedem commented in lp for posterity
15:16:58 melwitt yesss
15:21:17 lyarwood dansmith: nope not bcache or CAS tbh
15:22:35 dansmith lyarwood: well, CAS is six months old and just exists as source to create kernel modules, so I assume we can ignore that
15:22:50 dansmith lyarwood: bcache requires a specially-formatted backing device.. it's not transparent
15:23:35 dansmith lyarwood: meaning brick would have to *convert* your volume before hooking it up under bcache, and then apparently it's difficult to reverse if you attach that volume elsewhere that isn't using bcache
15:23:54 dansmith which to me makes it nearly useless for nova
15:32:42 melwitt seems like the gate is back to being angry again
15:42:15 mriedem efried: i did my professional sleuthing on that arcane NOTE in the resource tracker https://review.opendev.org/#/c/693435/ - let me know if you want me to just address those comments inline or follow up
15:42:21 mriedem tl;dr the note should be removed
15:44:41 efried ack, thanks.
15:48:18 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/693554
15:49:30 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Revert "openstack server create" to "nova boot" in nova docs https://review.opendev.org/693240
15:49:30 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/693554
15:58:44 lyarwood dansmith: sorry calls-- , agreed on CAS, seems like it's the main driver behind Intel looking into this even if it's brand spanking new.
15:58:57 lyarwood dansmith: eww okay so bcache also wouldn't work
15:59:10 dansmith lyarwood: yes, another "sell more hardware" feature for nova for sure:/
16:00:01 dansmith lyarwood: yeah, so I think the only way to do this reasonably is with CAS, which is crazy new and unproven, would require compiling kernel modules from source, etc
16:00:05 mriedem when is the last time zvm or powervm ci actually reported and worked? is it time for a warning log on startup of those drivers?
16:00:22 lyarwood dansmith: isn't dm-cache also an option here?
16:00:28 dansmith lyarwood: I love their FAQ page which says "Why do filesystem benchmark tools report that there is disk corruption? Oh don't worry about those silly errors!"
16:00:38 lyarwood L O L
16:00:39 dansmith lyarwood: not AFAIK, see my comments
16:01:07 dansmith lyarwood: dm-cache is static partitioning only as far as I know.. i.e. one cache device per block device to be cached
16:02:10 lyarwood dansmith: ack
16:02:32 mriedem oh i see a recent zvm ci job, took 2 hours, failed

Earlier   Later