Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-01-11
19:19:06 claudiub|2 artom: i see. :)
19:19:09 JayF I don't think you could even lift me, much less hand me over to someone ;)
19:19:32 claudiub|2 mriedem: replied. :)
19:21:22 claudiub|2 mriedem: although, not exactly sure how to treat a certain comment: Andrey Volkov was asking if there could be a way to warn operators that if they want to use hyper-v vnuma, they'd have to disable the numa spanning on the host. is there any other better way to do this (except release notes)?
19:24:21 openstackgerrit Gábor Antal proposed openstack/nova: Transform instance.volume_detach notification https://review.openstack.org/408676
19:38:33 EmilienM I think I found something "weird" in nova-compute & nova-placement
19:39:01 EmilienM if you start n-cpu *before* n-placement, n-compute will never retry to register the placement services
19:41:02 diana_clarke dansmith: ^ was that what you were talking about yesterday? That you were going to change to keep retrying? Ignore me if I've missunderstood.
19:41:10 openstackgerrit Lee Yarwood proposed openstack/nova: libvirt: Ensure to_xml always returns an ASCII string https://review.openstack.org/402382
19:41:12 openstackgerrit Lee Yarwood proposed openstack/nova: libvirt: utils.get_disk return the target dev of the root device https://review.openstack.org/402716
19:41:13 openstackgerrit Lee Yarwood proposed openstack/nova: libvirt: Use the mirror element to detect when to pivot https://review.openstack.org/399735
19:41:15 openstackgerrit Lee Yarwood proposed openstack/nova: libvirt: Make guest.BlockDevice only use the target dev of a disk https://review.openstack.org/402163
19:41:15 openstackgerrit Lee Yarwood proposed openstack/nova: libvirt: Wait for abort_job to complete when not resizing https://review.openstack.org/382449
19:42:09 diana_clarke dansmith: perhaps this patch? https://review.openstack.org/#/c/418590/
19:43:00 dansmith EmilienM: diana_clarke yep
19:43:04 EmilienM oh wow
19:43:07 EmilienM but it's not only scheduler
19:43:09 EmilienM it's also compute
19:43:19 dansmith EmilienM: it's only compute, and it was intentional, but a little naive
19:43:50 EmilienM https://review.openstack.org/#/c/418590/4/nova/scheduler/client/report.py is run by nova-compute?
19:43:56 dansmith EmilienM: yes
19:44:02 EmilienM nice
19:44:06 EmilienM thanks!
19:44:07 dansmith EmilienM: it's the scheduler *client* which is the client of the scheduler, which would be compute
19:44:17 EmilienM we need this patch :D
19:44:25 dansmith EmilienM: I wrote it for you dude :)
19:44:48 EmilienM dansmith: yeah? I just found the bug in tripleo :-P
19:45:00 EmilienM we deploy placement but in some random cases, it was not starting.
19:45:18 dansmith EmilienM: I did, specifically to make the upgrade case a little more graceful
19:45:24 dansmith EmilienM: planning to backport that to newton
19:46:26 EmilienM dansmith++
19:47:25 EmilienM dansmith: do you think https://review.openstack.org/#/c/418590/ can merge today?
19:52:16 dansmith EmilienM: you should hassle someone that can do something about it
19:52:28 dansmith EmilienM: jaypipes once was +2 on that and may grant your wish
19:56:02 EmilienM jaypipes: in case you have some time to review, please: https://review.openstack.org/#/c/418590/
20:01:18 mriedem claudiub|2: you could emit a warning in the logs...
20:02:16 mriedem claudiub|2: i'd also get it into the numa docs in the admin guide: http://docs.openstack.org/admin-guide/compute-cpu-topologies.html - so tag the patch with DocImpact and leave a comment in the commit message about updating those docs?
20:02:31 diana_clarke mriedem: I wasn't sure what to work on instead, so I am going to keep playing with microversions & osc. We'll see how it goes. https://review.openstack.org/#/c/419108/
20:03:28 mriedem dansmith: are you going to tweek the docs in a follow up to https://review.openstack.org/#/c/418590/ ?
20:03:38 mriedem i.e. http://git.openstack.org/cgit/openstack/nova/tree/doc/source/placement.rst?id=f5d7ea52c681611c2d85f344da643f9f630d1ef9#n108
20:04:30 dansmith mriedem: tbh, it's not high on my list considering time available and the rush to unfsck the current stuff
20:04:38 dansmith mriedem: if you think it's a priority for right now then I will,
20:04:49 dansmith but if not, maybe we could find someone interested in doing it?
20:05:14 diana_clarke dansmith, mriedem: I can try and do it.
20:05:41 dansmith diana_clarke: cool, thanks
20:06:18 mriedem i don't really see how spamming the logs fixes the complaints i was hearing from the red hat people yesterday,
20:06:27 mriedem but if it's something you guys have agreed it makes your lives easier somehow, then ok
20:06:40 mriedem that's all, re: why i'm not rushing it in
20:07:01 dansmith mriedem: spamming the logs doesn't fix anything for us, continuing to try does
20:07:09 openstackgerrit Dan Smith proposed openstack/nova: Make metadata server know about cell mappings https://review.openstack.org/417962
20:07:11 openstackgerrit Dan Smith proposed openstack/nova: WTFWIP: Multicell support for instance listing https://review.openstack.org/396775
20:07:11 openstackgerrit Dan Smith proposed openstack/nova: Don't bypass cellsv1 replication if cellsv2 maps are in place https://review.openstack.org/418141
20:09:05 mriedem ok so in a point release rhops rolls out creds to the newton computes and restarts nova-compute on them, right? which as long as they keep trying to connect to placement, it just happens whenever rhosp rolls out the placement service in an update for newton? i guess i just figured the restart that you need to do for the computes would come after placement is available, at which point we don't need the continual retry
20:09:52 dansmith mriedem: we can't really enforce that ordering,
20:10:06 dansmith in general, and for a couple different approaches to the upgrade
20:10:33 mriedem ok so you/we might avoid an unnecessary restart of the computes depending on the order of the upgrade
20:10:35 mriedem which is the point
20:11:10 mriedem w/o the patch, you could have deployed creds updates to the computes, restarted them, then rolled out placement, at which point you have to go back and restart all of the computes again
20:11:22 mriedem and it's that potential 2nd mass restart that is the point of the patch (trying to avoid it)
20:12:52 dansmith at least that yeah
20:13:09 mriedem ok
20:13:16 mriedem i think i can smell what you're cooking now
20:14:18 dansmith so, that and this gets us a little more nag spam in the logs to make sure that people running newton see that this is required before they have kicked off their upgradeand decided to skim the release notes once things are broken
20:14:32 dansmith which goes back to our not so awesome messaging, retroactive "oh and also..." thing in the point release, etc
20:21:40 EmilienM dansmith: look at what we have to do when we missed nova cells v2 in newton https://review.openstack.org/#/c/419121/1/elements/puppet-stack-config/puppet-stack-config.pp
20:22:31 jaypipes mriedem, claudiub|2: hyper-v osvif patch reviewed.
20:23:20 dansmith EmilienM: what about it?
20:23:44 edleafe jaypipes: got a resource provider question if you have a few minutes
20:23:49 EmilienM dansmith: I just wanted to show you the complexity of things about nova cells v2 upgrade from newton to ocata
20:23:58 dansmith EmilienM: to make me feel bad?
20:24:10 EmilienM dansmith: no :-) so you can maybe give feedback
20:24:55 EmilienM and yes, hopefully you feel very sad
20:25:16 EmilienM because we missed it in tripleo/newton, now we have to do that thing
20:25:16 edleafe jaypipes: if I have a reference to a provider, say 'rp', and call rp.set_inventory(inv), that 'rp' reference is now stale, since it now 1 generation behind the DB.
20:25:16 openstackgerrit Diana Clarke proposed openstack/nova: docs - Connect to placement service & retries https://review.openstack.org/419123
20:25:41 EmilienM jaypipes: thx for the +w :-)
20:26:05 edleafe jaypipes: my question is: should methods that change generation also update the generation field in the current object?
20:26:18 claudiub|2 jaypipes: ty, will upload a new ps shortly. :D
20:26:42 diana_clarke dansmith: It's hard to write about things you don't know much about, but there goes ;) ^^
20:26:48 dansmith diana_clarke: looks good tome
20:26:51 dansmith also to me
20:27:41 dansmith diana_clarke: mriedem will -1 one time to eff up my stats, but otherwise I think it covers all the things
20:29:30 mriedem dansmith: EmilienM: jaypipes: postmortem https://review.openstack.org/#/c/418590/ +1 fwiw with a more detailed recap of my understanding of the issue the patch is trying to avoid/fix
20:29:39 mriedem i just got distracted with PTG talk in another channel
20:30:00 dansmith mriedem: it's not dead yet.. it's like peri-mortem
20:30:17 EmilienM mriedem: ack
20:30:40 EmilienM I like the idea of https://review.openstack.org/#/c/418590/ being backported fwiw
20:30:58 EmilienM I'm pretty sure it will really help deployers
20:31:23 jaypipes edleafe: https://github.com/openstack/nova/blob/master/nova/objects/resource_provider.py#L306
20:31:56 jaypipes EmilienM: np :)
20:32:10 edleafe jaypipes: that's a static method that only updates the DB
20:32:30 mriedem EmilienM: so in retrospec, i guess we're going to need to figure out a working relationship on how to communicate to the tripleo guys that even though something is optional now, you're going to want to bake it into your tooling, because release notes aren't going to cut it
20:32:33 edleafe jaypipes: the object that called it is now stale
20:32:35 mriedem *retrospect
20:32:49 EmilienM mriedem: I do agree
20:32:58 mriedem i.e. if nova has to touch grenade for ANY reason, someone from tripleo needs to know
20:33:08 mriedem s/tripleo/any of the 40 openstack deployment projects/
20:33:12 dansmith mriedem: I was just about to say, it's in the grenade history
20:33:23 jaypipes edleafe: https://github.com/openstack/nova/blob/master/nova/objects/resource_provider.py#L372
20:33:32 jaypipes edleafe: the object that called it is self.

Earlier   Later