Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-20
17:15:59 melwitt woo, thanks
17:16:51 gibi mriedem, melwitt: hm, this means that sending a notification causes a REST call to glance. interesting...
17:17:07 melwitt the trace says 'internal' and 'public' though so it seems like they haven't changed that
17:17:12 mriedem File "/opt/stack/nova/nova/notifications/base.py", line 398, in info_from_instance
17:17:12 mriedem     context)
17:17:12 mriedem   File "/opt/stack/nova/nova/image/api.py", line 65, in generate_image_url
17:17:33 melwitt gibi: yeah, that surprised me
17:17:57 gibi melwitt: me too :)
17:18:33 mriedem there was also something related to this which we fixed at the end of queens
17:18:48 gibi mriedem: this one? https://review.openstack.org/#/c/511397/14/nova/notifications/base.py
17:18:50 mriedem https://github.com/openstack/nova/commit/62ef6cfcf01d84813f71d1e8252b86c170ee39f0
17:19:20 gibi mriedem: OK yours is different and seems more relevant
17:20:41 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Allow to specify granular CPU feature flags https://review.openstack.org/534384
17:20:42 mriedem they likely need to update nova, or restack their devstack env, and try again
17:22:59 melwitt the commit id they pulled is from march 4 though, so they should have that fix
17:23:30 openstackgerrit Merged openstack/nova master: Remove old flavor_extra_specs_get db api method https://review.openstack.org/539706
17:24:33 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Correct documentation for '[pci] passthrough_whitelist' https://review.openstack.org/552874
17:26:11 mriedem ildikov: a real hum dinger https://bugs.launchpad.net/nova/+bug/1757190
17:26:13 openstack Launchpad bug 1757190 in OpenStack Compute (nova) "resize fails with volume multiattach using with libvirt 4.0.0 (and qemu 2.11.1): Failed to get shared "write" lock" [Medium,Triaged] - Assigned to Matt Riedemann (mriedem)
17:26:14 mriedem will fix after lunch
17:26:32 mriedem from what i can tell, it has nothing to do with the actual libvirt/qemu versions, except maybe older libvirt/qemu were masking a bug in nova
17:27:00 mriedem during resize we blow away our special 'multiattach' flag in the connection_info here https://github.com/openstack/nova/blob/f80b4e50093002f84b43ff245a605fbe44d34711/nova/virt/block_device.py#L639
17:27:02 mriedem derp
17:28:00 ildikov mriedem: nasty... :/
17:28:53 ildikov mriedem: I guess it gives us another reason to find a better way to pass the multiattach info
17:30:16 gibi melwitt, mriedem: we call glance API from the notification sending due to the else branch here https://github.com/openstack/nova/blob/24379f1822e3ae1d4f7c8398e60af6e52b386c32/nova/image/glance.py#L120
17:30:45 gibi melwitt,mriedem: s/glance/keystone/
17:31:48 gibi melwitt, mriedem: and that ksa code was introduced here https://review.openstack.org/#/c/488137/23/nova/image/glance.py
17:33:18 melwitt okay, yeah, and we're failing on L126 on endpoint = utils.get_endpoint(ksa_adap)
17:33:19 gibi melwitt, mriedem: before that ^^ the notification url generation only depened on the config params but after that it can fall back to keystone
17:33:51 openstackgerrit Surya Seetharaman proposed openstack/nova master: Add CellMappingList.get_enabled_or_disabled(disabled=False) query method https://review.openstack.org/550188
17:33:52 openstackgerrit Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527
17:34:00 gibi melwitt: I think so yes
17:34:43 melwitt I meant that's what's in the trace. so that fails before we ever get to the code that will try to strip the version from the url
17:35:41 gibi melwitt: this also means that you can workaround the problem by setting CONF.glance.api_servers in the nova.conf
17:35:59 openstackgerrit Merged openstack/nova master: Remove old flavor_access_remove db api method https://review.openstack.org/539709
17:37:50 melwitt yeah, true. on the surface, it seems like there must be something wrong with their service catalog, because this is a straight call to keystone, or so it seems
17:38:13 openstackgerrit Merged openstack/nova master: Remove old flavor_access_add db api methods https://review.openstack.org/539714
17:38:56 gibi melwitt: yes, hence my classification of this change as only a workaround and not a real solution
17:39:15 gibi mriedem, melwitt: I have to leave now, but I will read back tomorrow to see if we have to do someting about not talking to keyston from the notification sending codepath
17:39:32 melwitt thanks gibi
17:48:14 edmondsw melwitt there are a ton of unapproved specs... is there any focus on reviewing those and knocking down that queue?
17:48:24 edmondsw I know the powervm spec has been sitting without comment since Feb
17:57:10 melwitt edmondsw: I think several people have been reviewing specs. I've been busy with PTG summary writeups and getting the runways proposal going. spec freeze is Apr 19 so I'm going to send email this week to get input on a spec review day date where everyone will focus on spec reviews
17:58:56 edmondsw melwitt I know a bunch of spec reviews are going on. Just wondering if there is a concerted focus on burning down the queue. Sounds like you're thinking about that
17:59:04 edmondsw kinda a prereq for the runways stuff
17:59:09 edmondsw gotta get the spec approved first :)
18:00:04 dansmith edleafe: around?
18:01:33 melwitt yeah. as usual, the beginning of the cycle people are reviewing specs more because as you said, have to approve things before we can focus on reviewing the implementations. I think people are already doing that. last cycle we had a 79% approved spec/bp completion percentage so I think this cycle we're probably going to approve fewer things to increase that percentage significantly
18:02:19 melwitt like last cycle, we'll have a dedicated spec review day to burn through a lot of them before spec freeze. that's what I'm going to send email about this week so we can pick a date that works for most
18:02:59 edleafe dansmith: yeah - somewhat distracted atm
18:03:41 dansmith edleafe: okay, I'm having trouble getting member_of to work and there's kindof a missing case in your tests, which I thought maybe was covering up a bug, but I tweaked it and it still passes, so .. I'm still digging
18:04:01 dansmith edleafe: but, I've associated a provider with an aggregate, but when I member_of it, I get back no candidates
18:04:36 edleafe dansmith: that's... odd
18:04:47 edmondsw melwitt just a little anxious not seeing reviews on the powervm spec. I'd have thought it was a fairly easy review. I believe the nova team is already committed to this effort and it's just a matter of how much or how little we bite off per release, not whether. I'd like to get the rocky content locked down.
18:05:08 edleafe dansmith: the gabbi tests show that exact case
18:05:31 edmondsw we've been putting up commits for review, but until the spec is approved hard to ask anyone to look at functional commits
18:05:33 melwitt edmondsw: has it been previously approved? I don't know the history about it
18:05:49 edmondsw melwitt yes, and the history is all in the spec at the end
18:05:55 dansmith edleafe: yeah, I know, that's why I'm trying to diff what I'm doing with those
18:06:02 edmondsw melwitt it's a multirelease effort
18:06:21 edmondsw melwitt since it was considered too much to bite off in one release from a reviewer bandwidth perspective
18:06:25 dansmith edleafe: you don't have a case where you only have one thing in member_of that actually returns results, but I shaved one down and it still seems to work (although it's hard to know if it's returning the thing we expect
18:06:48 edmondsw melwitt https://review.openstack.org/#/c/545111/
18:08:08 artom So, do we handle *any* NUMA stuff during live migration? I know hugepages and CPU pinning aren't, but what about plain old NUMA topology?
18:09:03 openstackgerrit Merged openstack/nova master: Remove old flavor_access_get_by_flavor_id db api method https://review.openstack.org/539720
18:09:04 melwitt edmondsw: okay, probably just needs some reminding then. looks like mriedem has reviewed it last month so maybe he can take another look at it soon
18:09:21 edleafe dansmith: you could inspect the returned a-c, but that's always too complicated to write in gabbi
18:09:46 dansmith edleafe: yeah, I'm doing that in my real test and getting back [], but I dunno how to do much with the gabbit
18:09:47 cdent edleafe: untrue, either make the test verbose: True or make it fail intentionally in the response_json_paths
18:09:54 openstackgerrit Merged openstack/nova master: Remove old flavor_destroy db api method https://review.openstack.org/539729
18:10:00 dansmith but it's checking the length of allocation_requests
18:10:04 cdent and the error message will show the full response
18:10:23 openstackgerrit Merged openstack/nova master: Remove old flavor_get_by_flavor_id db api method https://review.openstack.org/539733
18:10:35 openstackgerrit Merged openstack/nova master: Remove old flavor_get_by_name db api method https://review.openstack.org/544379
18:10:36 dansmith cdent: ah, duh, trying
18:10:47 openstackgerrit Merged openstack/nova master: Remove old flavor_get db api method https://review.openstack.org/544621
18:10:58 openstackgerrit Merged openstack/nova master: Remove old flavor_get_all db api method https://review.openstack.org/544688
18:11:10 openstackgerrit Merged openstack/nova master: Remove old flavor_create db api method https://review.openstack.org/544703
18:11:35 openstackgerrit Merged openstack/python-novaclient master: Fix local test fails with pypy https://review.openstack.org/553426
18:11:45 dansmith cdent: I tried just setting the asserted length to something wrong, but it doesn't show me anything else other than 17 != 1
18:12:03 cdent dansmith: yeah, I meant something more breaking that that:
18:12:19 cdent response_json_paths:\n $: foo
18:12:39 cdent will try to compre the whole json object to foo
18:14:11 vivsoni__ In case of 'nova create' - NO cinder API is called
18:14:18 vivsoni__ In case of 'nova delete' - NO cinder API is called
18:15:10 vivsoni__ Hi Team, please correct if my understanding is wrong
18:15:25 mriedem if you're booting from volume then of course cinder is called
18:15:44 mriedem if you've attached volumes before you delete the instance, of course cinder is called
18:15:46 vivsoni__ mriedem: ok
18:16:36 edleafe cdent: /me learns new gabbi trick
18:16:51 dansmith cdent: edleafe: yeah, well, that clearly shows it's working as expected
18:16:56 dansmith so I be stumped
18:17:04 dansmith if I don't pass member_of, I get back the things I expect
18:17:42 vivsoni__ mriedem: so if my instance is attached to volume, then if i shutoff the nova instance and restart the instance... then cinder api of detach and attach is called is it ?
18:17:53 dansmith interestingly, the request logging from placement is urlencoded from inside my functional test, but not when the gabbit runs
18:18:10 dansmith so I wonder if it's getting more than the uuid
18:18:12 mriedem vivsoni__: no, if you're just stopping and starting the instance, nova doesn't detach the volume
18:19:00 edleafe dansmith: are you sending a single uuid string, or a 1-element list?
18:19:30 dansmith edleafe: I've tried both, initially just one, then tried in:$uuid

Earlier   Later