| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-01 | |||
| 19:58:20 | efried | jaypipes: People in the know, know. | |
| 19:59:33 | efried | ya know? | |
| 20:00:15 | jaypipes | heh | |
| 20:14:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup launch instance and manage IPs docs https://review.openstack.org/540121 | |
| 20:17:05 | mriedem | this needs another +2 https://review.openstack.org/#/c/531517/ | |
| 20:29:49 | dansmith | yes, yes it does | |
| 20:44:13 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: Update Provider Tree https://review.openstack.org/540111 | |
| 20:47:04 | melwitt | mriedem: do you know what causes neutron to send events? is it reacting to os-vif calls or? | |
| 20:47:24 | melwitt | I updated the bug with my latest findings on the timeout waiting for network-vif-plugged | |
| 20:47:32 | dansmith | melwitt: it's when neutron does things | |
| 20:47:36 | mriedem | melwitt: the neutron agent sends info to the server via rpc | |
| 20:47:45 | mriedem | the neutron agent is polling for changes, as far as i understand | |
| 20:48:00 | melwitt | okay, that makes sense. I'm looking in the agent logs trying to piece things together | |
| 20:48:01 | mriedem | the server then gets the rpc message from the agent(s) and turns those into rest api calls (server external events) to nova | |
| 20:48:31 | dansmith | mriedem: I dunno that it's polly, | |
| 20:48:34 | dansmith | er, polling | |
| 20:48:37 | melwitt | because what's happening is, when we start the reboot, while we're tearing down the libvirt domain, BEFORE we unplug the vif, we are getting a network-vif-plugged event from neutron | |
| 20:48:51 | melwitt | and it's not from the initial boot because we have one from that too, earlier in the log | |
| 20:48:54 | dansmith | I think it's like when you ask it to do a vif plug, it does it, and then sends an RPC to neutron server to send it out | |
| 20:49:30 | melwitt | and I'm wondering, why is it sending us network-vif-plugged if we haven't even tried to plug anything yet | |
| 20:49:31 | dansmith | melwitt: network-changed comes a lot, are you sure it was vif-pluged? | |
| 20:49:36 | melwitt | I'm sure | |
| 20:49:54 | melwitt | we do get network-changed FIRST though, right before we initiate the reboot in compute | |
| 20:50:02 | melwitt | then we get network-vif-plugged | |
| 20:50:06 | dansmith | well, it might be that they do it any time they refresh a vif into a bridge and thus send it sometimes when we're not asking for it | |
| 20:50:07 | melwitt | which makes no sense | |
| 20:50:27 | dansmith | like not always just in response to our asking for it | |
| 20:50:45 | melwitt | vif-plugged you mean? | |
| 20:51:22 | melwitt | yeah, I figured there might be some reasons it sends it randomly. but, it's also not sending us one after we plug the vif after the reboot | |
| 20:52:06 | melwitt | I was thinking the next step is to learn how it decides to send them and find what's missing there | |
| 20:52:25 | dansmith | did we unplug first? | |
| 20:52:40 | melwitt | yes. we unplugged but did not wait for an unplug event | |
| 20:52:46 | melwitt | which I wonder if that's a problem | |
| 20:53:56 | dansmith | well, no, not waiting for the event shouldn't make it not send the corresponding plug | |
| 20:54:03 | dansmith | because unless it's for an instance that doesn't exist, the api doesn't tell the person sending the event that we weren't waiting for it | |
| 20:54:29 | melwitt | yeah. agreed, just that's the only thing I could think of as a potential reason why | |
| 20:54:51 | melwitt | you know what though, looking at this again | |
| 20:55:02 | melwitt | the vif shows up as active=False for both the unplug and the plug | |
| 20:55:43 | melwitt | I wonder if that's a problem | |
| 20:56:03 | melwitt | well, actually it's *supposed* to be active=False before we plug | |
| 20:56:09 | melwitt | based on our event waiting logic | |
| 20:56:27 | melwitt | *our plug event waiting logic | |
| 20:56:45 | melwitt | we only register to wait for vifs that have active=False. so that checks out | |
| 20:58:17 | melwitt | active=True when the reboot begain, then it changed to active=False somewhere between the libvirt destroy and the unplug | |
| 20:58:27 | melwitt | *began | |
| 21:00:01 | mriedem | nova meeting time | |
| 21:00:09 | melwitt | ah thanks | |
| 21:05:51 | openstackgerrit | Merged openstack/nova master: SchedulerReportClient._delete_provider https://review.openstack.org/535463 | |
| 21:18:52 | openstackgerrit | Merged openstack/nova master: ProviderTree.get_provider_uuids: Top-down ordering https://review.openstack.org/536624 | |
| 21:19:02 | openstackgerrit | Merged openstack/nova master: set_{aggregates|traits}_for_provider: tolerate set https://review.openstack.org/536625 | |
| 21:22:16 | mriedem | efried: there all your things are merged | |
| 21:25:48 | openstackgerrit | Merged openstack/nova master: Fix nits in support traits changes https://review.openstack.org/537351 | |
| 21:25:58 | openstackgerrit | Merged openstack/nova master: Mention required traits in the flavors user docs https://review.openstack.org/539300 | |
| 21:26:02 | mriedem | sdague: if you're around, this is a simple docs migration that we missed in pike https://review.openstack.org/#/c/540048/ - content is unchanged except for fixing links; the patch on top cleans up some of the content to modernize it | |
| 21:26:13 | openstackgerrit | Merged openstack/nova master: add "--until-complete" option for nova-manage db archive_deleted_rows. https://review.openstack.org/539654 | |
| 21:26:16 | edleafe | efried: I told you to bribe the people in infra! | |
| 21:26:25 | openstackgerrit | Merged openstack/nova master: Use util.validate_query_params in list_traits https://review.openstack.org/539407 | |
| 21:26:38 | openstackgerrit | Merged openstack/nova master: Deduplicate aggregate notification samples https://review.openstack.org/531162 | |
| 21:26:51 | openstackgerrit | Merged openstack/nova master: Deduplicate service status notification samples https://review.openstack.org/534335 | |
| 21:27:17 | openstackgerrit | Merged openstack/nova master: Log options at debug when starting API services under wsgi https://review.openstack.org/519462 | |
| 21:28:13 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Regenerate context during targeting https://review.openstack.org/540145 | |
| 21:31:44 | traskat | hey, i have a question about nova and ceph. how can cinder volume gets resized to the flavor disk size if the VM image gets cloned by cinder? | |
| 21:31:51 | sdague | mriedem: done | |
| 21:34:30 | mriedem | sdague: cool thanks | |
| 21:34:43 | mriedem | wow this is a super latent bug, and goes to show no one uses the IsolatedHostsFilter https://bugs.launchpad.net/nova/+bug/1746483 | |
| 21:34:44 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Provide support matrix and doc for VGPU https://review.openstack.org/539266 | |
| 21:34:44 | openstack | Launchpad bug 1746483 in OpenStack Compute (nova) "Not able to boot from Volume / Volume snapshot when using isolated_images" [Medium,Triaged] | |
| 21:35:12 | mriedem | the IsolatedHostsFilter assumes the request spec has a valid image id in it, which isn't the case for boot from volume when you don't provide an image id | |
| 21:38:18 | openstackgerrit | Merged openstack/nova master: De-duplicate _numa_get_flavor_XXX_map_list https://review.openstack.org/385074 | |
| 21:38:35 | openstackgerrit | Merged openstack/nova stable/pike: Add 'delete_host' command in 'nova-manage cell_v2' https://review.openstack.org/513717 | |
| 21:54:45 | tssurya | mriedem: regarding the scatter_cells bug thanks for backporting it to pike, we have upgraded to ocata today, so far so good (will keep you posted if we find anything major/interesting) | |
| 21:57:11 | mriedem | tssurya: which one? | |
| 21:57:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Ensure the JSON-Schema covers the legacy v2 API https://review.openstack.org/540154 | |
| 21:57:51 | tssurya | mriedem : https://bugs.launchpad.net/nova/+bug/1746509 | |
| 21:57:53 | openstack | Launchpad bug 1722404 in OpenStack Compute (nova) ocata "duplicate for #1746509 Database transactions can fail with "TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction" because of scatter_gather_cells" [Undecided,In progress] - Assigned to Matt Riedemann (mriedem) | |
| 21:57:59 | tssurya | the one you mentioned in the meeting | |
| 21:58:48 | mriedem | ah that's not scatter/gather | |
| 21:59:05 | mriedem | and the ocata backport isn't ready yet, unit tests are failing, i haven't dug into how to fix those yet | |
| 21:59:35 | mgagne | mriedem: I don't use IsolatedHostsFilter because you would need to redeploy your config to add new hosts/images, that's suboptimal. | |
| 22:00:18 | mgagne | but good point about image_ref not being available with BFV, I'm not sure how we didn't catch that one in a similar filter we wrote | |
| 22:02:37 | mriedem | the request spec is also not good at telling anything if it's boot from volume or not | |
| 22:03:10 | mriedem | it doesn't even have the original bdms requested in it | |
| 22:03:21 | mgagne | I would have to dig into it a bit | |
| 22:03:26 | mriedem | so you'd basically have to assume that if the request spec doesn't have an image, it's volume-backed | |
| 22:04:00 | mriedem | maybe bauzas has thoughts on that one | |
| 22:07:48 | mriedem | rgerganov: can you take a look at this? (or someone from vmware?) https://bugs.launchpad.net/nova/+bug/1744182 | |
| 22:07:49 | openstack | Launchpad bug 1744182 in OpenStack Compute (nova) "can not create instance when using vmware nova driver" [Undecided,New] | |
| 22:09:40 | melwitt | mriedem: commented on the review, I think yielding the copy of the context won't help because the callers aren't using the yielded context | |
| 22:10:09 | melwitt | I'm not sure if there's any clever thing we can do instead of having to fix all the callers in ocata | |
| 22:11:20 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional tests for traits-based scheduling https://review.openstack.org/539310 | |
| 22:12:02 | mriedem | oh right... | |
| 22:12:07 | mriedem | guh | |
| 22:12:14 | melwitt | yeah :( | |
| 22:14:44 | melwitt | mriedem: looks like you've already documented the lack of neutron event in certain situations https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7508-L7512 | |
| 22:15:51 | mriedem | that's the 2nd time i've looked at that today for different reasons | |
| 22:16:24 | mriedem | i don't remember writing that though | |
| 22:16:28 | mriedem | might have been blacked out | |
| 22:16:51 | melwitt | yeah. ugh, it's my fault I even told lyarwood not to pass vifs_already_plugged=True :( | |
| 22:17:17 | melwitt | suck | |
| 22:19:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional tests for traits-based scheduling https://review.openstack.org/539310 | |