| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-21 | |||
| 12:54:19 | aspiers | sean-k-mooney: not adding, more like moving and updating | |
| 12:54:25 | sean-k-mooney | that looks like a potentally good approch to take | |
| 12:54:37 | sean-k-mooney | refactoring :) | |
| 12:56:01 | sean-k-mooney | im in two minds about your test__get_machine_types fucntion | |
| 12:56:46 | sean-k-mooney | i dont like use hardcoding data like that in tests that are that coupled but i guess its ok | |
| 12:57:30 | sean-k-mooney | i guess its fine since it matching the fixture but it would be nic if you could get teh expected arch/machine types form the fixture | |
| 12:58:35 | sean-k-mooney | ill think about it over lunch but ill proably +1 goes to actully have lunch now | |
| 12:59:07 | aspiers | sean-k-mooney: thanks :) | |
| 13:17:26 | mdbooth | dansmith: I've got a much better feeling about this one: https://bugs.launchpad.net/nova/+bug/1840912 | |
| 13:17:27 | openstack | Launchpad bug 1840912 in OpenStack Compute (nova) "libvirt calls aren't reliably using tpool.Proxy" [Undecided,New] | |
| 13:20:45 | dulek | Hi! This is probably trivial, but I see this in n-cpu logs on a fresh DevStack instance: `libvirtError: Cannot access storage file '/opt/stack/data/nova/instances/403824ef-422f-4e23-9e3a-d897766aa70d/disk' (as uid:107, gid:107)`. Any ideas how to get rid of that? | |
| 13:29:33 | dansmith | mdbooth: okay, have to do something first but then I'll look | |
| 13:30:36 | mdbooth | dansmith: That's just the bug description, btw. Patch incoming. Patch might touch a couple of testing weird spots, so I'm expecting discussion/rework. | |
| 13:33:03 | dansmith | mdbooth: I'll be looking for why this wasn't a thing in the first eight years of its life, of course :) | |
| 13:33:39 | mdbooth | dansmith: I think it was ;) I suspect there's a point 2 to this bug which is why libvirt is being slow enough for this customer to trigger it. | |
| 13:33:52 | mdbooth | However, we tested it and it's definitely not using the proxy | |
| 13:42:58 | dansmith | mdbooth: ah, I see, looks like a good lead | |
| 13:46:05 | openstackgerrit | Adam Spiers proposed openstack/nova master: Make memtune parameters consistent with libvirt docs and code https://review.opendev.org/636301 | |
| 13:53:23 | openstackgerrit | Adam Spiers proposed openstack/nova master: Track libvirt host/domain capabilities for multiple machine types https://review.opendev.org/673151 | |
| 13:57:35 | openstackgerrit | Matthew Booth proposed openstack/nova master: libvirt: Fix service-wide pauses caused by un-proxied libvirt calls https://review.opendev.org/677736 | |
| 14:10:25 | openstackgerrit | Adam Spiers proposed openstack/nova master: Provide HW_CPU_X86_AMD_SEV trait when SEV is supported https://review.opendev.org/638680 | |
| 14:10:25 | openstackgerrit | Adam Spiers proposed openstack/nova master: Extract new base class for provider usage functional tests https://review.opendev.org/676964 | |
| 14:10:35 | openstackgerrit | Adam Spiers proposed openstack/nova master: Add extra spec parameter and image property for memory encryption https://review.opendev.org/664420 | |
| 14:10:49 | openstackgerrit | Adam Spiers proposed openstack/nova master: Extract SEV-specific bits on host detection https://review.opendev.org/636334 | |
| 14:11:03 | openstackgerrit | Adam Spiers proposed openstack/nova master: Add |
|
| 14:11:14 | openstackgerrit | Adam Spiers proposed openstack/nova master: Apply SEV-specific guest config when SEV is required https://review.opendev.org/644565 | |
| 14:11:39 | openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | |
| 14:20:34 | openstackgerrit | Matthew Booth proposed openstack/nova master: libvirt: Fix service-wide pauses caused by un-proxied libvirt calls https://review.opendev.org/677736 | |
| 14:36:54 | openstackgerrit | Adam Spiers proposed openstack/nova master: libvirt: Parse the 'os' element from domainCapabilities https://review.opendev.org/673790 | |
| 14:39:53 | mriedem | mdbooth: can we avoid the libvirt-python test dependency? i think that will mean local dev needs to install a native package, from what i remember years ago anyway | |
| 14:40:16 | mdbooth | mriedem: Yeah, I threw that in there to have a starting point, tbh | |
| 14:40:20 | mriedem | and it was a source of problems before too | |
| 14:40:22 | mriedem | ok | |
| 14:40:31 | mdbooth | mriedem: I mostly wanted to get a full tempest run against it. | |
| 14:40:42 | mriedem | so WIP it? | |
| 14:40:55 | mdbooth | mriedem: Will do. | |
| 14:41:01 | mriedem | thanks | |
| 14:41:28 | mdbooth | mriedem: I really want eyes on it, though :) Also, ideas for useful testing without using libvirt. | |
| 14:42:00 | mdbooth | Might look at something using fakelibvirt | |
| 14:42:07 | mriedem | i assumed fakelibvirt | |
| 14:42:12 | mriedem | since that's what we always use | |
| 14:42:28 | sean-k-mooney | mdbooth: you porbaly could mock the call to inspect | |
| 14:42:48 | sean-k-mooney | and pass in a set of different moduels for it to use and assert they were wrapped | |
| 14:43:13 | mdbooth | sean-k-mooney: The test I really want to do is that Host.list_instance_domains() returns <list of tpool.Proxy(virDomain)> | |
| 14:43:56 | mdbooth | If I can get that, I'll be confident that a bunch of things are lined up | |
| 14:44:08 | sean-k-mooney | i think that would be doable with out the really libvirt module | |
| 14:44:23 | mdbooth | I think so | |
| 14:44:39 | mriedem | were you going to open an issue against eventlet for the not-handling-list thing? | |
| 14:44:51 | sean-k-mooney | we might have to asser it returns <list of tpool.Proxy(mock)> | |
| 14:45:36 | sean-k-mooney | or fakelibvirt.virDomain but we should be able to cover the behaivor i think | |
| 14:48:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props' https://review.opendev.org/674894 | |
| 14:48:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples https://review.opendev.org/675338 | |
| 14:48:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Fold in argument to '_update_provider_tree_for_vgpu' https://review.opendev.org/676729 | |
| 14:48:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 14:48:10 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 14:49:31 | dtantsur | hey folks. TheJulia and I might be seeing a race condition during ironic node take over. | |
| 14:50:00 | dtantsur | it seems that when the hash ring rebalances, the old n-cpu may clean up the node records, including the resource provide in placement | |
| 14:50:10 | sean-k-mooney | node takeover e.g. when a node is moved to a different ironic compute service | |
| 14:50:13 | dtantsur | yep | |
| 14:50:48 | sean-k-mooney | is this for all node or just ones in mantaince mode | |
| 14:50:56 | dtantsur | it was for a normal node | |
| 14:51:11 | dtantsur | apparently under some condition the new (receiving) n-cpu may not re-create the resource provider for the node it took over | |
| 14:51:19 | sean-k-mooney | dtantsur: ok mnaser noted there is an issue with nodes in mainance mode being deleted | |
| 14:51:34 | dtantsur | I think it was fixed by mriedem recently | |
| 14:52:03 | sean-k-mooney | proably i just head about it a day or two ago | |
| 14:52:39 | mnaser | sean-k-mooney, dtantsur, TheJulia: https://review.opendev.org/#/c/675704/ is the stack you might want to look at | |
| 14:52:41 | sean-k-mooney | dtantsur: its possible the reciving node check if the RP exists pefore teh souce node deletes it | |
| 14:52:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove deprecated [neutron]/url option https://review.opendev.org/677755 | |
| 14:53:27 | dtantsur | yep, that's my guess | |
| 14:53:39 | dtantsur | relevant log lines: https://etherpad.openstack.org/p/zCQ8vPcGX6 | |
| 14:53:59 | sean-k-mooney | the first patch looks relevent https://review.opendev.org/#/c/675704/ | |
| 14:54:00 | jroll | ? | |
| 14:54:09 | mriedem | mordred: efried: i'm removing the old deprecated [neutron]/url option in https://review.opendev.org/677755 and i noticed that the neutron install guide still uses it: https://bugs.launchpad.net/neutron/+bug/1840930 - as i mentioned in that neutron bug, we could just replace with endpoint_override but i don't think that's what we want, right? | |
| 14:54:10 | openstack | Launchpad bug 1840930 in neutron "Networking service in neutron - install guide says to configure nova with [neutron]/url which is deprecated" [Undecided,New] | |
| 14:54:26 | mriedem | mordred: efried: we want to avoid endpoint_override and just have config for letting KSA do it's thing with the service catalog | |
| 14:54:33 | dtantsur | jroll: see scrollback, potential race condition on take over | |
| 14:54:46 | dtantsur | you may know this code better than me | |
| 14:54:50 | TheJulia | So an important note from what dtantsur is mentioning is that we're seeing this in our grenade job, which means this is when stein is still running from what we see in the logs | |
| 14:55:32 | mriedem | stein doesn't have the fix yet | |
| 14:55:36 | mriedem | waiting for stable reviews | |
| 14:55:49 | mriedem | https://review.opendev.org/#/q/topic:bug/1839560+branch:stable/stein | |
| 14:55:50 | sean-k-mooney | TheJulia: is this only before we do the upgrade. | |
| 14:55:52 | jroll | there we go | |
| 14:56:00 | jroll | dtantsur: side question: does the new compute service re-add it on the following RT run? | |
| 14:56:06 | dtantsur | jroll: nope, never | |
| 14:56:11 | TheJulia | sean-k-mooney: well, we never actually get to perform the upgrade | |
| 14:56:12 | jroll | oh boy | |
| 14:56:13 | dtantsur | it becomes orphaned essentially | |
| 14:56:16 | efried | mriedem: Right; for people who just gotta specify, they can s/url/endpoint_override/, but it is better to just let the service catalog do its thing. | |
| 14:56:38 | mriedem | efried: yeah, i think the question is - for an install guide, we don't want to document the exceptional path | |
| 14:56:45 | sean-k-mooney | TheJulia: i was wondering if ye tried it manually but ya it would fail tempest and not upgrade so make sense | |
| 14:56:51 | mriedem | so i was just going to remove the url mention from the install guide | |
| 14:57:09 | mriedem | dtantsur: TheJulia: i'm curious, this has been a thing since rocky, so why are you hitting it now? | |
| 14:57:12 | mriedem | new tests? | |
| 14:57:13 | TheJulia | sean-k-mooney: we wouldn't even make it to tempest, there is a resource check where placement is consulted | |
| 14:57:32 | TheJulia | if we don't have the number of "baremetal" nodes, we eventually time out and fail | |
| 14:57:37 | mriedem | https://review.opendev.org/#/c/571535/ is since rocky i mean | |
| 14:57:37 | dtantsur | mriedem: I'm curious as well. The simplest explanation is that it's become more likely for some unrelated reason. | |
| 14:57:50 | efried | mriedem: We should just be able to document "do this the same way you would do any service backed by the catalog". mordred might have a central document somewhere that recommends best practices for that. If we don't have that doc, we should. | |
| 14:58:12 | sean-k-mooney | TheJulia: well if we can recreate it reliably we proably should add a depens on against the stien backport and see if that resolves it | |