| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-23 | |||
| 21:14:21 | lyarwood | there's another error when trying to use multipathd but again that looks like a poorly logged issue in os-brick and nothing to do with fips | |
| 21:14:24 | lyarwood | Sep 08 17:16:49.575982 centos-8-stream-rax-dfw-0026368401 cinder-volume[109619]: INFO cinder.volume.manager [None req-e52cdd22-a649-4134-9a83-75b41fc5f2d3 tempest-TestEncryptedCinderVolumes-1784657899 None] Created volume successfully. | |
| 21:16:04 | lyarwood | but that said | |
| 21:16:05 | lyarwood | 2021-09-08 17:21:22,863 117508 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ' |
|
| 21:16:13 | lyarwood | ^ that's dumped when we fail the test | |
| 21:16:20 | lyarwood | so the guestOS didn't boot | |
| 21:22:50 | lyarwood | ade_lee: Yeah this is weird, the volume creation looks fine but there's nothing being logged to the console of the instance | |
| 21:23:01 | lyarwood | ade_lee: You might want to ask the Cinder folks to look at this tbh | |
| 21:23:31 | lyarwood | from a Nova POV the instance boots correctly so the disk is at least currently encrypted with the expected passphrase etc | |
| 21:23:40 | lyarwood | correctly* | |
| 21:24:22 | ade_lee | lyarwood, ack thanks for looking. it is weird. | |
| #openstack-nova - 2021-09-24 | |||
| 00:26:32 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| 06:27:55 | frickler | good morning nova, it seems you are running the l-c job in gate as non-voting, which doesn't make sense to me, see e.g. https://review.opendev.org/809955 | |
| 07:31:06 | bauzas | good Friday, Nova | |
| 07:31:26 | bauzas | frickler: tell me, we had issues with this job due to some dep | |
| 07:32:04 | bauzas | frickler: saw the mailing thread from gibi ? can find it if you want | |
| 07:36:08 | bauzas | actually, the problem is on placement, not nova | |
| 07:39:52 | elodilles | well, the problem is there in several projects | |
| 07:40:05 | bauzas | elodilles: I don't disagree | |
| 07:40:17 | elodilles | bauzas: good to hear that :) | |
| 07:40:33 | bauzas | but I'd somehow appreciate that we could only make the job non-voting only when needed | |
| 07:40:56 | bauzas | for nova, ussuri and later provide decorator>=4.0.0 | |
| 07:40:57 | elodilles | and nova has it too in ussuri and older branches | |
| 07:41:33 | bauzas | oh, strange https://github.com/openstack/nova/blob/stable/ussuri/lower-constraints.txt#L21 | |
| 07:41:35 | bauzas | oh shit | |
| 07:41:45 | bauzas | 22 to 24 is 3 releases | |
| 07:42:05 | bauzas | xena being the 24th, ussuri is 4 releases older | |
| 07:42:13 | bauzas | I forgot about victoria :D | |
| 07:42:33 | bauzas | probably the effect of not having the PTG physically located for the first time :) | |
| 07:42:54 | elodilles | anyway, gibi has a solution, which actually solves the situation: https://review.opendev.org/c/openstack/nova/+/810461 | |
| 07:43:52 | elodilles | maybe we should use that one directly and not merge the 'set l-c as non-voting' patch (which meant to be as 'temporary') | |
| 07:44:54 | bauzas | maybe | |
| 08:01:17 | gibi | morning | |
| 08:10:45 | kashyap | lyarwood: Morning; that error about "Could not allocate dynamic translator buffer" -- I recall seeing it in the past (in OpenStack CI) -- it was due to out-of-memory mostly | |
| 08:30:49 | lyarwood | ACK, just getting my haircut now, I'll look again at the memory tracking log in the job when I get back | |
| 08:40:04 | kashyap | Have a good one | |
| 09:24:31 | bauzas | gibi: does that ring a bell to you if I say that when the virt driver returns a list of inventories, we don't remove the existing RPs that are no longer used ? | |
| 09:24:44 | bauzas | context : https://bugs.launchpad.net/nova/+bug/1944031 | |
| 09:25:48 | bauzas | hmmmm, looking at https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L8226 | |
| 09:26:10 | bauzas | we update the provider tree | |
| 09:26:28 | bauzas | but I guess if we have resource providers, we don't verify whether they're still used | |
| 09:47:14 | opendevreview | Pierre Riteau proposed openstack/nova master: Create empty pcpuset for unpinned instances https://review.opendev.org/c/openstack/nova/+/810849 | |
| 10:11:17 | gibi | bauzas: hm, it could be that we never had to delete any RP before VGPU move to its own RP | |
| 10:11:29 | gibi | from nova-compute | |
| 10:11:36 | gibi | so we might missed that case | |
| 11:06:03 | opendevreview | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1944759 https://review.opendev.org/c/openstack/nova/+/810763 | |
| 11:14:55 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Walk database migrations in correct order https://review.opendev.org/c/openstack/nova/+/810291 | |
| 11:14:55 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Add migration to resolve shadow table discrepancies https://review.opendev.org/c/openstack/nova/+/805738 | |
| 11:14:56 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Address some nits with database migration series https://review.opendev.org/c/openstack/nova/+/810856 | |
| 11:14:56 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Silence noise from database tests https://review.opendev.org/c/openstack/nova/+/810857 | |
| 11:41:52 | opendevreview | Lee Yarwood proposed openstack/nova master: Add check job for FIPS https://review.opendev.org/c/openstack/nova/+/790519 | |
| 12:05:23 | gibi | sean-k-mooney: do you remember where we are with mixing numa aware live migration with sriov live migration? Does src_compute(PF-numa0) -> dest_compute(PF-numa1) works? | |
| 12:05:42 | gibi | I do remember that we had issues but I don't if we solved them or just punted them for later | |
| 12:06:14 | gibi | and I only have lab nodes where both PF on numa1 so I cannot test it | |
| 12:10:10 | sean-k-mooney | gibi: yes it should | |
| 12:10:53 | sean-k-mooney | although most of the testing i did was with nic that did not report numa affinity | |
| 12:11:30 | sean-k-mooney | but i did force cross numa migration viat the cpu_dedicated_set and test that with sriov | |
| 12:12:06 | sean-k-mooney | but i would have been using the prefer policy effectivly by relaying on the fact my nics did not report numa affinity | |
| 12:12:42 | sean-k-mooney | gibi: you can try testing it with the prefer policy in your case | |
| 12:13:14 | sean-k-mooney | the ohter way to test it is technially the numa filed in /sys is writable | |
| 12:13:25 | gibi | hm, interesting :D | |
| 12:13:29 | sean-k-mooney | so if you echo 0 into the file and restart libvirt ... | |
| 12:13:42 | sean-k-mooney | i have done that in the past to fake it too with my hardware | |
| 12:13:59 | gibi | OK, I can try that, thanks for the idea | |
| 12:14:26 | opendevreview | Lee Yarwood proposed openstack/nova-specs master: Repropose flavour and image defined ephemeral storage encryption https://review.opendev.org/c/openstack/nova-specs/+/810867 | |
| 12:14:27 | opendevreview | Lee Yarwood proposed openstack/nova-specs master: Repropose Add libvirt support for flavor and image defined ephemeral encryption https://review.opendev.org/c/openstack/nova-specs/+/810868 | |
| 12:15:13 | sean-k-mooney | gibi: since i have you have you seen nova.exception.PortNotUsable: Port 123c94fa-a71a-48c1-9195-ec2a1d6d2f40 not usable for instance 36441284-b272-439d-b8cf-f7c62efc9151. before | |
| 12:15:54 | gibi | hm I saw but I have to dig, I think there are multiple reasons for that | |
| 12:16:13 | sean-k-mooney | i must have been lucky to never hit it till now | |
| 12:16:27 | sean-k-mooney | im currently trying to test vdpa with ovn | |
| 12:16:38 | gibi | one reason is that the port is in a different project than the instance | |
| 12:16:39 | sean-k-mooney | ill dig into it on the neutron side and see whats going on | |
| 12:16:50 | gibi | nova explicitly checks that | |
| 12:17:03 | sean-k-mooney | it is failing in _validate_requested_port_ids so maybe | |
| 12:17:21 | gibi | https://github.com/openstack/nova/blob/c8940f9d60f1b0290ebea94fb6174efac9a1632e/nova/network/neutron.py#L737 | |
| 12:17:28 | gibi | yepp that will be it | |
| 12:17:34 | sean-k-mooney | yep https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L736-L738 | |
| 12:19:12 | sean-k-mooney | ya ok prot is owned by demo and vm is owned by admin | |
| 12:19:45 | sean-k-mooney | ok so horrizon does not work with vdpa ports | |
| 12:20:14 | sean-k-mooney | they dont show up in the port list in the instance create as demo | |
| 12:20:24 | sean-k-mooney | maybe i shoudl just recreated it and test this form the cli | |
| 12:22:17 | sean-k-mooney | oh active :) | |
| 12:22:24 | sean-k-mooney | i was not expecting that :P | |
| 12:31:42 | opendevreview | Pierre Riteau proposed openstack/nova master: Create empty pcpuset for unpinned instances https://review.opendev.org/c/openstack/nova/+/810849 | |
| 13:06:48 | bauzas | gibi: I guess we would have the same concern (deleting old and not used RPs) for vpmems, nope ? | |
| 13:08:24 | gibi | vpmems has its own RP? | |
| 13:08:33 | gibi | or is it tracked on the compute RP? | |
| 13:21:58 | opendevreview | sean mooney proposed openstack/nova master: [DMN] test removal of CAP_DAC_OVERRIDE https://review.opendev.org/c/openstack/nova/+/810906 | |
| 13:22:17 | bauzas | gibi: my bad, those are just inventories from the same RP | |
| 13:22:19 | sean-k-mooney | gibi: i think its on the compute node currently | |
| 13:22:31 | gibi | yepp that what was I assumed | |
| 13:22:33 | bauzas | sean-k-mooney: yeah, https://github.com/openstack/nova/blob/62406b5728077afa9cd38d5c5d510bba64c43bd7/nova/virt/libvirt/driver.py#L8379 | |
| 13:22:33 | sean-k-mooney | with 1 inventory per namespace | |
| 13:22:50 | sean-k-mooney | *namespace size | |
| 13:23:06 | bauzas | then, maybe bandwidth-aware resource providers could be impacted or is that only a VGPU thing ? | |
| 13:23:23 | bauzas | actually, this doesn't come from the virt driver | |
| 13:23:28 | gibi | bauzas: bwm RPs are managed by neutron agents | |
| 13:23:32 | bauzas | yeah | |
| 13:23:38 | sean-k-mooney | what are you currently talking about by the way | |
| 13:23:39 | bauzas | so I guess this is maybe unrelate | |