| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-08-04 | |||
| 09:21:34 | gibi | bauzas: I think that only means that mypy did not assign any type to either of those variables but mypy needed to use some type for mdev_class_mapping for something else later so mypy asked the typehint from you. When you added reveal_type you forced mypy to try to assign type to both variables hence is asked typehint for both from you | |
| 09:22:21 | bauzas | gibi: yup, that's what I found | |
| 09:22:30 | bauzas | static typing, my love | |
| 09:22:57 | bauzas | anyway, this is fixed in the last rev, I added an annotation | |
| 09:23:56 | gibi | ack | |
| 09:25:04 | gibi | I'm done with the re-review of the mdev series so far so good :) | |
| 09:25:24 | bauzas | see, I haven't grumbled about mypy | |
| 09:26:20 | bauzas | sure, it's important to tell that a variable using a collections.defaultdict is a dict :p | |
| 09:26:43 | bauzas | just in case people don't know :D | |
| 09:41:38 | bauzas | gibi: I replied to your (good) concern https://review.opendev.org/c/openstack/nova/+/803233/3//COMMIT_MSG#17 | |
| 09:54:07 | stephenfin | kashyap: not an option, I'm afraid https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html | |
| 09:57:12 | kashyap | Yeah, I know "standards" | |
| 10:32:27 | sean-k-mooney | kashyap: well in this case its openstacks standard | |
| 10:32:47 | sean-k-mooney | which nova does follow | |
| 10:36:46 | sean-k-mooney | kashyap: for what its worth if qemu upstream dont remove it im not really in a rush to remove it form nova. deprecating floppy usage sure, but it does not cost us much if anything to keep it and distros can still drop support even if we support it upstream | |
| 10:39:05 | kashyap | sean-k-mooney: Hey; I see what you mean, but as noted it is more of a liability than anything at this point | |
| 10:39:17 | kashyap | (Given past CVEs. Upstream QEMU too, it's discouraged) | |
| 10:40:05 | kashyap | (Downstream distros of OpenStack can also simply declare it out of scope / deprecated.) | |
| 10:55:07 | opendevreview | Merged openstack/nova master: zuul: Increase GLANCE_LIMIT_IMAGE_SIZE_TOTAL for nova-lvm https://review.opendev.org/c/openstack/nova/+/803322 | |
| 11:00:02 | lyarwood | \o/ | |
| 11:37:28 | opendevreview | Lee Yarwood proposed openstack/nova master: libvirt: Handle silent failures to extend volume within os-brick https://review.opendev.org/c/openstack/nova/+/801714 | |
| 11:37:55 | opendevreview | Lee Yarwood proposed openstack/nova master: Add functional test for bug 1937375 https://review.opendev.org/c/openstack/nova/+/802011 | |
| 11:37:56 | opendevreview | Lee Yarwood proposed openstack/nova master: compute: Avoid duplicate BDMs during reserve_block_device_name https://review.opendev.org/c/openstack/nova/+/801990 | |
| 11:37:56 | opendevreview | Lee Yarwood proposed openstack/nova master: fup: Move _wait_for_volume_attach into InstanceHelperMixin https://review.opendev.org/c/openstack/nova/+/802623 | |
| 11:38:26 | opendevreview | Lee Yarwood proposed openstack/nova master: Add regression test for bug 1938326 https://review.opendev.org/c/openstack/nova/+/802801 | |
| 11:38:26 | opendevreview | Lee Yarwood proposed openstack/nova master: compute: Do not mark disabled but down services as in maintenance https://review.opendev.org/c/openstack/nova/+/802317 | |
| 11:38:46 | lyarwood | gibi / bauzas ; now the gate is fixed reviews on ^ would be appreciated this week if you have time | |
| 11:45:26 | gibi | lyarwood: on it | |
| 12:01:03 | gibi | lyarwood: after your lunch, is this extended race scenario possible? https://review.opendev.org/c/openstack/nova/+/801990/6/nova/compute/manager.py#6985 | |
| 12:12:11 | lyarwood | gibi: yeah that's also possible but I wonder if we want to treat it as a separate fix? | |
| 12:13:45 | lyarwood | actually thinking about it, is it an issue if we have racing requests against the same volume | |
| 12:14:58 | lyarwood | multiattach races would be caught by c-api later | |
| 12:15:00 | gibi | lyarwood: I would treat it as a separate fix. whay you proposed is a valid fix for a scenario reported int he bug | |
| 12:15:59 | gibi | hm, I guess we have the instance.uuid lock as a pattern, we have that for most of the operations | |
| 12:16:15 | gibi | but here a volume.id lock would be better | |
| 12:16:26 | gibi | I'm not sure we even need the instance.uuid lock here | |
| 12:16:32 | gibi | but could be | |
| 12:27:20 | stephenfin | bauzas: Can we trade reviews for your generic mdev series and my DB series? I'm really eager to close out as much as I can this week, especially with a few people on PTO next week | |
| 12:27:50 | bauzas | stephenfin: for the moment, I'm working on the functest | |
| 12:27:57 | bauzas | for the mdev series, asked by gibi | |
| 12:28:08 | bauzas | but in one hour, we could do | |
| 12:29:07 | sean-k-mooney | gibi: i think we take an instace lock for volume stuff to avoid concurrent voluem attach/detach operation on the same instance | |
| 12:30:34 | sean-k-mooney | multi attach is interesting edgecase | |
| 12:30:40 | gibi | sean-k-mooney: how a reserve that only create a new bdm can race with an attach that already has the bdm created or a detach that will delete a dbm tha should already exists | |
| 12:31:09 | gibi | still, I'm not against keeping the instance.uuid lock | |
| 12:31:24 | sean-k-mooney | gibi: well i was just using attach as example not suggesting it would race with it | |
| 12:31:36 | gibi | it is a sensible safety measure | |
| 12:34:50 | opendevreview | Alexey Stupnikov proposed openstack/nova master: Fix incorrect reservation IDs in unit tests https://review.opendev.org/c/openstack/nova/+/803363 | |
| 13:09:05 | aarents | Hi nova, | |
| 13:09:26 | sean-k-mooney | o/ | |
| 13:09:31 | aarents | I'm a bit stuck with this change https://review.opendev.org/c/openstack/nova/+/799606 (see my self comment) which is only a partial fix to the bug https://bugs.launchpad.net/nova/+bug/1934742 I have in operation, | |
| 13:09:42 | aarents | I'm not sure how to properly fix that, if someone have idea I can dig.. | |
| 13:10:33 | sean-k-mooney | let me read but you having issue with pepole delete port via neutron? rather then port detach right | |
| 13:10:43 | sean-k-mooney | technially we dont really support that | |
| 13:11:10 | aarents | yep port deletion when attached | |
| 13:11:14 | sean-k-mooney | we have the network-vif-deleted event to try and help fix things | |
| 13:11:25 | sean-k-mooney | btu deletion when attached is strongly discuraged | |
| 13:12:03 | sean-k-mooney | the correct way to fix it would be to either block it in neutron or have nova call detach internally | |
| 13:14:28 | sean-k-mooney | aarents: stirckly speaking form a nova perpective deleting an interface that is attach to an instance via neutorn is a user error | |
| 13:15:10 | sean-k-mooney | and the code path you are looking at is a workaround we put in place to try an partly handel it | |
| 13:15:43 | aarents | So we can imagine calling detach_interface but it will try to to unbind and will probably got 404 from neutron which is not a big deal | |
| 13:16:23 | sean-k-mooney | we would have to handel that ya but its something we could handel | |
| 13:16:54 | sean-k-mooney | this came up a few weeks ago though in the neutron driver meeting an i was pushing to block deleteing attach ports in the neutron api | |
| 13:17:00 | sean-k-mooney | long term that is the correct approch | |
| 13:18:06 | sean-k-mooney | this is really a neutorn bug not a nova one but nova can do a better jobs at cleaning up when you incorrectly detelete attach ports | |
| 13:18:25 | sean-k-mooney | the same is true for attach volumes by the way | |
| 13:18:40 | sean-k-mooney | you should never try to delete an attach volume but in that case i belive cinder will protect you | |
| 13:19:50 | sean-k-mooney | gibi: you rememebr that conversation about 2-3 weeks ago in the drivers meeting right? i think the resolution was to write a neutron spec for the new extnsion | |
| 13:19:51 | aarents | sean-k-mooney: I see your point about not a nova issue, My issue is I'like to avoid this possible leak because it generates un live_migrable instances in infra.. | |
| 13:20:14 | sean-k-mooney | yep im sure it does and yes that is why we try to clean it up | |
| 13:20:36 | sean-k-mooney | aarents: it has other proplems it hte port that was delete uses sriov | |
| 13:20:52 | sean-k-mooney | since the pci device will not be released properly | |
| 13:21:12 | aarents | I see | |
| 13:21:32 | sean-k-mooney | aarents: sriov deattach does not work properly in any release before wallaby | |
| 13:22:09 | sean-k-mooney | and i think it sitll wont work properly if you delete an attach neturon port although ill admit i have not looked at that code path to check | |
| 13:22:35 | sean-k-mooney | i just suspect that we did not test that when we added sriov attach/detach support | |
| 13:23:01 | sean-k-mooney | aarents: anyway your partial fix is to just add a lock | |
| 13:24:34 | sean-k-mooney | that might help i in limited cases but i dont think it will in general fix this | |
| 13:24:47 | aarents | I aggre it is not enought | |
| 13:30:12 | aarents | sean-k-mooney: thks! for the usefull info, I will dig more on blocking or try a detach | |
| 13:30:44 | sean-k-mooney | aarents: by the way if we cant use detach directly because all files cant be constructed we have 2 options. | |
| 13:31:14 | sean-k-mooney | we could have neutron send the full port info in the detach payload or we could contocut as much data as we can and pass incompelte objects | |
| 13:31:43 | sean-k-mooney | but blocking it in nueton would be the best approch or having neutron call detach which is rahter messy but would work better | |
| 13:32:15 | aarents | "neutron send the full port info in the detach payload" is interesting yes | |
| 13:32:37 | sean-k-mooney | one thing we disucssed was moving when neutron sends the event | |
| 13:32:56 | sean-k-mooney | if neutorn sent the deleted event and waited for nova to ack it | |
| 13:33:01 | sean-k-mooney | then that would also fix it | |
| 13:33:06 | sean-k-mooney | to a degree at least | |
| 13:33:38 | sean-k-mooney | anyway this is a know issue that we just have not focused on so sorry your are hitting it | |
| 13:34:16 | aarents | "could contocut as much data as we can and pass incompelte objects" <- I try that but it needs details like vif_type bridge name this info are no more available when port is deleted, I don't know how regenerate that | |
| 13:35:03 | sean-k-mooney | ah yes we do need those | |
| 13:35:25 | sean-k-mooney | which may or may not be still in the info cache as you have found | |
| 13:35:59 | opendevreview | Merged openstack/nova master: Add functional test for bug 1937375 https://review.opendev.org/c/openstack/nova/+/802011 | |
| 13:36:27 | sean-k-mooney | its its a hack but we might be able to reconstuct the required data by inspecting the libvirt xml but i would prefer to avoid that | |
| 13:38:38 | aarents | sean-k-mooney: yes.. it was the workaround I was considering, kind of detach_device_by_mac, we intereate on interfaces nad deleted the good one | |
| 13:39:27 | sean-k-mooney | ya by mac works for most things excption sriov pfs but in that case you can kind of still figure it out in some cases | |
| 13:39:47 | sean-k-mooney | it really depend on how much of the db info we have acess too | |
| 13:40:33 | sean-k-mooney | we cant get the pci address form the port since its gone and the info cache may be empty but we recently started recoring the neutron port uuid as the requester id in the pci_devices table i belive | |
| 13:41:13 | aarents | the info we can grab in db are entry in virtuales_interfaces: | |
| 13:41:14 | sean-k-mooney | which means we should be able to get the pci adress by the port uuid | |