| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-26 | |||
| 12:03:58 | gibi | and I do believ that RDO missed the deadline not to raise the issue on the TC ruling | |
| 12:09:50 | sean-k-mooney | im also kind of confused why RDO have not been testign with 3.8 as well | |
| 12:10:11 | sean-k-mooney | it has been a testing requirement for some time | |
| 12:10:38 | sean-k-mooney | i guess they were usign the lower requiremtn for centos of 3.6 | |
| 12:12:02 | sean-k-mooney | but give our downstream will be using 3.9 form wallaby+ that feels like a gap someone form redhat would have been woking on fixing | |
| 12:13:47 | sean-k-mooney | py 3.9.8 currrenlty py 3.8 will not be supproted on centos 9 | |
| 12:20:28 | lyarwood | sorry back | |
| 12:20:41 | lyarwood | just noticed I missed stephenfin 's thread about this on the ML | |
| 12:30:06 | gibi | thread even | |
| 14:19:29 | artom | Huh, looks like at some point between... victoria? and master ovs+hybrid_plug became plug-time Neutron events... | |
| 14:20:55 | artom | https://zuul.opendev.org/t/openstack/build/2f75f41da4df48f8a8abc5087fd29efb/log/controller/logs/screen-n-cpu.txt#9233 | |
| 14:23:18 | tobias-urdin | lyarwood: should we ping in somebody from RDO on the above topic or did u take it already? | |
| 14:23:51 | lyarwood | tobias-urdin: Looks like folks from RDO already raised this on the ML, I hadn't seen it until the end of the discussion earlier | |
| 14:30:52 | tobias-urdin | lyarwood: ack, good to hear | |
| 14:36:09 | sean-k-mooney | artom: ovs hybrid plug was always a plugtime event | |
| 14:36:21 | artom | sean-k-mooney, not for revert resize | |
| 14:36:33 | artom | Remember 7a7a223602ca5aa0aca8f65a6ab143f1d8f8ec1b? | |
| 14:36:34 | sean-k-mooney | for revert its already plugged | |
| 14:36:39 | artom | Looks like we don't need it anymore | |
| 14:36:40 | sean-k-mooney | so its bind time | |
| 14:37:12 | artom | Apparently that's no longer the case, see the log link | |
| 14:37:23 | artom | It's from https://review.opendev.org/c/openstack/nova/+/819349 | |
| 14:38:22 | sean-k-mooney | for resize we dont touch the netwroking on the souce host until you confrim | |
| 14:38:43 | sean-k-mooney | we do update the port binding (there is only one) before we go to resize verify | |
| 14:38:53 | sean-k-mooney | then on revert we update the host back to the soruce | |
| 14:38:56 | artom | Well I have no idea what happened, but we now get plug-time events, and not bind-time events | |
| 14:39:08 | sean-k-mooney | but we should not need to set up the networking on the host again | |
| 14:39:23 | artom | I mean, I don't disagree | |
| 14:39:44 | artom | But based on what my CI job run is telling me, revert resize is now plug-time like everything else (again) | |
| 14:39:56 | sean-k-mooney | we are going to get plug time events as well i think | |
| 14:40:03 | sean-k-mooney | but we should get bind time events first | |
| 14:40:12 | artom | Never get the former | |
| 14:40:22 | sean-k-mooney | we will call effectivly hard reboot as part of starting the instance again on the source | |
| 14:40:29 | sean-k-mooney | that is where the plug time event comes form | |
| 14:40:46 | sean-k-mooney | at least that is what im guessing | |
| 14:41:01 | sean-k-mooney | have not looked at that code regently how are we starting the souce vm | |
| 14:42:04 | artom | Tbh I'm sick of the whole thing, want to get CI coverage on it so when it changes again under us we at least notice it, and revert 7a7a223602ca5aa0aca8f65a6ab143f1d8f8ec1b because we apparently don't need it anymore | |
| 14:42:51 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L11238-L11240 | |
| 14:43:33 | sean-k-mooney | artom: im not sure we should | |
| 14:43:37 | sean-k-mooney | revert 7a7a223602ca5aa0aca8f65a6ab143f1d8f8ec1b | |
| 14:43:46 | sean-k-mooney | to me this is a neutron regresssion | |
| 14:44:30 | sean-k-mooney | its a bug that it is not sendign a event at bind time when we change the host_id | |
| 14:48:00 | artom | sean-k-mooney, hellz yeah, less work for us | |
| 14:48:09 | artom | :P | |
| 14:48:10 | sean-k-mooney | artom: so ya https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L7262 we call plug vifs again when starting the ports on the source but that is idempotent and will only update the network backend (ovs) if the ports are not present | |
| 14:49:49 | sean-k-mooney | artom: i suspect the issue is the l2 agent recice the port update event but does not set the port status as up because its already up and therefor does not trigger the event | |
| 14:52:25 | artom | sean-k-mooney, huh, so I can actually hand this over to #neutron via the downstream CIX call, as originally it was a CIX | |
| 14:54:30 | sean-k-mooney | it might be related to this logic https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L2696-L2699 | |
| 14:54:52 | sean-k-mooney | that looks wrong to me as it will filter out host chagnes on the port | |
| 14:57:25 | artom | sean-k-mooney, we can try a quick DNM change to verify that... what should we change it to? | |
| 14:58:17 | sean-k-mooney | give me a minute to trace teh code | |
| 14:58:47 | sean-k-mooney | i can push something but i feel like its ignoring the host chage or somehting like that since the port and flows hould alreuady be configured | |
| 14:59:10 | sean-k-mooney | eighter that or its on the other side where setting the port status up is skipped if the port is up | |
| 14:59:34 | sean-k-mooney | eiter case woudl lead to the provisioning blocks code not beeing called | |
| 15:02:40 | sean-k-mooney | artom: we are not seeing this logged right https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L2069 | |
| 15:09:50 | artom | sean-k-mooney, lemme check | |
| 15:11:30 | sean-k-mooney | this i think is where the update shoudl come from https://github.com/openstack/neutron/blob/4b0a225e8011c904dfa7c4e41fac13cea0aa872b/neutron/plugins/ml2/rpc.py#L303-L327 | |
| 15:11:39 | artom | (Sorry, teaching daughter albegra at the same time, aah COVID how we love thee) | |
| 15:12:08 | artom | Though it is fun to figure out a way to make them understand, and see them get it | |
| 15:12:30 | artom | They're essentially exploring the commutativity of division | |
| 15:12:40 | artom | Without those terms, obviously, grade 2 :P | |
| 15:13:23 | sean-k-mooney | i see | |
| 15:13:46 | artom | So 14/2 = 10/2 + 5/2 | |
| 15:13:55 | artom | For example | |
| 15:13:59 | artom | Aaaaanyways | |
| 15:14:10 | sean-k-mooney | ya out side of an acadmic context they are not really used | |
| 15:14:38 | sean-k-mooney | well as written that should be != | |
| 15:14:49 | sean-k-mooney | unless we are talking about integer aritmatic | |
| 15:15:10 | sean-k-mooney | sicne 7 != 7.5 | |
| 15:15:11 | artom | Err, 4/2 | |
| 15:15:15 | artom | That was a typo :P | |
| 15:15:15 | sean-k-mooney | :) | |
| 15:15:15 | stephenfin | gmann: lyarwood: gibi: Looks like I opened an somewhat unrelated can of worms by bringing up that question about python_requires /o\ | |
| 15:16:24 | stephenfin | reading back through your chat from earlier (been fighting with local auth issues all afternoon) | |
| 15:17:49 | artom | sean-k-mooney, but to answer your question, no we do not see 'changing status to down' logged for our port that's resize reverting | |
| 15:18:00 | sean-k-mooney | in the l2 agent | |
| 15:18:02 | sean-k-mooney | ok | |
| 15:18:23 | sean-k-mooney | do you have the link to the l2 agent log and the port uuid | |
| 15:18:51 | sean-k-mooney | we shoudl see if we get the notificaiton for the put update with the host change | |
| 15:19:45 | artom | sean-k-mooney, uh... wanna hop on a gmeet? I have the logs from the job run locally | |
| 15:19:57 | artom | 'cuz half of those words I didn't understand :P | |
| 15:20:30 | sean-k-mooney | sure | |
| 15:20:40 | sean-k-mooney | also i wonder if we are hitting https://github.com/openstack/neutron/blob/1ad9ca56b07ffdc9f7e0bc6a62af61961b9128eb/neutron/plugins/ml2/drivers/agent/_common_agent.py#L302-L311 | |
| 15:20:52 | sean-k-mooney | from _process_device_if_exists | |
| 15:23:34 | sean-k-mooney | meet.google.com/oen-kxyp-cwa when it suits | |
| 15:48:31 | gmann | gibi: bauzas on RBAC BP, ack. I will try to join next meeting but might not due to hospital appointment(let's see how long it takes me to drive in snow :)) but dansmith can answer the queries on that if he joining. | |
| 15:48:41 | bauzas | ack | |
| 15:49:57 | gmann | stephenfin: yeah :). I am not sure why people did not brought those py3.6 which we discussed since PTG and re-re-updated the testing runtime - https://review.opendev.org/c/openstack/governance/+/815851/3..6/reference/runtimes/yoga.rst | |
| 15:50:56 | opendevreview | Merged openstack/nova stable/wallaby: Add a WA flag waiting for vif-plugged event during reboot https://review.opendev.org/c/openstack/nova/+/818519 | |
| 15:52:32 | gibi | stephenfin, gmann: I think it is a good discussion on the ML, it is probably a late discussion but a good one :) | |
| 15:55:01 | gmann | gibi: IMO, it is not late but revised one :) I have kept centos-stream 8 in my updating the testing runtime because it was not clear for TC in PTG if centos-stream 9 is released or not and later in patch review one of TC checked with centos team and confirmed it is fine to move to centos stream9 | |
| 15:55:34 | gmann | or different opinion from centos team may be | |
| 15:56:38 | gibi | I think there is some missunderstanding / miscommunication around the centos 9 readyness | |
| 15:57:02 | gmann | yeah | |
| 15:57:25 | gmann | anyways let's see if we end up reverting then we can revert the changes inclduing https://review.opendev.org/c/openstack/nova/+/819415 | |
| 15:58:00 | gmann | bauzas: I am +2 on lyarwood changing python_requires = >=3.8 just leaving +A in case you are ok - https://review.opendev.org/c/openstack/nova/+/819415 | |
| 15:58:32 | lyarwood | do we want to see what happens with the thread first | |
| 15:59:25 | lyarwood | in case the TC need to revert back to py36 and CS8 | |
| 15:59:26 | bauzas | gmann: lyarwood: yeah we can hold | |
| 15:59:28 | gmann | lyarwood: either is ok but if we revert to py36 then anyways we have to revert 819194 too | |