Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-11-18
18:49:29 sean-k-mooney they are the same but the os-vif on uses master and the neutorn one uses the released version
18:49:59 melwitt ah ok, sounds cool
18:50:55 sean-k-mooney im kind of surprised we dont have any iptables testing in nova actully but i guess it has not been the default for a few releases at this point
18:51:41 sean-k-mooney ya even the greade job uses the ovs firewall at this point https://zuul.opendev.org/t/openstack/build/ec9d4bf702c84bcf9ae355ae0a735cd1/log/controller/logs/etc/neutron/plugins/ml2/ml2_conf.ini#299
18:52:12 sean-k-mooney although to be faire we also dont have any ovn testing so its not surprising
18:53:52 melwitt interesting. so the original bug only affects deployments using iptables. I hadn't realized that
18:55:02 sean-k-mooney oh wait sorry
18:55:08 sean-k-mooney i had that backwards
18:55:18 sean-k-mooney it only affect deployment not using iptables
18:55:23 sean-k-mooney so ya all the jobs test this
18:55:34 sean-k-mooney for iptables ovs already plugs the interface
18:55:39 sean-k-mooney when not using iptables libvirt does
18:56:09 sean-k-mooney now os-vif will always do it regardesll fo if you use iptables or not
18:56:52 sean-k-mooney melwitt: we fixed it for iptables alreeady via os-vif but we could not fix it via os-vif in this case since os-vif did not do the plugging libvirt did
18:57:12 melwitt I see, thanks
19:00:12 sean-k-mooney for context the vlan specification defines vlan 4095 as reserved for implemantion use. in ovs its used for the "dead vlan" per the specification swithc whould not forward packets with vlan 4095 so ovs adopted vlan 4095 as a reserved vlan on which all traffic will be droped
19:00:41 sean-k-mooney so the mitigation is to place allports on vlan 4095 until they are moved to the tenant vlan by neutron
19:01:48 sean-k-mooney you enable that behavior by setting isolate_vif but we cant do it via libvirt since it will conflict with neutron if you set vlan 4095 in the xml
19:02:26 sean-k-mooney * isolate_vif is set in the nova.conf in the os vif plugin section
19:04:34 sean-k-mooney so [os_vif_ovs]/isolate_vif=true which i should proably have destack set by default at somepoint
19:04:45 melwitt ah cool, thanks for those details, nice context for the fix approach
19:08:57 sean-k-mooney anyway to your orginial question all the tempest jobs are exefcuting the code change so i wont submit a DNM patch
19:12:16 melwitt sean-k-mooney: is there any change we would see in the logs as a result of the os-vif plug vs the old libvirt plug?
19:26:39 melwitt looks like in the guest xml the device changes to <interface type="ethernet"> from <interface type="bridge">
19:29:01 sean-k-mooney yes the xml would be different
19:29:12 sean-k-mooney and that is in the nova compute log
19:29:20 melwitt cool, yeah I am looking at it
19:30:21 sean-k-mooney sorry was away just putting on dinner
19:30:38 melwitt np, I found it :)
19:34:09 sean-k-mooney huh i wonder if infra changed something in zuul
19:34:28 sean-k-mooney its not rendering the xmls correctly anymore
19:34:46 sean-k-mooney https://zuul.opendev.org/t/openstack/build/00ff397ca5374a0b9441036d43ee3416/log/controller/logs/screen-n-cpu.txt#7371 should be the full xml but you can only see it in the raw view
19:34:58 melwitt I'm looking at the raw files so I didn't see the zuul thing. but I did notice the UI changed recently in other ways
19:35:23 sean-k-mooney ya i might need to repot that as a bug since you cant link to lines in the raw view
19:35:38 sean-k-mooney i guess they have an issue with multi line stringes or soemthing
19:36:47 melwitt hm yeah, need to ask in -infra
20:41:25 melwitt sean-k-mooney: I'm trying to confirm that the nova-grenade-multinode fails on PS25 is unrelated to the change. I can't tell why the live migrations failed. do you expect there would be any issues with a live migration going from N to N+1 or vice versa during an upgrade wrt to the plugging mechanism?
21:13:36 sean-k-mooney the port would be created on ovs twice but that is ok
21:14:08 sean-k-mooney os-vif would create it via pre live migate on the new host then libvirt would create it again by deleteing it and readding it
21:14:14 sean-k-mooney that should not break anything
21:14:48 sean-k-mooney libvirt basiclaly does ovs-vsctl del-port --if-exist ... ovs-vsctl add-port
#openstack-nova - 2020-11-19
00:42:24 openstackgerrit Brin Zhang proposed openstack/nova-specs master: Re-proposes 'Proposal for a safer remote console with password authentication https://review.opendev.org/759828
01:07:16 openstackgerrit Brin Zhang proposed openstack/nova-specs master: Remove tenant_id https://review.opendev.org/737241
01:10:42 openstackgerrit Brin Zhang proposed openstack/nova-specs master: [Trivial] update the upgrade release goal https://review.opendev.org/763294
06:20:31 ninad Hi
06:20:32 ninad I am using linuxbridge for the neutron services and follow the installation guide as documented but during manila instance creation I am getting [Errno 113] EHOSTUNREACH _test_server_connection
06:21:03 ninad can someone please help me?
07:58:40 openstackgerrit Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264
08:06:20 openstackgerrit Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264
09:24:40 lyarwood aarents: apologies, had a fun day downstream yesterday, your test changes LGTM, I missed that you had already pulled in the devstack change within the tempest change so assuming it's still passing this should be good to go now.
09:31:16 aarents lyarwood: many thanks, I just repush with nit fix sugested by gmann
10:21:35 stephenfin sean-k-mooney: I think there's something broken in https://review.opendev.org/#/c/602432/. It appears grenade hasn't passed on that for the last couple of revisions. Haven't gone diving through logs yet
11:13:45 openstackgerrit Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264
11:14:24 openstackgerrit Lee Yarwood proposed openstack/nova master: zuul: Add devstack-plugin-ceph-compute-local-ephemeral to experimental https://review.opendev.org/743220
11:45:38 sean-k-mooney stephenfin: if we were to go from new to old i guess it would break
11:46:09 sean-k-mooney the xml would be generated on the souce with type ethernet
11:46:28 sean-k-mooney and the os-vif on the dest would be a noop
11:46:38 sean-k-mooney so nothing would plug the interface
11:48:02 sean-k-mooney the singel node grenade have been passing but not multinode
11:50:22 sean-k-mooney the only way i think of to fix that is to first backport the neutron change then flip the default for the option in os-vif/depercate it for removal and backport that too before we merge on master
11:50:55 sean-k-mooney and i would then have to keep backporting the neutron and os-vif patches first for every release.
11:51:56 sean-k-mooney stephenfin: lyarwood ^ is there a better way around that
12:06:50 lyarwood sean-k-mooney: we shouldn't be going from new to old in the tests right?
12:07:02 lyarwood sean-k-mooney: only old to new, or is that not defined in grenade
12:07:07 sean-k-mooney we do live migration back and fort
12:07:14 sean-k-mooney in the multi node job
12:08:06 stephenfin sean-k-mooney: You could make this dependent on a service version check
12:08:23 sean-k-mooney not really i would have to keep changin it
12:08:50 stephenfin why? Once everything's on Wallaby with a suitably new version of os-vif, we're good, no?
12:08:52 sean-k-mooney i thory this should go to osp 10 so all the way back to queens
12:08:55 stephenfin ah
12:09:22 sean-k-mooney im thinking of stoping at train to be honest
12:09:34 sean-k-mooney but it defiently needs to be backported a few releases
12:09:41 stephenfin (osp 10 is newton, fwiw)
12:09:55 sean-k-mooney ... of course it is
12:10:32 sean-k-mooney since its related to a security issue we are still ment to backport it althou technically i guess 13 would be enough at this point
12:10:33 stephenfin what you've proposed would require os-vif be upgraded before nova, right? I don't think we can count on that
12:10:45 stephenfin *os-vif and neutron
12:11:34 sean-k-mooney without the neturon patch you just get erros in the neutron logs but it fixes its self
12:11:44 sean-k-mooney the os-vif change however would be needed first ya
12:13:05 sean-k-mooney lyarwood: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ff2/602432/26/check/nova-grenade-multinode/ff2bd9a/logs/new/tempest_conf.txt
12:13:12 sean-k-mooney live_migrate_back_and_forth = True
12:13:37 sean-k-mooney that is why its failing
12:14:06 gibi stephenfin: do you have a lauchpad bp filled for this? https://review.opendev.org/#/c/755109
12:14:24 sean-k-mooney its off in live migration job and that all the new version so it passes the https://zuul.opendev.org/t/openstack/build/e2fa8ae7802f41309877b537f51ba660/log/controller/logs/tempest_conf.txt#76
12:14:26 stephenfin gibi: I though I did but I'm not sure
12:14:39 gibi the bp link in the specs does not seem to work for me
12:16:28 sean-k-mooney ya cant find it either
12:16:45 sean-k-mooney but it should be a simple copy past of the into section
12:17:01 stephenfin gibi: Didn't look like it. Apologies. Have created one now https://blueprints.launchpad.net/nova/+spec/modernize-os-hypervisors-api
12:17:07 gibi thanks
12:19:13 sean-k-mooney weird i cant add the spec via the spec link so i put it in the whiteborad
12:19:34 sean-k-mooney i tought i could do that once i was on the bug team but i guess not
12:28:30 gibi sean-k-mooney: I guess we edited the whiteboard in parallel and my change overwrote yours, sorry.
12:30:52 sean-k-mooney no launchpad gave me a permission error
12:31:38 gibi interesting, I got the following mail from launchpad
12:31:38 gibi Blueprint changed by sean mooney:
12:31:39 gibi Whiteboard set to:
12:31:39 gibi spec: https://review.opendev.org/#/c/755109
12:31:55 gibi so at least some part of your update went through

Earlier   Later