Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-11-18
16:06:59 owalsh I'm saying we can't use a similar strategy to separate files
16:10:04 dansmith owalsh: okay I'm sorry, I've really lost track of who "we" and what the problem is. You're saying "we=tripleo" deploy a metadata per cell and can't, for some reason, point it at the cellN conductor's config file?
16:11:44 owalsh dansmith: rpms/debs in particular are the issue, but yes, can't point metadata-api at the conductor's config file
16:14:19 openstackgerrit Lee Yarwood proposed openstack/nova-specs master: WIP libvirt: Allow the default machine type to be changed https://review.opendev.org/762199
16:15:02 dansmith owalsh: okay, I'm not sure why not currently, because we have that other knob
16:15:30 owalsh dansmith: nova.conf is hardcoded in wsgi_api.py
16:21:07 owalsh dansmith: TBH it would make a lot more sense to just say: use nova.conf for everything except nova-compute....
16:22:25 dansmith owalsh: I'm really lost, I'm sorry
16:22:38 dansmith you can, of course, just use nova.conf for everything,
16:23:00 dansmith but if they're all on the same host and you have cell and super conductors, they can't all use the same config file
16:29:57 bauzas grmblbl, I get exceptions about RPC API 6.0 that I don't understand...
16:31:10 bauzas "ERROR nova.api.openstack.wsgi oslo_messaging.rpc.client.RPCVersionCapError: Requested message version, 5.0 is incompatible. It needs to be equal in major version and less than or equal in minor version as the specified version cap 6.0"
16:31:19 bauzas WTF
16:36:43 bauzas dansmith: do you have any idea why the RPC API only supports 6.0 as the minor RPC version while I wrote a 5.12 proxy in https://review.opendev.org/#/c/761452/2 ?
16:37:03 bauzas I got tons of tempest and func tests failing because of this
16:37:59 bauzas in theory, I should expose 5.x as a valid RPC target thanks to the added service RPC endpoint
16:50:06 gmann stephenfin: bauzas dansmith can you review this, sample config file fix - https://review.opendev.org/#/c/762721/
17:01:59 noonedeadpunk hey!
17:02:14 noonedeadpunk having a question regarding http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018663.html
17:03:18 noonedeadpunk to be specific - was trying to provide --config-file for uwsgi via pyargv but nova seems to ignore it...
17:04:12 noonedeadpunk and it indeed works for neutron... so not sure if it's just me having issues or what, as things are set via oslo anyway....
17:39:54 dansmith noonedeadpunk: funny you should ask
17:40:32 dansmith noonedeadpunk: I think it should, I dunno why it doesn't, and I'm about to caffeinate myself and write a tome to the ML thread about it and other things
18:32:40 openstackgerrit sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432
18:32:57 sean-k-mooney melwitt: i hope that adresses your comments ^
18:33:25 sean-k-mooney stephenfin: if you have time to have a look again at ^ it basicaly the same + some typo fixes and a rebase
18:33:33 melwitt thanks
18:34:40 sean-k-mooney based on your question i set the min verion to 1.15.2 instaed of 1.15.1 since it has imporant bugfixes for linuxbridge and added that to the commit
18:35:46 sean-k-mooney 1.15.0 was broken for ovs 1.15.1 fixed that but still was broken in some cases for linux bridge 1.15.2 is what we released with and support on stable stine in upper constratints
18:37:19 melwitt thanks for explaining that, makes sense now. I had been confused by the gap in the release notes and couldn't find the significance of 1.15.1
18:39:07 melwitt sean-k-mooney: oh, one more question that I had on the review that I think was missed, is this new delegation to os-vif for the plug, does it run during tempest tests in the gate? or is it for a config that is not running in the gate
18:41:04 sean-k-mooney oh right i will run in any job using iptables
18:41:12 sean-k-mooney i think we have on that does that but ill check now
18:41:26 sean-k-mooney that used to be the default
18:47:36 sean-k-mooney still checking look like we change the default in devstack but we have an iptable job in experimental so i can kick that off or add a DNM testing patch on top
18:47:49 sean-k-mooney melwitt: which would you prefer
18:48:28 melwitt sean-k-mooney: whatever is easier
18:49:14 sean-k-mooney experimental is eaiser but it was a bunch of resouce so ill do a dnm patch with both the os-vif and neutron iptables tempest job
18:49:29 sean-k-mooney they are the same but the os-vif on uses master and the neutorn one uses the released version
18:49:59 melwitt ah ok, sounds cool
18:50:55 sean-k-mooney im kind of surprised we dont have any iptables testing in nova actully but i guess it has not been the default for a few releases at this point
18:51:41 sean-k-mooney ya even the greade job uses the ovs firewall at this point https://zuul.opendev.org/t/openstack/build/ec9d4bf702c84bcf9ae355ae0a735cd1/log/controller/logs/etc/neutron/plugins/ml2/ml2_conf.ini#299
18:52:12 sean-k-mooney although to be faire we also dont have any ovn testing so its not surprising
18:53:52 melwitt interesting. so the original bug only affects deployments using iptables. I hadn't realized that
18:55:02 sean-k-mooney oh wait sorry
18:55:08 sean-k-mooney i had that backwards
18:55:18 sean-k-mooney it only affect deployment not using iptables
18:55:23 sean-k-mooney so ya all the jobs test this
18:55:34 sean-k-mooney for iptables ovs already plugs the interface
18:55:39 sean-k-mooney when not using iptables libvirt does
18:56:09 sean-k-mooney now os-vif will always do it regardesll fo if you use iptables or not
18:56:52 sean-k-mooney melwitt: we fixed it for iptables alreeady via os-vif but we could not fix it via os-vif in this case since os-vif did not do the plugging libvirt did
18:57:12 melwitt I see, thanks
19:00:12 sean-k-mooney for context the vlan specification defines vlan 4095 as reserved for implemantion use. in ovs its used for the "dead vlan" per the specification swithc whould not forward packets with vlan 4095 so ovs adopted vlan 4095 as a reserved vlan on which all traffic will be droped
19:00:41 sean-k-mooney so the mitigation is to place allports on vlan 4095 until they are moved to the tenant vlan by neutron
19:01:48 sean-k-mooney you enable that behavior by setting isolate_vif but we cant do it via libvirt since it will conflict with neutron if you set vlan 4095 in the xml
19:02:26 sean-k-mooney * isolate_vif is set in the nova.conf in the os vif plugin section
19:04:34 sean-k-mooney so [os_vif_ovs]/isolate_vif=true which i should proably have destack set by default at somepoint
19:04:45 melwitt ah cool, thanks for those details, nice context for the fix approach
19:08:57 sean-k-mooney anyway to your orginial question all the tempest jobs are exefcuting the code change so i wont submit a DNM patch
19:12:16 melwitt sean-k-mooney: is there any change we would see in the logs as a result of the os-vif plug vs the old libvirt plug?
19:26:39 melwitt looks like in the guest xml the device changes to <interface type="ethernet"> from <interface type="bridge">
19:29:01 sean-k-mooney yes the xml would be different
19:29:12 sean-k-mooney and that is in the nova compute log
19:29:20 melwitt cool, yeah I am looking at it
19:30:21 sean-k-mooney sorry was away just putting on dinner
19:30:38 melwitt np, I found it :)
19:34:09 sean-k-mooney huh i wonder if infra changed something in zuul
19:34:28 sean-k-mooney its not rendering the xmls correctly anymore
19:34:46 sean-k-mooney https://zuul.opendev.org/t/openstack/build/00ff397ca5374a0b9441036d43ee3416/log/controller/logs/screen-n-cpu.txt#7371 should be the full xml but you can only see it in the raw view
19:34:58 melwitt I'm looking at the raw files so I didn't see the zuul thing. but I did notice the UI changed recently in other ways
19:35:23 sean-k-mooney ya i might need to repot that as a bug since you cant link to lines in the raw view
19:35:38 sean-k-mooney i guess they have an issue with multi line stringes or soemthing
19:36:47 melwitt hm yeah, need to ask in -infra
20:41:25 melwitt sean-k-mooney: I'm trying to confirm that the nova-grenade-multinode fails on PS25 is unrelated to the change. I can't tell why the live migrations failed. do you expect there would be any issues with a live migration going from N to N+1 or vice versa during an upgrade wrt to the plugging mechanism?
21:13:36 sean-k-mooney the port would be created on ovs twice but that is ok
21:14:08 sean-k-mooney os-vif would create it via pre live migate on the new host then libvirt would create it again by deleteing it and readding it
21:14:14 sean-k-mooney that should not break anything
21:14:48 sean-k-mooney libvirt basiclaly does ovs-vsctl del-port --if-exist ... ovs-vsctl add-port
#openstack-nova - 2020-11-19
00:42:24 openstackgerrit Brin Zhang proposed openstack/nova-specs master: Re-proposes 'Proposal for a safer remote console with password authentication https://review.opendev.org/759828
01:07:16 openstackgerrit Brin Zhang proposed openstack/nova-specs master: Remove tenant_id https://review.opendev.org/737241
01:10:42 openstackgerrit Brin Zhang proposed openstack/nova-specs master: [Trivial] update the upgrade release goal https://review.opendev.org/763294
06:20:31 ninad Hi
06:20:32 ninad I am using linuxbridge for the neutron services and follow the installation guide as documented but during manila instance creation I am getting [Errno 113] EHOSTUNREACH _test_server_connection
06:21:03 ninad can someone please help me?
07:58:40 openstackgerrit Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264
08:06:20 openstackgerrit Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264
09:24:40 lyarwood aarents: apologies, had a fun day downstream yesterday, your test changes LGTM, I missed that you had already pulled in the devstack change within the tempest change so assuming it's still passing this should be good to go now.
09:31:16 aarents lyarwood: many thanks, I just repush with nit fix sugested by gmann
10:21:35 stephenfin sean-k-mooney: I think there's something broken in https://review.opendev.org/#/c/602432/. It appears grenade hasn't passed on that for the last couple of revisions. Haven't gone diving through logs yet
11:13:45 openstackgerrit Jorhson Deng proposed openstack/nova master: To deal instance with soft-deleting in _init_instance https://review.opendev.org/761264
11:14:24 openstackgerrit Lee Yarwood proposed openstack/nova master: zuul: Add devstack-plugin-ceph-compute-local-ephemeral to experimental https://review.opendev.org/743220
11:45:38 sean-k-mooney stephenfin: if we were to go from new to old i guess it would break
11:46:09 sean-k-mooney the xml would be generated on the souce with type ethernet
11:46:28 sean-k-mooney and the os-vif on the dest would be a noop
11:46:38 sean-k-mooney so nothing would plug the interface
11:48:02 sean-k-mooney the singel node grenade have been passing but not multinode
11:50:22 sean-k-mooney the only way i think of to fix that is to first backport the neutron change then flip the default for the option in os-vif/depercate it for removal and backport that too before we merge on master
11:50:55 sean-k-mooney and i would then have to keep backporting the neutron and os-vif patches first for every release.
11:51:56 sean-k-mooney stephenfin: lyarwood ^ is there a better way around that

Earlier   Later