| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-11-18 | |||
| 15:23:30 | dansmith | oh, then I'm thinking of something else | |
| 15:24:37 | owalsh | dansmith: re cell nova-conductor still requiring api db for affinity checks, should I assume that will eventually change? | |
| 15:24:39 | dansmith | oh, right, this probably really shouldn't even be a CLI thing, we probably added it there so we could poke it for testing | |
| 15:24:40 | rm_work | owalsh: oooo will take a look, thanks | |
| 15:24:59 | stephenfin | okay, that was my gut feel too; I'll skip it so | |
| 15:25:04 | stephenfin | dansmith++ thanks | |
| 15:26:19 | dansmith | stephenfin: I was thinking of this: https://docs.openstack.org/api-ref/compute/#reset-networking-on-a-server-resetnetwork-action | |
| 15:26:40 | dansmith | note the "no longer supported by any virt driver" because xen | |
| 15:26:56 | stephenfin | aha, yes, I've done work on that recently | |
| 15:28:47 | dansmith | owalsh: probably not any time soon.. it was hinging on affinity in placement, and I gather that people aren't interested in furthering those goals these days, | |
| 15:28:55 | dansmith | owalsh: so probably no change on the horizon | |
| 15:39:40 | owalsh | dansmith: ack, thanks, so it wouldn't be crazy to use these creds to create the cell mapping when deploying a new cell conductor? | |
| 15:40:09 | dansmith | owalsh: nope, that's reasonable at the moment I'd say | |
| 15:46:12 | owalsh | dansmith: ack, great. Also does metadata api need the api db config even when local_metadata_per_cell=True? | |
| 15:47:06 | dansmith | owalsh: shouldn't, no | |
| 15:48:08 | dansmith | owalsh: yeah, if that's set, we never run the mapping lookup, which is the api hit | |
| 15:48:25 | dansmith | owalsh: so unless there's a bug or something buried deep, that's the intent.. no api db hit if =True | |
| 15:49:21 | owalsh | dansmith: ack, yea, didn't initially make sense that local_metadata_per_cell existed when everything else just checks if the api db is configured, but need to support cell conductor & metadata api sharing nova.conf | |
| 15:53:27 | dansmith | owalsh: well, I think metadata was the first thing to need that determination, so we added a conf knob for it specifically.. it's also potentially security-related because the workloads can talk to it, so explicit-ness is good | |
| 15:53:47 | dansmith | owalsh: but we could deprecate that conf knob and actually just honor the presence of the api creds if you think it'd be better for consistency | |
| 15:58:11 | owalsh | dansmith: but that wouldn't work in cases where the cell conductor shares nova conf with cell metadata-api | |
| 15:58:32 | dansmith | cell conductor, yeah | |
| 16:01:25 | owalsh | dansmith: which goes back to the recommendation on the ML to follow the devstack approach to conf - it doesn't seem to work when using cell local metadata api... | |
| 16:02:01 | owalsh | since metadata-api always uses nova.conf, never nova-cellN.conf | |
| 16:02:36 | dansmith | owalsh: but devstack runs a global metadata api right? | |
| 16:03:02 | owalsh | dansmith: yup, doesn't seem to support cell local metadata api | |
| 16:03:31 | dansmith | devstack's goal, of course, is not to support every possible deployment arrangement... | |
| 16:03:59 | owalsh | dansmith: indeed, and for CI/dev running multiple cells makes sense but in production it does not... | |
| 16:04:09 | owalsh | multiple cells on the same host that is | |
| 16:04:45 | dansmith | I've lost track of what we're arguing about | |
| 16:06:14 | owalsh | dansmith: http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018732.html | |
| 16:06:59 | owalsh | I'm saying we can't use a similar strategy to separate files | |
| 16:10:04 | dansmith | owalsh: okay I'm sorry, I've really lost track of who "we" and what the problem is. You're saying "we=tripleo" deploy a metadata per cell and can't, for some reason, point it at the cellN conductor's config file? | |
| 16:11:44 | owalsh | dansmith: rpms/debs in particular are the issue, but yes, can't point metadata-api at the conductor's config file | |
| 16:14:19 | openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: WIP libvirt: Allow the default machine type to be changed https://review.opendev.org/762199 | |
| 16:15:02 | dansmith | owalsh: okay, I'm not sure why not currently, because we have that other knob | |
| 16:15:30 | owalsh | dansmith: nova.conf is hardcoded in wsgi_api.py | |
| 16:21:07 | owalsh | dansmith: TBH it would make a lot more sense to just say: use nova.conf for everything except nova-compute.... | |
| 16:22:25 | dansmith | owalsh: I'm really lost, I'm sorry | |
| 16:22:38 | dansmith | you can, of course, just use nova.conf for everything, | |
| 16:23:00 | dansmith | but if they're all on the same host and you have cell and super conductors, they can't all use the same config file | |
| 16:29:57 | bauzas | grmblbl, I get exceptions about RPC API 6.0 that I don't understand... | |
| 16:31:10 | bauzas | "ERROR nova.api.openstack.wsgi oslo_messaging.rpc.client.RPCVersionCapError: Requested message version, 5.0 is incompatible. It needs to be equal in major version and less than or equal in minor version as the specified version cap 6.0" | |
| 16:31:19 | bauzas | WTF | |
| 16:36:43 | bauzas | dansmith: do you have any idea why the RPC API only supports 6.0 as the minor RPC version while I wrote a 5.12 proxy in https://review.opendev.org/#/c/761452/2 ? | |
| 16:37:03 | bauzas | I got tons of tempest and func tests failing because of this | |
| 16:37:59 | bauzas | in theory, I should expose 5.x as a valid RPC target thanks to the added service RPC endpoint | |
| 16:50:06 | gmann | stephenfin: bauzas dansmith can you review this, sample config file fix - https://review.opendev.org/#/c/762721/ | |
| 17:01:59 | noonedeadpunk | hey! | |
| 17:02:14 | noonedeadpunk | having a question regarding http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018663.html | |
| 17:03:18 | noonedeadpunk | to be specific - was trying to provide --config-file for uwsgi via pyargv but nova seems to ignore it... | |
| 17:04:12 | noonedeadpunk | and it indeed works for neutron... so not sure if it's just me having issues or what, as things are set via oslo anyway.... | |
| 17:39:54 | dansmith | noonedeadpunk: funny you should ask | |
| 17:40:32 | dansmith | noonedeadpunk: I think it should, I dunno why it doesn't, and I'm about to caffeinate myself and write a tome to the ML thread about it and other things | |
| 18:32:40 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | |
| 18:32:57 | sean-k-mooney | melwitt: i hope that adresses your comments ^ | |
| 18:33:25 | sean-k-mooney | stephenfin: if you have time to have a look again at ^ it basicaly the same + some typo fixes and a rebase | |
| 18:33:33 | melwitt | thanks | |
| 18:34:40 | sean-k-mooney | based on your question i set the min verion to 1.15.2 instaed of 1.15.1 since it has imporant bugfixes for linuxbridge and added that to the commit | |
| 18:35:46 | sean-k-mooney | 1.15.0 was broken for ovs 1.15.1 fixed that but still was broken in some cases for linux bridge 1.15.2 is what we released with and support on stable stine in upper constratints | |
| 18:37:19 | melwitt | thanks for explaining that, makes sense now. I had been confused by the gap in the release notes and couldn't find the significance of 1.15.1 | |
| 18:39:07 | melwitt | sean-k-mooney: oh, one more question that I had on the review that I think was missed, is this new delegation to os-vif for the plug, does it run during tempest tests in the gate? or is it for a config that is not running in the gate | |
| 18:41:04 | sean-k-mooney | oh right i will run in any job using iptables | |
| 18:41:12 | sean-k-mooney | i think we have on that does that but ill check now | |
| 18:41:26 | sean-k-mooney | that used to be the default | |
| 18:47:36 | sean-k-mooney | still checking look like we change the default in devstack but we have an iptable job in experimental so i can kick that off or add a DNM testing patch on top | |
| 18:47:49 | sean-k-mooney | melwitt: which would you prefer | |
| 18:48:28 | melwitt | sean-k-mooney: whatever is easier | |
| 18:49:14 | sean-k-mooney | experimental is eaiser but it was a bunch of resouce so ill do a dnm patch with both the os-vif and neutron iptables tempest job | |
| 18:49:29 | sean-k-mooney | they are the same but the os-vif on uses master and the neutorn one uses the released version | |
| 18:49:59 | melwitt | ah ok, sounds cool | |
| 18:50:55 | sean-k-mooney | im kind of surprised we dont have any iptables testing in nova actully but i guess it has not been the default for a few releases at this point | |
| 18:51:41 | sean-k-mooney | ya even the greade job uses the ovs firewall at this point https://zuul.opendev.org/t/openstack/build/ec9d4bf702c84bcf9ae355ae0a735cd1/log/controller/logs/etc/neutron/plugins/ml2/ml2_conf.ini#299 | |
| 18:52:12 | sean-k-mooney | although to be faire we also dont have any ovn testing so its not surprising | |
| 18:53:52 | melwitt | interesting. so the original bug only affects deployments using iptables. I hadn't realized that | |
| 18:55:02 | sean-k-mooney | oh wait sorry | |
| 18:55:08 | sean-k-mooney | i had that backwards | |
| 18:55:18 | sean-k-mooney | it only affect deployment not using iptables | |
| 18:55:23 | sean-k-mooney | so ya all the jobs test this | |
| 18:55:34 | sean-k-mooney | for iptables ovs already plugs the interface | |
| 18:55:39 | sean-k-mooney | when not using iptables libvirt does | |
| 18:56:09 | sean-k-mooney | now os-vif will always do it regardesll fo if you use iptables or not | |
| 18:56:52 | sean-k-mooney | melwitt: we fixed it for iptables alreeady via os-vif but we could not fix it via os-vif in this case since os-vif did not do the plugging libvirt did | |
| 18:57:12 | melwitt | I see, thanks | |
| 19:00:12 | sean-k-mooney | for context the vlan specification defines vlan 4095 as reserved for implemantion use. in ovs its used for the "dead vlan" per the specification swithc whould not forward packets with vlan 4095 so ovs adopted vlan 4095 as a reserved vlan on which all traffic will be droped | |
| 19:00:41 | sean-k-mooney | so the mitigation is to place allports on vlan 4095 until they are moved to the tenant vlan by neutron | |
| 19:01:48 | sean-k-mooney | you enable that behavior by setting isolate_vif but we cant do it via libvirt since it will conflict with neutron if you set vlan 4095 in the xml | |
| 19:02:26 | sean-k-mooney | * isolate_vif is set in the nova.conf in the os vif plugin section | |
| 19:04:34 | sean-k-mooney | so [os_vif_ovs]/isolate_vif=true which i should proably have destack set by default at somepoint | |
| 19:04:45 | melwitt | ah cool, thanks for those details, nice context for the fix approach | |
| 19:08:57 | sean-k-mooney | anyway to your orginial question all the tempest jobs are exefcuting the code change so i wont submit a DNM patch | |
| 19:12:16 | melwitt | sean-k-mooney: is there any change we would see in the logs as a result of the os-vif plug vs the old libvirt plug? | |
| 19:26:39 | melwitt | looks like in the guest xml the device changes to <interface type="ethernet"> from <interface type="bridge"> | |
| 19:29:01 | sean-k-mooney | yes the xml would be different | |
| 19:29:12 | sean-k-mooney | and that is in the nova compute log | |
| 19:29:20 | melwitt | cool, yeah I am looking at it | |
| 19:30:21 | sean-k-mooney | sorry was away just putting on dinner | |
| 19:30:38 | melwitt | np, I found it :) | |
| 19:34:09 | sean-k-mooney | huh i wonder if infra changed something in zuul | |
| 19:34:28 | sean-k-mooney | its not rendering the xmls correctly anymore | |