| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-02-03 | |||
| 20:05:20 | sean-k-mooney | its a bit late here but we should proably disucss this some more next week | |
| 20:05:30 | dvo-plv | another way, we can check qemu version, this feature was implemented from qemu 4.2 | |
| 20:06:24 | dvo-plv | or grep funtionality like in this port http://blog.vmsplice.net/2020/05/how-to-check-virtio-feature-bits-inside.html | |
| 20:07:06 | dvo-plv | Thank you, let's move it to next weekÑ” | |
| 20:07:06 | dvo-plv | Thank you, let's move it to next weekє | |
| 20:07:33 | sean-k-mooney | the issue is we need to make the scudleing work before we have selected any host | |
| 20:08:22 | sean-k-mooney | so in general we cannot check the qemu/libvirt verion at the schduling step | |
| 20:08:39 | dvo-plv | I tried to get this feature at this method static_traits in the nova/virt/libvirt/driver.py | |
| 20:09:18 | sean-k-mooney | the best i can think of would be to stash a flag in the instance_system_metadta to recored it was booted on a host that supported the pact format | |
| 20:09:36 | sean-k-mooney | and then include a request for the trait if that is found in the instance_system_metadata | |
| 20:09:54 | sean-k-mooney | that is proably the best we can do since we dont know if its used by the guest | |
| 20:10:31 | sean-k-mooney | the static_traits function is the correct plase to report the trait for the compute host | |
| 21:06:28 | gmann | bauzas: stephenfin: need one more review in this to unblock stable/wallaby gate (removing broken sdk job) https://review.opendev.org/c/openstack/nova/+/871798 | |
| 22:39:50 | melwitt | dansmith: I have look-see'd | |
| 22:40:31 | opendevreview | Merged openstack/nova stable/wallaby: [stable-only] Remove broken sdk job from wallaby https://review.opendev.org/c/openstack/nova/+/871798 | |
| 22:54:55 | dansmith | melwitt: um | |
| 22:55:04 | dansmith | I'm not sure you can past-tense-verb look-see | |
| 22:55:22 | dansmith | (but thanks :) | |
| 22:55:35 | melwitt | :) | |
| 22:56:49 | sean-k-mooney | its english there is always a way to do things | |
| 23:00:52 | sean-k-mooney | 023-02-03 23:00:25.664 41 ERROR nova.virt.driver [None req-80a00ce3-332e-4a98-be2e-d0a1bd9329d7 - - - - - -] Compute driver option required, but not specified | |
| 23:01:07 | sean-k-mooney | lol ok that explains why its not working | |
| 23:05:37 | opendevreview | Sylvain Bauza proposed openstack/nova master: Enable cpus when an instance is spawning https://review.opendev.org/c/openstack/nova/+/868237 | |
| 23:34:24 | EugenMayer4 | I'aim running OVS on zed. What i'am trying to do is. I have 2 networks (or 3, if you also count the provider lan). lets call them provider_wan, intranet and DMZ. Now i want DMZ to only be able to talk to some very few clients on some specific port in intranet (ip / port allow list), but not nothing else in intranet. Also, a client in DMZ should be | |
| 23:34:24 | EugenMayer4 | able to access the internet via the provider lan. I finished setting up the internet access, i can also access the intranet, but i;am not able to limit what a DMZ client is able to access in the intranet. Any hints? All clients in this case are nova vms (qemu) | |
| 23:36:02 | sean-k-mooney | the only way i can think of to do that without the firewall as a service project which is dead is to use a vm as a router | |
| 23:36:30 | sean-k-mooney | so dont interconnect the networks on the datacenter side or with neutron routhers | |
| 23:37:04 | sean-k-mooney | but run a pfsence or similar vm and make it the default gateway for the DMZ and intranet | |
| 23:37:15 | sean-k-mooney | and connet it to thr wan network | |
| 23:37:24 | sean-k-mooney | then you can implemnte whatever network policy you like there | |
| 23:37:45 | sean-k-mooney | EugenMayer4: in general the neutorn channel would proably be able to help more | |
| #openstack-nova - 2023-02-04 | |||
| 07:04:45 | EugenMayer4 | sean-k-mooney that sounds like way too much effort for that. I'am fairly familiar with OPNsense but deploying it (manually, no tf suppport) just for that purpose seems over the top). I assumed that a DMZ should be something that neutron should be dealing with without bigger issues. | |
| 07:06:04 | sean-k-mooney[m] | not really that is not something that they support | |
| 07:06:25 | sean-k-mooney[m] | your only way to do this in neutron is security groups | |
| 07:06:47 | sean-k-mooney[m] | or the firewall as a service project which as i said is nolonger activly developed | |
| 07:08:13 | sean-k-mooney[m] | so if security groups dont work for your usecase then you need to create something your self. i assume you have tried security groups and that was not enough | |
| 07:09:57 | sean-k-mooney[m] | security groups can match on ports/adreees adn on ingress vs egress | |
| 07:11:00 | sean-k-mooney[m] | but if you want to enforce that policy regradless of how the vms are booted on the dmz you cant reallly do that at the network level in neutron out of the box | |
| 07:16:27 | sean-k-mooney[m] | you could try https://docs.openstack.org/neutron/latest/admin/fwaas.html but as i said im not sure if that is even maintaiend anymore you shouold talk to the neutron team about it first because the project was being discontinued at one point | |
| 09:53:40 | opendevreview | melanie witt proposed openstack/nova master: Reproducer for bug 2003991 unshelving offloaded instance https://review.opendev.org/c/openstack/nova/+/872470 | |
| 09:53:40 | opendevreview | melanie witt proposed openstack/nova master: Enforce quota usage from placement when unshelving https://review.opendev.org/c/openstack/nova/+/872471 | |
| 12:06:10 | EugenMayer4 | sean-k-mooney[m] thank you for clarifying. The scope of security groups is hard to grasp. I tried secgroups on the VM, but i cannot limit the ingress to the intranet lan while opening the egress for internet via 0.0.0.0/0 | |
| 12:06:53 | EugenMayer4 | also, i assume that port (router port) based SG with OVS are entirely broken | |
| 12:07:25 | sean-k-mooney[m] | not broken security groups were only ever intended for vm ports | |
| 12:07:54 | sean-k-mooney[m] | the firewall as a service api was for router based firewalling. | |
| 12:07:59 | EugenMayer4 | i understand that (know) after digging int things. But that is not how they are exposed, described and "advertised" | |
| 12:08:15 | sean-k-mooney[m] | with that said you have differnt behavior with iptables and openflow in some cases | |
| 12:09:08 | sean-k-mooney[m] | the openflow firewall is more permissive interms fo what packets are allowed | |
| 12:09:09 | EugenMayer4 | yes, the iptables vs openflow stuff is entirely on me. We use OVS and i'am actually only okish with iptables in general, understand the concepts and know how to debug and isolate. Openflow is rather blowing my mind up | |
| 12:09:47 | EugenMayer4 | So that one is very much on me. I seem to struggle a lot of netns/openflow and all the tools one needs (and a lot of more complex concepts) | |
| 12:10:34 | sean-k-mooney[m] | i learnd openflow and ovs at the same time i was learning linux networking so i gnereally unserdstand the ovs side better | |
| 12:11:16 | sean-k-mooney[m] | if you were using the iptables firewalll driver you might be able to open egrees to the world and limit ingress | |
| 12:11:45 | sean-k-mooney[m] | the way the connection tracker stuff works is different between the two | |
| 12:12:24 | EugenMayer4 | well yes, i could limit the other side, but that seems overkill closing down the ingress on all the other targets. But surely doable somewhat | |
| 12:13:21 | sean-k-mooney[m] | https://docs.openstack.org/neutron/latest/admin/config-ovsfwdriver.html#differences-between-ovs-and-iptables-firewall-drivers | |
| 12:13:51 | EugenMayer4 | I'am used to limit what networks can do, since that really helps reducing the extra cost. I mean networks in openstack, beside the segmentation, has no real usage otherwise, if you cannot really control the flow, right. I mean you could also opt in for a huge 16 network or whatever and then use ingress for limitations (somewhat) | |
| 12:14:52 | sean-k-mooney[m] | neutron model is all based aroudn the ports not the networks | |
| 12:15:02 | sean-k-mooney[m] | and doign qos/firewallign on the ports | |
| 12:15:26 | EugenMayer4 | interesting, reading the OVS part it would mean, if i have an egress 0.0.0.0/0 and on 10.10.5.5/32 and i want to talk to 10.10.5/32 it might block it (but it does not make sense) | |
| 12:15:40 | sean-k-mooney[m] | partly because form most of its life it only had contol at the endpoints | |
| 12:15:42 | EugenMayer4 | yes, maybe i have to adopt that more. I usually tend to do more with networks | |
| 12:15:44 | sean-k-mooney[m] | i.e. linux bridge and ovs really could not influcance the core network at all | |
| 12:16:26 | EugenMayer4 | So you would, potentially, instead of controlling the egress of my DMZ network, rather control the ingress of the VMS in the the intranet network | |
| 12:16:45 | sean-k-mooney[m] | so there were those that wanted to do that and they created the firewall as a service and service function chaning projects to try and do that | |
| 12:16:46 | EugenMayer4 | that would be more the neutron way, right? | |
| 12:16:54 | sean-k-mooney[m] | the problem is they didnt keep maintianing them | |
| 12:17:16 | sean-k-mooney[m] | contolling ingress to the vms is the normally way yes | |
| 12:17:28 | sean-k-mooney[m] | security groups by defaull block all trafffic | |
| 12:17:45 | sean-k-mooney[m] | and you are expected to only open the port to the clinets that need acess | |
| 12:17:53 | EugenMayer4 | Yeah, i think the openstack hype was 2018 and has fallen quiet a bit. Also looked for some literature recently. People basically stopped publishing in 2018 | |
| 12:18:33 | sean-k-mooney[m] | i tought the hype died before that but ok :) | |
| 12:19:01 | EugenMayer4 | Well i'am not the one to define that, i came in here very much after that. I think we run openstack since jan 2021 | |
| 12:19:04 | sean-k-mooney[m] | the problem escpailly on the neutron side si none of the network vendors wanted to work and maintian the core | |
| 12:19:20 | sean-k-mooney[m] | they wanted to integrate there network stack so they could sell you that | |
| 12:19:52 | EugenMayer4 | well i think, the "vendors" wanted to make a difference in their services they offer, and to have that, they stopped sharing to be "different and better" then the competitor vendor | |
| 12:20:22 | EugenMayer4 | looking at what is really missing in openstack, and looking who is using openstack big scale, i see that most of the things i miss, have been solved on-platform by that vendor | |
| 12:20:43 | sean-k-mooney[m] | right but since neutron allows vendor extentions in the api cisco or juniper would just add a vendor exteion instead of implement a common shared api | |
| 12:20:59 | EugenMayer4 | yeah, i see that. | |
| 12:21:27 | sean-k-mooney[m] | there is less of a porblem for that in cinder | |
| 12:21:52 | sean-k-mooney[m] | they do not allow arbitray api extentions and use microverions like nova and keystone | |
| 12:22:01 | sean-k-mooney[m] | so there is driver to driver variance | |
| 12:22:11 | sean-k-mooney[m] | but the api is much more uniform | |
| 12:23:20 | EugenMayer4 | I would also say that in terms of API, openstack seems like designed by sysops people opting on microservices, while forgetting / not caring about all the dowsides and complications. There are so much lose ends, nobody is responsible for a task. Creating a backup with glance of a machine in nova ends up to be a task nobody has control over. One | |
| 12:23:20 | EugenMayer4 | starts it, the other "does something but has no clue who and what it belongs to" .. and that funny thing then, if anything errors, does not know how and what to recover or where to report the error to even | |
| 12:24:58 | EugenMayer4 | I think the hypversor/network/encapsulation part of Openstack is really solid, that's where a lot of the sysops people could do a lot of the good stuff. But the API and Software stack, the architecture, seems to have been neglected and as an "end user" that really bothers me | |
| 12:29:00 | EugenMayer4 | A war story, from time to time, no specific pattern, on of our VMs is just shutdown. In the VM audit log i see an anon user (so it will be something systemic) telling the VM to shutdown (cleanly). That's it. No trails no nothing .So something told something to do something somewhere - and i cannot even track the "next layer". Or i have a | |
| 12:29:00 | EugenMayer4 | hard-anti-affinity of a server-group. 3 VMS, 3 computes. first deployment, happy time, they land on 1 compute each. From time to time, openstack decides to reschedule them, and on VM lands on the same compute, so one compute has none. This happened about 3 times now and finally i will basically do the stupid thing and tie each of them to one | |
| 12:29:00 | EugenMayer4 | specific compute. What is responsible for the re-scheduling, IDK. | |
| 12:29:19 | EugenMayer4 | Enough of the ranting (if it was any). Thank you a lot for your insight (as always!) | |
| 13:19:03 | opendevreview | melanie witt proposed openstack/nova master: Reproducer for bug 2003991 unshelving offloaded instance https://review.opendev.org/c/openstack/nova/+/872470 | |
| 13:19:04 | opendevreview | melanie witt proposed openstack/nova master: Enforce quota usage from placement when unshelving https://review.opendev.org/c/openstack/nova/+/872471 | |
| #openstack-nova - 2023-02-05 | |||
| 16:54:19 | prometheanfire | working on an updated constraints patchset, so far it's just nova and tempest that are still failing, anyone around to help figure out which lib update is causing the fail? | |
| 16:54:23 | prometheanfire | https://zuul.opendev.org/t/openstack/build/4ba64ca08116467daca31551ce8a9753 | |
| 22:32:37 | opendevreview | Takashi Natsume proposed openstack/placement master: Fix a wrong assertion method https://review.opendev.org/c/openstack/placement/+/861489 | |
| #openstack-nova - 2023-02-06 | |||
| 03:22:23 | opendevreview | yzp proposed openstack/nova master: Fix bug after executing cinder retype https://review.opendev.org/c/openstack/nova/+/872726 | |
| 06:20:19 | opendevreview | yzp proposed openstack/nova master: fix the bug of volume_id is not correct about existed instance https://review.opendev.org/c/openstack/nova/+/872728 | |
| 08:59:09 | ralonsoh | sean-k-mooney, good morning! do you think we should reduce the concurrency in the os-vif tempest tests? | |
| 08:59:12 | ralonsoh | https://review.opendev.org/c/openstack/os-vif/+/872655 | |
| 08:59:50 | ralonsoh | increasing the swap size reduce the occurrence of oom but kills the performance | |
| 10:27:42 | opendevreview | yzp proposed openstack/nova master: fix the bug of volume_id is not correct about existed instance https://review.opendev.org/c/openstack/nova/+/872728 | |