Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-04-24
12:56:00 bauzas then they can skip-level upgrade to Antelope
12:56:17 sean-k-mooney yes and then upgrade there os to 22.04
12:56:25 bauzas then, before skip-level upgrading again to C, they need to upgrade their computes to 22.04 and py310
12:56:31 sean-k-mooney then slurp to 2024.1
12:56:32 bauzas cool, then I accept the plan
12:56:51 bauzas sean-k-mooney: ok, then I'll clarify the context
12:57:03 bauzas sean-k-mooney: fwiw, I'm horrified we're pulling tooz as a dep for nova
12:57:15 bauzas just because of the ironic virt driver
12:57:18 sean-k-mooney its just for ironic and its going away
12:57:58 sean-k-mooney likely in 2024.2 assumign we get the shared supprot done in B or C
12:58:22 sean-k-mooney our aim shoudl be to deprecate teh peer list in b/C so it can go out in C and we can drop it in D
12:58:44 sean-k-mooney ralonsoh: what is the problem with ndested virt by the way
12:58:56 ralonsoh https://bugs.launchpad.net/neutron/+bug/1999249
12:59:00 sean-k-mooney ralonsoh: you are aware that we are not ment to have any voting jobs that use it
12:59:06 ralonsoh timeouts in the jobs
12:59:50 ralonsoh so how we implement neutron-tempest-plugins jobs?
13:00:03 sean-k-mooney ralonsoh: they shoudl be useing QEMU not kvm
13:00:09 sean-k-mooney like all the nova jobs
13:00:42 sean-k-mooney ralonsoh: infra provied the abltiy to run josb with nested virt but we shoudl not have any voting jobs in any project that depend on them
13:01:08 sean-k-mooney ralonsoh: can you link to the job so i can confirm its actully using nested virt
13:01:14 ralonsoh sean-k-mooney, one sec
13:01:39 sean-k-mooney https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/857031/8/zuul.d/base-nested-switch.yaml#32
13:01:42 sean-k-mooney so ya they are
13:01:49 ralonsoh https://github.com/openstack/neutron-tempest-plugin/blob/master/zuul.d/base-nested-switch.yaml
13:01:53 ralonsoh exactly
13:02:21 sean-k-mooney im kind of surpised that you have those since we were ask not to use the nested virt lables in any voting job
13:02:37 sean-k-mooney ralonsoh: what is the reason you are using nested virt there
13:02:50 ralonsoh sean-k-mooney, I can't reply to this, tbh
13:02:53 sean-k-mooney what kvm only funcitonality is neutron testing tha twould justify that
13:03:03 ralonsoh yk^^
13:03:08 ralonsoh one sec
13:03:33 sean-k-mooney to be clare we have some kvm only fucntionliyt in nova we do not test sicne we did not have enough cloud provider to be comfortable with having a votign job
13:03:40 ralonsoh ykarel, hi!
13:03:46 ykarel hi ralonsoh
13:03:48 ralonsoh let me ask you the same question
13:04:00 ralonsoh what is the reason you are using nested virt there
13:04:08 ralonsoh in n-t-p
13:05:18 ykarel ralonsoh, context in https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/821067
13:05:34 ykarel linked etherpad have more details
13:07:20 ralonsoh ykarel, then I think, for now, we should move to Jammy and non nested
13:07:44 sean-k-mooney ykarel: you realsie that we are not ment to have any voting job that use nested virt since we have not enough ci priorse for it. we were asked in teh past not to make any voting jobs unless we had 3 providers for it
13:08:05 sean-k-mooney it looks like we now have 2 clouds form vexhost and one form ovh
13:08:15 sean-k-mooney is this why this was made to use nested virt
13:08:22 ykarel sean-k-mooney, when we moved i recall there were 3 providers, need to check current status
13:08:26 sean-k-mooney i am not aware fo any anochment that it was now ok to do this
13:08:50 sean-k-mooney ykarel: so nested virt should only be used if a job needs it
13:09:02 sean-k-mooney what in this tempest plug need nested virt
13:09:59 sean-k-mooney we were previosuly asked not to enable it just to make the job faster to concerve the capsity
13:10:09 ykarel sean-k-mooney, yes right nothing in those tests need nested virt, the switch was done just for better perf/times in jobs, and that worked great for us for almost an year before hte switch to jammy
13:10:09 ralonsoh no one in particular, that was to improve the current stability of our jobs
13:10:29 sean-k-mooney ykarel: right so per/jobs times was specfic not a valid reason to use it in the past
13:11:00 ralonsoh ok, let's revert that for now and move to jammy
13:11:15 ralonsoh I'll push a DNM patch to test that
13:11:42 lajoskatona +1 let's have fresh results, and base any forward steps on that
13:14:23 ykarel sean-k-mooney, yes right i discussed this few months back with infra some months back and i was said it should be avoided just for perf but if team knows the issues with using nested virt(no support, less providers etc) it can be used as those nodes have resources available to use
13:15:08 sean-k-mooney ykarel: it used ot be stated in the docs somewhere too
13:15:53 ykarel and at that time we switched to focal until infra issue get's fixed(it requires infra compute nodes to be upgraded)
13:15:54 sean-k-mooney ykarel: my concern is we have thing that at least previous could only be tested with nested virt in nova
13:16:09 sean-k-mooney and it was previously not considerd ok to use the nested virt lables to do that
13:17:55 ykarel ralonsoh, lajoskatona due to qemu/libvirt version just revert will not work as we run many concurrent guest vms (6+) while qemu-5.0.0 it uses too much memory
13:18:03 ykarel 1gb per guest vm extra by default
13:18:29 sean-k-mooney because fo the tcib cache issue
13:18:33 sean-k-mooney when not using nested
13:18:52 ykarel libvirt-8.0.0 supports customizing it and for that DNM patch https://review.opendev.org/c/openstack/nova/+/868419
13:19:10 ykarel will update it as per sean-k-mooney comments
13:19:35 sean-k-mooney this should be a specless blueprint
13:19:38 ykarel but even with we need to adjust(split into multiple ) our jobs
13:19:47 sean-k-mooney can we get this on the team meetign adjenda for tomorrow
13:19:51 sean-k-mooney bauzas: ^
13:20:29 sean-k-mooney ykarel: i think we shoudl start a mail thread on using nested virt in votign jobs
13:22:01 sean-k-mooney https://lists.openstack.org/pipermail/openstack-discuss/2019-April/004681.html was the last time we discussed it on the list i think
13:26:25 bauzas sean-k-mooney: I'm a bit diverted by some meeting now, but agenda is up for everyone : https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
13:27:30 sean-k-mooney bauzas: so tooz shoudl be blocked by UC right
13:28:06 sean-k-mooney i guess we dont have any focal based job that blocked the update
13:28:39 sean-k-mooney so in the short term i think we are goign to have to downgrwade the tooz verion in uc untill we can move the ceph jobs to focal or resolve the uwsgi issue
14:21:55 dansmith bauzas: any ideas on who else we could get to review this and the following patch? https://review.opendev.org/c/openstack/nova/+/880632
14:22:04 dansmith I'm sort of waiting to rebase my actual compute ids stuff on that
14:22:28 bauzas dansmith: good question, let's say gibi or sean-k-mooney
14:22:39 ykarel sean-k-mooney, ack will add it to meeting agenda. and wrt mail thread sure could be done but as discussed in past voting should be avoided unless and until nested-virt is really needed as it's support is best effort and only few providers compared to other providers, but may be it can help in getting the messaging more clear to everyone
14:22:52 dansmith bauzas: ack, I just know they're both very busy with other things.. I added sean-k-mooney a while back
14:45:54 frickler sean-k-mooney: there could also be a specific py38 pin in u-c for tooz, like we had for py2.7 for some time. you may want discuss with reqs team (or maybe TC)
14:47:51 frickler also reminder that tooz isn't the only lib affected, oslo.db has the same situation except that u-c adoption is still blocked by other things stephenfin is having fun with
15:07:25 opendevreview Jorge San Emeterio proposed openstack/nova master: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/873127
15:23:04 opendevreview Balazs Gibizer proposed openstack/nova master: Revert "Temporary skip some volume detach test in nova-lvm job" https://review.opendev.org/c/openstack/nova/+/881389
15:43:44 opendevreview Balazs Gibizer proposed openstack/nova master: Revert "Temporary skip some volume detach test in nova-lvm job" https://review.opendev.org/c/openstack/nova/+/881389
16:07:29 bauzas sean-k-mooney: dansmith: lemme get it right, so https://zuul.opendev.org/t/openstack/build/5f269ed5244d47be800dab988253c72c is failing because we bumped py to 3.9 and apache httpd complains ? https://review.opendev.org/c/openstack/nova/+/881339/3/.zuul.yaml#586
16:07:52 sean-k-mooney yes
16:08:04 sean-k-mooney if we move tooz to an extra package
16:08:10 sean-k-mooney we can drop the bump to 3.9
16:08:21 sean-k-mooney and that job should work since it does not have ironic
16:08:27 sean-k-mooney or barbican
16:08:38 sean-k-mooney so castalan and tooz wont be installed
16:09:10 sean-k-mooney or we can move the nova-ceph-multistore to 22.04
16:09:20 dansmith I don't actually see the failure logged anywhere, other than keystone can't be imported
16:09:55 bauzas yeah me too, hence my question
16:10:05 dansmith oh I see that's the bump to 3.9
16:10:09 bauzas this looks to me that keystone can't work with 3.9
16:10:22 dansmith or uwsgi is using 3.8 more likely I think
16:10:30 bauzas because RegionOne isn't an UUID
16:10:38 dansmith it says importerror but doesn't actually show that it's not a valid module, so I'm not positive

Earlier   Later