| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-04 | |||
| 13:09:18 | gmann | bauzas: ah, good point on yoga spec link. in that case, let me keep it same name then and in detail i can mention what all things this BP is targeting. I will update zed proposed spec file | |
| 13:09:33 | bauzas | gmann: ack ok | |
| 13:47:12 | opendevreview | Rico Lin proposed openstack/nova-specs master: Add vIOMMU device support for libvirt driver https://review.opendev.org/c/openstack/nova-specs/+/840310 | |
| 14:11:00 | sean-k-mooney | gibi: so rodolfo has a patch to make this the default but do we ant to proceed with https://review.opendev.org/c/openstack/nova/+/840446 anyway and perhaps backport that to the relevent branches ? | |
| 14:11:55 | gibi | I haven't see hits of this bug on stable. do we have the hybrid job on stable? | |
| 14:12:29 | sean-k-mooney | i think artom is adding it we had before yoga too since that was the devstack default | |
| 14:14:38 | sean-k-mooney | gibi: https://github.com/openstack/nova/blob/stable/yoga/.zuul.yaml#L650= | |
| 14:15:17 | gibi | then I think it make sense to land this now and backport it | |
| 14:15:28 | gibi | then we can drop the flag from master when ralonsoh's patch lands | |
| 14:16:05 | sean-k-mooney | this https://review.opendev.org/c/openstack/nova/+/828413/2 and https://review.opendev.org/c/openstack/nova/+/828418 will be adding it to xena and wallaby | |
| 14:16:18 | sean-k-mooney | cool | |
| 16:04:28 | opendevreview | Rico Lin proposed openstack/nova-specs master: Add vIOMMU device support for libvirt driver https://review.opendev.org/c/openstack/nova-specs/+/840310 | |
| 16:05:56 | ricolin | sean-k-mooney: Thanks for your very detailed review, just update the spec accordingly :) | |
| 16:34:35 | opendevreview | Merged openstack/nova-specs master: Re-propose remove tenant_id https://review.opendev.org/c/openstack/nova-specs/+/837789 | |
| 16:44:10 | opendevreview | Ghanshyam proposed openstack/nova-specs master: Re-propose allow Project admin to list allowed hypervisors https://review.opendev.org/c/openstack/nova-specs/+/833165 | |
| 16:45:25 | gmann | gibi: dansmith ^^ as you reviewed it in Yoga cycle. re-proposing the spec. | |
| #openstack-nova - 2022-05-05 | |||
| 09:00:45 | kashyap | gibi: Hi, the 'py310' and 'nova-next' failures seem nothing to do with my patch, yeah? - https://review.opendev.org/c/openstack/nova/+/838926 | |
| 09:01:46 | gibi | py310 is definitely not, as it fails at setting up mysql | |
| 09:02:07 | bauzas | wow, python 31.0 release number, time flies | |
| 09:02:16 | bauzas | :) | |
| 09:02:44 | gibi | the nova-next fails with some rsync issue so that is also not your patch | |
| 09:03:00 | gibi | I'm not sure we have a tracking bug for the py310 issue but we should | |
| 09:03:17 | gibi | the nova-next issue might be intermittent, but worth to check if it happens regurarly | |
| 09:03:50 | gibi | bauzas: yeah, I push the problem out to time when py 31.0 will be released until then py310 is py 3.10 for me :D | |
| 09:03:52 | kashyap | I see, yeah - I'll file a tracking bz for py310 'mysql' fail | |
| 09:04:22 | gibi | kashyap: try to check if the same job fails for other repos too maybe there is also a tracking bug somewhere outside of nova | |
| 09:04:37 | kashyap | Right, good point. I'll check before filing | |
| 09:15:32 | kashyap | gibi: Hmm, I checked a random Neutron patch from today, and py310 (non-voting) seem to pass: https://review.opendev.org/c/openstack/neutron/+/840565 | |
| 09:18:17 | gibi | interesting | |
| 09:18:27 | gibi | then we probably need a nova tracking bug for this | |
| 10:19:37 | sean-k-mooney | python 3.10 and eventlest are not partically happy currently. i dont think you can currenlty stack with devstack for example | |
| 10:19:49 | sean-k-mooney | im not sure if we wil see that in unit/functional tests | |
| 10:20:00 | sean-k-mooney | since we dont really use eventlets there the same way | |
| 10:20:04 | sean-k-mooney | just an fyi | |
| 10:20:37 | sean-k-mooney | deian are in the process of moving debian testing to 3.10 and have been having issue with this lately | |
| 10:22:13 | kashyap | sean-k-mooney: Thanks; so it's not just me. | |
| 10:24:41 | sean-k-mooney | kashyap: i think our unit test could pass but i know that there are some issue so i have not looked at your spcific failure but if its a devstack run i woudl expect it to fail currenlty | |
| 10:41:15 | gibi | but the current failue is somewhere in the job setup as it fails on mysql :) | |
| 10:50:35 | frickler | gibi: ah, these nice "unit" tests with external dependencies. py310 currently runs on f35, maybe your setup is ubuntu specific? | |
| 10:51:00 | gibi | frickler: yeah that seems to be the issue | |
| 10:51:05 | frickler | devstack on 22.04 runs the base services mostly without issues except for horizon | |
| 10:51:42 | frickler | https://review.opendev.org/q/topic:add-jammy2 | |
| 11:26:20 | ralonsoh | sean-k-mooney, hi! I'm playing again with live-migration and ceph | |
| 11:26:30 | ralonsoh | and, of course, during the compute installation I had this | |
| 11:26:31 | ralonsoh | 2022-05-05T11:18:22.104+0000 7fcd2a59c700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2] | |
| 11:26:47 | ralonsoh | I've copied the ceph ID from the controller (that is also a compute) | |
| 11:28:14 | ralonsoh | sean-k-mooney, never mind! what I was missing was the keyring files | |
| 11:28:24 | ralonsoh | I've copied them and now it works heheheh | |
| 11:31:53 | sean-k-mooney | cool | |
| 11:32:19 | opendevreview | Merged openstack/nova master: Fix segment-aware scheduling permissions error https://review.opendev.org/c/openstack/nova/+/839361 | |
| 11:32:22 | sean-k-mooney | by the way you might find https://github.com/SeanMooney/ansible_role_devstack interesting/helpful | |
| 11:33:27 | sean-k-mooney | the ceph support curerntly only works on ubuntu since the devstack pluging currently does not have c9s support | |
| 13:16:25 | sean-k-mooney | dansmith: by the way did anything come of the excessivly high keystone and neutron load on the db | |
| 13:16:58 | sean-k-mooney | from your perfoamcne data patches | |
| 13:34:48 | opendevreview | Merged openstack/nova master: Allow claiming PCI PF if child VF is unavailable https://review.opendev.org/c/openstack/nova/+/838555 | |
| 14:03:58 | dansmith | sean-k-mooney: I brought it up to the keystone people a couple days ago, and it was agreed that it was too high, but nothing more than that (yet) | |
| 14:04:43 | sean-k-mooney | ack so they did not see any smoking gun or anything else obviously wrong | |
| 14:11:08 | dansmith | well, nothing obvious off the top of their head, I dunno that anyone has done any real digging yet | |
| 14:41:55 | ralonsoh | sean-k-mooney, sorry again. Did you see something like this https://paste.opendev.org/show/bEMWSLCst6bhd4Ir3zwl/? | |
| 14:42:15 | ralonsoh | one of the nova-compute services is exiting | |
| 14:42:25 | ralonsoh | in a non very elegant way | |
| 14:42:28 | sean-k-mooney | ew segfault | |
| 14:42:46 | ralonsoh | this is the controller-compute node | |
| 14:42:47 | sean-k-mooney | no that is new to me | |
| 14:42:50 | ralonsoh | ok | |
| 14:43:01 | ralonsoh | I'll try to re install everything | |
| 14:43:03 | sean-k-mooney | where was that thrown | |
| 14:43:12 | ralonsoh | during, of course, the live migration | |
| 14:43:13 | sean-k-mooney | was it ci or a local vm/server | |
| 14:43:20 | ralonsoh | when I'm bringing the VM to this host | |
| 14:43:37 | ralonsoh | when the VM evacuates this host, all is OK | |
| 14:43:52 | ralonsoh | no sorry, it is when the VM evacuates the host | |
| 14:44:05 | sean-k-mooney | evacuate or live migrate | |
| 14:44:11 | sean-k-mooney | they are very differnt things | |
| 14:44:17 | ralonsoh | yeah sorry | |
| 14:44:19 | ralonsoh | live-migrate | |
| 14:44:36 | ralonsoh | when the VM is leaving the host | |
| 14:44:37 | sean-k-mooney | so you live migrate and then the source host segfaults? | |
| 14:44:56 | sean-k-mooney | are there any OOM errors | |
| 14:45:03 | sean-k-mooney | or other detail | |
| 14:45:09 | ralonsoh | let me check | |
| 14:45:22 | ralonsoh | nope, I have still 13GB free | |
| 14:45:32 | sean-k-mooney | like the segfault appears to by in python but im wondering if an allcoation failed because you ran out of memory | |
| 14:45:34 | sean-k-mooney | ok | |
| 14:52:14 | opendevreview | Balazs Gibizer proposed openstack/nova master: Adapt tools/test-setup to Fedora 35 https://review.opendev.org/c/openstack/nova/+/840684 | |
| 14:52:51 | gibi | frickler: I try to unblock openstack-tox-py310 job | |
| 14:52:57 | gibi | with ^^ | |
| 14:56:11 | kashyap | gibi: Thank you! | |
| 14:58:22 | opendevreview | Rico Lin proposed openstack/nova-specs master: Add vIOMMU device support for libvirt driver https://review.opendev.org/c/openstack/nova-specs/+/840310 | |
| 15:59:15 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Replace old URLs with new ones https://review.opendev.org/c/openstack/python-novaclient/+/840693 | |
| 16:10:22 | gibi | frickler, kashyap: my google foo failed me to figure out why the mysql password change fails on fedora 35 in openstack-tox-py310 So if you have ideas please shoot https://review.opendev.org/c/openstack/nova/+/840684/1#message-ffd5ac00ca235cfaebd12988b65dbc210c2b9ec8 | |
| 16:15:41 | clarkb | gibi: seems like your mysqladmin tool isn't compatible with mariadb | |
| 16:15:47 | clarkb | since it is generating the sql that fails | |
| 16:20:37 | mnaser | hrm | |
| 16:20:52 | gibi | hm, mariadb is on version 10.5 but mysqlclient and mysqladmin is on 8.0.28 but I'm not sure how to map these verison | |
| 16:21:16 | mnaser | i've got a really weird situation where a hypervisor stops getting vms scheduled to it | |
| 16:21:34 | mnaser | i checked `openstack resource provider inventory list 9fe525d9-df51-41c2-8ca7-8344ed0eee39` and that shows the resources available, and `openstack allocation candidate list --resource VCPU=4 --resource DISK_GB=64 --resource MEMORY_MB=2048 | grep 9fe525d9-df51-41c2-8ca7-8344ed0eee39` even returns that | |
| 16:21:51 | mnaser | so its not placement | |
| 16:21:56 | clarkb | gibi: mysqlclient and mysqladmin are probably the mysql tools and not the mariadb tools? Possible that mariadb has alternatives | |
| 16:22:47 | gibi | clarkb: yeah that make sense... try to figure out where are those alternatives | |