| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-08-24 | |||
| 19:37:27 | lyarwood | Tune the I/O APIC. Possible values for the driver attribute are: kvm (default for KVM domains) and qemu which puts I/O APIC in userspace which is also known as a split I/O APIC mode. Since 3.4.0 (QEMU/KVM only) | |
| 19:37:27 | lyarwood | ioapic | |
| 19:37:54 | lyarwood | https://libvirt.org/formatdomain.html#hypervisor-features | |
| 19:38:33 | sean-k-mooney | let me look at the kernel message again | |
| 19:38:45 | sean-k-mooney | the one i was fixing initally was a hagn related to the apic not the ioapic | |
| 19:39:21 | sean-k-mooney | ok so that case ya is deffinetly the io apic | |
| 19:39:53 | sean-k-mooney | am i would still be tempted to say driver=none | |
| 19:39:57 | sean-k-mooney | but that is not documented | |
| 19:40:22 | sean-k-mooney | lyarwood: i dont think changing it to qemu will help | |
| 19:41:08 | lyarwood | Why given it's suggesting you should use that when using QEMU | |
| 19:41:55 | sean-k-mooney | well libvirt woudl do that automtically | |
| 19:42:03 | sean-k-mooney | kvm is only the default for kvm domains | |
| 19:42:10 | sean-k-mooney | it wont use kvm on a qemu domain | |
| 19:43:08 | lyarwood | Are you sure libvirt will add this automatically to the domain? | |
| 19:43:55 | sean-k-mooney | well we are not requesting it if it provdie an api it will set the correct drvier im pretty sure | |
| 19:44:06 | sean-k-mooney | if no i would consider that a libvirt bug | |
| 19:44:24 | sean-k-mooney | one that we likely should not fix in nova we could but that not how i would expect this to work | |
| 19:44:43 | sean-k-mooney | without requesting an ioapic by the way i would not expect to get one | |
| 19:44:47 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/ddb6cf53e65a404c89dac4dc4ca1cd51/log/controller/logs/screen-n-cpu.txt#7489 | |
| 19:44:55 | sean-k-mooney | we are not requesting it | |
| 19:45:11 | sean-k-mooney | perhaps its nto considerd a toggelable feature | |
| 19:45:19 | sean-k-mooney | "All features are listed within the features element, omitting a togglable feature tag turns it off." | |
| 19:46:45 | lyarwood | okay so we've never enabled the ioapic somehow? | |
| 19:47:34 | lyarwood | https://paste.centos.org/view/e7a2611c - we've only had three hits of this recently in our gate and I think two of these were before the latest workaround apic patch landed | |
| 19:47:46 | lyarwood | so maybe this can wait until kashyap is back | |
| 19:49:27 | sean-k-mooney | we have not enabeld it in the xml no | |
| 19:51:26 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/ddb6cf53e65a404c89dac4dc4ca1cd51/log/controller/logs/libvirt/libvirt/qemu/instance-00000001_log.txt | |
| 19:51:36 | sean-k-mooney | we also done seam to be enablit it on the qemu commandline | |
| 19:52:54 | sean-k-mooney | so either we need to explictly disable it which there seams to be no documented way to do or this is somethign qemu can disabel but libvirt cant | |
| 19:56:15 | opendevreview | Artom Lifshitz proposed openstack/nova master: Reproduce bug 1941005 https://review.opendev.org/c/openstack/nova/+/805882 | |
| 19:56:29 | artom | sean-k-mooney, ^^ confirmed, resize with extra specs is b0rked | |
| 19:56:33 | artom | We don't update the request spec | |
| 19:56:49 | sean-k-mooney | still on master? | |
| 19:56:54 | artom | Yep | |
| 19:56:54 | sean-k-mooney | it used to work in the past | |
| 19:57:04 | artom | stephenfin's patch fixed the numa topology case, not the general case | |
| 19:57:15 | sean-k-mooney | even before stephens patch | |
| 19:57:25 | sean-k-mooney | we regressed this at some point | |
| 19:57:37 | sean-k-mooney | im sure we used to update it at least in memory | |
| 19:57:46 | artom | It's kinda tricky to see - *adding* stuff normally fails as expected if no hosts can provide it, because the resource claim uses the flavor and image directly | |
| 19:57:49 | sean-k-mooney | we may not have commited it back to the db | |
| 19:58:03 | sean-k-mooney | but we did use them at some point for scudliign and creatign the xml | |
| 19:58:33 | artom | But *removing* stuff can go wrong, because despite having hosts that can fit the new thing, the scheduler tries to schedule according to the old request spec | |
| 19:59:54 | sean-k-mooney | ok well its broke now so we should fix an backport in anycase | |
| 20:00:21 | artom | Yar | |
| 20:00:39 | sean-k-mooney | but im pretty sure this used to work in icehouse as im pretty sure i tested resizeing to and from vms with pci deviecs back then | |
| 20:01:12 | artom | Perhaps - icehouse was a while ago - and it's pretty specific to hit | |
| 20:01:58 | lyarwood | cuts to titanic 80 years gif | |
| 20:02:17 | sean-k-mooney | hehe | |
| 20:02:57 | sean-k-mooney | yes it was i just remember testing this in the past althogu hit was specific realted to cpu pinnign and pci passtough so i dont know | |
| 20:03:36 | sean-k-mooney | no one complained and the xmls seams to eb right althgu we know that there are several placese where this si broken for move operations in the past so im not that surprised | |
| 20:04:01 | sean-k-mooney | artom: dont we have whitebox test fo this by the way | |
| 20:04:23 | artom | sean-k-mooney, we might... | |
| 20:04:42 | artom | But say you have two hosts with PCI devices - and you resize the instance from "PCI" to "no PCI" | |
| 20:05:03 | artom | It'll work fine - scheduling will pass with the bad old request spec because all hosts have PCI devices | |
| 20:05:17 | artom | It'll only bit you if you only have left hosts with *no* PCI devices | |
| 20:05:24 | artom | *bite you | |
| 20:05:37 | artom | Because then the old request spec can't be scheduled anywhere | |
| 20:05:52 | sean-k-mooney | artom: we not if we were validating the pci claims in the test like we should be | |
| 20:05:58 | sean-k-mooney | but looking we dont have test for this | |
| 20:06:21 | artom | I think claims use the flavor and image directly, so that part's fine | |
| 20:06:35 | artom | By pure luck, really | |
| 20:06:38 | artom | But it's fine :) | |
| 20:06:39 | sean-k-mooney | well we do | |
| 20:06:41 | sean-k-mooney | https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_vpmu.py | |
| 20:06:46 | sean-k-mooney | the vpum tests | |
| 20:06:58 | sean-k-mooney | i know we added this recently for a feature | |
| 20:07:06 | sean-k-mooney | *knew | |
| 20:07:25 | sean-k-mooney | the probel with https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_vpmu.py#L64-L74 | |
| 20:07:39 | sean-k-mooney | is that we are not asserting anyting baout the request spec | |
| 20:07:41 | sean-k-mooney | just the xml | |
| 20:08:04 | sean-k-mooney | which means on the destination host we are correctly using the new flavor | |
| 20:08:24 | sean-k-mooney | but as you said the scudler is not nessisarly useing the correct extra specs | |
| 20:09:30 | sean-k-mooney | artom: so you have a repoducer do you know where the bug is? | |
| 20:09:34 | sean-k-mooney | or is that still TBD | |
| 20:14:12 | artom | sean-k-mooney, well, I know vaguely where it is - we went through that code together to figure out the OSP 10 resize issue | |
| 20:14:25 | artom | But I don't have a fix yet, will need ot go through it again to figure out the best place for it | |
| 20:14:47 | artom | That code is full of potential races and side effects | |
| 20:15:48 | sean-k-mooney | ack ya ok. | |
| 20:38:39 | opendevreview | Merged openstack/nova master: Fix documentation about cpu topologies https://review.opendev.org/c/openstack/nova/+/805833 | |
| 20:43:27 | lyarwood | elodilles: https://review.opendev.org/c/openstack/nova/+/795435 - would you mind taking a look at this if you have time? | |
| 20:43:46 | NobodyCam | sean-k-mooney: FYI "resource provider / generation" for the win | |
| 20:52:36 | opendevreview | Merged openstack/nova stable/wallaby: Fix 1vcpu error with multiqueue and vif_type=tap https://review.opendev.org/c/openstack/nova/+/805304 | |
| 21:38:50 | opendevreview | Merged openstack/nova master: api: Introduce microversion 2.89 adjusting os-volume_attachments https://review.opendev.org/c/openstack/nova/+/804275 | |
| 22:08:32 | opendevreview | Merged openstack/nova master: Follow up from bp/pci-socket-affinity series https://review.opendev.org/c/openstack/nova/+/779556 | |
| #openstack-nova - 2021-08-25 | |||
| 01:36:00 | NobodyCam | sean-k-mooney: I clarify that there is a reserve inventory instance that is tripping up our readiness checks.. | |
| 01:46:43 | opendevreview | Merged openstack/nova stable/stein: Move 'check-cherry-picks' test to gate, n-v check https://review.opendev.org/c/openstack/nova/+/804615 | |
| 02:25:34 | opendevreview | Merged openstack/nova stable/stein: Prevent archiving of pci_devices records because of 'instance_uuid' https://review.opendev.org/c/openstack/nova/+/760984 | |
| 02:25:43 | opendevreview | Merged openstack/nova master: fup: Remove unused legacy block_device_info format https://review.opendev.org/c/openstack/nova/+/804286 | |
| 02:25:54 | opendevreview | Merged openstack/nova master: fup: Increase service_down_time beyond INITIAL_REPORTING_DELAY in test https://review.opendev.org/c/openstack/nova/+/805667 | |
| 08:56:18 | opendevreview | test proposed openstack/nova master: Add some missing parameters in docs of os-cells https://review.opendev.org/c/openstack/nova/+/805972 | |
| 09:26:57 | opendevreview | Lee Yarwood proposed openstack/nova master: nova-manage: Introduce volume show, refresh, get_connector commands https://review.opendev.org/c/openstack/nova/+/800634 | |
| 09:51:38 | opendevreview | Balazs Gibizer proposed openstack/placement master: Restrict amqp indirect dep to speed up pip resolution https://review.opendev.org/c/openstack/placement/+/805979 | |
| 09:51:45 | opendevreview | Balazs Gibizer proposed openstack/placement master: Bump os-traits to latest 2.6.0 https://review.opendev.org/c/openstack/placement/+/805830 | |
| 09:55:59 | lyarwood | gibi: https://review.opendev.org/q/topic:remove_luks_workarounds some simple cleanups here if you have time | |
| 09:56:12 | elodilles | lyarwood: I've commented on the stable/train's CI patch. I'm not completely against it, but it is not necessary in Train so I wouldn't merge it in all cost, plus I see one difference that would reduce coverage if I'm not mistaken: py2 grenade | |
| 09:56:33 | gibi | lyarwood: ack, lookgin | |
| 09:56:37 | lyarwood | elodilles: ack I'll look | |
| 09:56:57 | lyarwood | gibi: thanks, I'm trying to get to your qos series btw, almost there :) | |
| 09:57:04 | gibi | lyarwood: thanks in advance :) | |