| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-08-24 | |||
| 17:22:09 | sean-k-mooney | rmart04: so ya https://github.com/openstack/nova/blob/stable/stein/nova/conf/configdrive.py#L30 you can use vfat if that is supported by the guests instead if you need too | |
| 17:23:38 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Add migration to resolve shadow table discrepancies https://review.opendev.org/c/openstack/nova/+/805738 | |
| 17:27:35 | rmart04 | ok thanks for all the info, will see how i get on tomorrow | |
| 18:04:56 | opendevreview | Lee Yarwood proposed openstack/nova master: fup: Fix os-volume_attachments api-ref parameters https://review.opendev.org/c/openstack/nova/+/805876 | |
| 18:06:34 | lyarwood | gmann: ack fixed in a fup on top of the api change to avoid the respin, apologies again! | |
| 18:06:45 | lyarwood | ganso: ack I'll review this evening | |
| 18:11:49 | opendevreview | Lee Yarwood proposed openstack/python-novaclient master: Microversion 2.89 - os-volume_attachments https://review.opendev.org/c/openstack/python-novaclient/+/805022 | |
| 18:14:47 | NobodyCam | Good Morning Nova folks, was disk_available_least hypervisor stat removed in Ussuri? | |
| 18:14:58 | NobodyCam | for Ironic hypervisors? | |
| 18:17:08 | sean-k-mooney | i dont think so in ussuri but you should not be use the disk filter in ussuri for ironic or otherwise if that is why you are asking | |
| 18:19:29 | NobodyCam | hey hey Sean, in queens that was a value I used to validate the node was ready for deployment. | |
| 18:21:15 | sean-k-mooney | why not use the node status in ironic for that | |
| 18:21:25 | sean-k-mooney | its ready for deployment when its in the avaiable state | |
| 18:22:35 | NobodyCam | with that we hit no valid hosts, I should add that when we provision we use force_node to deploy to specific nodes | |
| 18:24:46 | NobodyCam | here is a sample if what I was seeing: | |
| 18:24:57 | NobodyCam | https://www.irccloud.com/pastebin/fuSfO22M/ | |
| 18:25:39 | sean-k-mooney | NobodyCam: ironic stopped reporting resouces for cpu ram and disk several cycles ago | |
| 18:26:04 | NobodyCam | ack | |
| 18:26:24 | sean-k-mooney | i dont rember off the top of my head when we moved to trackign ironic nodes with custom resouce classes but im sure tha tlikely had an effect of os-hyperviors output eventually | |
| 18:26:39 | NobodyCam | yep | |
| 18:27:07 | sean-k-mooney | in wally we have removed disk_avaiable_lset https://specs.openstack.org/openstack/nova-specs/specs/wallaby/implemented/modernize-os-hypervisors-api.html | |
| 18:27:39 | NobodyCam | ++ | |
| 18:28:02 | sean-k-mooney | NobodyCam: what you could do is see if the node exist in os-hyperviors rather then check for that value specifically | |
| 18:28:29 | sean-k-mooney | but still the expected way to check this would be via ironic | |
| 18:28:52 | NobodyCam | yea, let me try a few thing... yea we do check ironic | |
| 18:29:09 | sean-k-mooney | NobodyCam: i would proably check the state filed | |
| 18:29:16 | sean-k-mooney | and mayby status | |
| 18:29:28 | sean-k-mooney | so check for sate:up status:enabled | |
| 18:29:57 | sean-k-mooney | that should mean its schduleable | |
| 18:30:33 | NobodyCam | ++ Awesome, let try some deploys with that... Thank you very much sean-k-mooney !!!! | |
| 18:30:41 | NobodyCam | I'll let you know what I find | |
| 18:30:47 | sean-k-mooney | cool | |
| 18:31:23 | sean-k-mooney | the other thing you could check woudl be placment to see that the node is there but i think os-hyperviors will only have the node after its in placment anyway | |
| 18:31:29 | opendevreview | Merged openstack/nova master: scheduler: Merge driver into manager https://review.opendev.org/c/openstack/nova/+/773644 | |
| 18:31:41 | opendevreview | Merged openstack/nova master: tests: Merge 'test_utils', 'test_scheduler_utils' https://review.opendev.org/c/openstack/nova/+/773646 | |
| 18:31:51 | opendevreview | Merged openstack/nova master: conf: Remove deprecated aliases https://review.opendev.org/c/openstack/nova/+/773647 | |
| 18:55:46 | lyarwood | melwitt: https://review.opendev.org/c/openstack/placement/+/679441/ - if you're happy to look at the nits from me and Sean in a fup I'm happy to +W this now | |
| 18:57:23 | melwitt | lyarwood: sure thing, sorry I had only skimmed them so far | |
| 18:58:57 | lyarwood | melwitt: no issues they are nits after all, I'll look at the AttributeCache change now | |
| 19:00:55 | melwitt | lyarwood: thanks, appreciate it. and I will look at the respin of the nova-manage volume patch today | |
| 19:01:49 | lyarwood | melwitt: thanks, https://review.opendev.org/c/openstack/nova/+/805667 and https://review.opendev.org/c/openstack/nova/+/802317 could also use another core if you get time | |
| 19:02:13 | lyarwood | the first fixing a gate-failure I introduced | |
| 19:02:41 | melwitt | will do, added to the queue. thanks | |
| 19:30:58 | lyarwood | sean-k-mooney: are you about? | |
| 19:31:45 | sean-k-mooney | yes | |
| 19:32:07 | lyarwood | sean-k-mooney: https://zuul.opendev.org/t/openstack/build/ddb6cf53e65a404c89dac4dc4ca1cd51/log/controller/logs/screen-n-cpu.txt#13396 & https://e31f9be4742ab987838e-5d50e8ef24c23b6b3512cdd707b25e49.ssl.cf1.rackcdn.com/804286/4/gate/tempest-ipv6-only/ddb6cf5/testr_results.html - So I'm seeing a test failure dump out console logs suggesting that CirrOS is still trying and failing to load apic related stuff even with the feature disabled | |
| 19:32:08 | lyarwood | in the domain | |
| 19:35:13 | lyarwood | I wonder if we actually want <ioapic driver='qemu'/> | |
| 19:36:09 | sean-k-mooney | i dont think soe but we migtht want driver='none' | |
| 19:36:56 | sean-k-mooney | wait ioapic | |
| 19:37:04 | sean-k-mooney | that is different from the apic | |
| 19:37:15 | lyarwood | yeah | |
| 19:37:27 | lyarwood | Tune the I/O APIC. Possible values for the driver attribute are: kvm (default for KVM domains) and qemu which puts I/O APIC in userspace which is also known as a split I/O APIC mode. Since 3.4.0 (QEMU/KVM only) | |
| 19:37:27 | lyarwood | ioapic | |
| 19:37:54 | lyarwood | https://libvirt.org/formatdomain.html#hypervisor-features | |
| 19:38:33 | sean-k-mooney | let me look at the kernel message again | |
| 19:38:45 | sean-k-mooney | the one i was fixing initally was a hagn related to the apic not the ioapic | |
| 19:39:21 | sean-k-mooney | ok so that case ya is deffinetly the io apic | |
| 19:39:53 | sean-k-mooney | am i would still be tempted to say driver=none | |
| 19:39:57 | sean-k-mooney | but that is not documented | |
| 19:40:22 | sean-k-mooney | lyarwood: i dont think changing it to qemu will help | |
| 19:41:08 | lyarwood | Why given it's suggesting you should use that when using QEMU | |
| 19:41:55 | sean-k-mooney | well libvirt woudl do that automtically | |
| 19:42:03 | sean-k-mooney | kvm is only the default for kvm domains | |
| 19:42:10 | sean-k-mooney | it wont use kvm on a qemu domain | |
| 19:43:08 | lyarwood | Are you sure libvirt will add this automatically to the domain? | |
| 19:43:55 | sean-k-mooney | well we are not requesting it if it provdie an api it will set the correct drvier im pretty sure | |
| 19:44:06 | sean-k-mooney | if no i would consider that a libvirt bug | |
| 19:44:24 | sean-k-mooney | one that we likely should not fix in nova we could but that not how i would expect this to work | |
| 19:44:43 | sean-k-mooney | without requesting an ioapic by the way i would not expect to get one | |
| 19:44:47 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/ddb6cf53e65a404c89dac4dc4ca1cd51/log/controller/logs/screen-n-cpu.txt#7489 | |
| 19:44:55 | sean-k-mooney | we are not requesting it | |
| 19:45:11 | sean-k-mooney | perhaps its nto considerd a toggelable feature | |
| 19:45:19 | sean-k-mooney | "All features are listed within the features element, omitting a togglable feature tag turns it off." | |
| 19:46:45 | lyarwood | okay so we've never enabled the ioapic somehow? | |
| 19:47:34 | lyarwood | https://paste.centos.org/view/e7a2611c - we've only had three hits of this recently in our gate and I think two of these were before the latest workaround apic patch landed | |
| 19:47:46 | lyarwood | so maybe this can wait until kashyap is back | |
| 19:49:27 | sean-k-mooney | we have not enabeld it in the xml no | |
| 19:51:26 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/ddb6cf53e65a404c89dac4dc4ca1cd51/log/controller/logs/libvirt/libvirt/qemu/instance-00000001_log.txt | |
| 19:51:36 | sean-k-mooney | we also done seam to be enablit it on the qemu commandline | |
| 19:52:54 | sean-k-mooney | so either we need to explictly disable it which there seams to be no documented way to do or this is somethign qemu can disabel but libvirt cant | |
| 19:56:15 | opendevreview | Artom Lifshitz proposed openstack/nova master: Reproduce bug 1941005 https://review.opendev.org/c/openstack/nova/+/805882 | |
| 19:56:29 | artom | sean-k-mooney, ^^ confirmed, resize with extra specs is b0rked | |
| 19:56:33 | artom | We don't update the request spec | |
| 19:56:49 | sean-k-mooney | still on master? | |
| 19:56:54 | artom | Yep | |
| 19:56:54 | sean-k-mooney | it used to work in the past | |
| 19:57:04 | artom | stephenfin's patch fixed the numa topology case, not the general case | |
| 19:57:15 | sean-k-mooney | even before stephens patch | |
| 19:57:25 | sean-k-mooney | we regressed this at some point | |
| 19:57:37 | sean-k-mooney | im sure we used to update it at least in memory | |
| 19:57:46 | artom | It's kinda tricky to see - *adding* stuff normally fails as expected if no hosts can provide it, because the resource claim uses the flavor and image directly | |
| 19:57:49 | sean-k-mooney | we may not have commited it back to the db | |
| 19:58:03 | sean-k-mooney | but we did use them at some point for scudliign and creatign the xml | |
| 19:58:33 | artom | But *removing* stuff can go wrong, because despite having hosts that can fit the new thing, the scheduler tries to schedule according to the old request spec | |
| 19:59:54 | sean-k-mooney | ok well its broke now so we should fix an backport in anycase | |
| 20:00:21 | artom | Yar | |
| 20:00:39 | sean-k-mooney | but im pretty sure this used to work in icehouse as im pretty sure i tested resizeing to and from vms with pci deviecs back then | |
| 20:01:12 | artom | Perhaps - icehouse was a while ago - and it's pretty specific to hit | |
| 20:01:58 | lyarwood | cuts to titanic 80 years gif | |