| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-12-06 | |||
| 19:35:34 | melwitt | sean-k-mooney: semi related to that, I had been meaning to ask you, why does using the UEC image work to avoid guest kernel panics? | |
| 19:36:15 | melwitt | just a curiosity question | |
| 19:40:09 | opendevreview | Merged openstack/nova-specs master: Add spec for PCI Groups https://review.opendev.org/c/openstack/nova-specs/+/899719 | |
| 19:41:03 | sean-k-mooney | melwitt: i dont actully know. i suspect it takign a slightly diffent path in both qemu and in the init shell script | |
| 19:41:30 | sean-k-mooney | melwitt: i know neutron had previosuly added apci=off or something like that | |
| 19:41:41 | sean-k-mooney | to disable the apic timer | |
| 19:41:47 | melwitt | ah ok | |
| 19:41:54 | sean-k-mooney | but they dont actully do that any more | |
| 19:42:07 | sean-k-mooney | basiclly one of the panics suggeted doign that and when you use the split image | |
| 19:42:24 | sean-k-mooney | nova supprot passing kernel command line args to qemu | |
| 19:42:52 | sean-k-mooney | so orginally they explored this just to follow the mesage output which said A X to the command line to turn this off | |
| 19:45:03 | sean-k-mooney | melwitt: https://bugs.launchpad.net/openstack-gate/+bug/1713162 | |
| 19:45:15 | sean-k-mooney | Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. | |
| 19:45:30 | sean-k-mooney | so ya they orgianlly tied setting noapic on the kernel commadn line | |
| 19:45:36 | melwitt | ohh I see | |
| 19:46:08 | sean-k-mooney | i "fixed" this a diffent way in devstack | |
| 19:46:39 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#workarounds.libvirt_disable_apic | |
| 19:47:53 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/766043 and https://github.com/openstack/devstack/commit/1e86a25cc28e34d7f73a4c6ccbbc3fc667598d50 | |
| 19:48:11 | sean-k-mooney | lee actully enabeld it in devstack | |
| 19:48:15 | melwitt | oh, so using the split image seems to be a way to alleviate the problem without disabling the apic timer? | |
| 19:48:41 | sean-k-mooney | so i tought at one point we were addign the noapic thign somewhere | |
| 19:49:09 | sean-k-mooney | i just dont know where that is exactly | |
| 19:49:54 | sean-k-mooney | melwitt: but ya apprently the split imag eis enouch i just cant prove that | |
| 19:50:00 | melwitt | well that's confusing. it's already been set to disabled but it was still panicking? 🤔 | |
| 19:50:00 | melwitt | well that's confusing. it's already been set to disabled but it was still panicking? 🤔 | |
| 19:50:26 | sean-k-mooney | so there are two tings | |
| 19:50:30 | sean-k-mooney | the apic and ioapic | |
| 19:50:44 | sean-k-mooney | we are disabling the apic | |
| 19:50:48 | melwitt | oh | |
| 19:51:19 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1939108 | |
| 19:51:45 | sean-k-mooney | im not sure if i orgianlly got those mixed up or not | |
| 19:52:06 | sean-k-mooney | in anycase i think thats a slighly idffent kernel panic | |
| 19:52:21 | melwitt | 🤯 | |
| 19:52:21 | melwitt | 🤯 | |
| 19:52:22 | sean-k-mooney | as that speicic one was a upstrema bug in the cirros iamge kernel that was fixed in a later release | |
| 19:52:52 | sean-k-mooney | basically this specific panicn in that bug should not happen in 6.x cirros iamges | |
| 19:53:32 | melwitt | I see | |
| 19:55:36 | sean-k-mooney | for what its worth it might be interesting to see if we can disabel that workaround once we have moved to the split miage | |
| 19:55:39 | sean-k-mooney | *image | |
| 19:56:43 | sean-k-mooney | we are using cirros 6.x now so it should not eb required | |
| 19:57:14 | melwitt | gotcha | |
| 19:57:15 | sean-k-mooney | so we could perhaps deprecate it this cycle and remove it in a future release | |
| 19:57:45 | melwitt | none of the panics I have seen had the message that specifically names the APIC timer so that seems to track | |
| 19:58:08 | melwitt | (from that bug) | |
| 19:58:55 | sean-k-mooney | ya same | |
| 20:00:22 | sean-k-mooney | ok im going to call it a day | |
| 20:00:31 | sean-k-mooney | ill chekc back on your patches tomorrow o/ | |
| 20:05:16 | melwitt | thanks sean-k-mooney, seeya tomorrow o/ | |
| 20:15:44 | JayF | sean-k-mooney: I know you're interested in getting sharding in this cycle; I'm trying to land stephen's chain here https://review.opendev.org/c/openstack/nova/+/659691/38 before re-landing sharding; if you want to take a look I think Ironic cores are happy (Ironic CI passes on the head of that chain) | |
| 20:24:19 | melwitt | dpawlik: hi, thanks for confirming! it was just a misunderstanding about swift log retention vs opensearch index retention. the swift log retention is 30 days but opensearch retains less. I think this is ok :) | |
| 21:32:14 | opendevreview | Merged openstack/nova master: Fix coverage issues with eventlet https://review.opendev.org/c/openstack/nova/+/900116 | |
| #openstack-nova - 2023-12-07 | |||
| 07:14:01 | dpawlik | melwitt ack. Let me know if you would like to increase the retention time | |
| 07:25:56 | opendevreview | Amit Uniyal proposed openstack/nova-specs master: Enforce console session timeout https://review.opendev.org/c/openstack/nova-specs/+/898553 | |
| 10:41:15 | opendevreview | Vasyl Saienko proposed openstack/nova master: Fix returning empty availability zone https://review.opendev.org/c/openstack/nova/+/902875 | |
| 10:58:40 | sfinucan | sean-k-mooney: Thanks for starting reviewing that ironic SDK migration series. Given it's a priority for the ironic folks and also relatively trivial, I wonder if I could get another "review sponsor" for the series? CC: gibi melwitt dansmith bauzas gmann | |
| 10:59:43 | stephenfin | It's been long enough since I wrote them that I could review myself at this point, but I'd rather not if I can avoid it | |
| 11:07:20 | sean-k-mooney[m] | ya they are failrly simple sofar. since you rebased them yesterday it would be good to land them before they start to bit rot | |
| 11:34:19 | sean-k-mooney[m] | ok done. im +2 all the way | |
| 11:51:22 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Use clearer syntax for empty generator https://review.opendev.org/c/openstack/nova/+/903076 | |
| 11:51:32 | stephenfin | sean-k-mooney[m]: "remove dark magic" patch | |
| 11:51:37 | stephenfin | ^ | |
| 11:52:42 | sean-k-mooney[m] | thanks | |
| 12:09:52 | opendevreview | Pranali Deore proposed openstack/nova master: Support glance's new location API https://review.opendev.org/c/openstack/nova/+/891036 | |
| 12:10:02 | opendevreview | Pranali Deore proposed openstack/nova master: DNM: Test glance new location api https://review.opendev.org/c/openstack/nova/+/891207 | |
| 13:36:55 | sean-k-mooney | hi can i get some eyes on https://review.opendev.org/c/openstack/devstack/+/890693 please. | |
| 13:37:06 | sean-k-mooney | oh that was for qa.. | |
| 13:40:48 | sean-k-mooney | tempest-integrated compute often seamm to get pretty close to its 2 hour timeout but there is also a lot of variance | |
| 13:40:54 | sean-k-mooney | https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-compute&project=openstack%2Fnova | |
| 14:10:13 | gibi | stephenfin: nice series, and now it is approved all the way up :) | |
| 14:10:40 | stephenfin | gibi: 👏 thank you! | |
| 14:10:40 | stephenfin | gibi: � thank you! | |
| 15:09:50 | bauzas | if people want to see my face live (after all, this is not like I see most of you everyday on gmeet) : https://www.youtube.com/live/thidlQGX29M?feature=shared | |
| 15:10:14 | bauzas | tl;dr: PTG recap OIL episode | |
| 16:15:07 | melwitt | gmann: if you get a moment https://review.opendev.org/c/openstack/nova/+/902809 | |
| 16:16:11 | melwitt | sean-k-mooney: I have noticed TIMED_OUT too for job running just a little over 2 hours | |
| 16:16:47 | sean-k-mooney | yep | |
| 16:16:51 | sean-k-mooney | i noticed that too | |
| 16:17:11 | sean-k-mooney | thats why i mentioned my zswap changes for devstack | |
| 16:17:15 | melwitt | I said "too" first, I saw your message in the scrollback :) | |
| 16:17:19 | sean-k-mooney | since that was ment to speed up the jobs | |
| 16:17:30 | melwitt | ah, cool, I'll check that out | |
| 16:17:43 | sean-k-mooney | it does not seam to be that common | |
| 16:57:16 | stephenfin | is it expected that adding a port will add the default security group to an instance if not already present? | |
| 16:58:06 | stephenfin | I'm guessing yes assuming a port is using the default security group | |
| 17:04:32 | bauzas | sean-k-mooney: I had zero time today to discuss with you about my mdev live-migration spec and the ping-pong game | |
| 17:09:42 | bauzas | long story short, the ping-pong game is made when the conductor calls check_dest() https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L8501-L8572 | |
| 17:12:26 | bauzas | if we really want to do a lockstep here by once, then we move the type check later in the post claim method, and we just return the current mdev and their types from the source in check_can_live_migrate_source() and we test and claim them in post_claim_migrate_data() | |
| 17:12:39 | bauzas | I'm OK with this change | |
| 17:30:30 | opendevreview | Merged openstack/nova master: Use SDK for add/remove instance info from node https://review.opendev.org/c/openstack/nova/+/659691 | |
| 17:33:32 | opendevreview | Merged openstack/nova master: Use SDK for getting network metadata from node https://review.opendev.org/c/openstack/nova/+/670213 | |
| 17:35:44 | opendevreview | Merged openstack/nova master: Use SDK for volume_target.* https://review.opendev.org/c/openstack/nova/+/867790 | |
| 17:35:50 | opendevreview | Merged openstack/nova master: Use SDK for node.set_provision_state https://review.opendev.org/c/openstack/nova/+/867791 | |
| 17:35:58 | opendevreview | Merged openstack/nova master: Use SDK for node.set_power_state https://review.opendev.org/c/openstack/nova/+/867792 | |
| 18:02:52 | sean-k-mooney | stephenfin: that is ill docuemtned buyt it depend on when the port is added | |
| 18:49:30 | gmann | melwitt: done. i opened it but somehow i forgot to review it yesterday | |
| 18:51:23 | melwitt | thanks gmann ! | |
| 21:08:31 | opendevreview | Merged openstack/nova master: Remove redundant setting of DEFAULT_IMAGE_NAME(_FILE) https://review.opendev.org/c/openstack/nova/+/902734 | |
| 21:08:38 | opendevreview | Merged openstack/nova master: Set UEC image vars for jobs not defined in Nova https://review.opendev.org/c/openstack/nova/+/902809 | |
| 22:28:46 | opendevreview | Merged openstack/nova master: Use SDK for node.vif* https://review.opendev.org/c/openstack/nova/+/867793 | |
| #openstack-nova - 2023-12-08 | |||
| 07:02:04 | opendevreview | Pranali Deore proposed openstack/nova master: Support glance's new location API https://review.opendev.org/c/openstack/nova/+/891036 | |