| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-11-29 | |||
| 19:47:57 | sean-k-mooney | we woudl have to change form the full disk image to the one with the splict out kernel and initram | |
| 19:48:10 | sean-k-mooney | i know that neutron use that in one of there jobs | |
| 19:48:22 | sean-k-mooney | to avoid some kernel panics. | |
| 19:48:29 | melwitt | oh huh | |
| 19:48:29 | sean-k-mooney | specifical this on ewith the apic timer | |
| 19:48:39 | melwitt | interesting | |
| 19:49:09 | sean-k-mooney | any unrecognised parmater on the kernel command line is set as an ENV var in the init procsss by the kernel | |
| 19:49:32 | sean-k-mooney | incase you didnt know that so if debug is based on an env var you sould jsut add that env var to the kernel command line | |
| 19:49:54 | melwitt | so theoretically the num_pcie_ports might fix the nova-next panics and the split disk image might fix the rest ... seems it would be too good to be true | |
| 19:50:17 | melwitt | ok yeah I didn't know that | |
| 19:50:22 | sean-k-mooney | well im 99% shure that the split disk will fix the apic one | |
| 19:50:52 | sean-k-mooney | the only down side to that is that we loose coverage of testign full disk images if we use that every where | |
| 19:51:10 | sean-k-mooney | ill check quickly if i can see where they enable it in there job | |
| 19:52:42 | melwitt | k. I wonder if we could get away with using the split image everywhere except nova-next | |
| 19:53:05 | opendevreview | alisafari proposed openstack/nova master: Fix traits to cpu flags mapping https://review.opendev.org/c/openstack/nova/+/902183 | |
| 19:53:15 | sean-k-mooney | proably | |
| 19:54:21 | melwitt | I feel like I'm seeing these panics all the time. not sure what changed bc they used to be so rare in the past. maybe OOM like you said or maybe if in the past everything was using the split image, that I don't know | |
| 19:56:02 | sean-k-mooney | no we were defienlty usign the whole disk image for several reelase | |
| 19:56:17 | sean-k-mooney | i dont know if we ever use d the split imate by default | |
| 19:56:18 | melwitt | ack | |
| 19:56:20 | sean-k-mooney | we may have | |
| 20:10:09 | sean-k-mooney | ah found it | |
| 20:10:10 | sean-k-mooney | https://github.com/openstack/neutron/commit/e04bd8fbdfa56320d16870b1f294b2cb62b8a828 | |
| 20:11:16 | sean-k-mooney | so if we add | |
| 20:11:20 | sean-k-mooney | CIRROS_VERSION: 0.6.2 | |
| 20:11:22 | sean-k-mooney | DEFAULT_IMAGE_NAME: cirros-0.6.2-x86_64-uec | |
| 20:11:24 | sean-k-mooney | DEFAULT_IMAGE_FILE_NAME: cirros-0.6.2-x86_64-uec.tar.gz | |
| 20:12:29 | sean-k-mooney | that shoud fix it | |
| 20:12:40 | sean-k-mooney | the other way to do that is move the job to use nested virt | |
| 20:12:43 | sean-k-mooney | https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/821067 | |
| 20:13:49 | sean-k-mooney | melwitt: what we coudl do is swap to the uec image for our default jobs | |
| 20:14:07 | sean-k-mooney | and move nova-next to nested virt with the whole disk image | |
| 20:14:59 | melwitt | sounds worth a try | |
| 20:15:02 | sean-k-mooney | although we woudl want ot use the nested virt jamy lable in our node set | |
| 20:15:04 | sean-k-mooney | https://opendev.org/openstack/project-config/src/branch/master/nodepool/nl04.opendev.org.yaml#L41-L42 | |
| 20:17:24 | melwitt | ok | |
| 20:18:55 | sean-k-mooney | do you know how ot make those changes. im going to finish for today and get dinner but i can update the jobs tomorow if you havent and we can see if it works | |
| 20:19:34 | melwitt | I think I can do it given the examples you linked. but yeah sounds good, if I do it wrong feel free to redo it or whatever tomorrow | |
| 20:21:12 | sean-k-mooney | we in theory have nested virt capablity for 2 vexhost clouds and ovh so i dont really feel as bad about using in in nova next as i did whne it was only one provider | |
| 20:22:53 | melwitt | ah ok | |
| #openstack-nova - 2023-11-30 | |||
| 03:01:08 | opendevreview | melanie witt proposed openstack/nova master: Use split kernel/initramfs Cirros UEC image by default in jobs https://review.opendev.org/c/openstack/nova/+/902217 | |
| 06:40:30 | melwitt | sean-k-mooney: there was an attempt ... https://review.opendev.org/c/openstack/nova/+/902217 but it's super not working 😆 | |
| 06:40:30 | melwitt | sean-k-mooney: there was an attempt ... https://review.opendev.org/c/openstack/nova/+/902217 but it's super not working 😆 | |
| 07:54:29 | opendevreview | alisafari proposed openstack/nova master: Fix traits to cpu flags mapping https://review.opendev.org/c/openstack/nova/+/902183 | |
| 09:26:14 | dvo-plv | gibi, dansmith Hello, gentlemens? Maybe you will have a time to review spec and code | |
| 09:26:18 | dvo-plv | https://review.opendev.org/c/openstack/nova-specs/+/895924 | |
| 09:26:22 | dvo-plv | https://review.opendev.org/c/openstack/nova/+/876075 | |
| 09:34:12 | gibi | bauzas: https://review.opendev.org/c/openstack/nova/+/901656 merged so I bobcat point release is good to go for me :) | |
| 09:34:26 | bauzas | cool | |
| 09:34:37 | bauzas | elodilles: can you please update the bobcat release patch then , | |
| 09:36:43 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.1: Do not manage CPU0's state https://review.opendev.org/c/openstack/nova/+/901567 | |
| 09:36:56 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.1: Allow enabling cpu_power_management with 0 dedicated CPUs https://review.opendev.org/c/openstack/nova/+/901660 | |
| 09:37:24 | gibi | elodilles: fixed your comment in https://review.opendev.org/c/openstack/nova/+/901567 | |
| 09:50:44 | dvo-plv | gibi, thanks. Should I do something else to get +1 workflow for patch ? | |
| 09:50:47 | frickler | melwitt: yes, you need to add more things to your nodeset definition, let me update | |
| 09:51:59 | gibi | dvo-plv: I'm waiting for the spec to land | |
| 09:52:03 | opendevreview | Merged openstack/nova-specs master: Re-propose using VirtIO PackedRing Configuration support for 2024.1 https://review.opendev.org/c/openstack/nova-specs/+/895924 | |
| 09:52:30 | dvo-plv | I see | |
| 10:17:44 | elodilles | bauzas: done: https://review.opendev.org/c/openstack/releases/+/899602 | |
| 10:17:55 | elodilles | gibi: ACK, will review it again then :) | |
| 10:19:29 | opendevreview | Dr. Jens Harbott proposed openstack/nova master: Use split kernel/initramfs Cirros UEC image by default in jobs https://review.opendev.org/c/openstack/nova/+/902217 | |
| 10:20:22 | frickler | sean-k-mooney: melwitt: ^^ fwiw I would rather not depend on the nested virt nodes, maybe just changing the image type would be good enough already? | |
| 10:22:38 | bauzas | elodilles: -1 for https://review.opendev.org/c/openstack/releases/+/899602 | |
| 10:22:42 | bauzas | looks you squashed two things | |
| 10:34:42 | opendevreview | alisafari proposed openstack/nova master: Fix traits to cpu flags mapping https://review.opendev.org/c/openstack/nova/+/902183 | |
| 11:08:01 | sean-k-mooney[m] | frickler: the concern is we dont want to change the image type everywhere we want to have at least one full disk image | |
| 11:08:35 | sean-k-mooney[m] | from a nova perspective its testing two very slightly different code paths | |
| 11:09:28 | sean-k-mooney[m] | so we would only be enabling nested virt on nova-next which would be the only job to test the full disk image that we are using today | |
| 11:12:46 | sean-k-mooney[m] | frickler: if we can have the ci stable without nested virt, either with tweaks to the image or the flavor (adding swap for exampel) we could do that instead for that job or use a diffent full disk image in that job | |
| 11:35:20 | frickler | all jobs should have swap already, or are you talking about swap for the cirros instance? | |
| 11:40:11 | sean-k-mooney[m] | yep swap for the cirros image. i dont think it reasonable to keep bumping the ram we keep allocating to cirros | |
| 11:40:20 | sean-k-mooney[m] | espically if its a very minor failure | |
| 11:40:42 | sean-k-mooney[m] | but we can add a small amount of swap to the flavor say 64 mb | |
| 11:41:11 | sean-k-mooney[m] | cirroz has zswap too so a small amount will go a long way | |
| 11:46:00 | frickler | hmm. will it activate that while still in initramfs phase? but I guess it can't hurt trying it out. I'll make a devstack patch | |
| 11:57:26 | elodilles | bauzas: oh, uh, sry :S i've updated the patch now | |
| 11:59:01 | elodilles | bauzas: btw, i hesitated a bit about the version bump, whether the last patch worth a MINOR bump (28.1.0), but I think we are still good with PATCH bump (28.0.1) | |
| 12:12:04 | opendevreview | Merged openstack/nova master: Packed virtqueue support was added. https://review.opendev.org/c/openstack/nova/+/876075 | |
| 13:00:31 | opendevreview | alisafari proposed openstack/nova master: Fix traits to cpu flags mapping https://review.opendev.org/c/openstack/nova/+/902183 | |
| 13:07:27 | sean-k-mooney | frickler: looking at the console output it at least loads zswap, wether swapon has been done or not i have no idea. it could be added to the init script i guess if its not already being done but ya you might be right that it will be too early for swap to have any impact | |
| 13:22:18 | opendevreview | alisafari proposed openstack/nova master: Fix traits to cpu flags mapping https://review.opendev.org/c/openstack/nova/+/902183 | |
| 13:25:52 | alisafari | sean-k-mooney: Would you possibly recheck this? https://review.opendev.org/c/openstack/nova/+/902183 | |
| 16:33:56 | dvo-plv | sean-k-mooney, gibi If you are aroung and have some time, glance patch, last one from the packed ring story already pass zuul | |
| 16:33:57 | dvo-plv | https://review.opendev.org/c/openstack/glance/+/882133 | |
| 16:36:33 | opendevreview | Sylvain Bauza proposed openstack/nova master: vgpu: Allow device_addresses to not be set https://review.opendev.org/c/openstack/nova/+/902084 | |
| 16:37:52 | opendevreview | Sylvain Bauza proposed openstack/nova master: vgpu: Allow device_addresses to not be set https://review.opendev.org/c/openstack/nova/+/902084 | |
| 16:53:39 | bauzas | JayF: heh, any news about ironic-shards for Caracal timeframe ? I see john's not connectedc | |
| 16:54:50 | JayF | bauzas: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/EUP3AITYONBPAMGZHXWT7TXLF7DYKC4Y/ we basically plan on doing it, and trying to fix some bugs in ironic/nova driver | |
| 16:55:12 | JayF | bauzas: I personally will be doing some of that work, but have been extremely busy; essentially my entire december for upstream is pointed at this :) | |
| 16:56:42 | bauzas | ++:) | |
| 16:56:44 | bauzas | awesome ! | |
| 16:56:55 | JayF | https://review.opendev.org/c/openstack/nova/+/900831 is up to resolve the bug mentioned there already, has no reviews, I was not going to recheck-spam to get it V+1 without any review feedback though; failures are not-ironic-related | |
| 17:04:25 | bauzas | JayF: feel free to add your patches up in https://etherpad.opendev.org/p/nova-caracal-status#L47 I'm confused about the exact paperwork we may need for this release | |
| 17:05:01 | bauzas | I'll sort out the right section to put those patches, whether those are bugfixes or features | |
| 17:05:35 | bauzas | but I want to be sure I'm able to track that effort for this release, and that etherpad is basically my braindump | |
| 17:05:40 | JayF | It's in there, that one patch is a bugfix for sure | |
| 17:05:50 | JayF | put it in proposed at the bottom | |
| 17:05:52 | bauzas | ++ | |
| 17:06:16 | bauzas | I'll then move it to approved bugfixes once I just check that the patch is ready for reviews | |
| 18:36:08 | opendevreview | melanie witt proposed openstack/nova master: Use split kernel/initramfs Cirros UEC image by default in jobs https://review.opendev.org/c/openstack/nova/+/902217 | |