Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-07-23
20:22:43 sean-k-mooney _colby: but they are based in france and done for today so they may be able to respond in the morning
20:28:37 _colby ok thanks @sean-k-mooney
21:23:11 opendevreview Merged openstack/nova master: conf: Clarify '[api] response_validation help' text https://review.opendev.org/c/openstack/nova/+/924501
23:39:59 opendevreview Merged openstack/nova master: Change force_format strategy to catch mismatches https://review.opendev.org/c/openstack/nova/+/924731
#openstack-nova - 2024-07-24
05:41:53 ykarel Hi after https://review.opendev.org/c/openstack/nova/+/924731 - Change force_format strategy to catch mismatches
05:42:07 ykarel many tests are failing involving snapshot/rescue, is this something known already?
05:43:56 ykarel may be now as don't see any recent bug for it, will report
05:57:28 ykarel reported https://bugs.launchpad.net/nova/+bug/2073944
06:11:06 ralonsoh sean-k-mooney, ^
06:11:18 ralonsoh bauzas, ^
06:11:33 ralonsoh upsss maybe you didn't see the message
06:11:39 ralonsoh https://bugs.launchpad.net/nova/+bug/2073944
06:11:46 ralonsoh since https://review.opendev.org/c/openstack/nova/+/924731
07:18:35 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: test downstream cross repo testing https://review.opendev.org/c/openstack/nova/+/924823
07:59:23 sean-k-mooney ralonsoh: is that on neutron ci
08:00:15 sean-k-mooney ykarel: ralonsoh: if i recall correctly neutron is using the uec images (ami format) and has not neessialy configure tempest correctly for that
08:00:36 ralonsoh let me check that
08:01:40 sean-k-mooney https://github.com/openstack/neutron/blob/master/zuul.d/tempest-multinode.yaml#L89-L90
08:02:49 ralonsoh sean-k-mooney, that change was to address https://bugs.launchpad.net/nova/+bug/1939108
08:02:51 sean-k-mooney tempest is by default configured ot expect qcow
08:02:53 ralonsoh https://review.opendev.org/c/openstack/neutron/+/916629
08:03:29 sean-k-mooney ralonsoh: right we have since determined that, using the uec image does not actully fix the probelm
08:03:44 sean-k-mooney it makes it less likely but we did it last cycle and the issue didnt go away for us
08:03:58 ralonsoh sean-k-mooney, ok, I'll revert the neutron patch then
08:05:10 sean-k-mooney reverting is one approch the other is to set the approrate disk/container formats in the tempest config
08:05:51 sean-k-mooney for 2024.1 we are also using thte uec images and im currently looking at regerting that to bring it inline with master
08:05:52 ralonsoh but if using uec images makes the issue lest likely to happen
08:06:05 ralonsoh what should I configure in tempest for that?
08:06:23 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/924746/2
08:06:26 sean-k-mooney i think that
08:06:35 sean-k-mooney but in general the uec image just change the timeign
08:06:57 sean-k-mooney which means if you dont land on a slow node you might get lucky
08:07:16 ralonsoh yeah, that's another issue
08:07:38 sean-k-mooney ralonsoh: so Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option.
08:07:50 sean-k-mooney is actully fixed by using cirros 0.6.2
08:08:44 sean-k-mooney that specific kernel panic in https://bugs.launchpad.net/nova/+bug/1939108 was a know kernel bug that was fixed in the cirros 0.6.x series
08:09:25 ralonsoh ok, so if this issue is solved in 0.6.2, I could also use qcow2 images too
08:09:27 ralonsoh right?
08:09:35 sean-k-mooney yep
08:09:40 ralonsoh that will fix the current problem
08:09:46 ralonsoh perfect then!
08:09:58 sean-k-mooney the other kernel panics im hoping have been fixed by https://review.opendev.org/c/openstack/devstack/+/924094
08:10:04 bauzas just saw _colby's question, I probably need more details
08:10:23 bauzas like the nvidia driver version they use
08:10:46 sean-k-mooney bauzas: ya i was hoping you recalled when nvidai split the driver in 2
08:10:56 bauzas was a long time ago
08:10:58 sean-k-mooney they proably didnt make that chagne for centos8
08:11:01 bauzas GRID16
08:11:18 sean-k-mooney but not that they are on centos 9 the preobaly pulled the latest dirvers
08:11:26 bauzas I can lookup the A40 support for GRID16
08:11:52 bauzas yeah
08:12:09 bauzas but GRID 16 and 17 transition around RHEL9
08:12:25 bauzas both support some RHEL9 versions, just not the same
08:12:36 bauzas GRID16 is also a LTE version
08:12:53 bauzas while GRID17 is only a production version, ie. with a 1-year term support
08:14:05 bauzas https://docs.nvidia.com/vgpu/16.0/product-support-matrix/index.html#abstract__red-hat-el-kvm
08:37:54 ykarel ralonsoh, sean-k-mooney but we still hit kernel panic with those 0.6.2 qcow images, i think i have never seen that issue with uec images
08:37:55 ykarel https://32b7a8914f85d2150a67-dfd2b77d4a65df68dc83e2cbebde05a8.ssl.cf1.rackcdn.com/924004/4/check/tempest-full-py3/861ecb0/testr_results.html
08:39:50 ykarel i mean never seen that issue with uec images in neutron jobs that we track
08:41:08 ralonsoh ykarel, we can also add https://review.opendev.org/c/openstack/nova/+/924746/2 in our jobs
08:41:55 ykarel ralonsoh, yeap +1 to that as we should also avoid those random kernel panics in our jobs
08:42:28 ralonsoh ykarel, perfect, I
08:42:34 ralonsoh I'll change the neutron patch
09:02:31 opendevreview Mark Goddard proposed openstack/nova master: ironic: Let Ironic handle deployment cleanup actions during destroy https://review.opendev.org/c/openstack/nova/+/883411
09:04:13 sean-k-mooney ykarel: we have see that issue with the uec images, when we swapped to them the kernel panices reduced but they never went away
09:05:16 sean-k-mooney ykarel: i will also poitn out that creating vms with uec images is not how 99% of people use openstack
09:06:06 sean-k-mooney its why nova in particaly cant just use uec images even if that was a 100% reliable solution which it is not.
09:38:53 frickler sean-k-mooney: iiuc the 2024.1 fix is blocked by https://review.opendev.org/c/openstack/nova/+/923831 now? so it could be rebase on top of it to get the tests to pass and reviewers could approve things in parallel?
09:41:05 sean-k-mooney its not blocked
09:41:11 sean-k-mooney frickler: well it is
09:41:16 sean-k-mooney but rebasing is not the answer
09:41:23 sean-k-mooney we can merge https://review.opendev.org/c/openstack/nova/+/923831
09:41:26 sean-k-mooney and then just rehceck
09:42:02 sean-k-mooney i woudl prefer to not have to respin the cherrypics for the sha but i can if needed
09:42:11 frickler which takes more time than to rebase now. so anyway, bauzas gibi could you look at ^^ please
09:43:09 sean-k-mooney frickler: i was about to ping bauzas, gibi is on pto today but i thinke we can proceed with this without there review
09:44:22 frickler yeah, sorry for being impatient, I was just hoping that with just a single commit per branch this whole thing would be done within hours and not take days again
09:45:07 sean-k-mooney frickler: well that why i started backporting https://review.opendev.org/c/openstack/nova/+/923831 2 weeks ago
09:45:34 sean-k-mooney i had hoped ot have that in place before this hit
09:45:57 sean-k-mooney the prvious cve was also impacted by theuse of the uec images
09:46:30 sean-k-mooney but that ended up bing partly mitigated
10:01:07 ykarel sean-k-mooney, ack atleast in our jobs we have not reproduced that issue with uec image
10:06:44 frickler sean-k-mooney: I noticed that I do have stable core powers, so added +2, but leaving the approval for you
10:13:25 sean-k-mooney ill give bauzas an hour or so and move forward if he or stephenfin dont have time to review https://review.opendev.org/c/openstack/nova/+/923831 by noon(utc+1)
10:16:08 bauzas here now
10:17:10 bauzas https://review.opendev.org/c/openstack/nova/+/923831 +Wd
10:17:22 sean-k-mooney bauzas: thanks
10:49:08 opendevreview Michael Still proposed openstack/nova master: libvirt: Add config option to require secure SPICE. https://review.opendev.org/c/openstack/nova/+/922544
10:49:09 opendevreview Michael Still proposed openstack/nova master: libvirt: allow concurrent access to SPICE consoles. https://review.opendev.org/c/openstack/nova/+/922546
10:49:09 opendevreview Michael Still proposed openstack/nova master: WIP: libvirt: Add guest devices to support SPICE USB. https://review.opendev.org/c/openstack/nova/+/922547
10:49:10 opendevreview Michael Still proposed openstack/nova master: WIP: libvirt: Optionally enable SPICE debug logging. https://review.opendev.org/c/openstack/nova/+/922548
10:49:10 opendevreview Michael Still proposed openstack/nova master: WIP: libvirt: Optionally support sound when using SPICE. https://review.opendev.org/c/openstack/nova/+/922549
10:49:11 opendevreview Michael Still proposed openstack/nova master: libvirt: allow direct SPICE connections to qemu https://review.opendev.org/c/openstack/nova/+/924844
10:49:44 mikal sean-k-mooney: https://review.opendev.org/c/openstack/nova/+/924844/1 is my first swing at the reworking of the Nova APIs for the SPICE consoles.
10:51:12 sean-k-mooney mikal: stephenfin is working on a paralle series to add openapi schema validation that might collide with that by the way but we can cross that bridge wehn we get there
10:51:55 mikal sean-k-mooney: yeah, fair enough. Imma just gonna keep rebasing and uploading for now.
10:53:38 sean-k-mooney mikal: currently my focuse is unfortuentlly on backporting a cve fix upstream and downstream so while i skimmed that i proably wont be able to start reviewing properly unitl next week
10:53:41 mikal sean-k-mooney: assuming that the bottom three of those pass CI (and the first two already have before), I'll move onto reimplementing the sound model as an extra spec next. There is also a patch in that stack which adds USB redirect devices, I am undecided if I should make the number of devices configurable in extra specs as well.
10:53:51 sean-k-mooney hopefullly we will get most of the cve patches landed today
10:54:39 mikal sean-k-mooney: yeah fair enough, I'm just a bit concerned about the code complete deadline, so I want to get them out there as quick as I can.

Earlier   Later