| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-03-10 | |||
| 11:59:24 | sean-k-mooney[m] | the running one in memory and an xml file on disk | |
| 11:59:44 | sean-k-mooney[m] | on ocation they get out of sync and cause issues in managing the vm | |
| 12:00:08 | sean-k-mooney[m] | so i would like to get rid of the file on disk and just use the in memory one which is the acurrate one | |
| 12:00:27 | noonedeadpunk | yeah, I do agree herem that makes sense | |
| 12:01:10 | noonedeadpunk | just sugested that it could be first step to raw qemu, as had to deal with such solution on some cloud platform | |
| 12:01:22 | sean-k-mooney[m] | if you have a patch or code to recreate the mdevs on boot using placement we would be happy to include it | |
| 12:02:00 | sean-k-mooney[m] | are you in favor of raw qemu or against? | |
| 12:02:08 | noonedeadpunk | I'm not at the moment as just found that out during incident, but will try to allocate some resources | |
| 12:02:14 | noonedeadpunk | I'm personally against :) | |
| 12:02:33 | sean-k-mooney[m] | ack just checkign libvirt does some useful stuff for us | |
| 12:02:42 | sean-k-mooney[m] | so we are not currently plannig to remove it | |
| 12:02:47 | noonedeadpunk | s/just suggested/just assumed/ | |
| 12:03:17 | sean-k-mooney[m] | we dont really want to have to manage cgroups directly for example | |
| 12:03:50 | noonedeadpunk | I guess it prings in selinux/apparmour rules as well | |
| 12:03:55 | noonedeadpunk | *brings | |
| 12:03:59 | sean-k-mooney[m] | yes | |
| 12:04:28 | sean-k-mooney[m] | it deals with selinux lables and a bunch of other things which we dont want to have to implement in nova | |
| 12:04:34 | noonedeadpunk | and some live migration features iirc | |
| 12:04:49 | sean-k-mooney[m] | yes libvirt does the cpu compatiblity check for us | |
| 12:05:13 | sean-k-mooney[m] | so removing the xml on disk simple. removign libvirt is basicaly a full rewrite of the dirver | |
| 12:05:26 | sean-k-mooney[m] | which is not something we have time for or really want to do | |
| 12:05:57 | noonedeadpunk | exeprience with raw qemu was quite painful | |
| 12:06:34 | sean-k-mooney[m] | the only real advantage to raw qemu would be if libvirt did not have support for feature x yet | |
| 12:06:55 | sean-k-mooney[m] | and at this point nova does not move fast enough for that to be an issue | |
| 12:07:10 | noonedeadpunk | just wanted to say the same:) | |
| 12:07:55 | noonedeadpunk | virtiofs great example of feature ppl love to get but nobody has capacity to help on with... | |
| 12:08:40 | sean-k-mooney[m] | actully that is on our radar | |
| 12:08:55 | sean-k-mooney[m] | it should hopefully get added in z | |
| 12:09:03 | sean-k-mooney[m] | for manilla shares | |
| 12:09:15 | sean-k-mooney[m] | but if you have other usecases for it let us know | |
| 12:10:03 | sean-k-mooney[m] | once we have the basic support in the libvirt driver we can extend it simply if cyborg or anything else wants to use it | |
| 12:10:30 | sean-k-mooney[m] | brb | |
| 12:12:41 | gibi | bauzas and others: I think we have a functional test instability https://paste.opendev.org/show/btHI7ErFfhKYFGdfoujl/ | |
| 12:12:50 | gibi | e.g.: https://6779a19d629122a2ad74-09e4be48fe62aca6e4b03d954e19defe.ssl.cf1.rackcdn.com/828570/8/check/nova-tox-functional-py38/99a1e4f/testr_results.html | |
| 12:13:33 | gibi | recently I saw this pretty frequently but It happened in the past too (I saw matches from mid february) | |
| 12:14:25 | gibi | Im filing a bug... | |
| 12:20:19 | gibi | https://bugs.launchpad.net/nova/+bug/1964472 | |
| 12:33:42 | gibi | ohh the failing testcase was added here https://review.opendev.org/c/openstack/nova/+/821840 | |
| 12:33:56 | gibi | this is one of the bug with rc1 tag, where the fix is still open | |
| 12:34:14 | gibi | so I guess the regression test added here is not stable and needs some tuning | |
| 12:35:24 | gibi | bauzas, sean-k-mooney[m] should we pull https://review.opendev.org/c/openstack/nova/+/815324/10 from the gate? | |
| 12:35:36 | gibi | or we expect a followup to stabilize the functional test in that? | |
| 12:41:22 | sean-k-mooney | is it unstable | |
| 12:41:44 | sean-k-mooney | oh the regerssion | |
| 12:42:24 | gibi | the test case is unstable | |
| 12:42:27 | sean-k-mooney | i see is it still unsable with the followup | |
| 12:43:07 | gibi | we only have data about it being unstable without the fix for the origina bug | |
| 12:43:14 | gibi | that bugfix is on the gate | |
| 12:43:32 | gibi | I tried to reproduce it locally but failed so far | |
| 12:43:40 | sean-k-mooney | In the last 7 days from 8 failed functional test run 6 was due to this in nova. But this happened as far back as 14th of February. | |
| 12:43:53 | sean-k-mooney | so the test has only merged yesterday | |
| 12:44:05 | sean-k-mooney | so those other failures would have been on the path itself | |
| 12:44:29 | sean-k-mooney | as in https://review.opendev.org/c/openstack/nova/+/821840/5 merged yesterday | |
| 12:44:43 | gibi | the test case merged on the 5th | |
| 12:44:56 | gibi | oh no | |
| 12:44:58 | gibi | it is 9th | |
| 12:45:01 | gibi | you are correct | |
| 12:45:02 | gibi | sorry | |
| 12:45:30 | gibi | and you are correct before yesterday it is only failed on 821840/5 | |
| 12:45:38 | gibi | after the merge it is failed on 3 other pathes | |
| 12:45:42 | gibi | see https://paste.opendev.org/show/btHI7ErFfhKYFGdfoujl/ | |
| 12:46:16 | sean-k-mooney | then ya we shoudl either revert or disable i guess | |
| 12:46:35 | opendevreview | Merged openstack/nova master: Add functional tests to reproduce bug #1960412 https://review.opendev.org/c/openstack/nova/+/830010 | |
| 12:47:39 | gibi | yeah, no point yanking the bugfix from the gate as that does not break anyithing, the patch that added the test case was already merged | |
| 12:48:07 | gibi | I can push a patch skipping the test case | |
| 12:52:32 | opendevreview | Balazs Gibizer proposed openstack/nova master: Skip TestRollbackWithHWOffloadedOVS.test_rollback_pre_live_migration https://review.opendev.org/c/openstack/nova/+/833079 | |
| 12:52:38 | gibi | bauzas, sean-k-mooney: ^^ | |
| 13:01:00 | sean-k-mooney | gibi: sorry was making coffee did you determin why its unstable by the way? i did not fully read your bug report | |
| 13:01:55 | gibi | sean-k-mooney: no, I had not time to actually look deeply into it. I has not been able to reproduce it locally yet. | |
| 13:02:03 | gibi | *was not able | |
| 13:03:23 | sean-k-mooney | ok its odd the test is pretty simple | |
| 13:03:42 | sean-k-mooney | i mean for a live migration test | |
| 13:05:13 | sean-k-mooney | gibi: did you mean to put that patch on top of master rather then the fix | |
| 13:05:31 | sean-k-mooney | actully i guess that makes sense too | |
| 13:07:20 | sean-k-mooney | if noone else reviews the skip by the end of the day ill fast approve to make sure its included in rc1 | |
| 13:07:46 | gibi | sean-k-mooney: thanks | |
| 13:08:12 | gibi | I have some calls this afternoon, so I'm not sure I will have time to figure out today why it fails | |
| 13:08:12 | sean-k-mooney | the followup failed grenade by the way | |
| 13:11:54 | bauzas | gibi: sean-k-mooney: sorry was afk, a tl;dr ? | |
| 13:12:18 | bauzas | gibi: I see your comment on https://review.opendev.org/c/openstack/nova/+/815324/10 | |
| 13:12:22 | gibi | bauzas: the functional test added here https://review.opendev.org/c/openstack/nova/+/821840 is unsatble | |
| 13:12:32 | gibi | so I pushed https://review.opendev.org/c/openstack/nova/+/833079 | |
| 13:12:43 | gibi | the we can figure out the stability issue | |
| 13:12:45 | gibi | *then | |
| 13:12:55 | bauzas | gibi: maybe we should revert the functional test then | |
| 13:13:17 | bauzas | gibi: hah | |
| 13:13:18 | gibi | bauzas: we can do that as well, revert the func test and the revert the fix | |
| 13:13:29 | bauzas | gibi: yeah I think it's better | |
| 13:13:34 | bauzas | gibi: can you revert it ? | |
| 13:13:39 | bauzas | erlon is not around | |
| 13:13:42 | gibi | I can | |
| 13:13:49 | bauzas | thanks | |
| 13:14:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Revert "Adds regression test for bug LP#1944619" https://review.opendev.org/c/openstack/nova/+/832902 | |
| 13:14:48 | gibi | bauzas, sean-k-mooney: ^^ revert then | |
| 13:15:48 | sean-k-mooney | ok that works for me too | |
| 13:16:10 | sean-k-mooney | although it would still be nice to fix this this cycle but i guess we can backport | |
| 13:16:30 | sean-k-mooney | i assuem we dont want to pull this into an RC2 right | |
| 13:16:39 | sean-k-mooney | since tis not really a regression form this cycle | |
| 13:17:09 | sean-k-mooney | its just a bug in the orginal sriov live migration feature | |
| 13:17:33 | sean-k-mooney | so it can wait and be backported as normla | |