| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-12-06 | |||
| 15:54:40 | gibi | OK, so our default jobs does not have barbican, but we have a separate job with barbican | |
| 15:55:26 | bauzas | correct, but we run barbican jobs on our check pipeline | |
| 15:55:45 | bauzas | https://zuul.opendev.org/t/openstack/builds?job_name=barbican-tempest-plugin-simple-crypto&skip=0 | |
| 15:56:01 | bauzas | (n-v) | |
| 15:56:23 | bauzas | but I'm not able to see a single job definition setting vtpm | |
| 16:10:08 | gibi | s/doc/job/ | |
| 16:10:08 | gibi | I will try to put together a simple tempest test that boot a VM with vtpm, and put it behind a tempest feature flag. Then we can try to enable that flag in our barbican doc | |
| 16:32:52 | bauzas | sean-k-mooney: I now remember why I needed to do the claim in post_claim_migrate_data() and not in one lockstep by check_can_live_migrate_destination() | |
| 16:33:28 | bauzas | tl;dr: when we call check_can_live_migrate_destination(), we don't know neither the used mdev from the guest nor its type | |
| 16:33:38 | bauzas | we just have an instance object | |
| 16:34:06 | bauzas | so we need this kind of tic-tac-toe between dest>src>dest | |
| 16:34:39 | bauzas | call it TCP handshake :p | |
| 16:43:42 | artom | gibi, we have vTPM tests in whitebox | |
| 16:45:10 | artom | gibi, example run: https://70d77d1b8efad238ddf5-24c1b4d0bad6f112f42872ab6905686e.ssl.cf5.rackcdn.com/824772/32/check/whitebox-devstack-multinode/2a8b9df/testr_results.html | |
| 16:45:15 | gibi | nice | |
| 16:45:15 | artom | (expand the last line) | |
| 16:45:16 | gibi | thanks | |
| 16:45:45 | gibi | now I just need to figure out if we have whitebox available in our tempest container used in openstack-k8s-operators testing... | |
| 16:46:24 | artom | Out of the box? I suspect not, but hopefully they have the framework in place to add arbitrary tempest plugins | |
| 16:56:00 | gibi | artom: we will see shortly https://github.com/openstack-k8s-operators/openstack-operator/pull/589 :D | |
| 17:07:39 | sean-k-mooney | bauzas: i see. thats less then ideal but if you can docuemnt it in the spec i guess i can live with that for now | |
| 17:07:51 | sean-k-mooney | im not really sure how to adress that otherwize | |
| 17:08:16 | sean-k-mooney | im not sure we reallly need to do the mdev type check at the conducor level | |
| 17:08:27 | sean-k-mooney | but if you think it helps then ok | |
| 17:08:43 | bauzas | sean-k-mooney: I can write something indeed explaining more the ping-pong | |
| 17:08:53 | sean-k-mooney | thanks | |
| 17:09:23 | bauzas | sean-k-mooney: we could just pass the source mdevs to the dest and then the dest would claim a type | |
| 17:09:31 | bauzas | so basically removing the first item | |
| 17:09:48 | bauzas | dest would claim a *mdev* | |
| 17:10:01 | sean-k-mooney | right that why i was orginlly suggetion jsut doing it in pre_live_migrate | |
| 17:10:17 | bauzas | pre_live_migration is after the claim | |
| 17:10:54 | bauzas | so I think we need to reserve the mdev in the conductor call to compute | |
| 17:11:06 | sean-k-mooney | but we dont have the mdev types at that point right | |
| 17:11:08 | sean-k-mooney | form the souce node | |
| 17:11:16 | sean-k-mooney | well i guess | |
| 17:11:26 | sean-k-mooney | we can claim them | |
| 17:11:34 | sean-k-mooney | but we wont detect if the type are diffent | |
| 17:11:39 | sean-k-mooney | until after we call the souce node | |
| 17:11:45 | sean-k-mooney | so we will claim them on the test | |
| 17:11:53 | sean-k-mooney | then apass the info to the soruce with the types | |
| 17:12:00 | sean-k-mooney | and it can check if they change and abort | |
| 17:12:10 | bauzas | hum | |
| 17:13:26 | bauzas | actually, this is not possible, damn | |
| 17:13:37 | bauzas | we call the destination first, then we call the sourcer | |
| 17:13:43 | bauzas | and then we claim | |
| 17:17:02 | sean-k-mooney | right but the call to the dest coudl do the claim | |
| 17:17:20 | sean-k-mooney | then pass to the souce to confirm if the types match | |
| 17:17:22 | sean-k-mooney | actully no | |
| 17:17:30 | sean-k-mooney | we are stuck with the 3 stpes i think | |
| 17:17:46 | sean-k-mooney | lets pick up this converstaion tomorrow | |
| 17:32:03 | gibi | artom: we have a chance https://github.com/openstack-k8s-operators/tcib/blob/4aef81881de941f0251aa848515243f666b18672/container-images/tcib/base/os/tempest/tempest-extras/tempest-extras.yaml#L4 | |
| 17:32:49 | artom | Oh wow | |
| 17:32:52 | artom | jparker++ | |
| 17:32:58 | artom | I'm assuming he's to blame :D | |
| 17:33:25 | artom | (Not literal `git blame`) | |
| 17:34:58 | gibi | almost :) https://github.com/openstack-k8s-operators/tcib/commit/199178028bba73548e21df5171a620df7981b451 | |
| 17:40:39 | artom | James was pulling the strings | |
| 17:41:33 | gibi | indeed, he is still pulling it https://github.com/openstack-k8s-operators/tcib/pull/110 | |
| 17:42:14 | artom | He's pull requesting them :D | |
| 17:46:19 | gibi | exactly :D | |
| 19:04:25 | opendevreview | melanie witt proposed openstack/nova master: Set UEC image vars for jobs not defined in Nova https://review.opendev.org/c/openstack/nova/+/902809 | |
| 19:15:58 | sean-k-mooney | melwitt: oh right i guess we should do that for those too | |
| 19:17:43 | sean-k-mooney | ill try and take a look at ^ tomorrow | |
| 19:18:10 | sean-k-mooney | one thing we might want to consider is actully makeing this change in the base devstack job | |
| 19:18:34 | sean-k-mooney | nova is really the only project that likely cares about testing the full disk image code path | |
| 19:18:58 | sean-k-mooney | so defaulting to the uec image might be worth considering in the tempest/devstack base job | |
| 19:19:23 | sean-k-mooney | gmann: ^ we should proably conisder that after we have run nova on them for a few weeks | |
| 19:20:41 | gmann | sean-k-mooney: melwitt yeah, i am thinking on that, if not default at least we can do that for most of the general integrated jobs in tempest and keep a few of them for example tempest-full-py3 on full image | |
| 19:21:00 | gmann | yes, let's observe it in nova for a few weeks first | |
| 19:34:16 | melwitt | sean-k-mooney, gmann: I realized the above (jobs not defined in nova) bc I saw tempest-integrated-compute fail on the follow up patch and it was the guest kernel panic. at first I was confused how/why could that be if we've got the UEC image but then I realized it is _not_ using the image bc I had only set the vars for the jobs defined by us in our .zuul.yaml | |
| 19:35:06 | sean-k-mooney | yep i forgot about the other jobs too | |
| 19:35:34 | melwitt | sean-k-mooney: semi related to that, I had been meaning to ask you, why does using the UEC image work to avoid guest kernel panics? | |
| 19:36:15 | melwitt | just a curiosity question | |
| 19:40:09 | opendevreview | Merged openstack/nova-specs master: Add spec for PCI Groups https://review.opendev.org/c/openstack/nova-specs/+/899719 | |
| 19:41:03 | sean-k-mooney | melwitt: i dont actully know. i suspect it takign a slightly diffent path in both qemu and in the init shell script | |
| 19:41:30 | sean-k-mooney | melwitt: i know neutron had previosuly added apci=off or something like that | |
| 19:41:41 | sean-k-mooney | to disable the apic timer | |
| 19:41:47 | melwitt | ah ok | |
| 19:41:54 | sean-k-mooney | but they dont actully do that any more | |
| 19:42:07 | sean-k-mooney | basiclly one of the panics suggeted doign that and when you use the split image | |
| 19:42:24 | sean-k-mooney | nova supprot passing kernel command line args to qemu | |
| 19:42:52 | sean-k-mooney | so orginally they explored this just to follow the mesage output which said A X to the command line to turn this off | |
| 19:45:03 | sean-k-mooney | melwitt: https://bugs.launchpad.net/openstack-gate/+bug/1713162 | |
| 19:45:15 | sean-k-mooney | Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. | |
| 19:45:30 | sean-k-mooney | so ya they orgianlly tied setting noapic on the kernel commadn line | |
| 19:45:36 | melwitt | ohh I see | |
| 19:46:08 | sean-k-mooney | i "fixed" this a diffent way in devstack | |
| 19:46:39 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#workarounds.libvirt_disable_apic | |
| 19:47:53 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/766043 and https://github.com/openstack/devstack/commit/1e86a25cc28e34d7f73a4c6ccbbc3fc667598d50 | |
| 19:48:11 | sean-k-mooney | lee actully enabeld it in devstack | |
| 19:48:15 | melwitt | oh, so using the split image seems to be a way to alleviate the problem without disabling the apic timer? | |
| 19:48:41 | sean-k-mooney | so i tought at one point we were addign the noapic thign somewhere | |
| 19:49:09 | sean-k-mooney | i just dont know where that is exactly | |
| 19:49:54 | sean-k-mooney | melwitt: but ya apprently the split imag eis enouch i just cant prove that | |
| 19:50:00 | melwitt | well that's confusing. it's already been set to disabled but it was still panicking? 🤔 | |
| 19:50:00 | melwitt | well that's confusing. it's already been set to disabled but it was still panicking? 🤔 | |
| 19:50:26 | sean-k-mooney | so there are two tings | |
| 19:50:30 | sean-k-mooney | the apic and ioapic | |
| 19:50:44 | sean-k-mooney | we are disabling the apic | |
| 19:50:48 | melwitt | oh | |