| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-28 | |||
| 13:51:02 | stephenfin | mriedem: We don't need to post to the mailing list when we add a parameter to the virt driver interface like that, do we? | |
| 13:51:12 | stephenfin | (the 'power_on' parameter) | |
| 13:51:52 | kashyap | aspiers: I completely agree on the confusion there, libvirt upstream definitely needs to address this. | |
| 13:52:13 | aspiers | kashyap: thanks for the moral support ;-) | |
| 13:52:34 | aspiers | kashyap: I've just pinged Jim Fehlig about it too | |
| 13:53:04 | mriedem | stephenfin: "need" to? no. but i can. | |
| 13:53:13 | artom | stephenfin, depends on who you ask, but it won't hurt | |
| 13:53:21 | kashyap | aspiers: Yeah, it is much needed frustration. As you can see, if we're duking around this damn schema so much, it's clearly not us :D | |
| 13:53:35 | kashyap | s/much needed/understandable/ | |
| 13:53:48 | stephenfin | mriedem: Yeah, I think it would be helpful at least, if you have time | |
| 13:56:06 | mriedem | stephenfin: done | |
| 13:56:11 | stephenfin | ta | |
| 13:58:20 | aspiers | kashyap: Actually this is not just a libvirt issue. The confusion is significantly compounded by placing 'type' and 'readonly' under os.loader.features. | |
| 13:58:43 | aspiers | kashyap: It should be os.loader.type and os.loader.readonly instead. | |
| 13:58:54 | kashyap | aspiers: In the Nova config class? | |
| 13:58:58 | aspiers | kashyap: yes | |
| 13:59:07 | aspiers | 'type' is not a feature | |
| 14:00:10 | aspiers | kashyap: Imagine future code like: if 'rom' in os.loader.features['type']: ... | |
| 14:00:20 | aspiers | kashyap: vs. if os.loader.type == 'rom': | |
| 14:00:46 | aspiers | or: if 'rom' in os.loader.types: ... | |
| 14:01:07 | aspiers | The latter two are instantly understandable; the former is a #wtf moment | |
| 14:02:45 | kashyap | aspiers: Nod. Okay, first step is to untangle this by moving type/readonly under os.loader, rather than os.loader.features | |
| 14:02:47 | aspiers | Public safety warning: I'm probably using overly antagonistic language right now, due to the mood which the ongoing clusterfuck in British politics has just put me into. Sorry in advance for any offense caused. | |
| 14:03:01 | aspiers | kashyap: Yes that would be a very good step I think | |
| 14:03:38 | aspiers | kashyap: and then if multiple types are allowed, we can ignore <type> in the libvirt schema and use os.loader.types (plural) instead | |
| 14:04:03 | kashyap | aspiers: Okay, Michal confirms - rom|pflash _are_ mutually exclusive. (As we guessed) | |
| 14:04:11 | aspiers | Ahah :-) | |
| 14:04:16 | aspiers | So the example XML is wrong | |
| 14:04:27 | aspiers | and hence our test fixtures too | |
| 14:05:41 | aspiers | kashyap: What about 'readonly' and 'secure'? | |
| 14:06:19 | kashyap | mprivozn: Yeah, you can't have secure "yes" and "no" at the same time -- that'll be "Schrödinger's XML"... | |
| 14:06:26 | kashyap | aspiers: They are also mutually exclusive. | |
| 14:06:47 | kashyap | aspiers: Okay, so on the example, this is what Michal says on _why_ both of them are listed: | |
| 14:06:48 | aspiers | kashyap: mprivozn isn't here ;-) wrong window? | |
| 14:07:20 | kashyap | aspiers: Here: | |
| 14:07:21 | kashyap | 16:04 < kashyap> mprivozn: Then, I'll submit a docs change to show just _one_ value under type|readonly|secure: https://libvirt.org/formatdomaincaps.html#elementsOSBIOS | |
| 14:07:24 | kashyap | 16:05 < mprivozn> kashyap: please don't | |
| 14:07:27 | kashyap | 16:05 < kashyap> (Because accepted values are shown in the below text) | |
| 14:07:29 | kashyap | 16:05 < kashyap> mprivozn: Hmm, why not? You wanted them there to show what are the possible values? | |
| 14:07:32 | kashyap | 16:05 < mprivozn> kashyap: the idea for domaincaps is to enumerate all accepted values; listing just one variant goes against that | |
| 14:07:35 | kashyap | 16:06 < mprivozn> kashyap: forget OSBIOS for a second and take a look at some other examples, diskDevice for instance | |
| 14:08:56 | kashyap | aspiers: So following the above example, diskDevice lists 4 different values (disk, cdrom, floppy, lun). | |
| 14:09:09 | aspiers | OK, so all the docs are wrong not just this? | |
| 14:09:15 | kashyap | aspiers: I.e. it can be _one_ of those values. | |
| 14:09:37 | aspiers | I'm sorry, but docs giving example XML which doesn't even validate against the schema (notional or otherwise) does not make sense | |
| 14:09:41 | kashyap | aspiers: No, the dev says docs are not wrong, and they intentionally enumerated as such | |
| 14:09:57 | aspiers | The allowed values can be listed *below* the example XML, or as comments in it | |
| 14:09:59 | kashyap | aspiers: See this point above: "the idea for domaincaps is to enumerate all accepted values; listing just one variant goes against that" | |
| 14:10:27 | aspiers | by "domaincaps" he means https://libvirt.org/formatdomaincaps.html ? | |
| 14:10:31 | aspiers | or something else? | |
| 14:10:41 | kashyap | aspiers: Yes, indeed | |
| 14:10:43 | aspiers | should we move this to #libvirt? | |
| 14:10:53 | kashyap | #virt, on OFTC, yes | |
| 14:10:58 | aspiers | oh OK | |
| 14:11:16 | sean-k-mooney | kashyap: isnt #virt internal? | |
| 14:11:31 | kashyap | sean-k-mooney: No. #virt is the community channel for libvirt on OFTC | |
| 14:11:42 | aspiers | weird, why not #libvirt? | |
| 14:11:48 | kashyap | (I mean, yes, too. But external exists, too) | |
| 14:12:17 | kashyap | aspiers: Because it includes adjacent projects like virt-manager, virt-tools, libosinfo, etc, in addition to libvirt | |
| 14:22:28 | mriedem | stephenfin: replied on https://review.opendev.org/#/c/637605/37 | |
| 14:22:57 | mriedem | and up, thanks for hitting those | |
| 14:23:29 | dansmith | mriedem: you wanna look at artom's latest changes on this per our discussion yesterday and opine on whether or not you can tolerate the hackery? https://review.opendev.org/#/c/635669/45 | |
| 14:24:14 | stephenfin | Ah, I meant would it make sense to add a 'should_raise' parameter to the 'delete' function (i.e. make it optional). It was a nit though | |
| 14:24:26 | stephenfin | should_raise or something along those lines | |
| 14:26:00 | mriedem | dansmith: sure | |
| 14:29:05 | dansmith | artom: left some clarifying questions in there, btw | |
| 14:32:53 | dansmith | artom: have you looked at why that DNM tempest test failed the live migration job? | |
| 14:33:02 | dansmith | looks like during setup, so maybe just something in the script | |
| 14:33:59 | dansmith | er, no | |
| 14:40:17 | mriedem | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_87/678887/6/check/nova-live-migration/d3d4357/ara-report/result/092050bb-201e-4657-ac3f-16aebd4578ff/ failed but not sure why yet | |
| 14:40:44 | mriedem | oh the job timed out | |
| 14:40:45 | mriedem | /home/zuul/workspace/devstack-gate/functions.sh: line 1018: 5091 Killed timeout -s 9 ${REMAINING_TIME}m bash -c "source $WORKSPACE/devstack-gate/functions.sh && $cmd" | |
| 14:41:14 | dansmith | yeah, but I figure it might be because it was polling for that to go active or something | |
| 14:43:36 | dansmith | looks like it started building an instance right around the on the subnode | |
| 14:44:27 | mriedem | the funny thing is, | |
| 14:44:34 | mriedem | it doesn't even seem to run the live migration stuff | |
| 14:44:41 | mriedem | it goes right to the test_evacuate.sh script | |
| 14:45:07 | dansmith | yeah | |
| 14:45:43 | dansmith | I didn't look to see the ordering | |
| 14:46:10 | mriedem | 2019-08-28 04:37:56.230 | 1. test with all local storage (use default for volumes) 2019-08-28 04:37:56.232 | NOTE: test_volume_backed_live_migration is skipped due to https://bugs.launchpad.net/nova/+bug/1524898 2019-08-28 04:37:57.817 | Missing value auth-url required for auth plugin password 2019-08-28 04:37:57.909 | + environment:post_test_hook:L1: /opt/stack/new/nova/gate/test_evacuate.sh | |
| 14:46:11 | openstack | Launchpad bug 1524898 in OpenStack Compute (nova) "Volume based live migration aborted unexpectedly" [High,In progress] - Assigned to Lee Yarwood (lyarwood) | |
| 14:46:16 | mriedem | maybe the flavor creation is blowing up? | |
| 14:46:52 | dansmith | ah | |
| 14:47:37 | mriedem | we're losing the tracing too | |
| 14:47:38 | dansmith | I still wanna know why this instance didn't build in case it is related to tweaking the claim stuff, | |
| 14:47:41 | mriedem | from one of the sourced scripts | |
| 14:47:45 | dansmith | but it seems like it's waiting for the neutron event | |
| 14:48:13 | dansmith | it's still in building state at the end of the run | |
| 14:48:55 | dansmith | ah, artom said something about that early this morning.. | |
| 14:49:17 | mriedem | instance c2b876e6-b616-4259-b03b-f0b2bd3d8908 right? | |
| 14:49:25 | dansmith | yup | |
| 14:50:46 | mriedem | yup start waiting for the event here https://zuul.opendev.org/t/openstack/build/d3d4357c13be452fa8e64a6b612c256d/log/logs/subnode-2/screen-n-cpu.txt.gz#1905 | |
| 14:50:55 | mriedem | Aug 28 04:38:17.325822 ubuntu-bionic-vexxhost-vexxhost-sjc1-0010477321 nova-compute[24765]: DEBUG nova.compute.manager [None req-4d7771be-f2e2-429d-87c7-1cd4469cede5 demo admin] [instance: c2b876e6-b616-4259-b03b-f0b2bd3d8908] Preparing to wait for external event network-vif-plugged-afd502d7-a21d-48f2-8607-881a86f0781f {{(pid=24765) prepare_for_instance_event /opt/stack/new/nova/nova/compute/manager.py:329}} | |
| 14:51:52 | mriedem | got the event here | |
| 14:51:53 | mriedem | https://zuul.opendev.org/t/openstack/build/d3d4357c13be452fa8e64a6b612c256d/log/logs/subnode-2/screen-n-cpu.txt.gz#1941 | |
| 14:51:58 | mriedem | Aug 28 04:38:21.643105 ubuntu-bionic-vexxhost-vexxhost-sjc1-0010477321 nova-compute[24765]: DEBUG nova.compute.manager [req-61851260-be42-4748-9d79-4b0ec5c08ed6 req-3c63a866-a638-4ec7-9f56-3a7aeb2e9958 service nova] [instance: c2b876e6-b616-4259-b03b-f0b2bd3d8908] Received event network-vif-plugged-afd502d7-a21d-48f2-8607-881a86f0781f {{(pid=24765) external_instance_event /opt/stack/new/nova/nova/compute/manager.py:8819}} | |
| 14:52:03 | mriedem | 4 seconds later | |
| 14:52:34 | dansmith | presumably unrelated to his change though if we've made it to the neutron event stage | |
| 14:54:22 | mriedem | i think during artom's revert resize ovs hybrid plug race thing that evacuate could have the same kind of race b/c of the time that we trigger the event by updating the port binding and when the driver's spawn method start waiting for it | |
| 14:55:30 | mriedem | although, if we didn't have a handler registered for the event callback we'd see "Received unexpected event" in the logs | |