| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-07-27 | |||
| 19:16:18 | gmann | yeah, it is difficult to make differentiation on those cases | |
| 19:16:43 | dansmith | so instance.host==None, 4xxsomething, but otherwise, I'd say 200 empty | |
| 19:16:57 | gmann | +1 | |
| 19:17:12 | dansmith | I don't think there's any useful distinction between "nothing has ever been written to the console" and "the console log is gone" | |
| 19:17:12 | sean-k-mooney | hehe ok fun | |
| 19:17:24 | sean-k-mooney | dansmith: so my boot form volume guest is not using a pty | |
| 19:17:25 | dansmith | like I said, *maybe* for a windows guest with no actual console, I dunno how that works really | |
| 19:17:38 | dansmith | even macos on uefi has a text console.. it's set to silent by default, but it's there | |
| 19:17:43 | gmann | I think I got your point of action is passed so it should be 200 irrespective of what output it give | |
| 19:17:57 | dansmith | gmann: yeah | |
| 19:18:10 | sean-k-mooney | dansmith: https://paste.opendev.org/show/bvGBKyn5fHQthz8TO13C/ | |
| 19:18:25 | dansmith | sean-k-mooney: interesting | |
| 19:18:36 | sean-k-mooney | dansmith: so its using tcp for the console | |
| 19:18:47 | sean-k-mooney | and the source is still there when its stoped | |
| 19:19:06 | dansmith | yeah, I dunno why that is set that way, but I can't imagine it has much to do with being lvm | |
| 19:19:26 | sean-k-mooney | that becasue its kolla-ansible i think | |
| 19:19:36 | dansmith | yeah *that* I'd believe :) | |
| 19:19:46 | sean-k-mooney | again this is related to how libvirt is configured | |
| 19:20:05 | sean-k-mooney | since its generating that so its connecting to virtlogd over tcp in this case | |
| 19:20:37 | sean-k-mooney | well to qemu i guess not virtlogd | |
| 19:21:10 | sean-k-mooney | this deployment is using molitic libvirt | |
| 19:22:03 | sean-k-mooney | dansmith: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4464 | |
| 19:22:16 | sean-k-mooney | so if we are usign tcp we get the path form log | |
| 19:22:20 | sean-k-mooney | not source | |
| 19:22:32 | sean-k-mooney | so thats why that is working | |
| 19:22:52 | dansmith | okay but virtlogd should hide this all from us and let us use the log file yeah? | |
| 19:23:08 | sean-k-mooney | yes for qemu/kvm | |
| 19:23:18 | sean-k-mooney | i would need to double check for lxc | |
| 19:23:22 | dansmith | ack | |
| 19:34:05 | sean-k-mooney | https://libvirt.org/formatdomain.html#consoles-serial-parallel-channel-devices is the relevent docs | |
| 19:34:31 | sean-k-mooney | Regardless of the type, character devices can have an optional log file associated with them. This is expressed via a log sub-element, with a file attribute. There can also be an append attribute which takes the same values described above. Since 1.3.3 . | |
| 19:34:56 | dansmith | that implies virtlogd works for everything yeah? | |
| 19:35:03 | dansmith | (as I would expect) | |
| 19:36:44 | sean-k-mooney | implies it yes btu does not actully say it | |
| 19:37:34 | sean-k-mooney | i also cant find anything that would imply how you tell libvirt to chooese the type | |
| 19:37:50 | sean-k-mooney | im going to check my kolla cloud incase it ahs something set in the libvirt configs | |
| 19:38:09 | sean-k-mooney | i suspect pty might be the default or the default in newer libvirts | |
| 19:40:07 | sean-k-mooney | ah found it | |
| 19:40:20 | sean-k-mooney | # The backend to use for handling stdout/stderr output from | |
| 19:40:22 | sean-k-mooney | # QEMU processes. | |
| 19:40:24 | sean-k-mooney | # | |
| 19:40:26 | sean-k-mooney | # 'file': QEMU writes directly to a plain file. This is the | |
| 19:40:28 | sean-k-mooney | # historical default, but allows QEMU to inflict a | |
| 19:40:30 | sean-k-mooney | # denial of service attack on the host by exhausting | |
| 19:40:32 | sean-k-mooney | # filesystem space | |
| 19:40:34 | sean-k-mooney | # | |
| 19:40:36 | sean-k-mooney | # 'logd': QEMU writes to a pipe provided by virtlogd daemon. | |
| 19:40:38 | sean-k-mooney | # This is the current default, providing protection | |
| 19:40:40 | sean-k-mooney | # against denial of service by performing log file | |
| 19:40:42 | sean-k-mooney | # rollover when a size limit is hit. | |
| 19:40:44 | sean-k-mooney | # | |
| 19:40:46 | sean-k-mooney | #stdio_handler = "logd" | |
| 19:41:07 | sean-k-mooney | my kolla cloud is using file | |
| 19:41:18 | dansmith | eesh man, pastebin c'mon | |
| 19:41:42 | dansmith | that's libvirt config? | |
| 19:41:54 | sean-k-mooney | :) sure ya that /etc/libvirt/qemu.conf | |
| 19:42:04 | dansmith | oh, qemu.conf, interesting | |
| 19:42:13 | dansmith | well, I dunno what to say | |
| 19:42:16 | dansmith | we don't support lxc | |
| 19:42:28 | dansmith | I don't want to break it for no reason, but... unless someone is going to try out a change... | |
| 19:42:31 | sean-k-mooney | ill try chanign my lvm devstack to file and see if it changes to tcp | |
| 19:46:19 | sean-k-mooney | no still using pty | |
| 19:46:27 | sean-k-mooney | ok im going ot leave it there today | |
| 19:47:08 | gmann | this test was wrong since starting and not sure how https://review.opendev.org/c/openstack/tempest/+/889109 unhide it on lvm and made it always failing | |
| 19:47:26 | gmann | 889109 does not change anything from this test perspective | |
| 19:47:32 | frickler | oh, interesting, iirc there is still a kolla bug about dos via console log filling the disk. I need to look into that | |
| 19:48:03 | sean-k-mooney | my cloud is not fully up to date | |
| 19:48:09 | frickler | gmann: interaction with other tests in the class? | |
| 19:48:19 | dansmith | frickler: I doubt it | |
| 19:48:24 | sean-k-mooney | but on the kolla sid i jsu tnotice that libvirtd.conf is configured to connect to libvirt with sasl over tcp | |
| 19:48:32 | dansmith | it may be because this change does increase the load with more parallelism, | |
| 19:48:37 | gmann | frickler: humm, no, it recreate the server in setup and keep it SSHABLE | |
| 19:48:58 | dansmith | however this test class was usually running early on enough that all the workers were slammed with work | |
| 19:49:04 | dansmith | I looked at a *lot* of stackviz reports before doing that :) | |
| 19:49:06 | sean-k-mooney | https://paste.opendev.org/show/bZMoOYeTqbcHyr79ZDwS/ | |
| 19:50:09 | gmann | dansmith: i doubt, server was active always in any parallel case in setup so it should have been working in same way it was working before | |
| 19:50:39 | dansmith | gmann: I agree, I'm just saying load can be higher with the split, but that was the point | |
| 19:51:07 | dansmith | I just rechecked sean-k-mooney's patch to get another data point.. i.e. to see if it's still only lvm and only that test | |
| 19:51:20 | gmann | anyways as 889895 corrent the tests and make it fail always. I am going to skip this test for now until we figure out the nova behavior in 'guest shutoff' case | |
| 19:51:25 | gmann | *coprrect | |
| 19:52:04 | dansmith | ack | |
| 21:10:25 | frickler | maybe this is related, since it has been moved to a different class https://opendev.org/openstack/tempest/src/branch/master/tempest/api/compute/servers/test_server_actions.py#L332-L337 | |
| 21:10:47 | frickler | also expected failures on the tempest patch going back to yoga, so not a recent regression at least | |
| 21:22:12 | gmann | but that was same before moving to different class also. this test s no issue, test_get_console_output_server_id_in_shutoff_status started failing after refactoring and not sure why | |
| 21:24:36 | gmann | or maybe you are right. the execution order of test in same class might be making the server console available before test_get_console_output_server_id_in_shutoff_status execute | |
| 21:29:41 | dansmith | how? | |
| 21:30:15 | dansmith | each class gets its own server to work on, and the tests within a class are single-threaded. if it matters now, it mattered before, so it must have already been broken | |
| 21:37:25 | gmann | one chance I see if that self.server_id server get through many operation before test_get_console_output_server_id_in_shutoff_status was executed before so console of that self.server_id was somehow available but now as it is in separate class, self.server_id is kind of fresh server and fail ? | |
| 21:37:46 | gmann | idk but that is something might be happening, i cannot find any other reasoning on this | |
| 21:50:29 | gmann | correcting and skipping this test for now https://review.opendev.org/c/openstack/tempest/+/889895 | |
| 22:15:53 | dansmith | yeah I don't think so.. at one point I was wondering if we had to call get_console once to process the pty data and start the log file, but with virtlogd that should definitely not be the case | |
| #openstack-nova - 2023-07-28 | |||
| 04:21:45 | opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | |
| 04:21:46 | opendevreview | melanie witt proposed openstack/nova master: Support resize with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870933 | |
| 04:21:46 | opendevreview | melanie witt proposed openstack/nova master: Add hw_ephemeral_encryption_secret_uuid image property https://review.opendev.org/c/openstack/nova/+/870935 | |
| 04:21:47 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936 | |
| 04:21:47 | opendevreview | melanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870937 | |
| 04:21:48 | opendevreview | melanie witt proposed openstack/nova master: Support rebuild and unshelve with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/870939 | |
| 04:21:48 | opendevreview | melanie witt proposed openstack/nova master: WIP Support rescue with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/873675 | |
| 04:21:49 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | |
| 04:21:50 | opendevreview | melanie witt proposed openstack/nova master: WIP raw imagebackend support https://review.opendev.org/c/openstack/nova/+/884313 | |