| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-03 | |||
| 08:58:34 | kashyap | gibi: Indeed; thanks for the ACK; the separate options would've made sense if we were doing it from scratch. | |
| 08:59:31 | kashyap | Thanks for bearing with me. | |
| 09:35:39 | songwenping__ | stephenfin: morning; there are two problems for the noVNC patch. 1. the password works after you reboot the guest 2. if you donnot pass the password when get-vnc-console, you need config `[vnc]auth_schemes = none, vnc` in /etc/nova/nova-cell1.conf for noVNC guest. | |
| 09:38:53 | songwenping__ | for the first issue, we can clarify in the docs that user need to reboot the server if set/reset password for vnc console. we can improve if the libvirt/qemu support live set/reset password. | |
| 09:41:47 | stephenfin | songwenping__: to be honest, it sounds like we shouldn't allow users to change the password if that's not supported so | |
| 09:41:59 | stephenfin | I don't think asking users to restart their instance is a reasonable request | |
| 09:44:25 | openstackgerrit | Lucas Alvares Gomes proposed openstack/nova master: [OVN] Explicitly set nova-next job to ML2/OVS https://review.opendev.org/c/openstack/nova/+/776944 | |
| 09:44:26 | songwenping__ | for the second issue, we plan to update the release note and the description of nova/conf/vnc.py for `auth_schemes`. | |
| 09:46:04 | songwenping__ | stephenfin: the libvirt docs[1] said it already support live update password, but i donnot know why it failed in my local ven. [1] https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainUpdateDeviceFlags | |
| 09:55:07 | brinzhang | stephenfin, songwenping__:IMHO, support reset VNC password is good for the user, it can be protected thire instance opening by unknown user, if it must be reboot to take effect, can we do the first mode to support [vnc]auth_schemes=vnc? | |
| 09:59:00 | lyarwood | stephenfin / gibi ; https://review.opendev.org/c/openstack/nova/+/778177 should be good now btw, I missed that the requirements job actually forces you to sync the in-direct LC deps to requirements that bloats the change a little | |
| 09:59:07 | stephenfin | brinzhang: The issue is that _set_ VNC password also requires a reboot | |
| 09:59:19 | lyarwood | as discussed with sean-k-mooney yesterday I've added a note to the PTG to discuss this mess | |
| 09:59:35 | stephenfin | Basically to touch anything to do with VNC passwords, the instance must be restarted | |
| 10:05:37 | brinzhang | stephenfin: can you check this libvirt docs? https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainUpdateDeviceFlags does it means it cannot be reset password if flags=LIVE(1)? | |
| 10:06:31 | brinzhang | *E.g. the hypervisor driver will return failure if LIVE is specified but it only supports modifying the persisted device allocation.*, so the VNC password isnot blongs to the *persisted device*? | |
| 10:09:36 | songwenping__ | stephenfin: we just need reboot first time to set password, if we reset the password again, we donnot need reboot the server any more. | |
| 10:10:03 | gibi | brinzhang: I agree to have a bug reported and fixing it in a separate patch | |
| 10:10:19 | songwenping__ | from none -> vnc need to reboot, from vnc -> vnc donnot need to reboot | |
| 10:11:29 | brinzhang | gibi: I have filed the bug, and will be submited the fix later ^, pls see bug 1917592 | |
| 10:11:32 | openstack | bug 1917592 in OpenStack Compute (nova) "Missed 'accel_uuids' when we the 'shelved_offload_time' time out in shelving instance periodic task" [Medium,New] https://launchpad.net/bugs/1917592 - Assigned to Brin Zhang (zhangbailin) | |
| 10:11:37 | gibi | brinzhang: thanks | |
| 10:11:56 | brinzhang | gibi: np, thanks for your mind too | |
| 10:13:13 | gibi | lyarwood: thanks I] | |
| 10:13:17 | gibi | lyarwood: thanks I'm +2 | |
| 10:19:45 | lyarwood | cheers | |
| 10:22:18 | lyarwood | random question, does anyone know why we don't use the instance UUID as the domain name within libvirt? | |
| 10:22:48 | lyarwood | trying to debug a live migration failure on stable/ussuri and hate having to lookup the domain name in the logs | |
| 10:26:29 | gibi | lyarwood: good question, we probably need a historian to answer itr | |
| 10:28:06 | kashyap | Yeah, I don't remember that answer either; and wondered the same in the past | |
| 10:28:26 | kashyap | But the guest XML records both instance-YYYYYYY and the UUID | |
| 10:28:46 | stephenfin | brinzhang: Yeah, looking now | |
| 10:30:10 | stephenfin | brinzhang: songwenping__: IMO we need to fix this and figure out how to get VNC password working with TLS. At the moment, enabling this means (a) no TLS, (b) you have to restart the instance to use the feature, and (c) you have to do some funky nova.conf changes for things to work | |
| 10:30:35 | stephenfin | Also (d) you're stuck with 8 characters (not our fault, but unfortunate all the same) | |
| 10:30:43 | stephenfin | Not a good UX /o\ | |
| 10:31:39 | stephenfin | brinzhang: I know it's a lot of work that you said you don't want to do it, but I would still seriously suggest switching the order of the VNC password and tenant_id -> project_id series | |
| 10:32:01 | lyarwood | kashyap: yeah thats fine in live envs, just a PITA when you only have logs to debug things | |
| 10:32:15 | lyarwood | kashyap: I think it might be that libvirt couldn't fit the entire UUID in the name or something? | |
| 10:32:18 | kashyap | lyarwood: Yeah, no denying; I just let the steam out by cussing | |
| 10:32:52 | kashyap | lyarwood: Hmm, that I don't remember. But lemme quickly test by creating a domain name with the UUID :) | |
| 10:33:27 | stephenfin | The VNC series is at huge risk due to all those concerns. I can't in good faith merge what I consider to be a broken feature :-( Hopefully we can address them before feature freeze but it will prevent the other series landing until we do | |
| 10:34:38 | kashyap | lyarwood: Ah, so: libvirt seems to be smart here: | |
| 10:34:53 | lyarwood | aaaaaaaaaah it's instance.name | |
| 10:35:17 | kashyap | lyarwood: When I tried to live-modify a guest's name from s/cvm1/64ba587c-57a7-44c0-94ec-5444e5268c4e/ --> libvirt tells me: "well, the guest already has a UUID" | |
| 10:35:59 | kashyap | I tried removing dashes and plug the dash-less UUID into <name/>, it still detects. | |
| 10:36:05 | lyarwood | okay, we can't really change this anyway, it's just annoying | |
| 10:37:08 | kashyap | Yeah, libvirt explicitly rejects a UUID for a domain "name" | |
| 10:37:21 | kashyap | s/Yeah,/Yeah, and/ | |
| 10:38:17 | lyarwood | not sure how we'd do that after the test run | |
| 10:38:56 | kashyap | lyarwood: You mean, you want a file logged a mapping of UUIDs with instance names? | |
| 10:40:11 | lyarwood | yeah but I don't think that's going to be possible *after* tempest has already deleted everything | |
| 10:40:35 | kashyap | Hmm, true | |
| 10:43:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: console: Improve logging https://review.opendev.org/c/openstack/nova/+/778407 | |
| 10:55:12 | lyarwood | gibi: https://review.opendev.org/q/topic:spec/libvirt-default-machine-type+status:open should be ready for review this week btw, not sure if you want to move it into a runway slot or not. | |
| 11:06:24 | gibi | lyarwood: I added to my queue, I see stephenfin is already +2 so I prioritize this | |
| 11:07:34 | lyarwood | gibi: yup thanks, I'm around all day for respins etc so let me know if you have any questions | |
| 11:07:42 | gibi | OK | |
| 11:23:07 | slaweq | hi nova team :) | |
| 11:23:11 | gibi | slaweq: hi! | |
| 11:23:14 | slaweq | I need Your help with some ci issue | |
| 11:23:31 | slaweq | we saw, mostly in stable/train errors like in https://a574f9c0fd4ca92b7603-2045be852d43868eb95da6cc3429b40d.ssl.cf2.rackcdn.com/777334/2/check/neutron-tempest-dvr-ha-multinode-full/44d0207/testr_results.html recently | |
| 11:23:59 | slaweq | do You have any idea why there is no host for cold-migration or resize in that job? | |
| 11:25:17 | slaweq | hi gibi :) | |
| 11:25:22 | bauzas | stephenfin: saw my weak -1 on your apidb change for Ocata ? https://review.opendev.org/c/openstack/nova/+/759402/4 | |
| 11:25:55 | bauzas | I don't really to vote -1 on a commit msg miss, but I feel this is important to comment the fact that we don't care about a specific DB migration | |
| 11:26:03 | bauzas | really like* | |
| 11:26:19 | bauzas | and a commit msg can't be patched by a FUP... | |
| 11:28:41 | gibi | slaweq: is it happens all the time or just sometimes? | |
| 11:29:46 | slaweq | gibi: on train I think that all the time | |
| 11:29:51 | slaweq | or at least very often | |
| 11:30:12 | slaweq | gibi: in scheduler log I see something like: | |
| 11:30:15 | slaweq | Mar 03 10:32:31.142569 ubuntu-bionic-rax-ord-0023272867 nova-scheduler[24545]: INFO nova.scheduler.host_manager [None req-657ecf60-4fba-4168-9613-3c57d01bdcb7 tempest-MinBwAllocationPlacementTest-1587030703 tempest-MinBwAllocationPlacementTest-1587030703-project-admin] Host filter ignoring hosts: ubuntu-bionic-rax-ord-0023272867 | |
| 11:30:34 | gibi | slaweq: ack, give me some time to dive into the logs | |
| 11:30:39 | slaweq | and then next message that " There are 0 hosts available but 1 instances requested to build." | |
| 11:30:52 | slaweq | gibi: sure, I will open LP for that to track it | |
| 11:30:55 | slaweq | thx for help | |
| 11:31:02 | gibi | slaweq: thanks for reporting it | |
| 11:31:10 | slaweq | it's not very urgent for us as this is non-voting job :) | |
| 11:31:16 | slaweq | so take Your time | |
| 11:31:36 | brinzhang | stephenfin: *(songwenping_)from none -> vnc need to reboot, from vnc -> vnc donnot need to reboot* I still think from docs or a bug trace this is a good choice | |
| 11:40:24 | slaweq | gibi: https://bugs.launchpad.net/nova/+bug/1917610 | |
| 11:40:26 | slaweq | reported | |
| 11:40:26 | openstack | Launchpad bug 1917610 in OpenStack Compute (nova) "Migration and resize tests from tempest.scenario.test_minbw_allocation_placement.MinBwAllocationPlacementTest failing in neutron-tempest-dvr-ha-multinode-full" [Undecided,New] | |
| 11:54:51 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add missed accel_uuids for _poll_shelved_instances https://review.opendev.org/c/openstack/nova/+/778440 | |
| 11:57:41 | stephenfin | brinzhang: I don't :( | |
| 11:57:58 | openstackgerrit | Stephen Finucane proposed openstack/nova master: apidb: Compact Ocata database migrations https://review.opendev.org/c/openstack/nova/+/759402 | |
| 11:58:03 | stephenfin | bauzas: Is that okay? ^ | |
| 11:58:12 | stephenfin | I'll rebase the rest of the series on top if so | |
| 11:58:24 | brinzhang | bauzas: fixed that bug, pls see https://review.opendev.org/c/openstack/nova/+/778440 | |
| 12:22:04 | brinzhang | stephenfin: we will re-search, and make sure whether can reset passwrod and without restart the server | |
| 12:22:44 | brinzhang | s/and// | |
| 12:28:08 | gibi | slaweq: responded in https://bugs.launchpad.net/nova/+bug/1917610 with alternatives to either fix the job definition in neutron or fix the test precoditions in tempest | |
| 12:28:09 | openstack | Launchpad bug 1917610 in OpenStack Compute (nova) "Migration and resize tests from tempest.scenario.test_minbw_allocation_placement.MinBwAllocationPlacementTest failing in neutron-tempest-dvr-ha-multinode-full" [Undecided,New] | |
| 12:37:28 | slaweq | gibi: thx | |
| 12:37:42 | slaweq | gibi: proposed fix https://review.opendev.org/c/openstack/tempest/+/778451 and dnm test patch https://review.opendev.org/c/openstack/neutron/+/778452 | |
| 12:37:49 | slaweq | let's check if that will help | |
| 12:38:01 | gibi | tht was fast :) | |
| 12:38:50 | gibi | wondering why this is only hit on stable/train. It should fail on any newer than train too | |
| 12:47:30 | gibi | lyarwood: I see an issue in https://review.opendev.org/c/openstack/nova/+/769548/12/nova/cmd/manage.py#2631 | |
| 12:53:53 | lyarwood | ah sorry, I thought I had sorted that out | |