| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-05-26 | |||
| 13:39:50 | gibi | sean-k-mooney, bauzas, tetsuro, melwitt: I made a step back and realized that non of you wanted a saftey check in the first place in https://review.opendev.org/c/openstack/nova-specs/+/788243 so I changed to propose not to add one. Lets see if this removes the impass | |
| 13:40:17 | gibi | impasse even | |
| 13:43:08 | sean-k-mooney | ack just finishing reviewing your qos spec. lyarwood's encypeted volumn on is next on my list then ill loop back to ^ | |
| 13:43:44 | gibi | sean-k-mooney: awesome, thanks | |
| 13:46:36 | bauzas | gibi: ack, will look again | |
| 13:59:03 | ozzzo | sean-k-mooney: I tried your grub trick and it allowed me to start the compute service with "cpu_models=Skylake-Server-IBRS,Cascadelake-Server-noTSX" | |
| 13:59:32 | ozzzo | but my cpu_info changed from "Cascadelake-Server-noTSX" to "Cascadelake-Server" whcih still doesn't match "Skylake-Server-IBRS" | |
| 13:59:59 | sean-k-mooney | they should not need to match | |
| 14:00:40 | sean-k-mooney | for live migration the xml that is used to start the dest server is a modifed one of the orginail server xml | |
| 14:00:51 | sean-k-mooney | we will not update the cpu model in that process | |
| 14:01:04 | sean-k-mooney | so the live migration will try to create the dest vm with Skylake-Server-IBRS | |
| 14:01:42 | sean-k-mooney | so you should be able to live migrate | |
| 14:02:29 | ozzzo | sean-k-mooney: I must be missing a step; I still get the "Host filter ignoring hosts" error | |
| 14:03:02 | sean-k-mooney | the host filter? that should only ignore the current host | |
| 14:03:03 | ozzzo | I did the grub trick, and then rebooted the compute, then deployed the compute | |
| 14:03:54 | sean-k-mooney | https://github.com/openstack/nova/blob/b0cd985f0c09088098f74cc0cb1df616cc0ef12b/nova/scheduler/filters/__init__.py#L22-L50 | |
| 14:04:15 | ozzzo | this is what I get in the scheduler log: https://paste.ubuntu.com/p/tX9W7dT3PH/ | |
| 14:04:57 | sean-k-mooney | actully its not that filter | |
| 14:06:05 | sean-k-mooney | thats coming form https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L490-L498 | |
| 14:06:38 | sean-k-mooney | so it means that the host has been added to the ignored host list | |
| 14:06:52 | sean-k-mooney | what release of openstack are you using by the way | |
| 14:06:56 | ozzzo | Train | |
| 14:07:16 | ozzzo | working on upgrading from centos 7-8 to prepare for train-ussuri | |
| 14:08:41 | sean-k-mooney | ok so you need to use debug logs in the schudler to see why the hsot got filtered out | |
| 14:08:51 | ozzzo | ok | |
| 14:09:20 | sean-k-mooney | something added the host to the ignored list becaue it failed a filter | |
| 14:21:32 | sean-k-mooney | gibi: im +1 on your pps spec but i have two things i woudl like to highlihgt quickly if you have time | |
| 14:22:04 | sean-k-mooney | https://review.opendev.org/c/openstack/nova-specs/+/785014/8/specs/xena/approved/qos-minimum-guaranteed-packet-rate.rst#328 and https://review.opendev.org/c/openstack/nova-specs/+/785014/8/specs/xena/approved/qos-minimum-guaranteed-packet-rate.rst#427 | |
| 14:22:41 | sean-k-mooney | gibi: neither are blocker to me i just want to make sure we are on the same page | |
| 14:23:44 | gibi | sean-k-mooney: sure | |
| 14:23:55 | gibi | sean-k-mooney: reading... | |
| 14:41:21 | ozzzo | sean-k-mooney: Here's the debug log: https://paste.ubuntu.com/p/4MrhjJs4r6/ | |
| 14:41:35 | ozzzo | at line 102 it says "Host filter ignoring hosts: chrnc-area51-os-compute-02.chtrse.com, chrnc-area51-os-compute-01.chtrse.com" | |
| 14:41:41 | ozzzo | but it's not obvious why | |
| 14:42:16 | ozzzo | compute-01 is the one where I did the grub trick | |
| 15:32:51 | stephenfin | sean-k-mooney: looks like that stable l-c patch for os-vif failed. I assume you're going to look at disabling those now, or should I do it? | |
| 15:33:52 | ozzzo | sean-k-mooney: turns out it's in the conductor log: https://paste.ubuntu.com/p/tNYS3jzXsT/ | |
| 15:34:24 | ozzzo | at 221 it says "nova.exception.MigrationPreCheckError: Migration pre-check error: CPU doesn't have compatibility." | |
| 15:34:33 | ozzzo | and then "internal error: Unknown CPU feature arch-facilities" | |
| 15:36:16 | ozzzo | actually it looks like the problem starts at line 35: | |
| 15:36:18 | ozzzo | 2021-05-26 14:30:35.457 23 DEBUG nova.conductor.tasks.live_migrate [req-d5ac7239-4326-4d9e-9731-e4fe53a19929 - - - - -] Skipping host: chrnc-area51-os-compute-02.chtrse.com because: Migration pre-check error: CPU doesn't have compatibility. | |
| 15:37:48 | sean-k-mooney | oh the arch-facilities thing | |
| 15:38:12 | openstack | bugzilla.redhat.com bug 1658406 in libvirt "mode="host-model" VMs include broken "arch-facilities" flag name [libvirt]" [Urgent,Closed: errata] - Assigned to jdenemar | |
| 15:38:12 | ozzzo | when I google it I find this: https://bugzilla.redhat.com/show_bug.cgi?id=1658406 | |
| 15:38:25 | sean-k-mooney | thats a different upgrade issue where they chagne that between centos 8 and centos 7 | |
| 15:45:56 | ozzzo | sean-k-mooney: how can I work around it? | |
| 15:46:20 | sean-k-mooney | stephenfin: we have two fixes that work | |
| 15:47:12 | sean-k-mooney | stephenfin: https://review.opendev.org/c/openstack/os-vif/+/792840 fixed lc and https://review.opendev.org/c/openstack/os-vif/+/793155 also fix it | |
| 15:47:38 | sean-k-mooney | zuul faild for other reasons | |
| 15:49:14 | sean-k-mooney | ozzzo: i think you need to update the version fo qemu then hard reboot the guest | |
| 15:49:34 | sean-k-mooney | ozzzo: i dont think there is a fix for that that does not invovle a guest reboot | |
| 15:50:25 | sean-k-mooney | ozzzo: so effectivly you cant avoid guest downtime so your simplest solution would be to cold migrate | |
| 15:51:02 | ozzzo | ok I'll experiment with cold-migration | |
| 15:51:11 | sean-k-mooney | ozzzo: unfortnetly this is a libvirt abi breakage that we cannot fix in nova | |
| 15:52:51 | ozzzo | sean-k-mooney: got it. thanks for your advice! | |
| 15:53:55 | gibi | sean-k-mooney: replied in https://review.opendev.org/c/openstack/nova-specs/+/785014 I think we are more or less on the same page | |
| 15:55:46 | sean-k-mooney | gibi: cool ill quickly skim over them one sec | |
| 15:58:07 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Repropose "vm-boot-with-unaddressed-port" spec https://review.opendev.org/c/openstack/nova-specs/+/793199 | |
| 16:02:40 | sean-k-mooney | gibi: cool i agree with youre respeonces so +w but just want to reinfoce that there is no way currently to make a neutron extention required unless nova is willign to stop supportin all neutron backends | |
| 16:03:17 | sean-k-mooney | gibi: i have brought up this problem at the last couple of ptgs | |
| 16:03:22 | gibi | sean-k-mooney: ack, then it means we will keep support for both format for a while | |
| 16:03:36 | gibi | and I will look into adding support for making someting mandatory in neutron | |
| 16:03:49 | sean-k-mooney | gibi: yes but if more them me are asking for this maybe we can actully get neutron to add a way | |
| 16:04:56 | sean-k-mooney | gibi: this type of exteniton can be implemented at the m2 2 level but since they still supprot monolity plugins the only way to make it mandatory is to implemetned above the ml2 laywer | |
| 16:05:11 | sean-k-mooney | as in move it to the core api | |
| 16:05:35 | sean-k-mooney | or start requiring that monolitich plugins implemenet a subset of the extentions | |
| 16:06:23 | bauzas | * bauzas needs to go off but I could chime on gibi's pps spec if needed | |
| 16:06:45 | sean-k-mooney | bauzas: i dont have any open issues with nova | |
| 16:06:52 | sean-k-mooney | bauzas: or the nova part of the spec | |
| 16:07:54 | sean-k-mooney | bauzas: its a neuton process/rules change that im raising | |
| 16:08:59 | bauzas | ack, will look at the spec | |
| 16:12:11 | sean-k-mooney | bauzas: i had just put a +w on it do you want me to remvoe that and leave it to you | |
| 16:13:49 | gibi | there are feedback in the neutron pps spec but I can only take that tomorrow so it is totally ok to keep the nova spec open | |
| 16:13:53 | gibi | * gibi leaves the building | |
| 16:14:49 | sean-k-mooney | you have a very well put together spec so i would also be happy addressing any nits that arise as a folow up if that works for you | |
| 16:18:23 | sean-k-mooney | i removed -w to let bauzas read the latest comments but feel free to re add it | |
| 16:21:34 | sean-k-mooney | lyarwood: minor point on your spec https://review.opendev.org/c/openstack/nova-specs/+/783806/2/specs/xena/approved/ephemeral-encryption.rst#187 | |
| 16:21:38 | stephenfin | sean-k-mooney: +w away | |
| 16:21:47 | stephenfin | we can always revert if needed | |
| 16:21:56 | stephenfin | IMO | |
| 16:21:57 | sean-k-mooney | lyarwood: are we going to have COMPUTE_EPHEMERAL_ENCRYPTION_LUKSV1 and COMPUTE_EPHEMERAL_ENCRYPTION_LUKSV2 or COMPUTE_EPHEMERAL_ENCRYPTION_LUKS and COMPUTE_EPHEMERAL_ENCRYPTION_LUKSV2 | |
| 16:23:03 | lyarwood | ENCRYPTION_LUKS and ENCRYPTION_LUKSV2 are already in os-traits | |
| 16:23:35 | sean-k-mooney | oh... | |
| 16:23:44 | lyarwood | I get that v1/v2 would be nicer but I think this was done to mirror the volume encryption types we have defined in os-brick | |
| 16:23:48 | sean-k-mooney | because we approve the spec last cycle | |
| 16:23:53 | lyarwood | that pre-date v2 so we started with luks and then added luksv2 later | |
| 16:24:00 | lyarwood | yeah | |
| 16:24:02 | sean-k-mooney | if the traits are already there then ok | |
| 16:24:24 | lyarwood | cool cool | |
| 16:25:13 | sean-k-mooney | i feel like im +w too many thing so i just +2'd https://review.opendev.org/c/openstack/nova-specs/+/783806 | |
| 16:25:41 | stephenfin | +W ALL the things! | |
| 16:25:58 | stephenfin | I did it for you ;) | |
| 16:26:21 | sean-k-mooney | * sean-k-mooney is trying to be conservitive with merge writes | |
| 16:26:45 | sean-k-mooney | * sean-k-mooney is also reading a lot of well trought out specs | |
| 16:26:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Enable 'vmcoreinfo' feature by default https://review.opendev.org/c/openstack/nova/+/752912 | |
| 16:26:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: libvirt: Make vmcoreinfo checks guest arch specific https://review.opendev.org/c/openstack/nova/+/790946 | |
| 16:26:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Make vmcoreinfo dependent on feature flag https://review.opendev.org/c/openstack/nova/+/779342 | |
| 16:26:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Remove 'create_fake_libvirt_mock' https://review.opendev.org/c/openstack/nova/+/790945 | |
| 16:27:15 | stephenfin | lyarwood: Could I get your +2 on the first of those again? ^ | |
| 16:27:19 | stephenfin | merge conflict :( | |