Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-05-26
12:23:38 sean-k-mooney at least untill we do the cahgne to external events
12:24:40 sean-k-mooney read ^ as a +0.5 sure but not may area :)
12:33:56 gibi I agree with sean-k-mooney here.
12:36:18 stephenfin sean-k-mooney: RE: the broken l-c jobs on os-vif stable branches: I see you're trying to fix them
12:36:23 stephenfin should we be bothering though
12:36:35 stephenfin if it's broken now then it was always broken
12:36:39 sean-k-mooney the inital backport https://review.opendev.org/c/openstack/os-vif/+/792840 failed to install hacking
12:36:43 stephenfin pip's broken resolver was just hiding that
12:37:07 sean-k-mooney i can ditch that and drop it for stable branchs but i would like elod and lyarwood to comment on that
12:37:27 stephenfin yeah, I would be in favour of dropping this rather than investing more effort in it, personally
12:37:33 stephenfin but stable cores should weigh in. melwitt too
12:37:46 lyarwood yup FWIW I'm fine dropping it
12:38:45 gibi our agreement was that we try to fix but drop it if we cannot
12:38:57 gibi so if the consensus is that it is now hard to fix then I'm fine to drop it
12:39:26 stephenfin yeah, that's my recollection also
12:39:26 sean-k-mooney from stable only or also form master?
12:39:37 stephenfin if master is still working, I see no reason to drop it there
12:39:39 stephenfin personally
12:39:47 sean-k-mooney master is yes
12:40:12 stephenfin but per gibi's comment above, if it breaks and is more than trivial to resolve, we should drop from master also
12:40:21 sean-k-mooney and stable wallaby is since it has your fix
12:40:31 sean-k-mooney so it would be victoria and older
12:40:35 gibi I agree with stephenfin
12:40:48 stephenfin sean-k-mooney: victoria and older wfm
12:41:32 sean-k-mooney ok i just reject the victoria fix if that fails to install hacking again or fails in general ill drop it and then propose the other patches for stable and rebase any pending patches
12:41:52 sean-k-mooney well the 2 i want to backport in anycase :)
13:07:04 openstackgerrit Stephen Finucane proposed openstack/nova master: Add 'hw:vif_multiqueue_enabled' flavor extra spec https://review.opendev.org/c/openstack/nova/+/792356
13:07:04 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Document virtio-net multiqueue https://review.opendev.org/c/openstack/nova/+/792362
13:07:05 openstackgerrit Stephen Finucane proposed openstack/nova master: Move 'hw:pmu', 'hw_pmu' parsing to nova.virt.hardware https://review.opendev.org/c/openstack/nova/+/792364
13:16:36 openstackgerrit Elod Illes proposed openstack/os-vif stable/victoria: [stable-only] Fix lower-constraints https://review.opendev.org/c/openstack/os-vif/+/793155
13:17:04 elod sean-k-mooney: this worked for me locally, let's see if zuul is also happy with it ^^^
13:17:36 sean-k-mooney elod: so did the patch i pushed :)
13:17:51 sean-k-mooney elod: https://review.opendev.org/c/openstack/os-vif/+/792840
13:17:58 elod oh did it meanwhile? :) I just saw that it failed :)
13:18:15 sean-k-mooney well it passed before i pushed it
13:18:24 sean-k-mooney it failed in teh gate because it could not install hacking
13:18:34 sean-k-mooney i think that was a limestone issue
13:20:10 sean-k-mooney elod: the recheck of min passed lower constratits for what its worth
13:20:46 elod it was passing before if I remember correctly
13:20:51 sean-k-mooney https://a23e1d5d8671587c8417-333846024fbe3cd00d8d8f9166c680e2.ssl.cf2.rackcdn.com/792840/1/check/openstack-tox-lower-constraints/f0bf4ed/
13:21:43 sean-k-mooney looks like other josb are failing to conenct to opendev.org
13:22:16 elod the only concern why I sticked to the fixing of constraints instead of removing most of them is I think it is more "future proof" to keep more constraints. otherwise the resolver could become slower and slower.
13:22:36 elod but, anyway, both patch works for me :)
13:26:04 sean-k-mooney elod: it could although os-vif tries to have relitivly minimal deps
13:27:16 sean-k-mooney i would even like to reduce the set we have futher by removng oslo.i18n
13:28:00 sean-k-mooney we have never hadn and translsation for os-vif and i dont think we do that anymore for logs/errors in general
13:33:19 elod sean-k-mooney: with the minimal l-c set, if you check the job time is 10 mins for the l-c job compared to py** which are 4 mins. at least that's what I saw after the first test results
13:34:25 sean-k-mooney the job rund the doc generation and unit tests
13:34:39 sean-k-mooney the py** jobs do far less work
13:34:54 sean-k-mooney well maybe not far less but it is less
13:35:27 sean-k-mooney we shoudl compare the runtime of the two version of the patch when zuul reports back
13:35:58 elod of course, but with more lower constraints the resolver is quicker and takes more or less the same time. :)
13:36:16 elod anyway, as I said, both patch works for me :)
13:37:44 sean-k-mooney the ussuri version will only need a minor change to add one pep8 rule to the ignore list by the way
13:38:00 sean-k-mooney so im ok with fixing it in the older branch provide its simple too
13:38:37 sean-k-mooney that said there are some patch that for downstream reason we kind of need to merge before june 2nd
13:39:17 sean-k-mooney specificaly https://review.opendev.org/c/openstack/os-vif/+/765912 and https://review.opendev.org/c/openstack/os-vif/+/765970/3
13:39:50 gibi sean-k-mooney, bauzas, tetsuro, melwitt: I made a step back and realized that non of you wanted a saftey check in the first place in https://review.opendev.org/c/openstack/nova-specs/+/788243 so I changed to propose not to add one. Lets see if this removes the impass
13:40:17 gibi impasse even
13:43:08 sean-k-mooney ack just finishing reviewing your qos spec. lyarwood's encypeted volumn on is next on my list then ill loop back to ^
13:43:44 gibi sean-k-mooney: awesome, thanks
13:46:36 bauzas gibi: ack, will look again
13:59:03 ozzzo sean-k-mooney: I tried your grub trick and it allowed me to start the compute service with "cpu_models=Skylake-Server-IBRS,Cascadelake-Server-noTSX"
13:59:32 ozzzo but my cpu_info changed from "Cascadelake-Server-noTSX" to "Cascadelake-Server" whcih still doesn't match "Skylake-Server-IBRS"
13:59:59 sean-k-mooney they should not need to match
14:00:40 sean-k-mooney for live migration the xml that is used to start the dest server is a modifed one of the orginail server xml
14:00:51 sean-k-mooney we will not update the cpu model in that process
14:01:04 sean-k-mooney so the live migration will try to create the dest vm with Skylake-Server-IBRS
14:01:42 sean-k-mooney so you should be able to live migrate
14:02:29 ozzzo sean-k-mooney: I must be missing a step; I still get the "Host filter ignoring hosts" error
14:03:02 sean-k-mooney the host filter? that should only ignore the current host
14:03:03 ozzzo I did the grub trick, and then rebooted the compute, then deployed the compute
14:03:54 sean-k-mooney https://github.com/openstack/nova/blob/b0cd985f0c09088098f74cc0cb1df616cc0ef12b/nova/scheduler/filters/__init__.py#L22-L50
14:04:15 ozzzo this is what I get in the scheduler log: https://paste.ubuntu.com/p/tX9W7dT3PH/
14:04:57 sean-k-mooney actully its not that filter
14:06:05 sean-k-mooney thats coming form https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L490-L498
14:06:38 sean-k-mooney so it means that the host has been added to the ignored host list
14:06:52 sean-k-mooney what release of openstack are you using by the way
14:06:56 ozzzo Train
14:07:16 ozzzo working on upgrading from centos 7-8 to prepare for train-ussuri
14:08:41 sean-k-mooney ok so you need to use debug logs in the schudler to see why the hsot got filtered out
14:08:51 ozzzo ok
14:09:20 sean-k-mooney something added the host to the ignored list becaue it failed a filter
14:21:32 sean-k-mooney gibi: im +1 on your pps spec but i have two things i woudl like to highlihgt quickly if you have time
14:22:04 sean-k-mooney https://review.opendev.org/c/openstack/nova-specs/+/785014/8/specs/xena/approved/qos-minimum-guaranteed-packet-rate.rst#328 and https://review.opendev.org/c/openstack/nova-specs/+/785014/8/specs/xena/approved/qos-minimum-guaranteed-packet-rate.rst#427
14:22:41 sean-k-mooney gibi: neither are blocker to me i just want to make sure we are on the same page
14:23:44 gibi sean-k-mooney: sure
14:23:55 gibi sean-k-mooney: reading...
14:41:21 ozzzo sean-k-mooney: Here's the debug log: https://paste.ubuntu.com/p/4MrhjJs4r6/
14:41:35 ozzzo at line 102 it says "Host filter ignoring hosts: chrnc-area51-os-compute-02.chtrse.com, chrnc-area51-os-compute-01.chtrse.com"
14:41:41 ozzzo but it's not obvious why
14:42:16 ozzzo compute-01 is the one where I did the grub trick
15:32:51 stephenfin sean-k-mooney: looks like that stable l-c patch for os-vif failed. I assume you're going to look at disabling those now, or should I do it?
15:33:52 ozzzo sean-k-mooney: turns out it's in the conductor log: https://paste.ubuntu.com/p/tNYS3jzXsT/
15:34:24 ozzzo at 221 it says "nova.exception.MigrationPreCheckError: Migration pre-check error: CPU doesn't have compatibility."
15:34:33 ozzzo and then "internal error: Unknown CPU feature arch-facilities"
15:36:16 ozzzo actually it looks like the problem starts at line 35:
15:36:18 ozzzo 2021-05-26 14:30:35.457 23 DEBUG nova.conductor.tasks.live_migrate [req-d5ac7239-4326-4d9e-9731-e4fe53a19929 - - - - -] Skipping host: chrnc-area51-os-compute-02.chtrse.com because: Migration pre-check error: CPU doesn't have compatibility.
15:37:48 sean-k-mooney oh the arch-facilities thing
15:38:12 ozzzo when I google it I find this: https://bugzilla.redhat.com/show_bug.cgi?id=1658406

Earlier   Later