| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-03-25 | |||
| 13:02:10 | noonedeadpunk | As I just realized they result in quite different xmls | |
| 13:03:04 | noonedeadpunk | In first case through "static" vcpu placement, in second through reall mapping of vCPUs to CPUs... | |
| 13:03:26 | sean-k-mooney | they shoudl result in the same | |
| 13:03:37 | noonedeadpunk | huh | |
| 13:03:50 | sean-k-mooney | both should do a one to one vcpu pinning with the vcpupin element | |
| 13:04:32 | noonedeadpunk | For resources:PCPU I got just <vcpu placement='static'>4</vcpu> | |
| 13:04:40 | noonedeadpunk | and no vcpupin | |
| 13:04:40 | sean-k-mooney | ya that is not correct | |
| 13:04:50 | sean-k-mooney | what release are you on | |
| 13:04:54 | noonedeadpunk | Xena | |
| 13:05:11 | sean-k-mooney | i wonder if the mixed cpu support regressed that | |
| 13:06:07 | noonedeadpunk | maybe I have some mistake in flavor, let me double-check | |
| 13:06:56 | sean-k-mooney | if you have resouces:PCPU it shoudl be all you need although i dont recommend that people use the resouces: syntax in general | |
| 13:07:07 | sean-k-mooney | https://github.com/openstack/nova-specs/blob/master/specs/victoria/implemented/use-pcpu-vcpu-in-one-instance.rst was implemented in victoria | |
| 13:07:20 | sean-k-mooney | its possibe that intoduced a regression but we shoudl have functional test for this | |
| 13:07:48 | noonedeadpunk | Yah I read https://docs.openstack.org/nova/latest/admin/cpu-topologies.html quite carefully :) | |
| 13:08:00 | noonedeadpunk | I just didn't find any difference in there | |
| 13:08:21 | noonedeadpunk | but `hw:cpu_policy='dedicated'` doing correct thing | |
| 13:09:13 | noonedeadpunk | Another question - I do remember some changelog, that suport of live migration for pinned instances was added - was I dreaming or it's a thing? | |
| 13:09:40 | sean-k-mooney | its a thing i know its in wallaby for sure but i think it was before that even | |
| 13:09:45 | sean-k-mooney | so xena has it | |
| 13:10:24 | sean-k-mooney | our functional test are not actully asserting things about the xml at first glace | |
| 13:12:29 | noonedeadpunk | Should we submit bug then? | |
| 13:13:44 | sean-k-mooney | for the pinnign yes please do | |
| 13:13:59 | sean-k-mooney | i was just looking at some other integration tests we have | |
| 13:14:34 | sean-k-mooney | whitebox has test to assert the xml generateion https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_cpu_pinning.py#L94-L110 | |
| 13:14:51 | sean-k-mooney | but i think that is only testing the dedicated case not pcpus | |
| 13:15:52 | sean-k-mooney | thats actully just the helper function | |
| 13:15:58 | sean-k-mooney | https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_cpu_pinning.py#L143-L172 is one of the test | |
| 13:16:48 | sean-k-mooney | noonedeadpunk: i was going to try and create a func test to repoduce something else but ill see if i can trigger this too. | |
| 13:17:06 | noonedeadpunk | yeah, it's testing only dedicated one | |
| 13:18:25 | sean-k-mooney | thanks for binging this up. i normally only test dedicated personally since i dont like using pcpus | |
| 13:18:51 | sean-k-mooney | i suspect when others spot check this form time to time its a simlar story | |
| 13:20:13 | noonedeadpunk | I was just wondering if there's any difference, so decided to try both and was quite surprised about the result) | |
| 13:21:06 | sean-k-mooney | ya am one question its still emiting the vcpu top level element and listing the n host cpus in the cpuset that were allcoated to the vm right | |
| 13:21:47 | sean-k-mooney | libvirt will internally generate vcpupin element but the behavior it will have is to let the cpu of the guest float over the n cpus in the set | |
| 13:23:11 | noonedeadpunk | oh, ok, I see | |
| 13:23:25 | sean-k-mooney | so the vm shoudl still be pinned to a dedicated set of cores that other vms wont use but each guest cpu wont be be pinned to a speicic core but a set of cores | |
| 13:24:07 | sean-k-mooney | that will potentally still have latency issue if the host schduer decided to move the vcpu threads between the cores so not ideal | |
| 13:24:15 | sean-k-mooney | but not terible | |
| 13:24:33 | sean-k-mooney | would you mind providign an xml or sanatised one as part of the bug | |
| 13:24:40 | noonedeadpunk | sure thing | |
| 13:32:52 | noonedeadpunk | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1966452 | |
| 13:34:01 | sean-k-mooney | <vcpu placement="static">4</vcpu> so ya that will still pin it but if that vm had 2 cores for exampel that woudl then be a range or list of hosts | |
| 13:34:12 | sean-k-mooney | e.g. <vcpu placement="static">4,6</vcpu> | |
| 13:34:23 | sean-k-mooney | or <vcpu placement="static">4-5</vcpu> | |
| 13:34:57 | sean-k-mooney | once we fix this you coudl fix the vms with a hard reboot and or live migration | |
| 13:34:58 | noonedeadpunk | but vm should have 4 cores actually | |
| 13:35:17 | noonedeadpunk | thankfully, I don't have that in production yet:) | |
| 13:35:20 | sean-k-mooney | oh just saw that | |
| 13:35:26 | sean-k-mooney | <nova:vcpus>4</nova:vcpus> | |
| 13:35:32 | sean-k-mooney | ok this is worse then i taught | |
| 13:36:07 | sean-k-mooney | ok i was going to mark this medium but i think ill triage this as high instead | |
| 13:36:08 | noonedeadpunk | or well, we likely have it until EOD, but jsut use hw:cpu_policy | |
| 13:37:30 | sean-k-mooney | ya the workaroudn is to use hw:cpu_policy dedicated which is my perfered option anyway so i don think this need to be critical but its valid and a high impact to performance | |
| 13:38:51 | sean-k-mooney | i have devstack running currently on my home server so ill see if i can recreate with master and we can go from there | |
| 13:40:00 | sean-k-mooney | trait:CUSTOM_HIGHCPU='required are you useing provider.yaml to set that or the cli out of interest | |
| 13:43:13 | noonedeadpunk | CLI | |
| 13:43:39 | sean-k-mooney | ack i dont think we have many using provider.yaml yet so just was wondering if you were | |
| 13:47:05 | noonedeadpunk | nah, not yet:) We already had some traits set in environment, so were following usual path | |
| 15:02:51 | outbrito | G'day folks! | |
| 15:03:02 | outbrito | About the problem I had running tox the other day, I filed this bug: https://bugs.launchpad.net/nova/+bug/1966477 | |
| 15:03:54 | outbrito | Not very sure how to handle it properly though. Any thoughts? | |
| 15:04:09 | outbrito | I left some options to fix it on the comments | |
| 15:11:10 | sean-k-mooney | that looks like an osbrick issue | |
| 15:11:37 | sean-k-mooney | the unit and func test shoudl not need iscsi | |
| 15:12:20 | sean-k-mooney | so we also shoudl be mocking any calls to os-brick in our unit tests | |
| 15:12:38 | outbrito | I believe theirs don't, but we do mock when we set that global | |
| 15:13:23 | outbrito | https://github.com/openstack/nova/blob/8f250f50446ca2d7aa84609d5144088aa4cded78/nova/test.py#L203 | |
| 15:13:38 | sean-k-mooney | my point wa that this code shoudl nto be running in install or during the tests | |
| 15:13:46 | outbrito | not iscsiadmin anyway, but the lock | |
| 15:14:54 | sean-k-mooney | so none fo this shoudl depend on the os you are on | |
| 16:20:05 | mloza | I know this isn't supported but I wanted to add server group to running VMs, I found `instance_group_member` table in nova_api needs to be modified. Any other tables I missed? | |
| 16:21:28 | mloza | also, is there any reason server group can't be added to running VMs? | |
| 16:34:51 | sean-k-mooney | noonedeadpunk: actully thinking about your pinning bug again i dont think that config enabeld pinning at all | |
| 16:35:13 | sean-k-mooney | that is asking for 4 floating cores | |
| 16:35:28 | sean-k-mooney | it does not have any cores specified in the cpuset atirbute | |
| 16:36:00 | sean-k-mooney | <vcpu placement='static' cpuset="1-4,^3,6" current="1">2</vcpu> is the alternitive way to pin | |
| 16:36:37 | sean-k-mooney | <vcpu placement="static">4</vcpu> just mean 4 cpus | |
| 16:36:49 | dansmith | mloza: we don't do it because of the affinity policies, if we just let you add them you could violate those, and we're not going to move things around to keep the policies straight | |
| 16:37:04 | sean-k-mooney | so ya resouces:PCPUs is totally broken | |
| 16:40:25 | opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Move implemented specs for the Yoga release https://review.opendev.org/c/openstack/nova-specs/+/835272 | |
| 16:47:14 | noonedeadpunk | sean-k-mooney: I just noticed a typo in flavor :( | |
| 16:48:02 | noonedeadpunk | So I just wasted your time likely | |
| 17:05:15 | sean-k-mooney | i havent spendt much on it yet currently debuging downstream issue and devstack problems | |
| 17:05:32 | sean-k-mooney | i was just looking at a different xml an realise i read your one wrong the first time | |
| 17:06:03 | sean-k-mooney | noonedeadpunk: if there isnt an issue great but we found we are missing coverage in whitebox in either case | |
| 17:16:13 | sean-k-mooney | noonedeadpunk: if the bug is not required feel free to close it but if its still does not work after you fix the typo let me know | |
| 17:17:45 | sean-k-mooney | oh s/resouces/resources/ | |
| 17:17:50 | sean-k-mooney | ya i would never spot that | |
| 21:24:06 | opendevreview | Merged openstack/nova master: Adds regression test for bug LP#1944619 https://review.opendev.org/c/openstack/nova/+/833166 | |
| #openstack-nova - 2022-03-26 | |||
| 03:03:27 | opendevreview | Ghanshyam proposed openstack/nova stable/yoga: DNM: test grenade skip job running https://review.opendev.org/c/openstack/nova/+/835350 | |
| 03:04:28 | opendevreview | Ghanshyam proposed openstack/nova master: DNM: test grenade skip job running https://review.opendev.org/c/openstack/nova/+/835351 | |
| 14:33:43 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/python-novaclient/+/835368 | |
| 14:43:23 | opendevreview | Takashi Natsume proposed openstack/osc-placement master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/osc-placement/+/835369 | |
| 15:06:19 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Add openssl in bindep.txt https://review.opendev.org/c/openstack/python-novaclient/+/835370 | |
| 15:10:12 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Add openssl in bindep.txt https://review.opendev.org/c/openstack/python-novaclient/+/835370 | |
| #openstack-nova - 2022-03-28 | |||
| 03:01:50 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Remove USE_PYTHON3 in .zuul.yaml https://review.opendev.org/c/openstack/python-novaclient/+/835405 | |
| 12:12:39 | viks__ | i'm facing the issue "Affinity policy violated with parallel requests".. i saw the resolution give in https://docs.openstack.org/nova/latest/admin/troubleshooting/affinity-policy-violated.html. I have tried setting `disable_group_policy_check_upcall`, but it does not seem to hve any effect.. Can someone please guide, why the solution provide there does not work? | |
| 12:14:10 | sean-k-mooney | so you have set disable_group_policy_check_upcall = False | |