| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-03-25 | |||
| 13:10:24 | sean-k-mooney | our functional test are not actully asserting things about the xml at first glace | |
| 13:12:29 | noonedeadpunk | Should we submit bug then? | |
| 13:13:44 | sean-k-mooney | for the pinnign yes please do | |
| 13:13:59 | sean-k-mooney | i was just looking at some other integration tests we have | |
| 13:14:34 | sean-k-mooney | whitebox has test to assert the xml generateion https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_cpu_pinning.py#L94-L110 | |
| 13:14:51 | sean-k-mooney | but i think that is only testing the dedicated case not pcpus | |
| 13:15:52 | sean-k-mooney | thats actully just the helper function | |
| 13:15:58 | sean-k-mooney | https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_cpu_pinning.py#L143-L172 is one of the test | |
| 13:16:48 | sean-k-mooney | noonedeadpunk: i was going to try and create a func test to repoduce something else but ill see if i can trigger this too. | |
| 13:17:06 | noonedeadpunk | yeah, it's testing only dedicated one | |
| 13:18:25 | sean-k-mooney | thanks for binging this up. i normally only test dedicated personally since i dont like using pcpus | |
| 13:18:51 | sean-k-mooney | i suspect when others spot check this form time to time its a simlar story | |
| 13:20:13 | noonedeadpunk | I was just wondering if there's any difference, so decided to try both and was quite surprised about the result) | |
| 13:21:06 | sean-k-mooney | ya am one question its still emiting the vcpu top level element and listing the n host cpus in the cpuset that were allcoated to the vm right | |
| 13:21:47 | sean-k-mooney | libvirt will internally generate vcpupin element but the behavior it will have is to let the cpu of the guest float over the n cpus in the set | |
| 13:23:11 | noonedeadpunk | oh, ok, I see | |
| 13:23:25 | sean-k-mooney | so the vm shoudl still be pinned to a dedicated set of cores that other vms wont use but each guest cpu wont be be pinned to a speicic core but a set of cores | |
| 13:24:07 | sean-k-mooney | that will potentally still have latency issue if the host schduer decided to move the vcpu threads between the cores so not ideal | |
| 13:24:15 | sean-k-mooney | but not terible | |
| 13:24:33 | sean-k-mooney | would you mind providign an xml or sanatised one as part of the bug | |
| 13:24:40 | noonedeadpunk | sure thing | |
| 13:32:52 | noonedeadpunk | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1966452 | |
| 13:34:01 | sean-k-mooney | <vcpu placement="static">4</vcpu> so ya that will still pin it but if that vm had 2 cores for exampel that woudl then be a range or list of hosts | |
| 13:34:12 | sean-k-mooney | e.g. <vcpu placement="static">4,6</vcpu> | |
| 13:34:23 | sean-k-mooney | or <vcpu placement="static">4-5</vcpu> | |
| 13:34:57 | sean-k-mooney | once we fix this you coudl fix the vms with a hard reboot and or live migration | |
| 13:34:58 | noonedeadpunk | but vm should have 4 cores actually | |
| 13:35:17 | noonedeadpunk | thankfully, I don't have that in production yet:) | |
| 13:35:20 | sean-k-mooney | oh just saw that | |
| 13:35:26 | sean-k-mooney | <nova:vcpus>4</nova:vcpus> | |
| 13:35:32 | sean-k-mooney | ok this is worse then i taught | |
| 13:36:07 | sean-k-mooney | ok i was going to mark this medium but i think ill triage this as high instead | |
| 13:36:08 | noonedeadpunk | or well, we likely have it until EOD, but jsut use hw:cpu_policy | |
| 13:37:30 | sean-k-mooney | ya the workaroudn is to use hw:cpu_policy dedicated which is my perfered option anyway so i don think this need to be critical but its valid and a high impact to performance | |
| 13:38:51 | sean-k-mooney | i have devstack running currently on my home server so ill see if i can recreate with master and we can go from there | |
| 13:40:00 | sean-k-mooney | trait:CUSTOM_HIGHCPU='required are you useing provider.yaml to set that or the cli out of interest | |
| 13:43:13 | noonedeadpunk | CLI | |
| 13:43:39 | sean-k-mooney | ack i dont think we have many using provider.yaml yet so just was wondering if you were | |
| 13:47:05 | noonedeadpunk | nah, not yet:) We already had some traits set in environment, so were following usual path | |
| 15:02:51 | outbrito | G'day folks! | |
| 15:03:02 | outbrito | About the problem I had running tox the other day, I filed this bug: https://bugs.launchpad.net/nova/+bug/1966477 | |
| 15:03:54 | outbrito | Not very sure how to handle it properly though. Any thoughts? | |
| 15:04:09 | outbrito | I left some options to fix it on the comments | |
| 15:11:10 | sean-k-mooney | that looks like an osbrick issue | |
| 15:11:37 | sean-k-mooney | the unit and func test shoudl not need iscsi | |
| 15:12:20 | sean-k-mooney | so we also shoudl be mocking any calls to os-brick in our unit tests | |
| 15:12:38 | outbrito | I believe theirs don't, but we do mock when we set that global | |
| 15:13:23 | outbrito | https://github.com/openstack/nova/blob/8f250f50446ca2d7aa84609d5144088aa4cded78/nova/test.py#L203 | |
| 15:13:38 | sean-k-mooney | my point wa that this code shoudl nto be running in install or during the tests | |
| 15:13:46 | outbrito | not iscsiadmin anyway, but the lock | |
| 15:14:54 | sean-k-mooney | so none fo this shoudl depend on the os you are on | |
| 16:20:05 | mloza | I know this isn't supported but I wanted to add server group to running VMs, I found `instance_group_member` table in nova_api needs to be modified. Any other tables I missed? | |
| 16:21:28 | mloza | also, is there any reason server group can't be added to running VMs? | |
| 16:34:51 | sean-k-mooney | noonedeadpunk: actully thinking about your pinning bug again i dont think that config enabeld pinning at all | |
| 16:35:13 | sean-k-mooney | that is asking for 4 floating cores | |
| 16:35:28 | sean-k-mooney | it does not have any cores specified in the cpuset atirbute | |
| 16:36:00 | sean-k-mooney | <vcpu placement='static' cpuset="1-4,^3,6" current="1">2</vcpu> is the alternitive way to pin | |
| 16:36:37 | sean-k-mooney | <vcpu placement="static">4</vcpu> just mean 4 cpus | |
| 16:36:49 | dansmith | mloza: we don't do it because of the affinity policies, if we just let you add them you could violate those, and we're not going to move things around to keep the policies straight | |
| 16:37:04 | sean-k-mooney | so ya resouces:PCPUs is totally broken | |
| 16:40:25 | opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Move implemented specs for the Yoga release https://review.opendev.org/c/openstack/nova-specs/+/835272 | |
| 16:47:14 | noonedeadpunk | sean-k-mooney: I just noticed a typo in flavor :( | |
| 16:48:02 | noonedeadpunk | So I just wasted your time likely | |
| 17:05:15 | sean-k-mooney | i havent spendt much on it yet currently debuging downstream issue and devstack problems | |
| 17:05:32 | sean-k-mooney | i was just looking at a different xml an realise i read your one wrong the first time | |
| 17:06:03 | sean-k-mooney | noonedeadpunk: if there isnt an issue great but we found we are missing coverage in whitebox in either case | |
| 17:16:13 | sean-k-mooney | noonedeadpunk: if the bug is not required feel free to close it but if its still does not work after you fix the typo let me know | |
| 17:17:45 | sean-k-mooney | oh s/resouces/resources/ | |
| 17:17:50 | sean-k-mooney | ya i would never spot that | |
| 21:24:06 | opendevreview | Merged openstack/nova master: Adds regression test for bug LP#1944619 https://review.opendev.org/c/openstack/nova/+/833166 | |
| #openstack-nova - 2022-03-26 | |||
| 03:03:27 | opendevreview | Ghanshyam proposed openstack/nova stable/yoga: DNM: test grenade skip job running https://review.opendev.org/c/openstack/nova/+/835350 | |
| 03:04:28 | opendevreview | Ghanshyam proposed openstack/nova master: DNM: test grenade skip job running https://review.opendev.org/c/openstack/nova/+/835351 | |
| 14:33:43 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/python-novaclient/+/835368 | |
| 14:43:23 | opendevreview | Takashi Natsume proposed openstack/osc-placement master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/osc-placement/+/835369 | |
| 15:06:19 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Add openssl in bindep.txt https://review.opendev.org/c/openstack/python-novaclient/+/835370 | |
| 15:10:12 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Add openssl in bindep.txt https://review.opendev.org/c/openstack/python-novaclient/+/835370 | |
| #openstack-nova - 2022-03-28 | |||
| 03:01:50 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Remove USE_PYTHON3 in .zuul.yaml https://review.opendev.org/c/openstack/python-novaclient/+/835405 | |
| 12:12:39 | viks__ | i'm facing the issue "Affinity policy violated with parallel requests".. i saw the resolution give in https://docs.openstack.org/nova/latest/admin/troubleshooting/affinity-policy-violated.html. I have tried setting `disable_group_policy_check_upcall`, but it does not seem to hve any effect.. Can someone please guide, why the solution provide there does not work? | |
| 12:14:10 | sean-k-mooney | so you have set disable_group_policy_check_upcall = False | |
| 12:14:19 | sean-k-mooney | on the compute nodes | |
| 12:14:33 | sean-k-mooney | and its still allowing the policy to be violated | |
| 12:17:51 | viks__ | yes... basically the instances are going in to same host | |
| 12:19:24 | viks__ | i'm using train version | |
| 12:20:05 | sean-k-mooney | and you have ensured the cell conductor has ascess to the api db | |
| 12:20:34 | sean-k-mooney | the way this up call works is one of the vm should boot but then the rest should fail and reschdule to other hosts | |
| 12:24:43 | viks__ | sean-k-mooney: `[api_database] connection=`?. yes i have set this | |
| 12:25:20 | sean-k-mooney | in the conductors? if so then the second vm shoudl fail to boot. | |
| 12:25:31 | sean-k-mooney | have you eneabled debug loging | |
| 12:25:44 | sean-k-mooney | do you see any references to upcalls in teh compute-agent log | |
| 12:34:52 | viks__ | sean-k-mooney: yes... in conductors..will check that | |
| 14:46:15 | dansmith | can someone +W this test fixture fix? https://review.opendev.org/c/openstack/nova/+/831084 | |
| 14:46:27 | dansmith | apparently it will reduce some CI failures | |
| 14:46:44 | sean-k-mooney | give me 2 mins and ill take a look | |
| 15:02:39 | bauzas | dansmith: done | |
| 15:04:36 | bauzas | cores : https://review.opendev.org/c/openstack/nova-specs/+/835272 (moving specs to implemented) | |
| 15:28:38 | opendevreview | Merged openstack/nova master: Retry in CellDatabases fixture when global DB state changes https://review.opendev.org/c/openstack/nova/+/831084 | |
| 15:51:20 | opendevreview | Erlon R. Cruz proposed openstack/nova master: Fix pre_live_migration rollback https://review.opendev.org/c/openstack/nova/+/815324 | |
| 15:57:32 | bauzas | folks, I know it's late but I'm asked by the Foundation folks to present the Nova Yoga higlights on Thurs | |
| 15:57:52 | bauzas | so I'm writing two slides, one for the yoga meat and one for the Zed plans we have | |
| 15:58:00 | bauzas | obviously the Zed slide is the beefy one | |