Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-03-25
13:04:54 noonedeadpunk Xena
13:05:11 sean-k-mooney i wonder if the mixed cpu support regressed that
13:06:07 noonedeadpunk maybe I have some mistake in flavor, let me double-check
13:06:56 sean-k-mooney if you have resouces:PCPU it shoudl be all you need although i dont recommend that people use the resouces: syntax in general
13:07:07 sean-k-mooney https://github.com/openstack/nova-specs/blob/master/specs/victoria/implemented/use-pcpu-vcpu-in-one-instance.rst was implemented in victoria
13:07:20 sean-k-mooney its possibe that intoduced a regression but we shoudl have functional test for this
13:07:48 noonedeadpunk Yah I read https://docs.openstack.org/nova/latest/admin/cpu-topologies.html quite carefully :)
13:08:00 noonedeadpunk I just didn't find any difference in there
13:08:21 noonedeadpunk but `hw:cpu_policy='dedicated'` doing correct thing
13:09:13 noonedeadpunk Another question - I do remember some changelog, that suport of live migration for pinned instances was added - was I dreaming or it's a thing?
13:09:40 sean-k-mooney its a thing i know its in wallaby for sure but i think it was before that even
13:09:45 sean-k-mooney so xena has it
13:10:24 sean-k-mooney our functional test are not actully asserting things about the xml at first glace
13:12:29 noonedeadpunk Should we submit bug then?
13:13:44 sean-k-mooney for the pinnign yes please do
13:13:59 sean-k-mooney i was just looking at some other integration tests we have
13:14:34 sean-k-mooney whitebox has test to assert the xml generateion https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_cpu_pinning.py#L94-L110
13:14:51 sean-k-mooney but i think that is only testing the dedicated case not pcpus
13:15:52 sean-k-mooney thats actully just the helper function
13:15:58 sean-k-mooney https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/whitebox_tempest_plugin/api/compute/test_cpu_pinning.py#L143-L172 is one of the test
13:16:48 sean-k-mooney noonedeadpunk: i was going to try and create a func test to repoduce something else but ill see if i can trigger this too.
13:17:06 noonedeadpunk yeah, it's testing only dedicated one
13:18:25 sean-k-mooney thanks for binging this up. i normally only test dedicated personally since i dont like using pcpus
13:18:51 sean-k-mooney i suspect when others spot check this form time to time its a simlar story
13:20:13 noonedeadpunk I was just wondering if there's any difference, so decided to try both and was quite surprised about the result)
13:21:06 sean-k-mooney ya am one question its still emiting the vcpu top level element and listing the n host cpus in the cpuset that were allcoated to the vm right
13:21:47 sean-k-mooney libvirt will internally generate vcpupin element but the behavior it will have is to let the cpu of the guest float over the n cpus in the set
13:23:11 noonedeadpunk oh, ok, I see
13:23:25 sean-k-mooney so the vm shoudl still be pinned to a dedicated set of cores that other vms wont use but each guest cpu wont be be pinned to a speicic core but a set of cores
13:24:07 sean-k-mooney that will potentally still have latency issue if the host schduer decided to move the vcpu threads between the cores so not ideal
13:24:15 sean-k-mooney but not terible
13:24:33 sean-k-mooney would you mind providign an xml or sanatised one as part of the bug
13:24:40 noonedeadpunk sure thing
13:32:52 noonedeadpunk sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1966452
13:34:01 sean-k-mooney <vcpu placement="static">4</vcpu> so ya that will still pin it but if that vm had 2 cores for exampel that woudl then be a range or list of hosts
13:34:12 sean-k-mooney e.g. <vcpu placement="static">4,6</vcpu>
13:34:23 sean-k-mooney or <vcpu placement="static">4-5</vcpu>
13:34:57 sean-k-mooney once we fix this you coudl fix the vms with a hard reboot and or live migration
13:34:58 noonedeadpunk but vm should have 4 cores actually
13:35:17 noonedeadpunk thankfully, I don't have that in production yet:)
13:35:20 sean-k-mooney oh just saw that
13:35:26 sean-k-mooney <nova:vcpus>4</nova:vcpus>
13:35:32 sean-k-mooney ok this is worse then i taught
13:36:07 sean-k-mooney ok i was going to mark this medium but i think ill triage this as high instead
13:36:08 noonedeadpunk or well, we likely have it until EOD, but jsut use hw:cpu_policy
13:37:30 sean-k-mooney ya the workaroudn is to use hw:cpu_policy dedicated which is my perfered option anyway so i don think this need to be critical but its valid and a high impact to performance
13:38:51 sean-k-mooney i have devstack running currently on my home server so ill see if i can recreate with master and we can go from there
13:40:00 sean-k-mooney trait:CUSTOM_HIGHCPU='required are you useing provider.yaml to set that or the cli out of interest
13:43:13 noonedeadpunk CLI
13:43:39 sean-k-mooney ack i dont think we have many using provider.yaml yet so just was wondering if you were
13:47:05 noonedeadpunk nah, not yet:) We already had some traits set in environment, so were following usual path
15:02:51 outbrito G'day folks!
15:03:02 outbrito About the problem I had running tox the other day, I filed this bug: https://bugs.launchpad.net/nova/+bug/1966477
15:03:54 outbrito Not very sure how to handle it properly though. Any thoughts?
15:04:09 outbrito I left some options to fix it on the comments
15:11:10 sean-k-mooney that looks like an osbrick issue
15:11:37 sean-k-mooney the unit and func test shoudl not need iscsi
15:12:20 sean-k-mooney so we also shoudl be mocking any calls to os-brick in our unit tests
15:12:38 outbrito I believe theirs don't, but we do mock when we set that global
15:13:23 outbrito https://github.com/openstack/nova/blob/8f250f50446ca2d7aa84609d5144088aa4cded78/nova/test.py#L203
15:13:38 sean-k-mooney my point wa that this code shoudl nto be running in install or during the tests
15:13:46 outbrito not iscsiadmin anyway, but the lock
15:14:54 sean-k-mooney so none fo this shoudl depend on the os you are on
16:20:05 mloza I know this isn't supported but I wanted to add server group to running VMs, I found `instance_group_member` table in nova_api needs to be modified. Any other tables I missed?
16:21:28 mloza also, is there any reason server group can't be added to running VMs?
16:34:51 sean-k-mooney noonedeadpunk: actully thinking about your pinning bug again i dont think that config enabeld pinning at all
16:35:13 sean-k-mooney that is asking for 4 floating cores
16:35:28 sean-k-mooney it does not have any cores specified in the cpuset atirbute
16:36:00 sean-k-mooney <vcpu placement='static' cpuset="1-4,^3,6" current="1">2</vcpu> is the alternitive way to pin
16:36:37 sean-k-mooney <vcpu placement="static">4</vcpu> just mean 4 cpus
16:36:49 dansmith mloza: we don't do it because of the affinity policies, if we just let you add them you could violate those, and we're not going to move things around to keep the policies straight
16:37:04 sean-k-mooney so ya resouces:PCPUs is totally broken
16:40:25 opendevreview Sylvain Bauza proposed openstack/nova-specs master: Move implemented specs for the Yoga release https://review.opendev.org/c/openstack/nova-specs/+/835272
16:47:14 noonedeadpunk sean-k-mooney: I just noticed a typo in flavor :(
16:48:02 noonedeadpunk So I just wasted your time likely
17:05:15 sean-k-mooney i havent spendt much on it yet currently debuging downstream issue and devstack problems
17:05:32 sean-k-mooney i was just looking at a different xml an realise i read your one wrong the first time
17:06:03 sean-k-mooney noonedeadpunk: if there isnt an issue great but we found we are missing coverage in whitebox in either case
17:16:13 sean-k-mooney noonedeadpunk: if the bug is not required feel free to close it but if its still does not work after you fix the typo let me know
17:17:45 sean-k-mooney oh s/resouces/resources/
17:17:50 sean-k-mooney ya i would never spot that
21:24:06 opendevreview Merged openstack/nova master: Adds regression test for bug LP#1944619 https://review.opendev.org/c/openstack/nova/+/833166
#openstack-nova - 2022-03-26
03:03:27 opendevreview Ghanshyam proposed openstack/nova stable/yoga: DNM: test grenade skip job running https://review.opendev.org/c/openstack/nova/+/835350
03:04:28 opendevreview Ghanshyam proposed openstack/nova master: DNM: test grenade skip job running https://review.opendev.org/c/openstack/nova/+/835351
14:33:43 opendevreview Takashi Natsume proposed openstack/python-novaclient master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/python-novaclient/+/835368
14:43:23 opendevreview Takashi Natsume proposed openstack/osc-placement master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/osc-placement/+/835369
15:06:19 opendevreview Takashi Natsume proposed openstack/python-novaclient master: Add openssl in bindep.txt https://review.opendev.org/c/openstack/python-novaclient/+/835370
15:10:12 opendevreview Takashi Natsume proposed openstack/python-novaclient master: Add openssl in bindep.txt https://review.opendev.org/c/openstack/python-novaclient/+/835370
#openstack-nova - 2022-03-28
03:01:50 opendevreview Takashi Natsume proposed openstack/python-novaclient master: Remove USE_PYTHON3 in .zuul.yaml https://review.opendev.org/c/openstack/python-novaclient/+/835405
12:12:39 viks__ i'm facing the issue "Affinity policy violated with parallel requests".. i saw the resolution give in https://docs.openstack.org/nova/latest/admin/troubleshooting/affinity-policy-violated.html. I have tried setting `disable_group_policy_check_upcall`, but it does not seem to hve any effect.. Can someone please guide, why the solution provide there does not work?
12:14:10 sean-k-mooney so you have set disable_group_policy_check_upcall = False
12:14:19 sean-k-mooney on the compute nodes
12:14:33 sean-k-mooney and its still allowing the policy to be violated
12:17:51 viks__ yes... basically the instances are going in to same host
12:19:24 viks__ i'm using train version
12:20:05 sean-k-mooney and you have ensured the cell conductor has ascess to the api db
12:20:34 sean-k-mooney the way this up call works is one of the vm should boot but then the rest should fail and reschdule to other hosts
12:24:43 viks__ sean-k-mooney: `[api_database] connection=`?. yes i have set this
12:25:20 sean-k-mooney in the conductors? if so then the second vm shoudl fail to boot.
12:25:31 sean-k-mooney have you eneabled debug loging

Earlier   Later