| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-09-14 | |||
| 12:59:23 | elodilles | bauzas: ACK. you can -1 the patch until it'll be updated: https://review.opendev.org/c/openstack/releases/+/894693 | |
| 12:59:52 | bauzas | oh missed it from my mails | |
| 13:00:01 | mgariepy | fails at running : _filter_pools_for_unrequested_pfs() | |
| 13:00:04 | bauzas | I was wondering why I wasn't seen it | |
| 13:00:44 | mgariepy | dev_type vs device_type ? | |
| 13:00:52 | mgariepy | or it's abstracted somewhere ? | |
| 13:01:18 | sean-k-mooney | its device_type in the alias | |
| 13:01:44 | ralonsoh | bauzas, yes, I'm going to mark this test as unstable again | |
| 13:01:50 | sean-k-mooney | https://github.com/openstack/nova/blob/stable/2023.1/nova/pci/request.py#L95C10-L104 that is the jsonschma fo rthe validation | |
| 13:02:00 | ralonsoh | we need to find a better way to set the subport as active after the live migration | |
| 13:02:01 | sean-k-mooney | but i think its dev type in the object | |
| 13:02:27 | mgariepy | let me logs the request and pool. to see if something seems odds. | |
| 13:02:38 | sean-k-mooney | https://github.com/openstack/nova/blob/stable/2023.1/nova/pci/request.py#L143C13-L145 | |
| 13:02:51 | bauzas | ralonsoh: ack | |
| 13:02:58 | sean-k-mooney | mgariepy: i would check the pci_devices table in teh cell db | |
| 13:03:13 | sean-k-mooney | and make sure the device has the expected type ectra | |
| 13:04:19 | mgariepy | nova.pci_devices tells me that the device is type-PF | |
| 13:05:02 | ralonsoh | bauzas, https://review.opendev.org/c/openstack/tempest/+/895167 | |
| 13:09:59 | sean-k-mooney | mgariepy: what does the pci whitelist look like for that host | |
| 13:11:09 | sean-k-mooney | based on the alias above it looks co unless you added physnet or remote managed or something im not sure why it would be remvoed | |
| 13:11:50 | sean-k-mooney | its the pci filter not the numa filter that is removing it right? | |
| 13:11:52 | mgariepy | https://paste.openstack.org/show/bij0SgRpSEVCTILyzIzf/ | |
| 13:12:10 | mgariepy | seems to be the pci filter | |
| 13:13:02 | sean-k-mooney | hum that all looks correct to me | |
| 13:14:15 | mgariepy | https://paste.openstack.org/show/bP2oY2IOe4lNvj6wGB8w/ | |
| 13:14:51 | mgariepy | what is resolving the alias ? is it dont by the scheduler directly ? | |
| 13:15:27 | sean-k-mooney | that depends on the request | |
| 13:15:47 | mgariepy | my flavor has alias stuff in in. | |
| 13:15:49 | sean-k-mooney | but no i think its doen in the compute or the api | |
| 13:16:53 | sean-k-mooney | https://docs.openstack.org/nova/latest/admin/pci-passthrough.html#configure-nova-api | |
| 13:17:07 | sean-k-mooney | so for ne vm request this is usd by the nova-api not the scheduler | |
| 13:17:30 | sean-k-mooney | and for rezize its used form teh compute | |
| 13:17:45 | sean-k-mooney | so if you are udataing this you need to restart the nova-api not the scheduler | |
| 13:18:26 | sean-k-mooney | the scheudle reciveds a fully popultated RequestSepc object which has the pci resuets embeeded in it | |
| 13:19:38 | mgariepy | but me resquest doesn't receive the device_type :/ | |
| 13:23:12 | sean-k-mooney | did you restart the nova-api processes after updating hte alias | |
| 13:24:02 | mgariepy | i did restart the nova slice. let me try juste putting the same config accross all the cluster to see if it changes something. | |
| 13:32:19 | mgariepy | what driver shoud be loaded fo the pci dev ? | |
| 13:32:56 | mgariepy | pci-stub or vfio-pci ? | |
| 13:32:58 | sean-k-mooney | the driver should not matter to the schduler | |
| 13:33:13 | sean-k-mooney | that woudl only matter on the compute node but yes either of those shoudl work | |
| 13:33:18 | mgariepy | ok | |
| 13:33:36 | sean-k-mooney | the important part is that the framebuffer is not initallised by xorg/wayland as that woudl prevent the passthough | |
| 13:33:53 | mgariepy | ha ok. | |
| 13:34:13 | sean-k-mooney | vfio-pci is what kvm/qemu will use when its being passthough through | |
| 13:34:26 | sean-k-mooney | so prebinding to that is generally not a bad idea but often not required | |
| 13:34:46 | mgariepy | i did some stuff with passthrough with el-cheapo gamer gpus. but with older openstack release. | |
| 13:35:50 | sean-k-mooney | thre is nothing really that has changed that would impact this | |
| 13:36:06 | sean-k-mooney | the alias and whitelist/device list syntax is the same | |
| 13:36:22 | sean-k-mooney | and your reporting that its fialing in the pci filter os its not related to the placment part | |
| 13:36:31 | sean-k-mooney | since it passes that to get to the point where its failing | |
| 13:37:09 | mgariepy | yeah i'll retest in like 30 minutes.. meeting time ;) | |
| 15:04:23 | mgariepy | sean-k-mooney, it works now Thanks a log for your help. | |
| 15:04:47 | mgariepy | with the type-PF and all my server with the same config it's almost magical :D | |
| 15:21:51 | sean-k-mooney | awsome | |
| 15:21:52 | opendevreview | Sylvain Bauza proposed openstack/placement master: Update 2023.2 reqs to support os-traits 3.0.0 as min version https://review.opendev.org/c/openstack/placement/+/895186 | |
| 15:22:21 | sean-k-mooney | the reqeust was proably being accpeted by a not restarted instace | |
| 15:22:29 | mgariepy | yeah | |
| 15:23:01 | bauzas | dansmith: sean-k-mooney: forgot to propose the requirements update for the new traits : https://review.opendev.org/c/openstack/placement/+/895186 | |
| 15:23:05 | mgariepy | any tips on debugging that sort of stuff when multiple instance of every service are running ? | |
| 15:23:08 | bauzas | we need it before RC1 | |
| 15:23:39 | dansmith | bauzas: aren't we past requirements freeze? | |
| 15:23:46 | dansmith | is that in u-c already? | |
| 15:24:41 | bauzas | dansmith: os-traits was delivered in March | |
| 15:24:46 | bauzas | and u-c already has it | |
| 15:24:52 | dansmith | okay | |
| 15:24:56 | bauzas | https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt | |
| 15:25:13 | dansmith | ack, cool | |
| 15:25:20 | bauzas | so we just need to say 'look, for Bobcat, we need to accept only 3.0 since we use a trait in it" | |
| 15:25:40 | bauzas | I haven't checked tho whether we really use this trait in Placement but I'm assuming it | |
| 15:26:11 | dansmith | bauzas: wait you're not sure if we need the new traits? :) | |
| 15:26:58 | dansmith | we just bumped for this a few weeks ago: https://review.opendev.org/c/openstack/nova/+/873221 | |
| 15:38:38 | opendevreview | Gorka Eguileor proposed openstack/nova master: Fix load_validators https://review.opendev.org/c/openstack/nova/+/895189 | |
| 15:39:12 | opendevreview | Gorka Eguileor proposed openstack/nova master: Logs cinderclient requests when debugging https://review.opendev.org/c/openstack/nova/+/895191 | |
| 15:39:12 | opendevreview | Gorka Eguileor proposed openstack/nova master: Fix debug options https://review.opendev.org/c/openstack/nova/+/895190 | |
| 15:39:40 | opendevreview | Gorka Eguileor proposed openstack/nova master: Fix guard for NVMeOF volumes https://review.opendev.org/c/openstack/nova/+/895192 | |
| 15:53:20 | bauzas | dansmith: so, I made a few researches, my bad, just followed blindly a pattern from Antelope when I did https://review.opendev.org/c/openstack/placement/+/874080 | |
| 15:54:09 | bauzas | so, https://review.opendev.org/c/openstack/nova/+/873221 is using traits-2.10 https://review.opendev.org/c/openstack/releases/+/873106 which is already supported as minimum | |
| 16:10:55 | bauzas | dansmith: so, I think I'll abandon https://review.opendev.org/c/openstack/placement/+/895186 | |
| 16:11:08 | dansmith | bauzas: okay | |
| 16:12:28 | bauzas | I'm actually lost, we created 3.0 in order to no longer support py2.6 and py2.7 | |
| 16:12:31 | sean-k-mooney | bauzas: so https://review.opendev.org/c/openstack/nova/+/831194/37/nova/share/manila.py#43 is returning the connection to via the sdk to talke to manilla | |
| 16:12:59 | bauzas | sean-k-mooney: sec, trying to see whether we need https://review.opendev.org/c/openstack/releases/+/894698 | |
| 16:13:06 | sean-k-mooney | its fine | |
| 16:13:20 | sean-k-mooney | focus on that i was just wonderign where the error was | |
| 16:13:54 | bauzas | sorry I meant https://review.opendev.org/c/openstack/placement/+/895186 | |
| 16:14:26 | bauzas | we'll add a new trait which isn't used yet by Nova | |
| 16:14:44 | bauzas | but we'll also make py3.6 and py3.7 unsupported | |
| 16:20:03 | bauzas | dansmith: sean-k-mooney: okay, I think I sorted it, we eventually need https://review.opendev.org/c/openstack/placement/+/895186 not because of the new trait, but because we want to unsupport 3.6 and 3.7 | |
| 16:20:09 | bauzas | in Bobcat | |
| 16:20:14 | dansmith | okay | |
| 16:20:38 | sean-k-mooney | ok so i should +2w that then right | |
| 16:20:43 | sean-k-mooney | the min version bump to 3.0.0 | |
| 16:24:35 | bauzas | sean-k-mooney: sorry, yeah | |
| 16:24:47 | bauzas | thanks | |
| 16:39:31 | sean-k-mooney | mgariepy: the only real tip is to use the request-id to corralate the requets across diffent services when looking at the logs | |
| 16:40:03 | sean-k-mooney | that and try and use automation like kolla-ansible to ensure you have the same config on all relevent nodes rather then doing manual changes | |
| 16:40:27 | sean-k-mooney | but no not really other then that | |
| 17:03:15 | mgariepy | sean-k-mooney, i'm using openstack-ansible | |
| 17:03:56 | mgariepy | when doing changes that i'm not quite sure i do it manually instead of doing it via osa. | |