| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-08-28 | |||
| 14:24:52 | sean-k-mooney | https://libvirt.org/formatdomain.html#vhost-user-interface | |
| 14:26:30 | sean-k-mooney | so if qemu is runing in client mode and you restart ovs to do a package upgrae it will break network connectivnty for all guests until you hard reboot them | |
| 14:26:38 | sean-k-mooney | have you tested this in an openstack envionment? | |
| 14:26:46 | dvo-plv | this is qemu command, -chardev socket,id=char0,path=/usr/local/var/run/stdvio6,server we set the server on the qemu side | |
| 14:27:16 | sean-k-mooney | ok then qemu is runing in server mode whch means qemu is creating the socket not dpdk | |
| 14:27:29 | sean-k-mooney | dpdk is runing in client mode and connecting to the socekt create by qemu | |
| 14:27:44 | sean-k-mooney | that is the way we recommend you use vhost user | |
| 14:28:39 | sean-k-mooney | what im suggetign is we add the path to the dpdk args when os-vif creates the interface in ovs | |
| 14:28:52 | sean-k-mooney | and the driver shoudl use that instead of trying to calulate it | |
| 14:29:36 | dvo-plv | yes, we formulate socket name by pci address according to the formula, because we reserve 4 first pcis for physical ports | |
| 14:30:24 | sean-k-mooney | yep but your not actully passing the vf number to dpdk in teh representor arg | |
| 14:30:40 | sean-k-mooney | what you are pasing is the intger offset to add to the pf adress | |
| 14:31:58 | sean-k-mooney | that is why your expect virtfn5 -> ../0000:2b:01.1/ to be 9 | |
| 14:32:40 | dvo-plv | yes, and this offset can not give us ability to work with sysfs | |
| 14:32:51 | dvo-plv | + socket name like stdvio + vf_num | |
| 14:33:05 | sean-k-mooney | that not the vf_num | |
| 14:33:13 | sean-k-mooney | its the pci endpoint number | |
| 14:33:21 | sean-k-mooney | its a diffent thing | |
| 14:33:29 | dvo-plv | sure | |
| 14:34:04 | sean-k-mooney | really the dpdk driver should jsut ascp representor=vf5 + the pf address | |
| 14:34:08 | sean-k-mooney | and compute that internaly | |
| 14:34:30 | dvo-plv | we can get this offset from here in some way root@server23:~# cat /sys/bus/pci/devices/0000\:2b\:00.0/sriov_offset | |
| 14:34:31 | dvo-plv | 4 | |
| 14:36:17 | sean-k-mooney | if we can read that form sysfs | |
| 14:36:39 | sean-k-mooney | then we can drop the algother in os-vif and use the exisitng fucntion and add the ofset | |
| 14:36:55 | sean-k-mooney | that still does not really help for the ml2 driver | |
| 14:37:42 | sean-k-mooney | do you have a document that describes the step by step process of adding the representor netdevs to ovs-dpdk manually | |
| 14:38:30 | dvo-plv | one moment | |
| 14:39:56 | dvo-plv | https://docs.napatech.com/r/Getting-Started-with-Napatech-Link-VirtualizationTM-Software/Create-the-OVS-Provider-Bridge-and-Start-2-VMs | |
| 14:41:48 | sean-k-mooney | that is incompelte. it does not have the port add comands for the represtor prots | |
| 14:42:45 | dvo-plv | item 4 Add the dpdkvp0 virtual port to the br-int bridge: | |
| 15:12:31 | dvo-plv | maybe we can create logic like with vhostuser_socket_dir, if vhostuser_socket_name is set, get this name from config + pci from sysfs? | |
| 16:59:28 | noonedeadpunk | hey folks. I wanna double-check one thing with you, that I'm not missing anything. So for volume to be attached to a VM with scsi bus - volume *must* be created from the image? | |
| 16:59:58 | noonedeadpunk | As I've found a spec to allow volume define that as well, but it was abandoned | |
| 17:11:56 | sean-k-mooney | the volume no | |
| 17:12:07 | sean-k-mooney | but the vm root disk must have hw_disk_bus=scsi | |
| 17:12:43 | sean-k-mooney | we only look at the metadata on the root disk be that a local disk or cinder volume root disk | |
| 17:12:57 | sean-k-mooney | and all other cinder volume must use the same disk bus | |
| 17:13:18 | sean-k-mooney | noonedeadpunk: so right now we do not support attching block devices with diffent busses | |
| 17:13:37 | sean-k-mooney | there is a way to ocationlaly make that work in a speicific edgecase but its not supported upstream | |
| 17:14:14 | noonedeadpunk | aha, ok, I see then | |
| 17:14:25 | sean-k-mooney | noonedeadpunk: we discussed in the last ptg addign a way to supprot per volume disk bueses | |
| 17:14:33 | sean-k-mooney | but no one actully worked on it | |
| 17:14:38 | noonedeadpunk | So if you found yourself in situation when 25 volumes are not enough - you should start from scratch kinda? | |
| 17:15:09 | sean-k-mooney | well. it depends | |
| 17:15:20 | sean-k-mooney | the vm i assume is using virtio-blk now | |
| 17:15:24 | noonedeadpunk | yup | |
| 17:15:28 | sean-k-mooney | is the vm bfv or local storage | |
| 17:15:57 | sean-k-mooney | and are you lookign for a admin solution or an end user one | |
| 17:18:07 | noonedeadpunk | I'm not sure if it's bfv or not as instance seem to be gone.... | |
| 17:18:17 | noonedeadpunk | but admin solution is fine | |
| 17:18:25 | sean-k-mooney | then you can use this https://docs.openstack.org/nova/latest/cli/nova-manage.html#image-property-set | |
| 17:18:27 | noonedeadpunk | (or I jsut can't find it somehow) | |
| 17:18:58 | noonedeadpunk | ugh... We're running xena (going to upgrade to 2023.1 in a month) | |
| 17:19:16 | noonedeadpunk | but that is really handy command I didn't know about | |
| 17:19:24 | noonedeadpunk | so thanks for pointing me to it! | |
| 17:19:28 | sean-k-mooney | its really ment for helping peopel upgrade | |
| 17:19:42 | sean-k-mooney | but it will update the ebeded image metadata | |
| 17:19:50 | sean-k-mooney | if its boot form volume | |
| 17:20:01 | sean-k-mooney | there is an other hack | |
| 17:20:31 | sean-k-mooney | tl;dr is using old micorvstion of the rebuild api | |
| 17:20:42 | sean-k-mooney | allow bfv guests to just update the image metadata | |
| 17:20:57 | sean-k-mooney | i.e. on microvsion where rebuild was not supproted for boot form volume | |
| 17:21:09 | sean-k-mooney | if you used the same image uuid we allowed the metadtaa to be updated | |
| 17:21:36 | noonedeadpunk | iirc rebuild is supported quite recently? like zed or 2023.1? | |
| 17:21:42 | sean-k-mooney | this is generally not that safe as it would break the vm in some caseses and in the new microvserion it will actully rebuild the volume | |
| 17:21:51 | sean-k-mooney | yes | |
| 17:22:30 | noonedeadpunk | ok, awesome, thanks a lot! | |
| 17:22:32 | sean-k-mooney | so preior to actully supproting it properly there was a poorly documented feature where rebuild to the same iamge "only for bfv guests" woudl not destory data and just update the metadata | |
| 17:22:50 | sean-k-mooney | but honestly htat should never have been a thing | |
| 17:23:11 | sean-k-mooney | we effectivly forgot it existed until we added real rebuild support | |
| 17:23:45 | noonedeadpunk | I can recall writing smth to the ML regarding that | |
| 17:25:24 | noonedeadpunk | but already forgot about this feature :D | |
| 17:27:35 | sean-k-mooney | having a behviaor for a normally distructive api not be only if its BFV is too easy to forget | |
| #openstack-nova - 2023-08-29 | |||
| 06:44:50 | dvo-plv | sean-k-mooney, Hello | |
| 06:45:55 | dvo-plv | regarding our yesterday discussion. I see two options. we can get back to the separate mech driver, like was suggested year ago | |
| 06:46:37 | dvo-plv | or we can pass socket name + pci endpoint in some way | |
| 07:25:02 | Uggla | Hello Nova. | |
| 07:34:45 | gibi | Uggla: o/ | |
| 08:24:42 | opendevreview | melanie witt proposed openstack/nova master: nova-manage: Add 'limits migrate_to_unified_limits' https://review.opendev.org/c/openstack/nova/+/891646 | |
| 12:56:44 | opendevreview | Maksim Malchuk proposed openstack/nova stable/xena: Fix to implement 'pack' or 'spread' VM's NUMA cells https://review.opendev.org/c/openstack/nova/+/829804 | |
| 14:47:28 | opendevreview | Dan Smith proposed openstack/nova master: WIP: Detach disks using alias when possible https://review.opendev.org/c/openstack/nova/+/893068 | |
| 14:49:02 | dansmith | kashyap: was just going to point you at this, first PoC of using the stable alias to detach disks: https://review.opendev.org/c/openstack/nova/+/893068 | |
| 14:49:19 | dansmith | seems to work, without having to muck with all the actual detach plumbing like I was originally thinking | |
| 14:49:33 | dansmith | I think something similar should work for interfaces too | |
| 14:49:59 | dansmith | there's a lot more looking-up-of-devices elsewhere that would need to change, but I'm hoping those places will be able to use that helper too | |
| 14:50:55 | opendevreview | Elod Illes proposed openstack/nova stable/yoga: Don't provide MTU value in metadata service if DHCP is enabled https://review.opendev.org/c/openstack/nova/+/879701 | |
| 14:50:56 | kashyap | dansmith: Nice that you preserve the fallback to using 'disk_dev' | |
| 14:52:04 | kashyap | dansmith: I guess you're (me for sure) pleasnatly surprised that it turned out to be much less of code-footprint than anticipated | |
| 14:52:12 | kashyap | (Unless there are no nasty surprises lurking somewhere) | |
| 14:52:43 | dansmith | yeah I originally had rewritten all the detach stuff to use that per-alias method, but realized it was really not necessary and that I could just change how we looked it up instead | |
| 14:53:00 | dansmith | which is much smaller *and* should apply to the other places where we just need to look it up and not detach | |
| 14:53:39 | kashyap | Smart. (On changing how we looked it up. I would've racked my brains longer) | |
| 14:53:54 | opendevreview | Pavlo Shchelokovskyy proposed openstack/nova master: Replace -- as 3-4 char in hostname with - https://review.opendev.org/c/openstack/nova/+/893072 | |
| 14:54:59 | dansmith | so I'm hoping that helper on guest that looks up all devices and then finds by alias will work for interfaces in a similar workflow | |
| 14:55:52 | kashyap | You already ran nova.tests.unit.virt.libvirt.* unit tests locallY? | |
| 14:57:02 | dansmith | no, no tests, it just works in a devstack | |
| 14:57:04 | opendevreview | Elod Illes proposed openstack/nova stable/xena: Don't provide MTU value in metadata service if DHCP is enabled https://review.opendev.org/c/openstack/nova/+/879703 | |
| 14:57:12 | opendevreview | Pavlo Shchelokovskyy proposed openstack/nova master: Replace -- as 3-4 char in hostname with - https://review.opendev.org/c/openstack/nova/+/893072 | |
| 14:57:43 | dansmith | kashyap: i.e.: Aug 29 14:40:22 jammy nova-compute[306476]: DEBUG nova.virt.libvirt.driver [None req-323b5960-d0c5-4729-b269-5743cd4fdf90 demo demo] Found disk vdb by alias ua-ada5af06-300e-4d07-931d-3cc2bff8a8a9 {{(pid=306476) _get_guest_disk_device /opt/stack/nova/nova/virt/libvirt/driver.py:2722}} | |