Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-23
14:25:10 sean-k-mooney which is not what dansmith was suggesting
14:25:48 sean-k-mooney if you take the approch of fixing it when you load the objects you need to keep that code for a few releases to not force peopel that are doing FFU to stop in ussuri
14:25:55 sean-k-mooney start the agent and have it fix the instnaces
14:27:05 sean-k-mooney so all i was really saying is eventaully we can remove that compatiablity code for old instnace but we will have to wait a releast or 3
14:29:33 shilpasd ok, got it
15:09:28 sean-k-mooney by the way we can recheck stuff currently right
15:09:57 sean-k-mooney there is not specific gate blockers at the moment out side of the intermitenly failing jobs
16:03:43 stephenfin sean-k-mooney: not that I'm aware of, no
16:25:35 KeithMnemonic good morning melwitt: should i keep playing recheck roulette for that patch on stable/pike or wait to see if the issue with migration might be found?
16:28:22 melwitt KeithMnemonic: I dunno what the pass rate for that job is on stable/pike at this point but it seems pretty terrible. sean-k-mooney and I looked at it at bit yesterday and no clues yet what's going wrong
16:29:04 KeithMnemonic are you ok with me continuing rechecks to see if it passes eventually
16:30:24 openstackgerrit Merged openstack/nova master: nova-net: Make the security group API a module https://review.opendev.org/696746
16:32:17 melwitt KeithMnemonic: yeah, you can
16:32:52 KeithMnemonic thanks
16:36:30 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unnecessary exception handling https://review.opendev.org/703973
16:36:31 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unused parameters https://review.opendev.org/703974
16:36:31 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unused nova-network objects https://review.opendev.org/697156
16:38:36 stephenfin efried: You saw https://zuul.opendev.org/t/openstack/build/471f2388615e4ee5806a85e19b79569d ?
16:38:42 stephenfin Have you a patch up to address that yet?
16:38:53 stephenfin Seeing as you were attacking them in a piecemeal fashion?
16:39:26 efried stephenfin: I hadn't seen that. I can throw out a patch quick...
16:39:34 sean-k-mooney melwitt: i got side tracked with backports so i did not look at it properly so ill try to do it today
16:39:48 stephenfin efried: if you could, that'd be good, yup :)
16:39:52 melwitt thanks sean-k-mooney
17:12:40 openstackgerrit Merged openstack/nova master: Remove remaining Python 2.7-only dependencies https://review.opendev.org/698895
17:15:19 deke997 Hey guys
17:15:34 deke997 I need to change how the guest pci addresses are generated. I have been trying to edit the config.py and driver.py files to add this functionality, but I cannot find the relevant code in the files
17:35:36 stephenfin deke997: What are you trying to achieve?
17:38:09 deke997 So I have a unique multi-function pci device that only works properly in the VM if the both devices are in the same slot but different functions. Like 0000:00:05.0 and 0000:00:05.1
17:38:36 deke997 by manually editing the libvirt.xml file, I verified that it works properly in the VM when set up like that
17:39:17 deke997 However, the libvirt.xml file is now generated automatically by nova, and by default it puts the devices in separate slots like: 0000:00:05.0 and 0000:00:06.0
17:40:49 deke997 So I want to change that behavior so that when I have a pci device that is passed through, it will pay attention to whether 2 devices were in the same slot with different functions on the host machine, and if so, it will repeat that behavior in the VM
17:47:19 deke997 stephenfin my understanding is that the xml file generation is handled by driver.py and config.py, but I cannot find the specific areas that deal with generating the guest pci address
17:47:29 stephenfin I'm trying to root that out at the moment
17:48:22 yoctozepto deke997: btw, is the device in question a secret or can you shed some light on it?
17:49:53 sean-k-mooney deke997: we dont do all the generation in nova
17:50:16 sean-k-mooney we actully leave it up to libvirt to assign the guesst pci adress and as a result define the toplogy
17:51:38 sean-k-mooney deke997: i have a backloag item to modify how that works for other reasons in the futre as i want o model 1 pci root complex per numa node to optimize how devices are exposed to vms
17:52:35 sean-k-mooney but it sould like you need to detect that the device is a multi fucntion device and then we would need to explcitly stat generating the toplogy and assing pci address in the guest to enable use to expose the device as a multi function device to the guest
17:52:58 sean-k-mooney deke997: this would be well into the catagory of a spec not a bugfix in my view
17:53:51 sean-k-mooney we might be able to skip the adress assignmentand still delegate that to libvirt but indicate that the device are both part of a multi function device but we would have to explore that to confirm
17:53:51 deke997 sean-k-mooney I see. This sounds like a fun project to work on. In the meantime, however, I have to find a solution that will let me get my VMs up
17:54:38 deke997 libvirt has a multifunction=on setting too
17:55:04 sean-k-mooney how to you request and whitelist the device at present
17:55:17 sean-k-mooney i assume you are using the pci whitelist and a flavor alias
17:55:21 deke997 yes
17:55:30 deke997 but I have to use a separate alias for each function
17:55:31 sean-k-mooney and you have 2 aliase right
17:55:35 sean-k-mooney one for each funciton
17:55:41 sean-k-mooney ya
17:55:57 deke997 We could add a multifunction alias type
17:56:07 sean-k-mooney so how are you ensuring the two VF/PF come form the same device on the hsot
17:56:29 sean-k-mooney deke997: we could add a tage but unless we can adress ^
17:56:37 sean-k-mooney we have bigger problems
17:56:44 deke997 right now, I am just using all of them, so it's not an issue haha
17:57:12 sean-k-mooney so you have only 1 device on the compute node and your booting 1 vm that use all the fucntion form that device
17:57:27 deke997 yes that's what I have set up for testing
17:57:42 sean-k-mooney ya in that specific case you whave worked around that issue
17:57:49 deke997 But in the libvirt XML it is very easy to see which devices from the host were passed through
17:57:58 deke997 and match them
17:58:02 sean-k-mooney this has come up in the context of gpus in the past
17:58:21 sean-k-mooney the GPU and audio contoler are usally 2 differnt pci endpoint on the same device
17:58:48 sean-k-mooney at pressent we dont have a way to passthough both form the same card and ensure they are form the same device
17:59:51 sean-k-mooney deke997: form a nova point of view you would basically want ot have an alsi that can refernce a second alias and request both as a pair
18:00:08 deke997 yea exactly
18:00:19 deke997 That doesn't seem super difficult
18:00:43 sean-k-mooney and then in nova/libvirt have it select the pair form the same device an make them a multifunction device in the xml if they are expose that way on the host
18:01:04 sean-k-mooney deke997: you say that now
18:01:08 deke997 haha
18:01:43 deke997 Is there a way to make nova play nice with edited libvirt.xmls?
18:01:44 sean-k-mooney so this all live in the pci manger code https://github.com/openstack/nova/tree/master/nova/pci
18:01:52 sean-k-mooney no
18:02:18 sean-k-mooney nova regenerates the xml based on seveal api actions
18:02:22 sean-k-mooney like hard reboot
18:02:36 sean-k-mooney so your eddit would last until someone stops and start the vm
18:02:56 deke997 Is there a way to disable that regeneration?
18:03:06 sean-k-mooney no
18:03:17 openstackgerrit Merged openstack/nova master: libvirt: Add a default VirtIO-RNG device to guests https://review.opendev.org/616603
18:04:12 sean-k-mooney so there are three parts. the pci mager woudl have to be enhanced to pars and track paired deivces, the pci pasthough filter would have to be update and the libvirt driver xml generateion would have to be updated
18:05:22 sean-k-mooney the alias schema coudl certenly be extended to referen a child device https://github.com/openstack/nova/blob/master/nova/pci/request.py#L66-L99
18:05:29 deke997 How would we go about updating the xml generation?
18:06:27 sean-k-mooney am ill see if i can find it. but likely i would see if we can still leave libvirt choose the target pci adders in the guest but mark a specicifc pci requiest as multi funcion
18:08:53 deke997 I need to know a bit more about how libvirt chooses those addresses
18:09:19 sean-k-mooney so you would have to modify https://github.com/openstack/nova/blob/6db486e9fd4f6b8dd02371b043e66808cdd1e0cd/nova/virt/libvirt/driver.py#L5871
18:10:10 sean-k-mooney the general docs for thi in libvirt are https://libvirt.org/formatdomain.html#elementsHostDevSubsys
18:11:25 sean-k-mooney normally we just generate a hostdev element like this
18:11:30 sean-k-mooney <hostdev mode='subsystem' type='pci' managed='yes'>
18:11:32 sean-k-mooney <source>
18:11:34 sean-k-mooney <address domain='0x0000' bus='0x06' slot='0x02' function='0x0'/>
18:11:36 sean-k-mooney </source>
18:11:38 sean-k-mooney <boot order='1'/>
18:11:40 sean-k-mooney <rom bar='on' file='/etc/fake/boot.bin'/>
18:11:42 sean-k-mooney </hostdev>
18:11:55 sean-k-mooney so we specify the source element and the host pci adress and libvirt chooes a target adress
18:12:07 sean-k-mooney that i assume is what you were changing with your manual edit
18:12:14 deke997 Yes
18:12:38 deke997 but I could only start that instance manually through libvirt after the edit, because nova just tries to overwrite it
18:13:39 sean-k-mooney https://libvirt.org/formatdomain.html#elementsAddress this is the docs for the device addressing
18:13:46 deke997 Also, adding multifunction = 'on' to the source address may be necessary. Libvirt docs say to use it, but I was able to have success either way
18:14:37 deke997 which would have to be done in nova

Earlier   Later