| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-12 | |||
| 12:52:34 | sean-k-mooney | e.g. as an alternive to hugepages | |
| 12:52:39 | sean-k-mooney | or file backed memeory | |
| 12:53:07 | sean-k-mooney | aspiers: looking | |
| 12:53:19 | alex_xu | good idea | |
| 12:53:34 | aspiers | I found _instance_to_allocations_dict() in scheduler/client/report.py which maybe could extract the allocations from the instance, but I don't know if that's right | |
| 12:53:47 | alex_xu | do you mean reject pmem resource class in extra spec resource:*? | |
| 12:53:55 | aspiers | also, nothing uses that method any more, since Iec02942d38 removed the last callers of it | |
| 12:54:46 | sean-k-mooney | alex_xu: we could but i was thinking just not populating the instance.vpmems from it | |
| 12:55:34 | alex_xu | that will do an allocation in the placement, but in the end, the instance won't attach the pmem | |
| 12:55:39 | sean-k-mooney | aspiers: do you need to cehck the resouce allocation or jsut the flavor | |
| 12:55:46 | aspiers | sean-k-mooney: the allocation | |
| 12:55:54 | aspiers | that's what efried said yesterday | |
| 12:56:37 | sean-k-mooney | im not sure why checking the request_sepc or flavor/image would not be enough | |
| 12:56:52 | aspiers | sean-k-mooney: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-06-11.log.html#t2019-06-11T16:49:51 | |
| 13:00:08 | aspiers | sean-k-mooney: but maybe when efried wrote that he didn't know that _sev_required() needs to be called from multiple locations in the driver code which don't have access to allocations | |
| 13:00:18 | aspiers | I guess I could set another instance variable | |
| 13:00:22 | sean-k-mooney | the instance contains the request_sepc and the request spec contains the request_resouces | |
| 13:00:29 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L98 | |
| 13:01:04 | aspiers | but requested_resources is different from allocations, no? | |
| 13:01:12 | sean-k-mooney | it is | |
| 13:01:24 | sean-k-mooney | and it does not currenly hav the info you want see the comment above | |
| 13:01:43 | aspiers | right | |
| 13:01:48 | sean-k-mooney | im just wondering is that better to use in the future | |
| 13:01:51 | aspiers | in fact efried wants me to help fix that | |
| 13:02:29 | aspiers | I'm guessing that code which wants to know whether to add SEV bits to the libvirt XML config should be checking what got allocated, not just what got requested | |
| 13:02:50 | sean-k-mooney | they should be the same thing | |
| 13:03:10 | sean-k-mooney | if placemnet returned an allcoation candatae then eveything that was requested was allcoated | |
| 13:03:32 | aspiers | I guess in the MEM_ENCRYPTION_CONTEXT case where only 1 unit is requested they have to be the same | |
| 13:03:34 | sean-k-mooney | looking at the instance object i dont see the allcoation stored anywhere | |
| 13:03:43 | aspiers | yeah I couldn't either | |
| 13:05:15 | aspiers | I found https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L2434 | |
| 13:07:18 | aspiers | that passes allocations to spawn() via https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L2203 | |
| 13:08:01 | sean-k-mooney | ya we do pass them to the driver | |
| 13:08:03 | sean-k-mooney | https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L2207 | |
| 13:08:16 | aspiers | that's 4 lines below my link :) | |
| 13:08:19 | aspiers | same thing | |
| 13:08:43 | aspiers | but then allocs doesn't get passed around to all the smaller methods called by spawn() | |
| 13:09:06 | sean-k-mooney | right its only used to get teh mdevs | |
| 13:09:07 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3237 | |
| 13:09:49 | aspiers | so maybe I can set an _sev_required instance variable in spawn() | |
| 13:10:26 | aspiers | or somewhere underneath that | |
| 13:10:26 | sean-k-mooney | well you could jsut pass the allcoation to _get_guest_xml | |
| 13:10:45 | aspiers | that's a good idea | |
| 13:10:57 | aspiers | I guess I only need it for constructing XML | |
| 13:11:02 | aspiers | so that would scope it correctly | |
| 13:11:20 | aspiers | hopefully efried will come online before I disappear down this rathole :) | |
| 13:12:33 | sean-k-mooney | its still a little messy | |
| 13:12:43 | sean-k-mooney | i think really we want to add the allcotion to the instance | |
| 13:13:02 | sean-k-mooney | that way you dont need to pass a new param all over the place | |
| 13:13:07 | aspiers | yeah | |
| 13:13:16 | sean-k-mooney | and i think more things will need the allocation in the future anyway | |
| 13:13:54 | sean-k-mooney | we track things like pci device allcotation in teh instance already so i think that is the best path forward. | |
| 13:14:18 | aspiers | sounds reasonable but I can't really comment | |
| 13:15:17 | sean-k-mooney | efried: when you are online any objection to adding the allocations form placmenet to the instnace object? | |
| 13:21:46 | artom | sean-k-mooney, sorta like network_info in info_cache? | |
| 13:22:14 | sean-k-mooney | not quite although kind of | |
| 13:23:11 | sean-k-mooney | artom: we store the list of pci devices allocated to an instance in the instance | |
| 13:23:13 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/objects/instance.py#L200 | |
| 13:23:33 | sean-k-mooney | so that we can use the same ones on reboot | |
| 13:23:58 | sean-k-mooney | it would be somewhat like storing the vifs in the info_cache | |
| 13:24:23 | sean-k-mooney | but keeping the allocation in the insantce so we dont have to look them up on swapn or reboot makes sense | |
| 13:24:59 | sean-k-mooney | *start or reboot | |
| 13:30:15 | sean-k-mooney | ... | |
| 13:30:31 | sean-k-mooney | i think our vgpu code is busted | |
| 13:31:21 | sean-k-mooney | bauzas: did you ever test stopping an instance with vgpus | |
| 13:31:29 | sean-k-mooney | then starting it again | |
| 13:32:48 | sean-k-mooney | power_on in the libvirt driver delegate to _hard_reboot which assumes the xml still exists to get the mdevs from | |
| 13:32:50 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L2897-L2900 | |
| 13:33:01 | sean-k-mooney | power_off calls destroy | |
| 13:33:33 | sean-k-mooney | so if you shutdown an instance with vgpus and then start it again it wont have vgpus... | |
| 13:39:19 | kashyap | Speaking of assigned devices, somepeople might find this tool ("mdevctl") useful: https://github.com/awilliam/mdevctl | |
| 13:39:28 | kashyap | More info and background about it here: | |
| 13:39:40 | kashyap | https://www.redhat.com/archives/libvir-list/2019-May/msg00722.html | |
| 13:39:44 | kashyap | ("mdevctl: A shoestring mediated device management and persistence utility") | |
| 13:40:09 | kashyap | It's a shell tool. So, be gentle with your eyes :D | |
| 13:40:11 | sean-k-mooney | ya maybe | |
| 13:41:44 | sean-k-mooney | they are not that hard to manage i personlaly would praobly do it with systemd rather then udev personally | |
| 13:42:24 | kashyap | I don't know, if the VFIO maintainer suggests that, then I'd go the route they suggest. | |
| 13:44:43 | sean-k-mooney | perhaps if it existited when i started managing mdev like 2 years ago maybe i think its still failly tivial to do yourself too | |
| 13:47:17 | sean-k-mooney | kashyap: this is the striped down lib we created for managian mdevs for smart nics last year https://github.com/intel-orchestration-software/networking-vhost-vfio/blob/master/networking_vhost_vfio/mdev/mdev.py | |
| 13:47:42 | kashyap | sean-k-mooney: If it were "trivial", why would they write that tool? | |
| 13:47:55 | sean-k-mooney | it does not do persistence but at the time i also looked into doing it with systemd | |
| 13:48:34 | sean-k-mooney | kashyap: because they felt like it? just because its trivial to do does not mean you dont have to do it alot and still want to automate it | |
| 13:49:41 | openstackgerrit | Merged openstack/nova master: Make get_provider_by_name public and remove safe_connect https://review.opendev.org/664062 | |
| 13:50:18 | rafaelweingartne | Hello guys, is there an "openstack server event list" command like that is able to list the events for deleted VMs? | |
| 13:52:04 | sean-k-mooney | im not sure that is what that is for | |
| 13:52:14 | rafaelweingartne | no? | |
| 13:52:41 | sean-k-mooney | i think it list the life cycle events a server has undergone | |
| 13:52:49 | sean-k-mooney | i dont think you can use it to wait for an event | |
| 13:53:02 | rafaelweingartne | It looks like it is presenting he vents I need/want such as attaching something, and or creating the vm | |
| 13:53:07 | rafaelweingartne | ah | |
| 13:53:11 | rafaelweingartne | no, I do not want to wait | |
| 13:53:29 | kashyap | sean-k-mooney: It's not "they felt like it"; there's a proper rationale there. | |
| 13:53:39 | kashyap | [quote] | |
| 13:53:40 | kashyap | everyone's needs on the first pass, but maybe we'll solve enough and | |
| 13:53:40 | kashyap | provide something and see where it goes. I doubt we'll solve | |
| 13:53:40 | kashyap | is largely left as an exercise for the user. This is an attempt to | |
| 13:53:40 | kashyap | Currently mediated device management, much like SR-IOV VF management, | |
| 13:53:41 | rafaelweingartne | I just want to list all events that already happened to a VM | |
| 13:53:42 | kashyap | provide helpers for the rest. Without further ado, I'll point to what | |