| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-03 | |||
| 14:44:38 | sean-k-mooney | no | |
| 14:44:52 | sean-k-mooney | the count of how many we can create is | |
| 14:45:28 | sean-k-mooney | not the mdev paths for the vgpu instnace | |
| 14:45:28 | artom | But we have to care about the specific individual ones like we do with PCPUs? | |
| 14:45:46 | sean-k-mooney | artom: yes and no | |
| 14:46:05 | sean-k-mooney | artom: if you have multiple devices or multipel mdev_types enabled on the host yes | |
| 14:46:15 | sean-k-mooney | artom: we create one RP per pGPU | |
| 14:46:41 | sean-k-mooney | so the allcoation need to be collated to the correct pgpu on the host and the mdev need to be created form that gpu | |
| 14:46:53 | sean-k-mooney | that info is stored only in memory | |
| 14:47:13 | sean-k-mooney | well we also parse the libvirt xmls when the agent restarts too | |
| 14:47:41 | artom | The host capabilities you mean? | |
| 14:48:18 | sean-k-mooney | no what do you mean by host capablities | |
| 14:48:25 | artom | You said XML | |
| 14:48:32 | sean-k-mooney | yes i ment the domain xml | |
| 14:48:32 | artom | I'm asking *which* XML :) | |
| 14:48:46 | artom | Ah, that lists the mdevs that each instance uses | |
| 14:49:01 | sean-k-mooney | when the agent restarts since we dont store the mdev a vm is using in the db we have to look it up for the running vms | |
| 14:49:14 | artom | Hrmpf :( | |
| 14:49:41 | sean-k-mooney | i argued it shoudl be stored in the db either in its own tabel or in the host numa toplogy blob | |
| 14:49:47 | sean-k-mooney | for what its worth | |
| 14:50:09 | sean-k-mooney | its true that in generall the uuid we use does not matter to the vm | |
| 14:50:19 | sean-k-mooney | but there are edgecase where we do care like agent restart | |
| 14:51:43 | sean-k-mooney | artom: for what its wort we have added a new generic resouce filed ot the instance_extra_tables | |
| 14:51:45 | sean-k-mooney | https://github.com/openstack/nova/commit/6a4d6ec786bb35b0bf81d91b867d55f4f8b1e32d | |
| 14:51:57 | sean-k-mooney | that could be used for mdev for vgpus too if we wanted | |
| 14:52:10 | sean-k-mooney | it was created to be generic so we could reuse it | |
| 14:52:34 | sean-k-mooney | its currently only used for pmem | |
| 14:53:24 | sean-k-mooney | the pmem resouce are also not in the comptue nodes table unless they are embeed in say the host numa_toplogy bolb | |
| 14:53:55 | sean-k-mooney | unless i missed something | |
| 14:54:19 | artom | I don't have enough context, but maybe they don't need to be persisted in the DB? | |
| 14:54:43 | artom | Like, what does it affect? For scheduling, we only need to know quantities of mdevs, not individual ones, no? | |
| 14:55:00 | sean-k-mooney | we do but we use placement for the sechduling aspect | |
| 14:55:19 | sean-k-mooney | artom: for vGPUs you mean | |
| 14:55:25 | artom | sean-k-mooney, yeah, for example | |
| 14:55:42 | artom | Building the XML of new instances? Do we need to now the paths of available mdevs? | |
| 14:56:03 | sean-k-mooney | quantity and mdev_type which can be tracked via triats is enough in that specific case | |
| 14:56:25 | sean-k-mooney | artom: we need to create an mdev and put the path in the xml | |
| 14:56:41 | artom | So based on that seems like not tracking them was maybe the correct choice? | |
| 14:56:59 | sean-k-mooney | yes and no | |
| 14:57:11 | sean-k-mooney | it creates a depenency on the domain xmls for running guest | |
| 14:57:21 | sean-k-mooney | but it was doable without the db level tracking | |
| 14:57:26 | sean-k-mooney | so we did not modify the db | |
| 14:57:48 | artom | You mean for stuff like hard reboot? | |
| 14:57:58 | openstackgerrit | Pavlo Shchelokovskyy proposed openstack/nova master: Raise 409 when removing security group from instance https://review.opendev.org/735572 | |
| 14:58:03 | sean-k-mooney | since we coudl rely on plamcnet for atomic claims of resouces we coudl skip doning that in nova | |
| 14:58:14 | artom | OTOH, it's not like vGPUs have state, so if it gets a different mdev after hard reboot, does it really matter? | |
| 14:58:35 | artom | sean-k-mooney, definitely agree with the general idea | |
| 14:58:36 | sean-k-mooney | if the mdev type does not change no | |
| 14:58:50 | sean-k-mooney | for pmem its much more important | |
| 14:59:03 | sean-k-mooney | which is why we now have https://github.com/openstack/nova/blob/c57d52e1978210fb3260bf8ea442237d548db576/nova/objects/resource.py | |
| 14:59:10 | sean-k-mooney | so that we can do this genericly | |
| 14:59:20 | artom | sean-k-mooney, ah, so for when it matters we *do* persist it in the DB | |
| 14:59:34 | sean-k-mooney | we do now yes | |
| 14:59:49 | sean-k-mooney | and we have a generic way to do it in the future without db modficiations | |
| 15:00:11 | sean-k-mooney | its one of the thing i pushed for durign the pmem stuff which alex_xu and other kindly implemented | |
| 15:01:22 | artom | sean-k-mooney, so yeah, NUMA in placement would be nice, and would solve a whole bunch of problems, but it's a mess to make work, so here we are ^_^ | |
| 15:02:07 | sean-k-mooney | this really doesnt have that much to do with numa in placment although this generic resouce field would allow use to too remove teh numa blobs if we had that | |
| 15:02:42 | sean-k-mooney | we would still have to do the resouce tracking in the db using this but its an alternitive | |
| 15:02:59 | sean-k-mooney | im not sure its worth the work to move it | |
| 15:03:34 | sean-k-mooney | but for example if we start support vdpa devices in the futrue | |
| 15:03:55 | sean-k-mooney | i would proably look to use this or extend the pci tracker | |
| 15:04:19 | sean-k-mooney | (just the type filed) | |
| 15:04:40 | sean-k-mooney | but we should not need to modify the db to add support for vdpa | |
| 15:05:06 | sean-k-mooney | or if we wanted to add orther host device passhtough support like block devices or usb devices | |
| 15:05:16 | sean-k-mooney | the resouce filed can be used for that | |
| 15:07:15 | sean-k-mooney | the limitation is that to use it you have to frist track the resouce in placment | |
| 15:13:55 | artom | sean-k-mooney, yeah, I meant more in the general sense of "using placement for atomic resource tracking" | |
| 15:14:10 | sean-k-mooney | artom: well it will never actully do that | |
| 15:14:19 | sean-k-mooney | it will track capastity | |
| 15:14:24 | sean-k-mooney | but not indiviual resouces | |
| 15:14:25 | artom | sean-k-mooney, yeah, inventories | |
| 15:14:36 | artom | (In retrospect, maybe that was a mistake?) | |
| 15:14:50 | sean-k-mooney | it was a deliberate design choice | |
| 15:14:55 | artom | I know | |
| 15:15:07 | sean-k-mooney | we knew the concequece was the resouce tracker would alwys be needed for assignment | |
| 15:15:15 | artom | Just saying, maybe not the correct one to make - though obviously hindsight is 20/20, yadda yadda | |
| 15:15:48 | openstackgerrit | Merged openstack/nova stable/rocky: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747975 | |
| 15:15:51 | sean-k-mooney | it not a hignsight thing we knew the resouce treacker and some level of assignment logic woudl be needed in nova | |
| 15:16:03 | openstackgerrit | Merged openstack/nova stable/ussuri: Removed the host FQDN from the exception message https://review.opendev.org/749607 | |
| 15:16:09 | openstackgerrit | Merged openstack/nova master: privsep: Add support for recursive chown, move_tree operations https://review.opendev.org/742864 | |
| 15:16:57 | sean-k-mooney | to be clear we coudl use palcment for indeivigual resouce tracking if we wanted via nesting and many many more resouce providers | |
| 15:17:24 | artom | Eww | |
| 15:17:29 | artom | sean-k-mooney, anyways, it's all moot at this point | |
| 15:17:29 | sean-k-mooney | its flexable enough to do that but we chose not too | |
| 15:49:20 | artom | Whoever did that is awesome | |
| 15:53:45 | gibi | nova meeting starts in 6 minutes on openstack-meeting-3 | |
| 16:00:18 | lyarwood | artom: it's great but I'd also love to write a script to open everything in vim (over http) using the raw links | |
| 16:01:02 | artom | lyarwood, ... everything? | |
| 16:01:04 | sean-k-mooney | lyarwood: i think lnav support direct links | |
| 16:01:31 | sean-k-mooney | it would automaticaly syntax highlith and alinge all the logs by time stamp too if that worked | |
| 16:02:44 | lyarwood | artom: well not everything but the stuff I typically look at, so n-* logs, tempest logs etc | |
| 16:02:59 | sean-k-mooney | anything ending in log | |
| 16:03:21 | sean-k-mooney | oh i guess most end in .txt | |
| 16:04:23 | sean-k-mooney | ah the gzip compression mess up usin lnav and likely vim with the raw urls | |
| 16:08:43 | lyarwood | sean-k-mooney: vim WORKSFORME with gzip FWIW | |
| 16:08:58 | sean-k-mooney | lyarwood: gzip is not the issue | |
| 16:09:09 | sean-k-mooney | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_557/746986/3/check/nova-live-migration/557b715/logs/screen-n-cpu.txt | |
| 16:09:17 | sean-k-mooney | us being served compressed | |
| 16:09:31 | lyarwood | ah yeah that borks vim | |
| 16:09:37 | sean-k-mooney | yep | |