Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-06-15
14:46:10 gibi <can I rewrite nova, please?>
14:47:12 sean-k-mooney well vGPU shoudl not be where it si now
14:47:39 sean-k-mooney and the pci module is ment to be shared across virt driver which is why its where it is
14:49:55 gibi so should there be a mdev_tracker in the resource tracker?
14:50:11 gibi and also a cpu/memory/disk tracker?
14:50:23 gibi then we would not need the update_provider_tree to run in the virt level
14:50:30 gibi it could run on the compute manager level
14:50:42 gibi but for some reasons we introduced update_provider tree down in the virt level
14:50:42 sean-k-mooney i wanted to track mdevs in the pci tracker or in the resouces table
14:50:49 sean-k-mooney that we use for pmem
14:50:55 sean-k-mooney we have multipel resouce tracker in nova already
14:51:17 sean-k-mooney cpus and memory are traced in the hsot numa toplogy blob vai the hardware.py module
14:51:33 sean-k-mooney pmem uses the resouces table
14:51:41 sean-k-mooney pci has its own tracker
14:51:50 sean-k-mooney and medev use the libvirt xml domain files
14:51:55 sean-k-mooney which is proably the worst of them
14:52:18 gibi so when numa (if ever) will be in placement then the cpu inventory reporting in update_provider_tree need to be moved also to the compute manager level?
14:52:43 gibi as most of the numa tracking is in the resource tracker not in the virt level
14:52:56 sean-k-mooney prbably
14:53:06 gibi so this shows that the concept of update_provider_tree is wrong
14:53:09 sean-k-mooney the resouce tracker is were most of the tackign happens
14:53:11 gibi it cannot really update anything
14:53:22 sean-k-mooney well we can compute it
14:53:28 sean-k-mooney the virt driver has the list of cores
14:53:32 sean-k-mooney and the list of instnaces
14:53:54 sean-k-mooney actully its simpelr then that
14:54:07 sean-k-mooney for cpus we jsut need cpu_share_set and cpu_dedicate_set
14:54:20 sean-k-mooney to define the capstiy
14:54:28 sean-k-mooney and all the traits are provided by the virt dirver
14:54:38 sean-k-mooney the same is technially true for pci devices
14:54:47 gibi yepp
14:54:54 sean-k-mooney so you can just import the pci module
14:55:00 sean-k-mooney and pass it the set of hypervior devices
14:55:06 sean-k-mooney and ask it for the set of pools
14:55:22 sean-k-mooney then use the pools to do the update
14:55:29 sean-k-mooney but that is duplciationg the data we already have
14:55:39 sean-k-mooney so it just comes down to efficnecy
14:56:03 gibi in my mind it comes down to coupling today the virt driver does not couple to the pci tracker at all but tomorrow it will
14:56:17 gibi it increase complexity
14:56:49 sean-k-mooney right so really the reosuce tracker and pci tracker are ment to be the source of truth for the resouce that are avlaibel
14:56:56 sean-k-mooney and the virt driver just provide the raw resoucs
14:57:01 sean-k-mooney btu that is not how it works today
14:57:19 sean-k-mooney we have the virt driver direclty updateign the tree
14:57:23 gibi yepp
14:57:51 sean-k-mooney so that is backwards architularly but we did it because how that tree will like is virt dirver depentent because ironic
14:58:14 sean-k-mooney well ironic vmware ectra they look differnt then libvirt or hyperv
14:58:29 sean-k-mooney e.g. clsuterd vs 1:1 drivers
14:58:51 gibi hm it is differnt per driver yes
14:59:02 gibi so that is baad abstraction
14:59:11 sean-k-mooney currently yes
14:59:21 sean-k-mooney if you jsut want to resue the code form the pci module that is fine
14:59:26 gibi we have the generic nova scheduler code that depends on the tree but the tree if virt driver dependent
14:59:28 sean-k-mooney i dont really think we shoudl port it
14:59:51 gibi I will play around more before I decide
14:59:56 sean-k-mooney ack
15:00:10 gibi thanks for talking to me about it
15:00:16 sean-k-mooney gibi: i will be happy to review working code
15:00:19 gibi sorry If I was sooo negative
15:00:19 sean-k-mooney what ever form that takes
15:00:34 gibi shit meeting :/
15:00:44 gibi I almost made progress today :D
15:06:41 opendevreview Merged openstack/nova stable/yoga: Allow claiming PCI PF if child VF is unavailable https://review.opendev.org/c/openstack/nova/+/840833
15:53:49 dansmith gibi: sean-k-mooney bauzas: glance is planning to make a change to how image locations are updated (for the tight integration with ceph)
15:54:06 dansmith right now we use the user's token and don't have a "talk to glance with this service account" sort of setup like we do for neutron
15:54:25 dansmith we'd need to create images with their token, but then update the location with the service account
15:54:35 dansmith which is similar I think to things we do for/with neutron
15:55:00 dansmith but the spec in glance is very close to merging and I don't think anyone from nova has looked at it or grokked it from a nova perspective (other than me)
15:55:10 dansmith anyone have any concerns over that or want to review it first?
15:56:20 bauzas dansmith: sorry, triage meeting atm
15:56:46 dansmith bauzas: yep, np, reply when you can
15:57:16 bauzas dansmith: pass me the spec
15:57:20 dansmith https://review.opendev.org/c/openstack/glance-specs/+/840882/15/specs/zed/approved/glance/new-location-info-apis.rst
15:58:41 bauzas dansmith: from what you described, the usecase seems legit
15:59:30 dansmith bauzas: I think so to and it matches what we do for neutron at least, but it's a bit of a departure from how we talk to glance today (which is always with the user's token) so I just wanted to make sure we're all in agreement
16:00:03 bauzas dansmith: I'm more concerned by who would be in charge of doing the necessary bits in nova
16:00:13 sean-k-mooney so we kidn of have this precedent for nuton because
16:00:19 sean-k-mooney we have to update the host-id field
16:00:35 bauzas dansmith: I don't want us to lag with old behaviour because of lack of resources
16:00:42 sean-k-mooney for glance dont we already need admin for reading the multi sotre backend locations
16:00:55 bauzas but if the spec owner signs off for doing the nova bits, I'm cool
16:00:58 sean-k-mooney this is one of the things that would be good to do with the service role in the futre
16:01:14 whoami-rajat bauzas, i might end up doing it since I'm the author of the glance spec but if not will find someone who will be willing to do it
16:01:17 dansmith sean-k-mooney: no, we use the internal api endpoint with the user's token
16:01:25 sean-k-mooney ah ok
16:01:26 bauzas whoami-rajat: cool, I understand
16:01:37 dansmith bauzas: yeah, I'm also curious about that, since it won't be super trivial given that it's buried pretty deep
16:01:43 sean-k-mooney so we cant depend on having the admin token today correct
16:01:49 whoami-rajat since I'm already in charge of the cinder changes, nova changes should not be too hard or different
16:01:58 bauzas dansmith: you get my point
16:02:15 bauzas dansmith: I'm afraid of this being not trivial and requiring some hard work, including testing
16:02:16 dansmith sean-k-mooney: right, this is to solve that problem, and let nova present actual credentials to do that thing, instead of letting unprivileged people do it, but only through the internal api endpoint
16:02:25 dansmith sean-k-mooney: basically real auth instead of host-based auth
16:02:59 dansmith bauzas: yeah, I don't think it'll be super hard since we only do this in a couple places, but it's deep, so not trivial
16:03:14 bauzas whoami-rajat: before saying "not too hard", it's quite self-contained
16:03:20 bauzas ah, burned by dansmith
16:03:25 bauzas but there be dragons
16:03:51 bauzas also
16:04:13 dansmith I really don't think there will much burning or dragons, but I haven't gone digging myself to see exactly what would be involved, so I'm hedging :)
16:04:14 bauzas I guess you'll stick with the existing auth'd roundrobins for a couple of releases ?
16:04:32 bauzas rolling upgrades is a bit of a thing

Earlier   Later