| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-04 | |||
| 16:46:16 | stephenfin | sean-k-mooney: Nah, mriedem makes a good point. It was rather broken before | |
| 16:47:48 | stephenfin | sean-k-mooney: I've just held off on the +W in case cfriesen gets a chance to wade in before tomorrow | |
| 16:48:05 | sean-k-mooney | ok cool thanks | |
| 16:51:35 | stephenfin | mriedem: I must have misunderstood something but weren't we planning to deprecate this option entirely, relying on alternatives for instance-host mapping use case? https://review.openstack.org/#/c/619953/10/nova/conf/libvirt.py | |
| 16:52:05 | sean-k-mooney | stephenfin: dan asked that we dont | |
| 16:52:10 | mriedem | stephenfin: see dansmith's comments in an earlier PS | |
| 16:52:21 | stephenfin | ack | |
| 16:57:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Follow up for per-instance serial number change https://review.openstack.org/634743 | |
| 17:29:37 | stephenfin | mriedem: Fancy checking this nit fix? https://review.openstack.org/#/c/626930/ | |
| 17:29:41 | stephenfin | *nit doc fix | |
| 17:31:53 | mriedem | done | |
| 17:32:34 | stephenfin | ta | |
| 17:33:23 | mriedem | stephenfin: a couple of nits in the one on top of that | |
| 17:34:11 | stephenfin | I can fix those now or in a follow-up. Your call | |
| 17:34:20 | mriedem | do it now and i'll re-approve | |
| 17:36:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Add solution to live migration ssh issues https://review.openstack.org/626931 | |
| 17:36:19 | stephenfin | mriedem: done ^ | |
| 17:49:09 | efried | artom jaypipes cdent sean-k-mooney | |
| 17:49:09 | efried | Is that still on ice? Has bp/numa-aware-live-migration brought it back into play? Or are we still doing that bp with the old style NUMA modeling? | |
| 17:49:09 | efried | Hey folks, where are we with NUMA modeling in placement? | |
| 17:50:00 | sean-k-mooney | the numa aware migration is entirely done via the resource tracker | |
| 17:50:13 | sean-k-mooney | efried: so its still on ice until train | |
| 17:50:37 | sean-k-mooney | efried: the RT changes will still be needed in eitehr case | |
| 17:51:05 | efried | So would it be appropriate to start proposing spec content for train around NUMA modeling in placement? (Again) | |
| 17:51:20 | sean-k-mooney | bauzas: ^ | |
| 17:51:50 | efried | i.e. using nested providers to represent NUMA nodes, figuring out a way to represent subtree affinity, etc. | |
| 17:51:52 | sean-k-mooney | efried: i guess but it would be good to take a more holistic aproach i think | |
| 17:52:01 | efried | holistic in what sense? | |
| 17:53:03 | sean-k-mooney | well there are a number of feature in the libvirt driver (pinning,hugepages,reatime,emulator threads,numn) all the nfv stuff that are not really placement friendly | |
| 17:53:54 | sean-k-mooney | efried: i have been wondering for a while if we shoudl create a parallel implmentation of new feature that are placment aware to replace teh existing ones with | |
| 17:54:13 | efried | not sure how we'll ever get there if we don't. | |
| 17:54:36 | cdent | yeah, I suspect a parallel leapfrog is the way to go | |
| 17:55:20 | sean-k-mooney | well my idea was to take the cellsv2 approcah and freeze/keep the exisiting feature set as is and work on a replacement in paralel and then swtich in U | |
| 17:55:32 | openstackgerrit | Merged openstack/nova master: Reject migrate with port having resource request https://review.openstack.org/630723 | |
| 17:56:41 | sean-k-mooney | it realy has become a problem lately that we have 2 seperate code patchs. 1 for numa awre guess and anotehr for non numa guests and i just feel like have one code path that works for all guesst in a common way might be the way to go | |
| 17:57:10 | sean-k-mooney | efried: anyway that is slightly different then the question you asked | |
| 17:57:48 | cdent | sean-k-mooney: I sometimes wonder the opposite: we should have two diferent paths: one for simple vms that normal people use, anothe for the crazy path :) | |
| 17:58:13 | sean-k-mooney | cdent: the issue is we also have 2 sets of config options and flavor atributes | |
| 17:58:22 | sean-k-mooney | cdent: and people mix and match them | |
| 17:59:06 | sean-k-mooney | i think we can all agree that there should only be one way of modeling the resocues in placement | |
| 17:59:23 | efried | s/resources in placement/resources: in placement/ | |
| 17:59:55 | sean-k-mooney | efried: placement does not do assignment | |
| 18:00:15 | sean-k-mooney | efried: so assignable resouces need to be tracked in the ResocueTracker also | |
| 18:00:16 | efried | at the RP level, not at the resource level, right. | |
| 18:00:29 | efried | Well, arguably they need to be tracked by the virt driver. | |
| 18:00:39 | sean-k-mooney | yes | |
| 18:01:01 | efried | I'm not sure there's a need for the resource tracker if all resources are tracked by the virt driver + placement. | |
| 18:01:25 | sean-k-mooney | well kind of. in the libvirt case the Resouce track is running in the compute aganet and is populated byt the virtdriver. | |
| 18:02:08 | cdent | efried: don't duck on that, fly that flag | |
| 18:02:18 | sean-k-mooney | efried: the virtdriver do not get there own db tables to track things so they have to use the resouce tracker to persist there tracked resources | |
| 18:03:37 | efried | sean-k-mooney: Oh, see, I disagree with that. The virt driver knows about its resources by looking at the VM. There shouldn't be a need to duplicate that information in an OpenStack database. | |
| 18:04:02 | sean-k-mooney | efried: the vm xml is not persisted acroudn vm reboots | |
| 18:04:19 | efried | oh, right, that stupid thing. We should fix *that* instead. | |
| 18:04:46 | sean-k-mooney | it could be maintianed in memory but im not sure that would work aross agent restarts | |
| 18:05:14 | sean-k-mooney | efried: well you just moving the state form the db to libvirt/xml or ram | |
| 18:05:48 | efried | Let me put it this way: Do any other virt drivers have this silly ephemeralness? | |
| 18:05:55 | efried | PowerVM doesn't. cdent, does VMWare? | |
| 18:06:12 | efried | If you reboot an ironic node, does it forget how much memory it had? | |
| 18:06:33 | sean-k-mooney | efried: all that info is stored in ironics db | |
| 18:06:47 | cdent | efried: vmware keeps track of itself, yes | |
| 18:06:52 | sean-k-mooney | libvirt db is the filesystem. | |
| 18:07:04 | efried | right, so why should the xml go away when you reboot it? | |
| 18:07:11 | efried | it's just a file in the file system, nah? | |
| 18:07:38 | sean-k-mooney | if the hsot is rebooted it does not | |
| 18:07:41 | efried | I may be tilting at windmills here, but that just seems goofy to me. | |
| 18:07:56 | sean-k-mooney | if the vm is rebooted it does and if we are recreated multipel vms in parallel it get racy | |
| 18:08:42 | sean-k-mooney | we could do it all in memory if we do there right locking to keep everything in sysnce or we can use a dbms | |
| 18:08:43 | efried | needing to keep track of the assignments in the nova database is just doomed. Because different platforms have different ways of identifying/tracking their resources. You'll never get the fields right for everyone. Unless they're just a big anonymous data blob. | |
| 18:08:58 | cdent | efried++ | |
| 18:09:09 | cdent | virtdriver should have it's own interaction with placement | |
| 18:09:09 | efried | this is why we were in such pain in PowerVM-land trying to get "PCI" passthrough to work | |
| 18:09:19 | efried | and why I started on my "generic device management" crusade three years ago. | |
| 18:09:22 | cdent | nova should just "use" that info as required | |
| 18:09:23 | sean-k-mooney | *cough* numa toplogy blob *couch* | |
| 18:09:37 | sean-k-mooney | but yes you are right | |
| 18:09:50 | sean-k-mooney | anyway we could chagne that if we did a parallel impementation | |
| 18:10:04 | sean-k-mooney | but its non triavial | |
| 18:10:12 | efried | smy point exactly. If the numa topology is modeled in placement, it's the virt driver's responsibility to draw that model via update_provider_tree, in whatever way is appropriate for that virt. | |
| 18:10:41 | sean-k-mooney | efried: placement cant sotre everything we need it too | |
| 18:10:58 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Follow up (#2) for the bw resource provider series https://review.openstack.org/634767 | |
| 18:10:59 | sean-k-mooney | but but it should be able to handel hugepges entirly | |
| 18:11:31 | sean-k-mooney | we could model cpus but we would have to have 1 RP per cpu which is not a good design | |
| 18:12:28 | efried | Oh, I'm not saying placement should be exploded to track individual resources. Counts ought to be fine. The individual resources should be tracked by the virt driver. And trying to store those mappings in some generic database table is going to be problematic. | |
| 18:12:36 | efried | s/going to be// | |
| 18:13:06 | sean-k-mooney | efried: ya. its legacy tech debt | |
| 18:13:48 | efried | sean-k-mooney: How do we do pinning these days? Like this: https://docs.openstack.org/nova/pike/admin/cpu-topologies.html ? | |
| 18:14:00 | sean-k-mooney | so i think there are two parally efforts. 1.) modelign what can and should be modled in placemetn and 2.) refacorting the libvirt dirver to do assignment sanly | |
| 18:14:36 | sean-k-mooney | efried: we use the numa toplogy blob to store the free cpus | |
| 18:15:13 | sean-k-mooney | and then the virt driver caluates the guest topology can claims them in the host numa toplogy blob via the resouce tracker | |
| 18:16:34 | sean-k-mooney | efried: an important thing to remember is that hw:cpu_socket hw:cpu_threads and hw:cpu_cores are all refering to the virtual toplogy and have no baring on placemnt, the host or schdueling | |
| 18:17:17 | sean-k-mooney | efried: e.g. if the host has 1 socket you can sping up a vm with hw:cpu_sockets=8 and its fine | |
| 18:18:08 | efried | ugh. I know I have a lot to learn here, but I suspect much of it is going to be tribal. | |
| 18:18:24 | efried | if there are any documents that will give me some reasonably sane picture of how all this works, please do share. | |
| 18:18:48 | sean-k-mooney | efried: stephenfin has dont some presentation on this at the summit | |
| 18:18:55 | sean-k-mooney | or fosdem | |
| 18:20:35 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add native implementation OVSDB API https://review.openstack.org/482226 | |
| 18:20:37 | cdent | that etcd-compute toy I've built takes the position of "you can only have stuff that can be represented in placement". It flows nicely. | |
| 18:21:38 | sean-k-mooney | cdent: link? | |
| 18:22:01 | sean-k-mooney | https://github.com/cdent/etcd-compute? | |
| 18:22:02 | cdent | sean-k-mooney: https://github.com/cdent/etcd-compute I'm surprised I haven't already shown you this. | |