Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-09
12:36:26 sean-k-mooney alex_xu: we still need to decised how to reconsile them but we intentionally did not use them for sriov live migration because we did not want to have them for live migration
12:39:28 alex_xu sean-k-mooney: emm...I need figure out what different between move clain and pci claim
12:39:57 sean-k-mooney the pci tracker has 3 state for the devices
12:40:01 luyao sean-k-mooney: vpmem need numa support, so it's ok in numa live migraion I think
12:40:19 sean-k-mooney basically avaiable allocated and in uses
12:40:57 alex_xu sean-k-mooney: both them are reconcile by rt.update_available_resource?
12:41:06 sean-k-mooney for the sriov migrating we move the dest pci deives that are claiimed too allocated and then go to in use in post live migration or get freed if we fail
12:41:46 sean-k-mooney the pci magnager work a littel differently
12:41:59 alex_xu and pci manager persistent the claim
12:42:05 sean-k-mooney yes
12:42:28 alex_xu vpmem need persistent also
12:42:45 sean-k-mooney well it persist it in that in the pci_stats? table we change the state of the device and recorred it as allocated fot the specic instnace
12:42:48 alex_xu and vpmem totally depends on instance.resources and instance.migration_context
12:43:23 sean-k-mooney ya. for vpmem at the moment i guess you have no other choice
12:43:27 alex_xu no vpmem claim persistent in instance.resources and instance.migration_context
12:43:35 alex_xu yea
12:44:00 sean-k-mooney i was hoping that wew woudl reconsile the different approches in sriov migraiton and numa migration by removing the live migration claim
12:44:54 alex_xu we still have RT at that time?
12:45:02 alex_xu I don't vision about that
12:45:13 sean-k-mooney and haneling numa without building on move claims but i guess i might have to do it the other way which make sme sad but
12:45:14 alex_xu s/don't vision/don't have vision'
12:45:39 sean-k-mooney i expect that we will always have the RT since plamcent does not handel assingment
12:46:07 alex_xu +1
12:47:53 sean-k-mooney i just think the RT will do less work over time.
12:48:52 alex_xu yea, that i can imagine
12:49:39 sean-k-mooney it partly why i was hoping the move/rebiuld/noop/live-migration claim would eventury be removed
12:49:57 sean-k-mooney i was hoping we could simply what it does over time to not need them
12:51:27 alex_xu ok, i see
12:57:19 sean-k-mooney is the only place we store the vpmem allocation the in instance_extra table in the resouces filed
12:58:14 sean-k-mooney then in memeoy the resouce tracker discovers the avialabel vpmem namespaces and uses the instnace extra info to determin which ones are free and which are in use
13:01:01 sean-k-mooney ah yes we model this using the generic Resocue object and the specalised vpmemdevcie metaddata
13:01:07 sean-k-mooney ths is all coming back to me https://github.com/openstack/nova/blob/a4311c401b55c27321de7b282b242661af73fa34/nova/objects/resource.py#L39-L110
13:03:34 alex_xu sean-k-mooney: yes
13:04:55 sean-k-mooney right so i rememebr we discussed if we need to do host side tracking like we do for pci device or if im memory + instastnce level tracking was enought like we do for vgpu
13:05:20 sean-k-mooney and we endup not doing host level tracking in the db
13:05:53 alex_xu yes
13:06:54 sean-k-mooney ya ok so the appoch in the spec makes sense then with that context.
13:07:00 alex_xu vgpu is more libvirt level tracking...
13:08:44 sean-k-mooney ya i was just looking at the vpmem discovery code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L429-L514
13:18:08 huaqiang 1;2C1;5C
13:20:46 openstackgerrit Lee Yarwood proposed openstack/nova stable/stein: libvirt: Add a rbd_connect_timeout configurable https://review.opendev.org/669167
13:21:56 openstackgerrit Lee Yarwood proposed openstack/nova stable/stein: Skip cpu comparison on AArch64 https://review.opendev.org/699116
13:50:02 efried Nova meeting in 10 minutes in #openstack-meeting
13:52:16 efried At a glance this morning it looked to me like nova-next is failing 100%. Has anyone looked at this?
13:52:35 efried ...before I dig in with my non-mriedem-ness?
13:53:42 openstackgerrit Matt Riedemann proposed openstack/nova master: Add a workaround config toggle to refuse ceph image upload https://review.opendev.org/657078
14:00:22 efried Nova meeting now.
14:01:55 efried stephenfin, sean-k-mooney, bauzas: ^
14:02:09 efried huaqiang: ^
14:02:10 efried who else?
14:03:39 brinzhang_ :)
14:06:16 brinzhang_ Can I send my questions? :P
14:07:04 alex_xu brinzhang_: meeting is at #openstack-meeting channel
14:07:25 brinzhang_ alex_xu: yeah, joined
14:07:29 efried brinzhang_: yes, for sure.
14:07:59 brinzhang_ alex_xu, efried: thanks
14:29:01 openstackgerrit Merged openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182
14:30:45 bauzas efried: sorry was on meeting
15:01:36 sean-k-mooney bauzas: every time you or somone else at redhat says they are sorry i was in a meeting i go crap did i miss one or was it a 1:1 :)
15:01:54 bauzas 1:1 indeed
15:02:09 sean-k-mooney although that is partly because im getting used to meeting again after the break
15:02:15 brinzhang_ hah
15:02:28 efried dansmith: Re specifying VCPUs in placement-ese and other stuff in hw:-ese...
15:02:28 efried We could make that work, if we had to, because the hw:-ese part could impart enough information that you could infer the rest about your shared CPUs.
15:02:28 efried I don't like that personally because I think the mix of placement-ese and hw:-ese is more confusing than just hw:-ese
15:02:58 efried point being: we're going to have to have snowflake syntax anyway, so might as well make it consistent.
15:03:36 sean-k-mooney efried: you coudld infer the amoung and then have a driver specifc convetion for how to select which ones would float and whcih woudl be pinned
15:03:44 efried Rather than asking people to understand which subset of placement-ese is side-effecty and which is not
15:03:45 sean-k-mooney but it make the code more complex
15:04:03 efried sean-k-mooney: Yes, exactly. Yes, exactly.
15:04:16 efried and also make it harder for the user to understand IMO.
15:05:36 sean-k-mooney if i was to code the algortiom it would be balance the shared cores evenly across numa nodes then make the rest pinned
15:05:40 dansmith I dunno, I'm just disappointed with the mess we've had, do have, and apparently will have
15:06:22 sean-k-mooney which in a singel numa node guest means all the shared cores are fist follow by the pinned cores
15:06:50 dansmith do the public clouds let you have this level of control over which cpus are pinned and to where?
15:06:54 sean-k-mooney and that is because the linux kernel and proably other has a prefernce for runing some code on core 0 of each numa node
15:07:38 efried sean-k-mooney: even allowing for future expansion where you can specify how many you want in each NUMA node, we could still do the math. My gripe is really about the UX.
15:08:14 efried dansmith: is that a leading question?
15:08:46 sean-k-mooney well today the just expose flaver with either pinning on or off or numa nodes. but you can configure this via the image too
15:08:52 dansmith efried: it's an honest one.. we used to have a shared value of "abstraction means not getting to turn every knob"
15:09:08 efried Isn't "our competitor can't do X, and neither can we, so let's make X possible so we're better than our competitor" how progress is made?
15:09:35 efried dansmith: and that abstraction still exists. You can still put up a flavor with no extra specs and get a VM. We're not taking away that simplicity.
15:09:57 dansmith efried: the same argument goes fo "some vendor shows up with a P2V'd ancient appliance that requires a specific hardware topology and we spent years trying to implement that instead of actually competing on something that matters"
15:10:04 sean-k-mooney so as an operate i can create a flavor with 8 cores and pinnnge and then as a user i can upload an image the says make the vm have 1 socket and, 2 thread per core, and 1 nuam node please
15:10:25 efried dansmith: I dig. But NUMA isn't going away.
15:10:52 efried and edge tuning is like the big thing, isn't it?
15:11:01 dansmith efried: no, numa isn't, that's not the point.. whether or not we need to provide this level of control over what the topology looks like instead of just providing a sane one with the asks we were given is.
15:11:26 dansmith efried: mostly because of legacy appliances afaik :)
15:11:32 efried if a good-enough VM was good enough, sure.
15:12:01 efried but in edge IIUC you need to be able to squeeze every cycle out of your hardware, no?
15:12:15 sean-k-mooney well if we were to provide a sane topology by default we would have to change our curren policy of 1 cpu socket per vcpu
15:12:25 sean-k-mooney but i get what you mean
15:12:27 dansmith that's not generally the reason they need to strictly control the topology.. it seems like the reason, but it's often not :)
15:12:33 sean-k-mooney sane default would be nice
15:12:48 efried I think we have sane defaults, don't we?
15:12:49 dansmith anyway, I'm not trying to stop the train on those grounds,
15:13:10 sean-k-mooney efried: i would not call the cpu toplogy nova generates sane
15:13:15 efried heh
15:13:22 efried well, it could be
15:13:34 dansmith I'm more looking for a signpost in the distance of what level of abstraction others have chosen

Earlier   Later