Posted	Nick	Remark
#openstack-nova - 2018-03-22
16:09:00	dansmith	I think someone said that wouldn't make sense, butnow I'm not sure I get that
16:09:23	jaypipes	sean-k-mooney: that's why I called it overhead_dedicated_Set
16:09:23	efried	dansmith: It makes sense. I just don't think we should do it. Because testability.
16:09:25	sean-k-mooney	dansmith: i would equate them as teh same form reading
16:09:35	dansmith	efried: hmm, okay
16:09:35	mriedem	efried: "To find images tagged with ready and approved, include tag=ready&tag=approved in your query string. (Note that only images containing both tags will be included in the response.)"
16:09:40	sahid	johnthetubaguy: ack thanks
16:09:46	cfriesen	jaypipes: currently if you enable "isolate" for the emulator then nova will internally allocate another physical host cpu to run the emulator threads
16:10:20	cdent	I said it didn't make sense, but that may be because I'm not fully understanding dan's use case
16:10:20	jaypipes	cfriesen: right, and I'm opposed to that, because it f**ks with resource tracking and inventory management.
16:10:53	dansmith	cdent: I'm on a call right now, but maybe a hangout with interested parties when I'm done?
16:10:58	cdent	and also because I wasn't really thinking and on the resources case, more concatenation of value
16:11:03	dansmith	if it's really a misunderstanding thing
16:11:26	cdent	dansmith: I'd like to understand more but today's not great, i'm in a meeting right now and have been in one or another since I started today
16:11:27	efried	The problem is simply that we used ',' to mean 'AND' in the `resources` qparam; but ',' is already taken in `member_of` (to mean 'OR', unfortunately, but disambiguated by requiring `in:`, which consumes another nice piece of punctuation, the ';')
16:11:31	jaypipes	cfriesen: I am attempting to create a new resource class that represents a physical dedicated CPU (sahid would rather this resource class be called CPU_DEDICATED insetad of PCPU) that can be used to inventory these kinds of resources (which are currently not tracked in the same way as other resources, much to my dismay)
16:11:51	efried	s/';'/':'/
16:11:53	efried	sigh
16:12:04	dansmith	efried: ah okay, although that's also kindof already done, but yeah I see that now
16:12:06	sean-k-mooney	jaypipes: yes, if we track the overhead sepreatly form the pcpus/vcpu resuces it could imply that the cpus the guest sees is the resouce:*CPU-overhead or we ask for resouce:*cpus + over head form placement
16:12:31	cfriesen	jaypipes: the problem is if I have X "dedicated" guest vcpus, and I want to allocate X+1 physical cpus, and run the emulator threads on a physical CPU that isn't running any vCPU thread.
16:12:40	jaypipes	sean-k-mooney: this has nothing to do with "what the guest sees". this is a resource accounting issue only.
16:12:42	coreycb	mriedem: special sauce indeeed! my bad.
16:12:42	efried	dansmith: so let's move ahead with repeatable member_of. Human reading query string will interpret '&' as 'AND', as it should be.
16:12:47	mriedem	nice
16:13:30	cdent	wait, efried, did you just make a new different decision? I thought we had gone the other way ten minutes ago?
16:13:39	efried	vay
16:13:40	cfriesen	jaypipes: I'd be fine with CPU_DEDICATED. Would we also have CPU_SHARED instead of VCPU?
16:14:01	efried	cdent: Where's your purple thingy?
16:14:04	jaypipes	sean-k-mooney: this is only about answering the question "how many dedicated CPUs are left on the host (or on a specific NUMA node)?" and "how many shared CPUs are left on the host (or specific NUMA node)"
16:14:06	sean-k-mooney	jaypipes: well it could what im asking is the resouces we counting are track soly by the vaules of resources:PCPU and resources:VCPU and that the overhead extra spec dont effect how we claim in placement
16:14:19	cdent	efried: do p!help
16:14:22	efried	p!help
16:14:26	jaypipes	cfriesen: no, we can't change VCPU to CPU_SHARED, unfortunately.
16:14:29	cdent	p!help
16:14:46	efried	p!log
16:14:53	jaypipes	cfriesen: which is why I wanted PCPU to be the class name for the dedicated CPU resource, since it matched VCPU
16:15:05	cdent	you can do it in a privmsg too, so we don't have to see it. there you can leave off the p! prefic
16:15:18	efried	cdent: http://p.anticdent.org/logs/openstack-nova?dated=2018-03-22%2016:05:18.169011#4hT5
16:16:37	efried	cdent: Basically, we collectively on average seem to hate the available options for intra-param punctuation more than we hate deviating from the rest of the API by supporting repeatable for this one new case.
16:16:52	sean-k-mooney	jaypipes: can you clear up somthing for me. if i have resources:VCPU=4 resources:PCPU=1 and hw:overhead_dedicated_set=0 do i claim 5 cpus in placement (4 shared and 1 dedicated)
16:16:53	cdent	oh, that's not how intepreted that
16:16:58	openstackgerrit	Eric Young proposed openstack/nova master: Support extending attached ScaleIO volumes https://review.openstack.org/554679
16:17:23	purplerbot	<cdent> efried: "fixable problem" was the immediate thing of "being able to accept duplicates on keys in general". I agree that addressign dan's problem in the least disruptive way is probably syntax in the param [2018-03-22 15:53:38.038412] [n 3cRQ]
16:17:23	cdent	I got what I said here about being least disruptive: [t 3cRQ]
16:17:35	jaypipes	sean-k-mooney: no, you claim 4 VCPU and 1 PCPU resources.
16:17:47	jaypipes	sean-k-mooney: there's no such things as a generic "CPU resource"
16:17:51	sean-k-mooney	jaypipes: yes that is what i ment
16:18:23	sean-k-mooney	i just wanted to make sure we dont claim 4VCPUs and 2 PCPUS
16:18:50	cfriesen	sean-k-mooney: jaypipes: hw:overhead_dedicated_set doesn't make sense since we may want to run emulator threads separate from all vCPU threads (this is what "isolate" does today)
16:19:05	sean-k-mooney	and in this case the guest recived 5 logical cpus and the emulator tread is pinned to the PCPU that was allocated to the vm
16:19:11	cdent	efried: it probably doesn't matter _that_ much as long as documentation etc
16:19:17	cdent	(whichever choice we make)
16:19:31	jaypipes	cfriesen: ack. I'm reworking this in the spec after your feedback.
16:19:56	sean-k-mooney	cfriesen when you say isolate which isolate do you mean
16:20:00	efried	cdent, dansmith, edleafe: Okay, let's make a call one way or another. And send an email. And propose a spec delta. And fix the code. And close the books.
16:20:00	jaypipes	cfriesen, sean-k-mooney: can there ever be >1 emulator thread, though?
16:20:08	cfriesen	jaypipes: yes
16:20:12	edleafe	cdent: efried: I didn't realize that we didn't accept multiple resources= params. But I do agree that changing that now is not a good use of time
16:20:15	jaypipes	sean-k-mooney: he is referring to hw:emulator_threads_policy=isolate
16:20:22	cfriesen	jaypipes: and there can be multiple I/O threads too
16:20:50	jaypipes	cfriesen: and how does "hw:emulator_threads_policy=isolate" account for >1 emulator thread? that was the whole point of the overhead_dedicated_set being a pinset -- allowing >1 emulator thread to be specified.
16:20:56	sean-k-mooney	jaypipes: ah ok hw:cpu_threads_policy=isolate is different just makeing sure
16:21:11	cfriesen	jaypipes: all the emulator threads get affined to a single physical cpu
16:21:44	jaypipes	cfriesen: that sounds like a libvirt-specific assumption that will likely change at any given time (as most of these things tend to do)
16:22:03	sahid	jaypipes: oh so you can't change VCPU? in that case i'm ok with PCPU
16:22:18	cfriesen	jaypipes: the emulator threads don't usually have a lot of work to do (the exception is during live migration I think). the goal of "hw:emulator_threads_policy=isolate" is to ensure that the minimal work they do doesn't interrupt the guest cpu threads
16:22:25	jaypipes	sahid: I mean, we *could*, but that would be a seriously annoying code change ;)
16:23:00	mriedem	melwitt: if you approve a blueprint like https://blueprints.launchpad.net/nova/+spec/hurrah-for-privsep-again you should set the 'series goal' to rocky so it shows up on the actual rocky blueprints page: https://blueprints.launchpad.net/nova/rocky
16:23:36	melwitt	mriedem: sorry, I missed that it wasn't set. setting now
16:24:05	jaypipes	cfriesen: so it never makes sense to have emulator threads on dedicated CPUs and guest vCPU threads on shared CPUs, right?
16:24:12	cfriesen	jaypipes: yes, I suppose that's qemu-specific
16:24:34	cfriesen	jaypipes: no, that wouldn't make sense.
16:24:37	sahid	jaypipes: yes i'm agreed with cfriesen, that never make sense
16:25:04	jaypipes	cfriesen: furthermore, from what I gather from your and sean-k-mooney's feedback, it also would never make sense to have emulator threads policy set to *anything* if the guest vCPU threads were *not* pinned to dedicated CPUs.
16:25:28	cfriesen	jaypipes: yes
16:25:35	jaypipes	k
16:26:16	cfriesen	jaypipes: but it might make sense to have vcpu threads on dedicated cpu, and emulator threads on shared cpu (lumped together with other emulator threads and shared vcpu threads)
16:27:01	stephenfin	yeah, that's one of the ideas that came up in reviews of sahid's spec ^
16:27:08	cfriesen	jaypipes: sahid: I'm just not sure if we need to keep the "allocate a whole host cpu for my emulator threads because I'm super important and don't want other guests to impact my emulator threads"
16:27:42	sahid	cfriesen: yes, you have asked me, but did not have responded. I think that is not necessary
16:27:44	stephenfin	assuming we gain 'pcpu_pin_set' option (or whatever tetsuro's spec called for)
16:28:22	sahid	we just want in some cases the emulthreads to run somewhere
16:28:26	jaypipes	stephenfin: CONF.cpu_dedicated_set
16:28:32	cfriesen	If we don't need that, then the simplest thing would be to enforce that we always have at least one shared host CPU, and if you ask for hw:cpu_threads_policy=isolate it runs there.
16:28:33	stephenfin	that's the one
16:28:44	jaypipes	stephenfin: is my proposal, along with CONF.cpu_shared_set being the other disjoint set.
16:28:59	stephenfin	or maybe not, but the idea's the same
16:29:13	jaypipes	cfriesen: you mean emulator_threads_policy, not cpu_threads_policy.
16:29:18	cfriesen	bah, yes
16:29:20	sahid	cfriesen: i see you point, we can't have cpu_shared_set empty
16:29:29	jaypipes	yet another reason all this stuff is so confusing...
16:30:09	stephenfin	So we'd completely remove the ability to have one core per guest for emulator threads (e.g. what we currently have)
16:30:19	stephenfin	not that I'm against that. Just making sure I understand
16:30:29	cfriesen	stephenfin: that's really an implementation detail currently, but yes
16:30:29	sahid	stephenfin: yes, no need of that anymore
16:30:37	stephenfin	(y)
16:31:38	jaypipes	stephenfin: I don't understand that last statement...
16:31:47	jaypipes	stephenfin: why would we be removing that ability?
16:32:03	cfriesen	jaypipes: if we assume that the emulator thread work scales with the number of vcpus, then we don't need to account for it separately, it's already in cpu_allocation_ratio. This would simplify things since you wouldn't need to ask for an extra VCPU from placement.

Earlier   Later