| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-22 | |||
| 18:49:24 | kmalloc | i've seen it used in non-administrative ways. | |
| 18:49:38 | sean-k-mooney | kmalloc: im just trying to tink of why a full could admin token would not be correct in this case | |
| 18:49:40 | kmalloc | though haven't looked at common cases lately (my knowledge there is out dated) | |
| 18:49:58 | openstackgerrit | Dan Smith proposed openstack/nova master: Add require_tenant_aggregate request filter https://review.openstack.org/545002 | |
| 18:49:59 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Honor availability_zone hint via placement https://review.openstack.org/546282 | |
| 18:50:23 | kmalloc | sean-k-mooney: trying to break the "manage infrastructure" and "admin can do things for an account/domain/project" | |
| 18:50:44 | sean-k-mooney | kmalloc: ah ok makes sense. | |
| 18:51:03 | kmalloc | yeah, it's something we've needed for a long time | |
| 18:51:14 | kmalloc | because admin via a magic project scope is ... kindof awful | |
| 18:51:46 | kmalloc | it leads to a lot of potential "oopse, this scope bled through and let something happen" making the concept of say "domain admin" hard. | |
| 18:51:53 | jaypipes | kmalloc: kinda awful that is the same as it was >6 years ago... | |
| 18:52:21 | kmalloc | jaypipes: yes, and through many many many many iterations... we have successfully moved the needle to "yeah we still need to fix it" | |
| 18:52:29 | kmalloc | jaypipes: which makes me a bit more than sad. | |
| 18:52:31 | jaypipes | kmalloc: not saying it's not a good idea, just that clearly everyone has worked around it or lived with admin-ness for time eternal at this point. | |
| 18:52:41 | sean-k-mooney | kmalloc: https://review.openstack.org/#/c/525772 will result in a api microversion change also right | |
| 18:52:59 | kmalloc | sean-k-mooney: hm. | |
| 18:53:28 | kmalloc | sure? but how are old microversions managed in policy... wait wait, going to stop going down this rabbithole for now | |
| 18:53:33 | mlavalle | rybridges, melwitt: Neutron creates the aggragates for you: https://github.com/openstack/neutron/blob/master/neutron/services/segments/plugin.py#L216 | |
| 18:54:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Always deallocate networking before reschedule if using Neutron https://review.openstack.org/520248 | |
| 18:54:02 | mriedem | oomichi: can you re-approve this? ^ it sat so long that the test needed to be updated. | |
| 18:54:04 | kmalloc | jaypipes: right, but i think it's time to provide a real fix rather than relying on "everyone has their own workaround" | |
| 18:54:20 | sean-k-mooney | kmalloc: do unscoped tokens work for all scoped apis. if so then you proable dont need on but it might be nice to have a microverion | |
| 18:54:22 | kmalloc | jaypipes: and with policy-in-code, and scope-types we're really moving the right way | |
| 18:54:35 | kmalloc | sean-k-mooney: no, unscoped tokens don't work anywhere | |
| 18:54:42 | kmalloc | excpet to get a scoped token in keystone. | |
| 18:54:47 | jaypipes | kmalloc: ok | |
| 18:55:14 | kmalloc | sean-k-mooney: we're trying to allow RBAC for system-scope actions, unscoped tokens are explicitly no-roles. | |
| 18:55:25 | sean-k-mooney | kmalloc: ok so ya that would be a micro version bump then for backwards compatiblity. | |
| 18:55:28 | melwitt | mlavalle: right, but what about compute hosts that go into those aggregates? is that something neutron can do automatically or is it left up to the admin to have to do that? | |
| 18:55:48 | melwitt | thanks for chiming in, btw :) | |
| 18:55:53 | kmalloc | sean-k-mooney: yeah i'm not sure how a microversion affects policy enforcement -- it becomes weird. | |
| 18:55:58 | jaypipes | kmalloc: I struggle to have confidence in the firmness of the ideas after seeing keystone v3 domains, federation, PKI and other things go back and forth. | |
| 18:56:29 | kmalloc | jaypipes: well, please chime in on the scope types stuff. | |
| 18:56:37 | mlavalle | melwitt, rybridges: Neutron adds the hosts to the aggregate: https://github.com/openstack/neutron/blob/master/neutron/services/segments/plugin.py#L223 | |
| 18:57:04 | sean-k-mooney | mlavalle: that is for routed networks correct | |
| 18:57:21 | mlavalle | sean-k-mooney: yes it is | |
| 18:57:56 | kmalloc | jaypipes: there are a number of things we have not changed once other services lean on it. [also not sure what you mean firmness of federation, that's stayed the same and been built on, it hasn't been taken out] | |
| 18:58:16 | melwitt | mlavalle: a-ha, okay. so rybridges is seeing behavior where no hosts get added to the aggregate | |
| 18:58:18 | kmalloc | jaypipes: but i'd rather have more eyes/comments than less on the scope-types work. | |
| 18:58:28 | rybridges | Hey mlavalle! Thanks so much for the reply. We just checked the segmenthostmappings table in the DB and we see that the host mappings are in the table and our compute hosts are listed. But they are not added in the outpute of aggregate show | |
| 18:58:56 | kmalloc | sean-k-mooney: i'll have to see how the policy stuff shakes out with microversions, i'm sure it'll be 100% ok, just not clicking right this moment. | |
| 18:59:14 | kmalloc | sean-k-mooney: which means, i need to poke at things more. hehe :) | |
| 18:59:58 | kmalloc | sean-k-mooney: thanks for your time! | |
| 19:00:25 | cfriesen | jaypipes: cpu thread isolation is actually a valid issue for resource accounting, because the amount of resources consumed is variable depending on whether the host has enabled HT or not. | |
| 19:00:53 | sean-k-mooney | kmalloc: ya no worres i just didnot notice any microversion bumps in https://review.openstack.org/#/c/525772/4 and sice it has 1+2 already i taught i would ask since it proably should not merge until that is checked | |
| 19:01:09 | jaypipes | cfriesen: I don't doubt that. I'm saying that the cpu_threads_policy has nothing to do with resource accounting. | |
| 19:01:23 | kmalloc | sean-k-mooney: yeah, it might also not require a microversion, because it doesn't actually change anything yet, the change might need a microversion. | |
| 19:01:41 | cfriesen | jaypipes: but it does, since it determines whether we need to allocate additional resources on some compute nodes | |
| 19:01:58 | kmalloc | sean-k-mooney: i'll make sure to get some time to ensure we aren't introducing any breaking changes there (i don't think we are) that would require microversion bumps and if we are... make sure it's added | |
| 19:02:01 | sean-k-mooney | kmalloc: oh adding the scope_types=['project'] does not change the behavior | |
| 19:02:11 | kmalloc | sean-k-mooney: yeah it shouldn't. | |
| 19:02:18 | jaypipes | cfriesen: whether the host has HT enable impacts the reporting of CPU inventory for the host (or its NUMA nodes), but cpu_threads_policy only affects which host processors the guest is willing to be pinned to. | |
| 19:02:48 | jaypipes | cfriesen: are you referring to cpu_allocation_policy? | |
| 19:02:56 | jaypipes | or cpu_threads_policy? | |
| 19:02:57 | kmalloc | sean-k-mooney: yeah it's meant to just allow for richer policy allowing for defining the scope-types in policy as well | |
| 19:03:04 | cfriesen | jaypipes: no. If I ask for 1 PCPU and set cpu_thread_policy=ISOLATE then it will actually consume two host CPUs (the main one and the HT sibling) | |
| 19:03:25 | kmalloc | sean-k-mooney: but that should just be allowing for more specific policy enforcement, not actually making a change to behavior. | |
| 19:04:20 | kmalloc | sean-k-mooney: that is why i was feeling confused, but you know, i am 100% willing to assume I was mis-understanding something since it's in nova's code base and i don't spend as much time there. | |
| 19:04:22 | sean-k-mooney | kmalloc: ok cool sound like you have it well in hand in anycase. just taught i would ask for my own knollage i avoid api changes unless i have no choice | |
| 19:04:35 | kmalloc | sean-k-mooney: cool. and again ty very much! | |
| 19:04:57 | cfriesen | jaypipes: basically by enabling cpu_thread_policy=ISOLATE I'm asking for the entire host core, which maps to two ht siblings. But each of those siblings was exposed to placement as a PCPU. | |
| 19:05:24 | jaypipes | cfriesen: it doesn't always map to 2 HTs. | |
| 19:05:37 | cfriesen | jaypipes: true, it could be more | |
| 19:05:38 | jaypipes | cfriesen: yet more hardware vendor undefined randomness. | |
| 19:05:40 | sean-k-mooney | jaypipes: unfrotuently yes | |
| 19:05:49 | cfriesen | jaypipes: or it could be 1, if HT is disabled on the host | |
| 19:06:01 | jaypipes | cfriesen: no, that is the host inventory of things. | |
| 19:06:03 | sean-k-mooney | or 8 HT if its powerpc | |
| 19:06:13 | jaypipes | cfriesen: again, I understand the host inventory part of this. | |
| 19:06:35 | jaypipes | cfriesen: my problem is with flavors that consume different amounts of resources on different hosts. | |
| 19:06:59 | cfriesen | yep, that's exactly what this does, since it depends on the host config | |
| 19:07:06 | sean-k-mooney | jaypipes: ya thats what happens for cpu_thread_policy=ISOLATE today | |
| 19:07:27 | jaypipes | cfriesen: which, due to the design of cpu_threads_policy, being a string of "isolate|prefer|share" is entirely impossible to predict an integer amaount of some CPU resources that will *actually* be consumed by the guest. | |
| 19:08:07 | sean-k-mooney | jaypipes: yes. this was not an issue in icehouse but its biting us now | |
| 19:08:23 | mlavalle | rybridges, melwitt: Look at slide 28 in https://www.slideshare.net/MiguelLavalle/routed-networks-sydney. That shows you what you should see in Placement. For each segment in a routed network, that is the Placement structure that you should see | |
| 19:08:28 | cfriesen | jaypipes: the original goal was to improve flexibility by enabling hyperthreads, while allowing instances to ask for whole cores if they need it for performance. | |
| 19:08:31 | tblakes | mriedem: For bug https://bugs.launchpad.net/nova/+bug/1756360, it looks like we're going to need to implement __repr__ for NovaExceptions. Do you have any input on the format we want to return? | |
| 19:08:33 | openstack | Launchpad bug 1756360 in OpenStack Compute (nova) "Serializer strips Exception kwargs" [Undecided,Incomplete] - Assigned to Tyler Blakeslee (tblakes) | |
| 19:09:30 | sean-k-mooney | jaypipes: cfriesen ya the original intel proposal made pinning a host config option with no flavor extra specs. then you would just use aggregate to make teh desision | |
| 19:09:31 | cfriesen | jaypipes: actually, it *is* possible to predict what will be consumed by the guest given the host information | |
| 19:09:58 | cfriesen | jaypipes: it's just that it could be different from one compute node to another | |
| 19:09:59 | mlavalle | rybridges, melwitt: a routed network is a network where segments are associated to its subnets. In other words, if you do a GET of those subnets, all of them should have a valua in their 'sehment_id' attribute | |
| 19:10:10 | sean-k-mooney | cfriesen: not before the placement allocate_candidates request | |
| 19:10:26 | mlavalle | rybridges, melwitt: 'segment_id'^^^^ | |
| 19:10:44 | cfriesen | sean-k-mooney: correct, unless we wanted to model siblings in placement. :) | |
| 19:11:10 | mriedem | tblakes: can you reply to gibi's question about reproducing this in comment 1 | |
| 19:11:20 | sean-k-mooney | cfriesen: lets not unless its a trait | |
| 19:11:29 | melwitt | rybridges: did you verify whether the nova_api.aggregate_hosts table has anything in it? in case it's some kind of scoping issue with doing the 'openstack aggregate show' not showing it? | |
| 19:11:45 | sean-k-mooney | event then its a state ful trait which is kind of a bad thing | |
| 19:11:46 | cfriesen | sean-k-mooney: actually, a trait of "number of HT siblings" might make sense | |
| 19:11:49 | openstackgerrit | Julia Kreger proposed openstack/nova master: WIP: Add microversion to ironic client wrapper call https://review.openstack.org/554762 | |
| 19:12:09 | sean-k-mooney | cfriesen: it could but its missueing tratits | |
| 19:12:17 | sean-k-mooney | and it changes the request | |
| 19:13:29 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: virt: allow instances to be booted with trusted VFs https://review.openstack.org/485522 | |
| 19:13:55 | sean-k-mooney | i guess you could tag the PCPU RP with HW_1_HT and HW_2_HT if HT was on | |
| 19:14:34 | sean-k-mooney | then maybe combine that with forbiden tratis to avoid HT hosts | |
| 19:14:47 | sdeath | Q: upgrade from Ocata->Pike; now services are showing up as duplicates (same ID, UUID, etc). Any ideas as to what might be causing it? | |
| 19:15:05 | sdeath | nova service-list, openstack compute servicec list | |
| 19:15:20 | sdeath | (asked last couple days running on #openstack) | |
| 19:15:22 | sean-k-mooney | its a strech jay im assuming you would not like tratis for HT amount | |