| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-27 | |||
| 11:04:58 | openstackgerrit | Takahito Hirose proposed openstack/python-novaclient master: api_version decorator becomes an error in Python 3.5.0. https://review.openstack.org/564731 | |
| 11:32:05 | sean-k-mooney | stephenfin: impresive http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2018-04-26.log.html#t2018-04-26T10:23:28 ill add them to my review queue but it could take a while to get to all of them. is it working end ot end? | |
| 11:34:00 | openstackgerrit | Martin Midolesov proposed openstack/nova master: vmware:Validation video ram against the max video ram allowed. https://review.openstack.org/564193 | |
| 11:59:45 | fried_rice | ō/ | |
| 12:00:23 | sean-k-mooney | fried_rice: o/ | |
| 12:01:37 | sean-k-mooney | fried_rice: am i correct in assuming that you are currently targeting Rocky 2 to complete the granular resouce request now that we are past rocky 1 | |
| 12:02:03 | fried_rice | sean-k-mooney: I hadn't really considered a "target" tbh | |
| 12:02:44 | fried_rice | sean-k-mooney: I'm not hurrying, because jaypipes had said he didn't want it to go in until nrp was done. | |
| 12:03:19 | fried_rice | sean-k-mooney: I personally don't think it needs to wait (we can still get some good scenarios with just sharing providers) but that's taken some of the steam off. | |
| 12:03:35 | sean-k-mooney | nrp is kind of done already. at least the ablity to model things with nested resouce providers but i gues you mean the ablity to use nRPs with allocation candiates | |
| 12:03:44 | mdbooth | LibvirtDriver._should_disconnect_volume() has no locking :( | |
| 12:03:58 | fried_rice | sean-k-mooney: The latter, without which nrp doesn't help us much. | |
| 12:03:59 | mdbooth | s/volume/target/ | |
| 12:04:29 | fried_rice | mdbooth: locking schmocking. It must be eventlet's fault. | |
| 12:04:48 | mdbooth | fried_rice: Every time... we never add locking to this stuff. | |
| 12:04:52 | sean-k-mooney | fried_rice: well it allows you to do all of the discovery part in things like cyborg or other project that will use NPRs in the future just not the schduling part | |
| 12:05:29 | fried_rice | sean-k-mooney: Right. And without the scheduling part... what can we do with it? | |
| 12:06:03 | cdent | gaze at our lovely towers? | |
| 12:06:04 | mdbooth | The rule of thumb is: if there's an opportunity for a race, it's almost always actually a race. | |
| 12:06:33 | sean-k-mooney | fried_rice: expose our inventories of resouces and starte to think about how to organise the tree and what to model as seperate resouces classes/ common with traits | |
| 12:07:17 | sean-k-mooney | fried_rice: so baicaly nothing beyond get a feel for how the api will work when we have requests | |
| 12:07:41 | fried_rice | sean-k-mooney: Yup. We should totally start to think about that. And like half of the specs open/approved for Rocky think about it. | |
| 12:07:43 | sean-k-mooney | cdent: i proably should | |
| 12:08:19 | fried_rice | sean-k-mooney: Which of course means that those specs are dead in the water until nrp works in alloc cands. | |
| 12:08:21 | sean-k-mooney | fried_rice: yes i just find its eaiser to do that kind of thing when you have something concreate to test it with | |
| 12:28:46 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Fix the policy argument in server-group-create https://review.openstack.org/564699 | |
| 12:31:19 | leakypipes | sean-k-mooney: https://review.openstack.org/#/c/554529/ | |
| 12:32:01 | sean-k-mooney | leakypipes: oh cool i see its jsut pending a +w | |
| 12:38:40 | fried_rice | sean-k-mooney: tbc, that patch does *not* enable nrp in alloc cands. There's half a dozen patches on top of it that get closer. But the actual microversion patch to make it work through the API has not yet been proposed (to my knowledge). | |
| 12:39:41 | sean-k-mooney | fried_rice: ok im just reading through the patch now | |
| 12:40:01 | fried_rice | sean-k-mooney: Best of luck. | |
| 12:40:14 | fried_rice | sean-k-mooney: If you understand this sqla stuff, we could use your help :) | |
| 12:40:29 | kashyap | sean-k-mooney: When you can, want to ACK this: https://review.openstack.org/#/c/563926/ | |
| 12:40:38 | kashyap | (It's the Lift the restriction of choices for `cpu_model_extra_flags`) | |
| 12:40:59 | kashyap | sean-k-mooney: After discussion with QEMU / libvirt folks, I've lifted restriction for all three CPU modes | |
| 12:41:01 | sean-k-mooney | fried_rice: i understad some of it but leakypipes still runs rings around me when it comes to sql | |
| 12:41:26 | kashyap | (See the config file help text for a valid use case.) | |
| 12:41:49 | sean-k-mooney | kashyap: adding flag to host-passthrough may be unsafe | |
| 12:42:03 | kashyap | sean-k-mooney: I have a good counter-argument | |
| 12:42:25 | kashyap | sean-k-mooney: I present you ... this test case: https://kashyapc.fedorapeople.org/CPU-flags-and-host-passthrough/readme.txt | |
| 12:42:54 | sean-k-mooney | kashyap: you mean the fixed the fact kvm can crash internally when using nested virt and emulating cpu instruction not supported by the host | |
| 12:42:55 | kashyap | sean-k-mooney: In brief: Sometimes QEMU disables certain CPU features (like Intel's "invtsc" (the Invariant TSC) | |
| 12:43:17 | kashyap | In that case, you _do_ want to specify explicitly the CPU flag *even with* 'host-passthrough' | |
| 12:43:34 | kashyap | sean-k-mooney: No, a different case ^ | |
| 12:43:59 | fried_rice | stephenfin: Remember I was having trouble rebuilding my docs env under tox? | |
| 12:44:15 | kashyap | sean-k-mooney: Even if your baremetal host has 'invtsc', if you do 'host-passthrough' and check /proc/cpuinfo, you won't find it: | |
| 12:44:18 | kashyap | $ cat /proc/cpuinfo | grep nonstop_tsc | |
| 12:44:21 | kashyap | $ echo $? | |
| 12:44:23 | kashyap | 1 | |
| 12:44:44 | kashyap | sean-k-mooney: However, if you do specify the flag explicitly with 'host-passthrough', and again check /proc/cpuinfo in the guest, you _will_ find it. | |
| 12:44:48 | sean-k-mooney | kashyap: hehe yes that case makes sense. i just wanted to point out that kvm + nested virt + emulating some instruction not supported by the host cpu has been buggy in the past so we have to be careful what you add | |
| 12:44:55 | fried_rice | stephenfin: I dorked with it some more this morning and was able to get around it by adding --no-cache to the pip command (in the [testenv] section of tox.ini). | |
| 12:45:09 | kashyap | sean-k-mooney: Okido; Yeah, I see your other point | |
| 12:45:36 | kashyap | sean-k-mooney: That is strictly in the wheelhouse of the Operator; if they insist on shooting themselves in the foot, we can't help | |
| 12:46:13 | sean-k-mooney | kashyap: true but we can at least tell them the gun is loaded when we give it to them in docs :) | |
| 12:46:47 | kashyap | Haha | |
| 12:46:59 | kashyap | These gun metaphors make me twitch | |
| 12:47:24 | sean-k-mooney | ya i guess in current climate they can sometimes be in poor taste | |
| 12:47:25 | kashyap | sean-k-mooney: If you have a preferred wording, please note in the review | |
| 12:48:22 | sean-k-mooney | kashyap: well im conflited on that as personaly i consider it to be a kvm/kernel bug so nova should not have to tell you about it. | |
| 12:48:28 | kashyap | Yeah | |
| 12:48:37 | kashyap | Probably let's not confuse | |
| 12:48:51 | kashyap | The help text for that already requires undivided attention to read and apply it properly | |
| 12:49:12 | kashyap | We'll get to it when we get the first bug filed in that area | |
| 12:49:54 | sean-k-mooney | the old example of what used to trigger it was adding AVX on a cpu that did not have it caused a fallback to sse4 instruction in qemu but that fallback was buggy when used with kvm and nested virt turned on | |
| 12:50:33 | sean-k-mooney | i think that has been fix however. | |
| 12:51:56 | kashyap | Aside: What does AVX do? | |
| 12:52:15 | kashyap | Ah, found it: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions | |
| 12:53:28 | sean-k-mooney | kashyap: yep its basicaly extra wide simd instructions. which is why you can emulate them with sse instrutions at a performace penalty but still better then not using avx at all | |
| 12:54:00 | kashyap | I see, thanks for the explanation | |
| 13:00:13 | sean-k-mooney | kashyap: out of interest does cpu model Haswell-noTSX-IBRS have meltdown/specter fixed | |
| 13:01:26 | sean-k-mooney | the fact that its IBRS and not noIBRS implies it doing speclitive execution | |
| 13:11:50 | mdbooth | jgriffith: https://bugs.launchpad.net/nova/+bug/1767363 | |
| 13:11:50 | openstack | Launchpad bug 1767363 in OpenStack Compute (nova) "Deleting 2 instances with a common multi-attached volume can leave the volume attached" [Undecided,New] | |
| 13:12:20 | kashyap | sean-k-mooney: Was AFK briefly | |
| 13:12:35 | kashyap | sean-k-mooney: Yes, and no | |
| 13:13:19 | sean-k-mooney | kashyap: yes to meltdown and not to specter because qemu/kvm/kernel is patch to prevent that? | |
| 13:13:35 | kashyap | Exactly | |
| 13:14:05 | sean-k-mooney | kashyap: cool then IBRS makes sense | |
| 13:14:07 | kashyap | sean-k-mooney: And more over: those -noIBRS models _doesn't_ include 'PCID' (some got confused) | |
| 13:16:01 | sean-k-mooney | kashyap: right well i guess it you dont do speculatve branch execution then you dont need pcid to create seperate user/kernel adress contextes in the guest to mitigate the security risk | |
| 13:16:43 | kashyap | sean-k-mooney: Hmm, didn't think that way; that's a good point. | |
| 13:23:26 | stephenfin | sean-k-mooney: Sure is. I tested on an AIO DevStack deployment with OVS and things get shunted to the correct NUMA node | |
| 13:23:43 | sean-k-mooney | stephenfin: cool | |
| 13:25:30 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: Lift the restriction of choices for `cpu_model_extra_flags` https://review.openstack.org/563926 | |
| 13:30:34 | sean-k-mooney | finucannot: did you see my comment on https://review.openstack.org/#/c/564439/1 | |
| 13:31:16 | kashyap | mriedem: Nice of you to fix the nits yourself | |
| 13:31:21 | sean-k-mooney | finucannot: also cool i might see if i can test it with ovs-dpdk but it will be next week at the earliest before i do | |
| 13:33:07 | finucannot | sean-k-mooney: That would be great. I was planning to do that myself. I also want to do some multi-host testing (to properly validate the scheduler aspects of things) though live migration is off the cards, seeing as this requires CPU pinning | |
| 13:33:49 | finucannot | sean-k-mooney: Question though: both you and jaypipes keep using the term "network backend". What do mean? | |
| 13:33:57 | finucannot | e.g. https://review.openstack.org/#/c/564439/1/nova/objects/numa.py@164 | |
| 13:35:21 | sean-k-mooney | finucannot: when i was using "network backend" i was referning to ovs vs linux bridge vs vpp vs sriov ... | |
| 13:35:29 | leakypipes | bauzas: so I've yet to see a single operator respond with a custom scheduler filter being used in the openstack-operators survey on scheduler filters. | |
| 13:35:50 | finucannot | sean-k-mooney: So neutron agents? | |
| 13:36:08 | sean-k-mooney | finucannot: so neutron can only have one tunnel enpoint ip for the ovs agent but if you also ran linux bridge on the host it could have a different one | |
| 13:36:13 | sean-k-mooney | finucannot: ya basically | |
| 13:36:22 | bauzas | leakypipes: maybe they're shy ? :p | |
| 13:38:41 | finucannot | sean-k-mooney: Yeah, I think we need to settle on terminology here. So to start, we've got neutron agents. Theoretically you can run many of these on one host but we're going to consider that out of scope here, I guess | |
| 13:39:13 | sean-k-mooney | finucannot: normally you will have sriov + one other maxium per host | |
| 13:40:20 | sean-k-mooney | finucannot: you can have more but gernally running more then two agent does not by you anything usefull and is a pain to mainatin | |