| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-16 | |||
| 08:13:01 | tssurya | that's okay right ? | |
| 08:13:04 | mriedem | oh yeah, right | |
| 08:13:06 | mriedem | yeah | |
| 08:13:31 | mriedem | nova CLI will do the right thing once you're using latest novaclient and server, | |
| 08:13:42 | mriedem | in that it will request the latest microversion which will get users the UNKNOWN status behavior | |
| 08:15:58 | tssurya | yes, speaking of which I am working on that right now: does the new microversion which skips certain keys need the api_samples functional tests ? or is it enough I clearly specify this in the api-ref docs ? | |
| 08:17:48 | mriedem | would you have a sample of what this looks like in the api-ref docs? | |
| 08:17:50 | mriedem | i think we probably should | |
| 08:17:56 | mriedem | and those are generated from the api-samples functional tests | |
| 08:18:24 | tssurya | mriedem: yea, okay it woud surely be nice to have sample jsons of what a response from down cell versus up cell looks like | |
| 08:18:35 | mriedem | yeah | |
| 08:18:42 | tssurya | okay then will add those tests too, but before that will do this pre-microversion thing config | |
| 08:18:44 | tssurya | and ping you bacl | |
| 08:18:46 | tssurya | back* | |
| 08:20:50 | mriedem | ok cool | |
| 08:20:53 | mriedem | not sure if this should be a bug or what | |
| 08:20:59 | mriedem | that's why i added it to the meeting agenda | |
| 08:21:34 | tssurya | right, we kind of skipped yesterday's meeting. so maybe we can discuss it with dansmith when he wkaes up ? | |
| 08:25:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove ChanceScheduler https://review.openstack.org/576384 | |
| 08:28:51 | sean-k-mooney | mriedem: o/ | |
| 08:29:46 | sean-k-mooney | mriedem: i tested the live migration between ovs with iptables and ovs with connectracker hosts and you multi port binding changes work well for that case. | |
| 08:30:14 | mriedem | ho | |
| 08:30:15 | mriedem | *hot | |
| 08:30:37 | sean-k-mooney | mriedem: i need to still test kernel ovs to ovs-dpdk an ovs to ovn but atleast the usecase called out in the spec is good | |
| 08:30:43 | mriedem | Kevin_Zheng: yikun: your cells perf email mentions needing to monkey_patch eventlet in wsgi mode, but didn't mention you needed to monkey patch the time module, right? | |
| 08:31:19 | Kevin_Zheng | Ah, yes, I forget that part | |
| 08:31:28 | mriedem | https://github.com/openstack/oslo.db/blob/9c6695969f6502f52bc86f22af0758715fdc57c0/oslo_db/sqlalchemy/engines.py#L51 | |
| 08:31:29 | mriedem | yeah | |
| 08:31:41 | Kevin_Zheng | yikun could you amend that mail to show why the time module matters? | |
| 08:32:26 | mriedem | i think you also only mention that all 10 cells are using a single DB at the end | |
| 08:32:30 | yikun | yeah, sure | |
| 08:32:32 | mriedem | probably good to mention that early and why | |
| 08:34:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove ChanceScheduler https://review.openstack.org/576384 | |
| 08:34:30 | Kevin_Zheng | OK, I will be more careful in the future | |
| 08:35:28 | sean-k-mooney | mriedem: what is the importance of time module in this case beyond causeing eventlet to yeild execution of the current green tread? | |
| 08:36:20 | mriedem | sean-k-mooney: we iterate the cells with greenthreads, | |
| 08:36:22 | sapd1 | sean-k-mooney: melwitt hi. I have a solution for octavia to use SR-IOV port. this is a workaround with custom in octavia. | |
| 08:36:38 | mriedem | but without monkey_patching the time module, we aren't actually iterating the cells in parallel | |
| 08:37:21 | mriedem | sapd1: how so? | |
| 08:37:29 | sean-k-mooney | mriedem: ah ok. well even with monkey patching its technicall concurent not parralel but i get the point | |
| 08:37:33 | mriedem | sapd1: see https://review.openstack.org/#/c/591898/ | |
| 08:38:06 | sean-k-mooney | sapd1: do you have a review available or a paste of the patch we can review? | |
| 08:38:14 | sapd1 | mriedem: I launch amphora instance using SR-IOV network instead of plug SR-IOV port to amphora instance. So I can launch amphora sucess. | |
| 08:38:24 | sapd1 | s/sucess/success/ | |
| 08:38:34 | mriedem | i don't know what an amphora instance is | |
| 08:38:50 | sean-k-mooney | mriedem: its a specifc load balancer impmentation | |
| 08:39:18 | sapd1 | mriedem: amphora instance is a instance running haproxy inside | |
| 08:39:23 | sapd1 | s/a/an | |
| 08:40:23 | sean-k-mooney | mriedem: there were a few specs in the past to actully make attach work for sriov. its non trival but doable in most cases | |
| 08:41:10 | sean-k-mooney | mriedem: we are missing the resouce tracker device claim and in future the placemnet allocation update to make it work correctly | |
| 08:41:46 | sean-k-mooney | mriedem: as part of the RT claim we also need to update the neutron port binding with pci adress info but thats a detail. | |
| 08:42:33 | sean-k-mooney | mriedem: sapd1 would like to resubmit the spec for stien so octavia can support loadblancers with sriov interfaces | |
| 08:42:57 | tssurya | Keving_Zheng, yikun: I like this part of the cells perf email "so, now the queries are in parallel, but the whole thing still seems serial.", we have been having the same experience :) and thanks for the efforts | |
| 08:43:36 | Kevin_Zheng | tssurya, interesting, since you guys got 70 cells, how does it work? | |
| 08:44:16 | tssurya | yea which is why we pushed for the patch of using inst_mappings to narrow it down to 2 or 3 cells per user | |
| 08:44:54 | tssurya | so that way it works fine for us on the user end | |
| 08:45:15 | tssurya | mainly because of the way we have our cells designed | |
| 08:45:46 | Kevin_Zheng | Hmm, that might not work for us, as I mentioned before, we are unable to limit our users to cells | |
| 08:46:24 | tssurya | Kevin_Zheng: yea I remember your comment on that patch, but I guess we still need to find a way to make the data gathering part faster then | |
| 08:47:55 | mriedem | sean-k-mooney: yeah i'm aware of the spec | |
| 08:48:48 | mriedem | sean-k-mooney: i also added you to an sriov-bond spec that our product team is interested in | |
| 08:49:07 | mriedem | tssurya: Kevin_Zheng: that reminds me, | |
| 08:49:29 | mriedem | tssurya: the reason we can't restrict users to cells in our public cloud is because flavors are defined by cell (among other things) | |
| 08:49:41 | mriedem | so if a user wants to use different flavors they can get instances in different cells | |
| 08:49:47 | mriedem | tssurya: how is that handled at CERN? | |
| 08:50:27 | sapd1 | sean-k-mooney: I think nova should care about SR-IOV instead of octavia. | |
| 08:50:35 | tssurya | mriedem: we actually map projects to cells | |
| 08:51:28 | tssurya | the default flavors are available everywhere and then the special flavors are available in certain cells which are then restricted for the tenants as well | |
| 08:51:32 | sean-k-mooney | sapd1: well i think this is a nova issue rather then octavia. i cant unfortunetly make nova care about sriov but enough of our users do that we try to support it :) | |
| 08:51:40 | tssurya | so only certain tenats end up in certain cells | |
| 08:51:49 | sean-k-mooney | mriedem: oh i had not seen it but i will take a look | |
| 08:53:54 | sean-k-mooney | sapd1: i started looking into this again last night. i think i know a slightly hacky way to fix this in the libvirt virt diriver. i am look to see if we can adress the missing logic in the compute manager instead as most of the code should not depend on the driver | |
| 08:54:03 | mriedem | tssurya: yeah i know cern mapped projects to cells, | |
| 08:54:16 | mriedem | oh ok so basic flavors are global, | |
| 08:54:34 | mriedem | then per-cell flavors have tenant access for the tenants in those cells | |
| 08:54:55 | mriedem | do you have any idea how many projects exist in CERN? | |
| 08:54:56 | mdbooth | I'm trying to enable debug logs in a functional test. Thought it would be as simple as self.flags(debug=True), but apparently not. | |
| 08:55:56 | sean-k-mooney | tssurya: are you using dansmith's prefilter to do the tenat to cell mapping or something else? | |
| 08:56:14 | mriedem | mdbooth: OS_DEBUG=True | |
| 08:56:23 | lyarwood | OS_DEBUG=1 tox -e functional [..] | |
| 08:56:26 | lyarwood | yeah sorry | |
| 08:56:39 | mdbooth | mriedem lyarwood: Thanks! You saved me some rtfs :) | |
| 08:57:59 | mriedem | sean-k-mooney: wouldn't most of the work for sriov attach be doing the RT claim for a pci_request and if that fails, we fail the attach request | |
| 08:58:16 | mriedem | and then set the port host binding details as you mentioned | |
| 08:58:24 | mriedem | most of ^ is probably already done in the port binding code | |
| 08:59:32 | openstackgerrit | jichenjc proposed openstack/nova master: Add zvm admin intro and hypervisor information https://review.openstack.org/533125 | |
| 08:59:33 | openstackgerrit | jichenjc proposed openstack/nova master: Add zvm CI information https://review.openstack.org/533512 | |
| 09:01:17 | sean-k-mooney | mriedem: ya. which is why i think we can do all the work from the compute manager rather then the virt driver | |
| 09:01:26 | sapd1 | sean-k-mooney: Will you have plan to re-implement this feature? :D | |
| 09:03:30 | sean-k-mooney | mriedem: sapd1 these are my notes form last night https://etherpad.openstack.org/p/sriov-attach | |
| 09:04:11 | openstackgerrit | Merged openstack/nova master: VMware: add missing os types in vSphere sdk 6.5 https://review.openstack.org/591771 | |
| 09:05:00 | openstack | Launchpad bug 1708433 in OpenStack Compute (nova) "Attaching sriov nic VM fail with keyError pci_slot" [Medium,In progress] - Assigned to Matt Riedemann (mriedem) | |
| 09:05:00 | sean-k-mooney | sapd1: https://bugs.launchpad.net/nova/+bug/1708433 was opened by my old team as we had a customer request to enable sriov attach generically | |
| 09:05:38 | sean-k-mooney | sapd1: that was before my team was pulled off upstream work for internal projects. i have now move to redhat i might be able to work on this but not sure | |
| 09:05:52 | sean-k-mooney | i would have to ask | |
| 09:10:05 | sean-k-mooney | mriedem: there are a few more edge cases that we would have to handel. e.g. vm runs out of pci ports, updating instace request spec, numa affintiy and updating placement allocation would be the main ones. a spec is the best way to capture all the edgecases | |
| 09:11:14 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Remove the deprecated API extensions policies https://review.openstack.org/586872 | |
| 09:12:37 | sapd1 | sean-k-mooney: good point. Because I am not a developer, So I can't help you. Because nova souce code is too complicated | |
| 09:13:03 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Remove the deprecated API extensions policies https://review.openstack.org/586872 | |
| 09:13:09 | mriedem | sean-k-mooney: that's why there was a spec :) | |