| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-16 | |||
| 08:06:44 | mriedem | b/c with the microversion we can list the down cell VMs with the UNKNOWN status | |
| 08:07:06 | tssurya | right so we need to change this now nad backport also I guess to Queens ? | |
| 08:07:06 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Make monkey patch work in uWSGI mode https://review.openstack.org/592285 | |
| 08:07:28 | tssurya | before we start merging the UNKNOWN stuff. | |
| 08:07:35 | mriedem | tssurya: not sure about backports, but maybe if it defaults to current behavior | |
| 08:08:33 | tssurya | mriedem: the previous behaviour was 500 for down cells, do you suggest the config option to return an empty list if the user has stuff in the down cell then ? | |
| 08:08:59 | tssurya | instead of skipping | |
| 08:09:02 | tssurya | that cell | |
| 08:09:09 | mriedem | originally the behavior was 500 right? | |
| 08:09:12 | tssurya | yes | |
| 08:09:13 | mriedem | you fixed that to skip down cells | |
| 08:09:16 | tssurya | yes | |
| 08:09:26 | mriedem | so yes, they want an option to say don't skip, but just return an empty list | |
| 08:09:32 | mriedem | well, | |
| 08:09:37 | mriedem | if the project has any instances in a down cell | |
| 08:09:55 | tssurya | okay, | |
| 08:10:01 | mriedem | i think it would only apply to non-admin instance list; an admin doing nova-list --all-tenants wouldn't apply here | |
| 08:10:09 | openstackgerrit | fupingxie proposed openstack/nova master: Support list for alias in pci section in nova.conf https://review.openstack.org/592243 | |
| 08:10:41 | tssurya | ack, | |
| 08:12:48 | tssurya | mriedem: in the agenda you say "show is OK", well show wouldn't work at all before the new microversion, would just be a 500 | |
| 08:13:01 | tssurya | that's okay right ? | |
| 08:13:04 | mriedem | oh yeah, right | |
| 08:13:06 | mriedem | yeah | |
| 08:13:31 | mriedem | nova CLI will do the right thing once you're using latest novaclient and server, | |
| 08:13:42 | mriedem | in that it will request the latest microversion which will get users the UNKNOWN status behavior | |
| 08:15:58 | tssurya | yes, speaking of which I am working on that right now: does the new microversion which skips certain keys need the api_samples functional tests ? or is it enough I clearly specify this in the api-ref docs ? | |
| 08:17:48 | mriedem | would you have a sample of what this looks like in the api-ref docs? | |
| 08:17:50 | mriedem | i think we probably should | |
| 08:17:56 | mriedem | and those are generated from the api-samples functional tests | |
| 08:18:24 | tssurya | mriedem: yea, okay it woud surely be nice to have sample jsons of what a response from down cell versus up cell looks like | |
| 08:18:35 | mriedem | yeah | |
| 08:18:42 | tssurya | okay then will add those tests too, but before that will do this pre-microversion thing config | |
| 08:18:44 | tssurya | and ping you bacl | |
| 08:18:46 | tssurya | back* | |
| 08:20:50 | mriedem | ok cool | |
| 08:20:53 | mriedem | not sure if this should be a bug or what | |
| 08:20:59 | mriedem | that's why i added it to the meeting agenda | |
| 08:21:34 | tssurya | right, we kind of skipped yesterday's meeting. so maybe we can discuss it with dansmith when he wkaes up ? | |
| 08:25:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove ChanceScheduler https://review.openstack.org/576384 | |
| 08:28:51 | sean-k-mooney | mriedem: o/ | |
| 08:29:46 | sean-k-mooney | mriedem: i tested the live migration between ovs with iptables and ovs with connectracker hosts and you multi port binding changes work well for that case. | |
| 08:30:14 | mriedem | ho | |
| 08:30:15 | mriedem | *hot | |
| 08:30:37 | sean-k-mooney | mriedem: i need to still test kernel ovs to ovs-dpdk an ovs to ovn but atleast the usecase called out in the spec is good | |
| 08:30:43 | mriedem | Kevin_Zheng: yikun: your cells perf email mentions needing to monkey_patch eventlet in wsgi mode, but didn't mention you needed to monkey patch the time module, right? | |
| 08:31:19 | Kevin_Zheng | Ah, yes, I forget that part | |
| 08:31:28 | mriedem | https://github.com/openstack/oslo.db/blob/9c6695969f6502f52bc86f22af0758715fdc57c0/oslo_db/sqlalchemy/engines.py#L51 | |
| 08:31:29 | mriedem | yeah | |
| 08:31:41 | Kevin_Zheng | yikun could you amend that mail to show why the time module matters? | |
| 08:32:26 | mriedem | i think you also only mention that all 10 cells are using a single DB at the end | |
| 08:32:30 | yikun | yeah, sure | |
| 08:32:32 | mriedem | probably good to mention that early and why | |
| 08:34:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove ChanceScheduler https://review.openstack.org/576384 | |
| 08:34:30 | Kevin_Zheng | OK, I will be more careful in the future | |
| 08:35:28 | sean-k-mooney | mriedem: what is the importance of time module in this case beyond causeing eventlet to yeild execution of the current green tread? | |
| 08:36:20 | mriedem | sean-k-mooney: we iterate the cells with greenthreads, | |
| 08:36:22 | sapd1 | sean-k-mooney: melwitt hi. I have a solution for octavia to use SR-IOV port. this is a workaround with custom in octavia. | |
| 08:36:38 | mriedem | but without monkey_patching the time module, we aren't actually iterating the cells in parallel | |
| 08:37:21 | mriedem | sapd1: how so? | |
| 08:37:29 | sean-k-mooney | mriedem: ah ok. well even with monkey patching its technicall concurent not parralel but i get the point | |
| 08:37:33 | mriedem | sapd1: see https://review.openstack.org/#/c/591898/ | |
| 08:38:06 | sean-k-mooney | sapd1: do you have a review available or a paste of the patch we can review? | |
| 08:38:14 | sapd1 | mriedem: I launch amphora instance using SR-IOV network instead of plug SR-IOV port to amphora instance. So I can launch amphora sucess. | |
| 08:38:24 | sapd1 | s/sucess/success/ | |
| 08:38:34 | mriedem | i don't know what an amphora instance is | |
| 08:38:50 | sean-k-mooney | mriedem: its a specifc load balancer impmentation | |
| 08:39:18 | sapd1 | mriedem: amphora instance is a instance running haproxy inside | |
| 08:39:23 | sapd1 | s/a/an | |
| 08:40:23 | sean-k-mooney | mriedem: there were a few specs in the past to actully make attach work for sriov. its non trival but doable in most cases | |
| 08:41:10 | sean-k-mooney | mriedem: we are missing the resouce tracker device claim and in future the placemnet allocation update to make it work correctly | |
| 08:41:46 | sean-k-mooney | mriedem: as part of the RT claim we also need to update the neutron port binding with pci adress info but thats a detail. | |
| 08:42:33 | sean-k-mooney | mriedem: sapd1 would like to resubmit the spec for stien so octavia can support loadblancers with sriov interfaces | |
| 08:42:57 | tssurya | Keving_Zheng, yikun: I like this part of the cells perf email "so, now the queries are in parallel, but the whole thing still seems serial.", we have been having the same experience :) and thanks for the efforts | |
| 08:43:36 | Kevin_Zheng | tssurya, interesting, since you guys got 70 cells, how does it work? | |
| 08:44:16 | tssurya | yea which is why we pushed for the patch of using inst_mappings to narrow it down to 2 or 3 cells per user | |
| 08:44:54 | tssurya | so that way it works fine for us on the user end | |
| 08:45:15 | tssurya | mainly because of the way we have our cells designed | |
| 08:45:46 | Kevin_Zheng | Hmm, that might not work for us, as I mentioned before, we are unable to limit our users to cells | |
| 08:46:24 | tssurya | Kevin_Zheng: yea I remember your comment on that patch, but I guess we still need to find a way to make the data gathering part faster then | |
| 08:47:55 | mriedem | sean-k-mooney: yeah i'm aware of the spec | |
| 08:48:48 | mriedem | sean-k-mooney: i also added you to an sriov-bond spec that our product team is interested in | |
| 08:49:07 | mriedem | tssurya: Kevin_Zheng: that reminds me, | |
| 08:49:29 | mriedem | tssurya: the reason we can't restrict users to cells in our public cloud is because flavors are defined by cell (among other things) | |
| 08:49:41 | mriedem | so if a user wants to use different flavors they can get instances in different cells | |
| 08:49:47 | mriedem | tssurya: how is that handled at CERN? | |
| 08:50:27 | sapd1 | sean-k-mooney: I think nova should care about SR-IOV instead of octavia. | |
| 08:50:35 | tssurya | mriedem: we actually map projects to cells | |
| 08:51:28 | tssurya | the default flavors are available everywhere and then the special flavors are available in certain cells which are then restricted for the tenants as well | |
| 08:51:32 | sean-k-mooney | sapd1: well i think this is a nova issue rather then octavia. i cant unfortunetly make nova care about sriov but enough of our users do that we try to support it :) | |
| 08:51:40 | tssurya | so only certain tenats end up in certain cells | |
| 08:51:49 | sean-k-mooney | mriedem: oh i had not seen it but i will take a look | |
| 08:53:54 | sean-k-mooney | sapd1: i started looking into this again last night. i think i know a slightly hacky way to fix this in the libvirt virt diriver. i am look to see if we can adress the missing logic in the compute manager instead as most of the code should not depend on the driver | |
| 08:54:03 | mriedem | tssurya: yeah i know cern mapped projects to cells, | |
| 08:54:16 | mriedem | oh ok so basic flavors are global, | |
| 08:54:34 | mriedem | then per-cell flavors have tenant access for the tenants in those cells | |
| 08:54:55 | mriedem | do you have any idea how many projects exist in CERN? | |
| 08:54:56 | mdbooth | I'm trying to enable debug logs in a functional test. Thought it would be as simple as self.flags(debug=True), but apparently not. | |
| 08:55:56 | sean-k-mooney | tssurya: are you using dansmith's prefilter to do the tenat to cell mapping or something else? | |
| 08:56:14 | mriedem | mdbooth: OS_DEBUG=True | |
| 08:56:23 | lyarwood | OS_DEBUG=1 tox -e functional [..] | |