| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-14 | |||
| 15:20:49 | sdague | dansmith / jroll it's not just a pbr limitation, there are other substantial issues there with the python packaging toolchain | |
| 15:21:13 | raj_singh | johnthetubaguy: yea running out of bandwidth for this cycle | |
| 15:22:05 | jroll | sdague: that's the understatement of the day :) | |
| 15:22:17 | jroll | I know pbr has hard blockers for it, that's all I wanted to point out | |
| 15:22:20 | mriedem | Kevin_Zheng: i know it's late for you, just fyi that https://review.openstack.org/#/c/429569/ could use some small improvements and then i'm +2 | |
| 15:25:51 | openstackgerrit | Dan Smith proposed openstack/nova master: Make conductor ask scheduler to limit migrates to same cell https://review.openstack.org/438025 | |
| 15:25:55 | dansmith | bauzas: can you look at this ^ ? | |
| 15:26:23 | dansmith | bauzas: I think that's what you're going for.. I still need to do some tests and fix the commit message and stuff, but I think that will efficiently make the caching scheduler behave the same as the filter scheduler | |
| 15:26:41 | bauzas | dansmith: sure, will do | |
| 15:26:47 | dansmith | i.e. filter based on cell per request, from the global collected list | |
| 15:26:58 | bauzas | dansmith: I need to go to the garage soon, but I'll look at it quickly | |
| 15:27:13 | dansmith | just a brief look to see if it's what you meant would be good | |
| 15:27:20 | dansmith | then I'll release you to the garage | |
| 15:28:55 | bauzas | dansmith: so https://review.openstack.org/#/c/438025/23/nova/scheduler/caching_scheduler.py | |
| 15:29:22 | bauzas | dansmith: lgtm, but I was also thinking about _get_by_hosts() returning a dict instead of a list | |
| 15:29:38 | bauzas | oops _get_up_hosts | |
| 15:29:53 | bauzas | dansmith: so we could just lookup by the cell name | |
| 15:30:00 | hamzy | hey mriedem I think that I am running into https://bugs.launchpad.net/nova/+bug/1670627 is there a way to clean up the instances from the CLI? I guess that I need to do some DB cleanup? | |
| 15:30:00 | openstack | Launchpad bug 1670627 in OpenStack Compute (nova) ocata "quota is always in-use after delete the ERROR instances " [Critical,In progress] - Assigned to Matt Riedemann (mriedem) | |
| 15:30:56 | dansmith | bauzas: okay, I need to see how much change that is to the other stuff, but probably okay | |
| 15:31:16 | bauzas | dansmith: it's just a nit tho | |
| 15:31:31 | bauzas | dansmith: but I'd prefer both of them rather than a filter | |
| 15:31:59 | mriedem | hamzy: the instances are deleted right? | |
| 15:32:13 | mriedem | hamzy: the problem is the quota usage isn't decremented | |
| 15:32:17 | hamzy | mriedem, yes "nova list --all-tenants" shows nothing | |
| 15:32:32 | hamzy | My compute overview page shows "RAM Used 48GB of 50GB" and 6 used instances... | |
| 15:32:57 | dansmith | bauzas: yeah, I don't know why I didn't think to do this when johnthetubaguy and I were talking about it.. probably just misunderstanding the cachingscheduler at the time | |
| 15:33:06 | mriedem | hamzy: yeah, and the "nova_cell0.usages" table should have that information in it, can you confirm | |
| 15:33:07 | mriedem | ? | |
| 15:33:14 | mriedem | oh wait, sorry, | |
| 15:33:16 | mriedem | not nova_cell0, | |
| 15:33:19 | mriedem | just nova.usages | |
| 15:33:34 | bauzas | dansmith: the problem with a filter is that operators would need to modify their config for an upgrade | |
| 15:33:38 | hamzy | mriedem, I have an AIO installation with a bunch of containers... do I need to attach to one of them> | |
| 15:33:45 | bauzas | dansmith: or they wouldn't check the cells | |
| 15:34:03 | bauzas | if they forget to modify their conf opt | |
| 15:34:09 | dansmith | bauzas: they wouldn't until they add another cell | |
| 15:34:10 | mriedem | hamzy: let me check something, we might have a CLI to run | |
| 15:34:19 | dansmith | bauzas: but I know, this is better, I'm not arguing | |
| 15:34:30 | bauzas | dansmith: sure, but they could also opt-out verifying cells, which I also dislike :) | |
| 15:34:40 | bauzas | anyway | |
| 15:34:42 | mriedem | cdent: replied to the ML thread on the placement client topic for completeness | |
| 15:34:51 | bauzas | mriedem: thanks for that, appreciatedf | |
| 15:34:52 | mriedem | and so i don't forget what we agreed on in 4 months time | |
| 15:34:52 | cdent | mriedem: yeah, saw that, thanks | |
| 15:34:53 | dansmith | bauzas: I'm not arguing for a filter | |
| 15:35:05 | bauzas | yeah no worries :) | |
| 15:35:20 | mriedem | hamzy: https://docs.openstack.org/developer/nova/man/nova-manage.html#nova-project | |
| 15:35:29 | mriedem | hamzy: i'm wondering if "nova-manage project quota_usage_refresh <project_id> [--user <user_id>] [--key <key>]" will help | |
| 15:35:47 | mriedem | hamzy: can you try that for the project in question | |
| 15:36:39 | hamzy | yeah let me figure out the project_id | |
| 15:38:25 | hamzy | mriedem, is there an openstack CLI equivalent? | |
| 15:38:32 | mriedem | hamzy: nope | |
| 15:38:35 | mriedem | it's direct to the nova db | |
| 15:38:53 | mriedem | and something in osc cli would require a rest api for this in the compute endpoint, which we don't have | |
| 15:41:06 | hamzy | mriedem, trying to figure out how to install it in my AIO environment | |
| 15:41:35 | mriedem | hamzy: nova-manage is available on the controller | |
| 15:41:42 | mriedem | it's a CLI | |
| 15:41:46 | mriedem | so "which nova-manage" | |
| 15:42:15 | mriedem | AIO == all in one? | |
| 15:42:20 | hamzy | yes | |
| 15:42:27 | hamzy | installed with openstack-ansible | |
| 15:44:06 | hamzy | mriedem, http://paste.openstack.org/show/602707/ that CLI is not in the utility-container | |
| 15:44:44 | mriedem | hamzy: i've never used openstack-ansible, so not sure how things are deployed, | |
| 15:45:09 | mriedem | hamzy: how about aio1_nova_conductor_container-41603c3d ? | |
| 15:45:55 | johnthetubaguy | not sure which container they run the nova-manage stuff in, worth asking andymccr in #openstack-ansible | |
| 15:45:57 | hamzy | nova command is there but not nova-manage | |
| 15:46:13 | hamzy | same with aio1_nova_console_container-fb9219e4... sigh | |
| 15:46:25 | andymccr | hamzy: it will be in the nova containers, but inside the venv for nova | |
| 15:46:34 | hamzy | ah | |
| 15:47:57 | hamzy | \o/ | |
| 15:48:54 | mriedem | now to see if that command fixes the usage | |
| 15:51:09 | raj_singh | johnthetubaguy: regarding https://review.openstack.org/#/c/438467/, I am wondering if new "force_complete" option could have choices of "abort, pause, post-copy", then we can deprecate live_migration_permit_post_copy? | |
| 15:51:44 | johnthetubaguy | raj_singh: but what about the force complete API that still uses that config? | |
| 15:51:47 | hamzy | mriedem, I don't think it did, but the overview page no longer shows the pie charts anymore | |
| 15:52:24 | johnthetubaguy | raj_singh: I think folks want to opt into post copy, most want it turned off, so a single config to do that makes sense I think | |
| 15:53:07 | raj_singh | johnthetubaguy: ok, let me rework the spec | |
| 15:53:29 | mriedem | hamzy: ok, we can check via nova cli, sec | |
| 15:54:27 | hamzy | mriedem, FYI I ran quota_usage_refresh on all four projects that I had | |
| 15:54:36 | mriedem | hamzy: https://docs.openstack.org/cli-reference/nova.html#nova-quota-show | |
| 15:54:37 | johnthetubaguy | raj_singh: we should add that context in the spec | |
| 15:54:51 | mriedem | nova quota-show --tenant <tenant id> | |
| 15:54:59 | mriedem | plus maybe --detail | |
| 15:55:05 | mriedem | nova quota-show --tenant <tenant id> --detail | |
| 15:55:16 | mriedem | see if the instances are back to 10 for limit and 0 for usage | |
| 15:55:37 | raj_singh | johnthetubaguy: yes, will update it soon. | |
| 15:57:31 | hamzy | mriedem, http://paste.openstack.org/show/602711/ | |
| 15:58:22 | johnthetubaguy | raj_singh: got a few nits on the follow on spec too | |
| 15:58:32 | mriedem | | instances | {u'limit': 10, u'reserved': 0, u'in_use': 6} | | |
| 15:58:40 | mriedem | yeah usage is still wrong | |
| 15:58:51 | raj_singh | johnthetubaguy: thanks for the review | |
| 16:00:09 | mriedem | hamzy: is the [database]/connection option in the nova.conf that the aio1-nova-conductor-container-41603c3d container is using pointing at the 'nova' database? | |
| 16:01:34 | openstackgerrit | sahid proposed openstack/nova master: libvirt: return a CPU overhead if isolate emulator threads requested https://review.openstack.org/385364 | |
| 16:01:34 | openstackgerrit | sahid proposed openstack/nova master: numa: fit instance NUMA node with cpus reserved onto host NUMA node https://review.openstack.org/400875 | |
| 16:01:36 | openstackgerrit | sahid proposed openstack/nova master: virt: update overhead to take into account vCPUs https://review.openstack.org/385024 | |
| 16:01:39 | hamzy | mriedem, there is a line "connection = mysql+pymysql://nova:random_password@172.29.236.100/nova?charset=utf8" | |
| 16:01:42 | openstackgerrit | sahid proposed openstack/nova master: libvirt: make emulator threads to run on the reserved pCPU https://review.openstack.org/400878 | |
| 16:01:44 | openstackgerrit | sahid proposed openstack/nova master: numa: take into account cpus reserved https://review.openstack.org/400876 | |
| 16:01:46 | openstackgerrit | sahid proposed openstack/nova master: numa: update numa usage incording the cpu reserved https://review.openstack.org/400877 | |
| 16:02:00 | mriedem | hamzy: yeah that's the nova db | |