| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-16 | |||
| 17:26:17 | mriedem | so we didn't have to guess | |
| 17:26:20 | dansmith | cell0 will grow indefinitely and need to be purged, api db will not.. that means different planning for those types of things | |
| 17:26:30 | dansmith | mriedem: that's fine too | |
| 17:27:21 | melwitt | dansmith: hm, okay. I hadn't thought of that | |
| 17:28:03 | openstackgerrit | OpenStack Proposal Bot proposed openstack/python-novaclient: Updated from global requirements https://review.openstack.org/420861 | |
| 17:44:32 | openstackgerrit | Dan Peschman proposed openstack/nova: nova-manage cell_v2 map_cell0 exit 0 https://review.openstack.org/420132 | |
| 17:45:41 | openstackgerrit | Gábor Antal proposed openstack/nova: Transform instance.reboot notifications https://review.openstack.org/382959 | |
| 17:47:51 | mriedem | dansmith: melwitt: so i don't have super strong feelings either way, would be nice if alaski were around to break any ties | |
| 17:48:23 | mriedem | i lean toward the name change trumping the chance of the cell0 and nova_api db connection creds being the same | |
| 17:48:48 | mriedem | i've already started working on backporting the fixes to stable/newton in case we want those | |
| 17:49:07 | mriedem | i tend to think that with most deployments not even at newton yet those would be good to have fixed by the time they upgrade | |
| 17:49:10 | dansmith | I think that we're leaning towards people not doing the simple thing, and thus not just arbitrarily using the same url/creds for everything, so I want us to be more sane | |
| 17:49:53 | mriedem | yeah which making the db connection arg required would fix that, but i'm not sure i want to backport that change | |
| 17:50:13 | dansmith | yeah | |
| 17:50:16 | mriedem | we could make db connection required in ocata, not backport anything, but put out a release note in newton saying you should be explicit when using map_cell0 | |
| 17:50:29 | melwitt | yeah, I don't think we should do that being that we know people don't want to be forced to pass connection strings on the command line | |
| 17:52:01 | mriedem | melwitt: sure, but then we're also forcing them to handle it in config, i guess they can pass the --config-file option to nova-manage too | |
| 17:52:13 | melwitt | if there's reason to have api db and cell0 db not be co-located (which has been mentioned today) then that addresses my concern about it. I had been thinking co-locating them would be most common, but if it wouldn't be, then I think it's fine not to default it that way | |
| 17:52:22 | mriedem | and by handle it in config, i mean if they want cell0 to use the same host/user/password as the nova_api db and not the main db | |
| 17:52:41 | melwitt | mriedem: yes, --config-file is key to setting up cells v1 anyway | |
| 17:52:55 | dansmith | if they don't want to use the command line, then they need to have two separate places to define it anyway | |
| 17:52:59 | dansmith | in the config I mean | |
| 17:53:39 | openstackgerrit | Gábor Antal proposed openstack/nova: Added instance.reboot.error to the legacy notifications https://review.openstack.org/411816 | |
| 17:53:46 | dansmith | the re-use of the config elements to avoid using command line args is super confusing and annoying to me | |
| 17:53:52 | mdbooth | I have a test which uses threading (it's a concurrency test, this is unavoidable) | |
| 17:53:54 | openstackgerrit | Gábor Antal proposed openstack/nova: Transform instance.reboot.error notification https://review.openstack.org/411791 | |
| 17:54:07 | dansmith | it would be better, IMHO, if we had a yaml format for creds or something that we just had people create for each insertion they want to use, | |
| 17:54:21 | mdbooth | I'm explicitly using threading.Thread rather than eventlet.greenthread, because the latter doesn't do what I need | |
| 17:54:35 | dansmith | instead of "create a fake config, put just this section/keys in there, then we'll use that to create a db row instead of reading that config at runtime like it was intended"... | |
| 17:54:49 | mdbooth | I create a thread and start it, but when the thread does threading.current_thread() it gets the main thread | |
| 17:55:03 | mdbooth | Is this some evilness related to eventlet? | |
| 17:55:57 | melwitt | dansmith: yeah, I guess cell0 is a special case. for all the other things, it makes sense to say "base this cell upon my old cells v1 cell conf". cell0 might probably be a whole new host/user/pass | |
| 17:56:06 | mdbooth | threading.current_thread() has the expected behaviour for me when not run in the test harness | |
| 17:56:30 | dansmith | melwitt: well, that case applies to like four operators.. the other billion will be creating fake configs when they want to configure or spin up new cells | |
| 17:57:19 | melwitt | dansmith: only if they don't want to pass on command line, yeah | |
| 17:57:34 | dansmith | melwitt: well, right, that's what I'm railing against :) | |
| 17:57:50 | mdbooth | (Pdb) threading.current_thread.__module__ | |
| 17:57:50 | mdbooth | 'eventlet.green.threading' | |
| 17:57:55 | mdbooth | eventlet: j'accuse! | |
| 18:00:13 | melwitt | dansmith: yeah, it would be good to have a real way of doing that | |
| 18:02:26 | jaypipes | gah, nested resource providers needs yet another rebase :) | |
| 18:02:40 | mriedem | melwitt: dansmith: sounds like a clouds.yaml kind of thing, but a bit different | |
| 18:02:45 | mriedem | nova-db.yaml :) | |
| 18:02:54 | dansmith | mriedem: yeah, like that | |
| 18:03:15 | ildikov | mriedem: I'm guessing if the version negotiation with the client can get sorted out then we can do the testing in the placement_api job as you seuggested earlier, right? | |
| 18:04:00 | mriedem | ildikov: yes, however, still need to solve the problem of how to configure nova to use cinder v3, which i think right now is going to be the job passing local.conf to d-g/devstack, and sdague is working on making that work | |
| 18:04:24 | mriedem | post-config or whatever it's called | |
| 18:04:42 | ildikov | yep, I remember that one too | |
| 18:05:12 | ildikov | I think we can sync back on it the client part in Cinder is sorted out | |
| 18:05:13 | mriedem | i can make scottda's patch work right now | |
| 18:05:48 | ildikov | mriedem: I won't say don't do it :) | |
| 18:09:17 | openstackgerrit | Lee Yarwood proposed openstack/nova: libvirt: Use the mirror element to detect job completion https://review.openstack.org/399735 | |
| 18:12:03 | mriedem | ildikov: scottda: done ^ | |
| 18:12:08 | mriedem | oops premature | |
| 18:12:10 | openstackgerrit | Matt Riedemann proposed openstack/nova: WIP Support cinder v3 API https://review.openstack.org/420201 | |
| 18:12:13 | mriedem | there | |
| 18:12:30 | openstackgerrit | Gábor Antal proposed openstack/nova: Transform instance.reboot.error notification https://review.openstack.org/411791 | |
| 18:13:01 | ildikov | mriedem: thanks!!! | |
| 18:14:08 | openstackgerrit | Matt Riedemann proposed openstack/nova: WIP Support cinder v3 API https://review.openstack.org/420201 | |
| 18:20:33 | openstackgerrit | Lee Yarwood proposed openstack/nova: libvirt: Reconnect volumes and encryptors during a hard reboot https://review.openstack.org/400384 | |
| 18:24:53 | mriedem | dansmith: so i'm kicking around ideas of how to make the multinode job run multiple cells, | |
| 18:25:12 | mriedem | i think at least part of that is calling map_cell_and_hosts from the subnode | |
| 18:25:22 | dansmith | mriedem: multiple cellsv2 cells you mean? | |
| 18:25:27 | mriedem | yeah | |
| 18:25:33 | mriedem | so host 1 is control + n-cpu, | |
| 18:25:38 | mriedem | host 2 (subnode) is just n-cpu | |
| 18:25:47 | mriedem | but both hosts are going to be a cell | |
| 18:26:41 | mriedem | or a combo of create_cell and discover_hosts | |
| 18:26:44 | mriedem | on the subnode | |
| 18:27:30 | dansmith | doesn't have to be on the subnode specifically, | |
| 18:27:38 | dansmith | but yeah, create_cell will make it more explicit I think | |
| 18:29:14 | mriedem | so currently we just create the dbs on the api node https://github.com/openstack-dev/devstack/blob/master/lib/nova#L683 | |
| 18:29:18 | mriedem | which is fine | |
| 18:29:52 | melwitt | I was thinkin once we have create_cell, we'd remove map_cell_and_hosts since that's really just create_cell + discover_hosts. to make things not overlap and be simpler | |
| 18:30:04 | dansmith | yep | |
| 18:30:12 | mriedem | then we do the mappings later in https://github.com/openstack-dev/devstack/blob/master/lib/nova#L948 | |
| 18:30:53 | mriedem | which is called later here https://github.com/openstack-dev/devstack/blob/master/stack.sh#L1368 | |
| 18:30:59 | mriedem | so we only do that stuff if on the api node | |
| 18:31:18 | mriedem | so i think i can put an else in there for when n-api isn't running (subnode) | |
| 18:31:21 | mriedem | and n-cpu is enabled, | |
| 18:31:49 | mriedem | but would probably want the subnode cell db on the api node, i'm not sure if it matters for CI? | |
| 18:31:58 | mriedem | given we're hitting OOM errors in the gate right now, | |
| 18:32:03 | mriedem | the cell db on the subnode is probably better | |
| 18:32:11 | dansmith | doesn't need to actually be a different place | |
| 18:32:21 | mriedem | yeah i know, | |
| 18:32:30 | mriedem | but devstack isn't topology aware, | |
| 18:32:40 | mriedem | and i'm not entirely sure how all of the subnode stuff gets setup by devstack-gate | |
| 18:32:51 | mriedem | so i'm not sure that when setting up the api node, we know about the subnode yet | |
| 18:33:05 | mriedem | but i guess maybe that doesn't matter? | |
| 18:33:42 | mriedem | we just have a db somewhere, and when we create the cell for the subnode we point that cell at that db connection | |
| 18:34:46 | dansmith | well, | |
| 18:35:01 | dansmith | the thing I ran into earlier trying to do something else was that the subnode didn't have all the stuff it needed to do certain things | |
| 18:35:05 | dansmith | like, not all the config | |
| 18:35:22 | dansmith | don't recall what exactly, but I couldn't start n-api on the subnode | |
| 18:35:29 | dansmith | might've been keystone and catalog related actually | |
| 18:36:42 | mriedem | this is what we have on the subnode http://logs.openstack.org/75/396775/39/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/f82fd9f/logs/subnode-2/etc/nova/nova.conf.txt.gz | |
| 18:40:04 | mriedem | somehow the subnode is getting the variables for the db host | |
| 18:41:14 | mriedem | ah d- | |
| 18:41:18 | mriedem | *d-g my old friend | |
| 18:44:36 | mriedem | alright time to hacky hacky | |