| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-07-24 | |||
| 18:03:18 | dansmith | s-dean: so if you grep transport_url out of your configs on all nodes, they match exactly what is in the db for the cell1 transport_url? | |
| 18:06:35 | s-dean | ok so i can confirm, that on my controller node the URL's are correct, however on the compute node it is not because of: https://docs.openstack.org/security-guide/messaging/security.html | |
| 18:07:32 | mriedem | sdague: did you ever report a bug for the request log thing with uwsgi? | |
| 18:07:52 | sdague | mriedem: I don't think I did actually | |
| 18:10:07 | mriedem | sdague: ok, did the same thing need to be done in other projects like cinder? | |
| 18:10:18 | mriedem | if so, could be good to track with a bug for those other projects too | |
| 18:13:50 | dansmith | s-dean: so...the compute node is not correct? | |
| 18:14:00 | s-dean | does the username need to be consistent across all nodes, or is having different users for each compute node ok ? | |
| 18:14:13 | s-dean | yes i gave the compute node its on users | |
| 18:14:15 | s-dean | user | |
| 18:14:23 | s-dean | and assigned it all permissions | |
| 18:14:31 | s-dean | own* | |
| 18:14:59 | dansmith | s-dean: they can be different, they just all have to point at the same vhost of course | |
| 18:15:12 | s-dean | in my case my vhost is / | |
| 18:15:31 | dansmith | as long as that is the same for all the nodes then that's fine | |
| 18:16:34 | dansmith | s-dean: aside from that, we need more context on which thing is failing, so more detailed logs, and a sanitized dump of your config and cell_mappings table are needed | |
| 18:17:09 | s-dean | paste bin urls ok ? | |
| 18:17:19 | dansmith | yeah | |
| 18:18:55 | s-dean | https://pastebin.com/WEu7JCWW | |
| 18:20:20 | dansmith | s-dean: are you running n-net? | |
| 18:20:36 | dansmith | s-dean: and looks like you're configured for cellsv1, is that intentional or did you think that was related? | |
| 18:20:57 | dansmith | because this timeout is trying to call the cellsv1 service, which if you're not running it, will always time out :) | |
| 18:21:55 | s-dean | oh ok | |
| 18:22:06 | s-dean | i didnt know i was running cells v1 | |
| 18:22:24 | s-dean | is this because of the following config line compute_api_class=nova.compute.cells_api.ComputeCellsAPI | |
| 18:22:32 | s-dean | https://bugs.launchpad.net/nova/+bug/1706118 | |
| 18:22:32 | s-dean | my config is here | |
| 18:22:33 | dansmith | s-dean: you definitely know if you're running cellsv1 because it's a huge friggin nightmare, | |
| 18:22:34 | openstack | Launchpad bug 1706118 in OpenStack Compute (nova) ocata "Adding TLS Connection String To Database Connection Causes malformed URL in cell_mappings DB" [High,Confirmed] | |
| 18:22:44 | dansmith | so I'm guessing maybe you configured that while trying to resolve this problem? | |
| 18:23:00 | s-dean | yeah i have been poking about quite a bit | |
| 18:23:18 | dansmith | s-dean: this is an upgrade from a previously working deployment, right? | |
| 18:23:57 | s-dean | no fresh install, but because of that DB error, i have been trying to debug and find the right settings | |
| 18:24:13 | dansmith | oh, okay | |
| 18:24:16 | dansmith | s-dean: well, you went down a bad path | |
| 18:24:32 | openstackgerrit | Matt Riedemann proposed openstack/nova master: deprecate ``wsgi_log_format`` config variable https://review.openstack.org/486623 | |
| 18:24:33 | openstackgerrit | Matt Riedemann proposed openstack/nova master: request_log addition for running under uwsgi https://review.openstack.org/485602 | |
| 18:24:34 | s-dean | haha, is my install screwed ? | |
| 18:24:34 | dansmith | s-dean: might want to start fresh if it's not too much trouble and avoid any of that cellsv1 stuff, depending | |
| 18:24:43 | s-dean | ok i will od | |
| 18:24:45 | s-dean | do | |
| 18:24:54 | mriedem | fear leads to anger, anger leads to hate, hate leads to cells v1 | |
| 18:24:55 | dansmith | it's definitely rescue-able, but if you're trying to nail down a procedure it'll be confusing from here | |
| 18:25:04 | dansmith | cellsv1 leads to? | |
| 18:25:13 | mriedem | showing up in nova freenode irc? | |
| 18:25:15 | openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Remove the useless FakeExt https://review.openstack.org/486415 | |
| 18:25:21 | s-dean | xD | |
| 18:25:26 | mriedem | which is equivalent to the dark side | |
| 18:25:38 | openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Remove the useless extension block_device_mapping_v1 object https://review.openstack.org/486069 | |
| 18:25:49 | openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Remove the useless fake ExtensionManager from API unittests https://review.openstack.org/486416 | |
| 18:25:52 | s-dean | awww man wish i came here sooner, i dont like asking for help i prefer to bash it out, and use google | |
| 18:25:58 | openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Move the note about '/os-volume_boot' to the correct place https://review.openstack.org/486071 | |
| 18:27:04 | mriedem | sdague: requestlog thing is approved | |
| 18:27:13 | mriedem | sdague: you want to skim over this hot little number? https://review.openstack.org/#/c/485435/ | |
| 18:27:20 | openstackgerrit | Ed Leafe proposed openstack/nova master: Migrate Ironic Flavors https://review.openstack.org/484949 | |
| 18:35:51 | dansmith | jaypipes: edleafe: I was thinking that the ironic node class was required now.. do we really need to concern ourselves with that not being set? | |
| 18:35:58 | s-dean | shouldnt that config option be removed from the docs ? | |
| 18:36:03 | s-dean | https://docs.openstack.org/ocata/config-reference/compute/cells.html | |
| 18:36:13 | dansmith | s-dean: we can't yet | |
| 18:36:27 | dansmith | s-dean: people still run that stuff and have to for a bit longer | |
| 18:36:55 | s-dean | a fair enough | |
| 18:42:22 | mriedem | s-dean: we're in the process of moving those docs into nova, at which point we can put a big fat warning at the top saying that's for cells v1 | |
| 18:43:04 | edleafe | jaypipes: regarding your comments about splitting the patch for https://review.openstack.org/#/c/486215/, do you want me to build the list of alternates, but continue to only return the selected hosts? And then change all the method signatures in https://review.openstack.org/#/c/486253/ ? | |
| 18:43:57 | edleafe | dansmith: I asked dtantsur|afk about that. He said that while adding the resource_class attribute was required, populating it completely was not | |
| 18:44:27 | dansmith | edleafe: okay so, we should check it and if it's not present, log a warning and bail? | |
| 18:45:38 | edleafe | dansmith: if it's not present, just skip | |
| 18:46:26 | dansmith | edleafe: we have to log a warning, since not having it set will lead to brokenness later, but.. your code currently is not skipping right? | |
| 18:46:47 | dansmith | in PS10 I mean | |
| 18:47:26 | edleafe | dansmith: correct. Want me to add some safety stuff there? | |
| 18:47:57 | dansmith | edleafe: yeah, warn and skip if unset, test for those cases and then I think we can land this and argue about the batching after | |
| 18:48:38 | s-dean | mriedem: that would be brilliant. before i head home for the night, can i specify in keystone, glance and nova their database connections to be localhost skipping the need for TLS, or do they need to be specified with a network address like in the docs? do the other service such as neutron and and nova-compute need to access the DB over the network? | |
| 18:49:04 | edleafe | dansmith: ok cool. I asked in -ironic about how often the compute service would be restarted. Got one answer from jroll, and that was he only did it during upgrades | |
| 18:49:12 | edleafe | IOW, not a common thing for him | |
| 18:49:22 | mriedem | s-dean: nova-compute definitely does not use the db locally | |
| 18:49:35 | mriedem | i don't know about the various neutron agents | |
| 18:49:37 | dansmith | edleafe: heh, sure, but jroll knows what he's doing | |
| 18:49:39 | mriedem | or cinder-volume | |
| 18:49:58 | dansmith | edleafe: like I said there are other config management approaches that are pretty darn restart happy | |
| 18:50:13 | dansmith | I know of one very specifically | |
| 18:50:15 | dansmith | one that has lots of Os in it | |
| 18:53:36 | mriedem | s-dean: got it https://review.openstack.org/#/c/486724/ | |
| 18:53:43 | jaypipes | edleafe: re: separate patch, yeah, I'm thinking it's best to generate that list of alternates in the first patch (but not change the returned value from select_destinations() and then in the followup patch, change the RPC API signature and the corresponding conductor stuff | |
| 18:53:55 | s-dean | mriedem: I'm installing a general compute cloud, nothing to fancy, | |
| 18:54:03 | openstackgerrit | Merged openstack/nova master: placement: add retry tight loop claim_resources() https://review.openstack.org/486170 | |
| 18:54:06 | s-dean | mriedem: cheers :) | |
| 18:54:13 | dansmith | jaypipes: edleafe agreed | |
| 18:54:16 | mriedem | s-dean: but the nova-compute services are running on separate hosts right/ | |
| 18:54:19 | mriedem | ? | |
| 18:54:24 | s-dean | yes | |
| 18:54:29 | mriedem | nova-compute will interact with the db via nova-conductor | |
| 18:54:33 | s-dean | compute01 | |
| 18:54:38 | s-dean | ok sweet | |
| 18:54:45 | edleafe | jaypipes: ok, so the last change will go from the current return of a list of hosts to the monsterous list of hosts+alternates+allocation_candidates, right? | |
| 18:54:49 | mriedem | so if nova-conductor is on the control nodes local to the db, then you could use localhost for nova.conf for conductor | |
| 18:54:51 | mriedem | but not nova-compute | |
| 18:55:06 | mriedem | s-dean: nova-compute shouldn't even need the [database] section filled in for nova.conf | |
| 18:55:20 | s-dean | ok awsome, yeah all the scheduler conductor novncproxy etc are on 1 controller node | |
| 18:55:30 | mriedem | then localhost or 127.0.0.1 is fine for those | |
| 18:55:43 | s-dean | looks like neutron needs access to the DB from across the network :/ | |
| 18:55:59 | s-dean | ill cross that brige when i come to it i guess | |