| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-18 | |||
| 15:55:18 | mriedem | backporting fixes from master to juno is fun | |
| 16:04:22 | dansmith | mriedem: these are left over from when juno /was/ master | |
| 16:09:28 | mriedem | you're old | |
| 16:13:33 | dansmith | fact | |
| 16:15:48 | mriedem | btw, i came home to my office having dolls all lined up in the window and all over my desk | |
| 16:15:51 | mriedem | it was a horror show | |
| 16:16:47 | dansmith | lol | |
| 16:16:57 | dansmith | I came home to my office having ... the same mess I left there | |
| 17:23:49 | leitan_ | Hi guys, quick question, im trying to work with SSL enabled rabbitmq, without SSL everything works fine, but when converted my deploy to SSL, and moved the rabbitmq listen port from 5672 to 5671, now im getting from nova api that cannot connect to "myrabbits:5672" BUT the transport URL says 5671 and also the rabbit_port on the oslo_messaging_rabbit section says that, what am i missing here ? | |
| 17:24:03 | leitan_ | its like nova-api is ingnoring all the rabbit port indications | |
| 17:26:58 | mriedem | dtantsur: do you plan on writing up a ptg recap for the nova/ironic discussions? because if so, i'll skip writing that, especially as i didn't follow the discussion as closely | |
| 17:27:33 | dtantsur | mriedem: hi, I have such plan, yes. I'll poke the folks here if I don't remember some details | |
| 17:27:49 | mriedem | excelltn | |
| 17:27:52 | mriedem | *excellent even | |
| 17:27:53 | mriedem | thanks | |
| 17:28:08 | dtantsur | np | |
| 17:28:28 | dtantsur | mriedem: we also have a traits discussion with johnthetubaguy in our room, so you don't have the whole context for sure | |
| 17:28:32 | dtantsur | s/have/had/ | |
| 17:38:17 | cfriesen | mriedem (or anyone): I heard something about a "prioritization discussion" from last tuesday that pushed shared resources down in priority...is there an etherpad/link for that? | |
| 18:01:47 | leitan_ | is nova caching the rabbit connection string somewhere ? | |
| 18:10:35 | mriedem | leitan_: yeah, check the nova_api.cell_mappings table | |
| 18:10:44 | mriedem | and there is a context cache in the service | |
| 18:11:19 | leitan_ | mriedem: holy molly thanks ! | |
| 18:11:33 | mriedem | leitan_: #2 here https://docs.openstack.org/nova/latest/user/cells.html#faqs | |
| 18:12:16 | mriedem | cfriesen: not much details but it's at the bottom here https://etherpad.openstack.org/p/nova-ptg-queens-placement | |
| 18:12:37 | leitan_ | mriedem: yes, now i see that | |
| 18:12:45 | leitan_ | it should be there where i persisted the rabbit | |
| 18:12:55 | leitan_ | without ssl | |
| 18:13:25 | leitan_ | and when i change the config ... i forgot updating the cell | |
| 18:14:04 | mriedem | leitan_: there is an update_cell CLI https://docs.openstack.org/nova/latest/cli/nova-manage.html#nova-cells-v2 | |
| 18:14:47 | leitan_ | mriedem: thanks a lot ! | |
| 18:15:16 | mriedem | yw | |
| 18:15:59 | mriedem | mikal: just a heads up, i don't plan on sending a ptg recap on the privsep stuff. feel free if you want. | |
| 18:26:59 | sdague | he's still in a mountain somewhere | |
| 18:39:45 | dansmith | hopefully on a mountain and not in it | |
| 18:53:55 | avolkov | mriedem: yes, will try to work on osc for placement, thanks for the info | |
| 19:37:23 | mriedem | ew https://bugs.launchpad.net/nova/+bug/1717916 | |
| 19:37:25 | openstack | Launchpad bug 1717916 in OpenStack Compute (nova) "Nova won't reschedule when specific hypervisor is set and request failed" [Undecided,New] | |
| 19:38:24 | mriedem | "if len(force_hosts) == 1 and (len(nodes) == 1 and len(force_nodes) == 1) or not force_nodes" | |
| 19:41:01 | sdague | mriedem: I'm confused why that's a new issue | |
| 19:41:15 | mriedem | we don't reschedule across nodes for the same host | |
| 19:41:27 | mriedem | so, create a server with the AZ:HOST thing, | |
| 19:41:29 | mriedem | but no NODE | |
| 19:41:32 | sdague | ok | |
| 19:41:37 | mriedem | if it's ironic, you could fail the first node and we won't retry | |
| 19:41:40 | sdague | but, that test has existed for over a year | |
| 19:41:55 | sdague | that's how we test that multinode is actually doing that | |
| 19:44:05 | mriedem | i don't know why it's a recent problem for ironic, unless they've been blacklisting that test this whole time | |
| 19:45:08 | sdague | yeh, I feel like that is the specific ask back. Because to my knowledge that test has been pretty static for a while. | |
| 19:45:35 | sdague | and this might be the team expanding CI and not running long enough before voting on something to realize this was going to happen | |
| 19:46:13 | mriedem | yeah just asked for details in the bug report | |
| 19:49:28 | mriedem | dansmith: look familiar at all? https://bugs.launchpad.net/nova/+bug/1717915 | |
| 19:49:29 | openstack | Launchpad bug 1717915 in OpenStack Compute (nova) "nova services and transport_url, cannot connect to vhost if specified" [Undecided,New] | |
| 19:50:44 | dansmith | mriedem: well, no, but I think he broke his config | |
| 19:50:46 | dansmith | I'll comment | |
| 19:50:58 | mriedem | i just did too, transport_url isn't a list opt | |
| 19:51:30 | dansmith | mriedem: I think it is supposed to take a list of hosts for clustered rabbit, I don't know if that's the problem | |
| 19:52:14 | mriedem | ah the docs aren't clear on the options | |
| 19:52:21 | mriedem | rabbit_hosts is deprecated in favor of transport_url | |
| 19:52:42 | dansmith | right | |
| 19:53:26 | dansmith | so yeah, I think this is just him getting the right format for transport_url with multiple hosts | |
| 19:53:44 | dansmith | i.e. hopefully not a cells issue but just a transport_url one | |
| 19:55:50 | mriedem | heh yeah, because we aren't parsing that in nova as a list | |
| 19:57:06 | mriedem | from the oslo.messaging code, it looks like it splits on ; | |
| 19:58:27 | mriedem | no tests on the change that deprecated the option... | |
| 20:01:12 | mriedem | https://github.com/openstack/oslo.messaging/commit/2f0d53b | |
| 20:02:48 | dansmith | well they'd be tests on the transport_url change right? | |
| 20:10:52 | mriedem | yeah, and looks like it does split on comma on the parsed netloc https://github.com/openstack/oslo.messaging/blob/393ecff3451091404832dd6b8a088e1bec760101/oslo_messaging/transport.py#L512 | |
| 20:12:25 | mriedem | and i think it applies the virtual host to all of them | |
| 20:37:40 | mriedem | rhosp is clustered rabbit right? so i'd assume you guys would have hit this with cells v2 by now if it were an obvious issue | |
| 20:38:48 | jaypipes | mriedem: tyvm for doing the denver recaps. very useful. | |
| 20:39:11 | jaypipes | mriedem: and no, I don't think rhosp is clustered rabbit, unless I'm mistaken... dansmith? | |
| 20:39:33 | dansmith | it absolutely is | |
| 20:40:17 | dansmith | mriedem: tripleo HA does it as well, so we can probably just ask a tripleo person, although the one I'd ask is on PTO this week, AFAK | |
| 20:40:20 | dansmith | *AFAIK | |
| 20:41:05 | mriedem | rhosp is tripled everything | |
| 20:41:08 | mriedem | even the deployer | |
| 20:41:26 | dansmith | owalsh likely knows, but he's probably not around this late | |
| 20:41:27 | mriedem | it's like the ihop breakfast of openstack deployments | |
| 20:41:28 | dansmith | in general, you can assume owalsh knows everything about this stuff | |
| 20:41:36 | dansmith | he lazy-loads things he doesn't know off the top of his head | |
| 20:42:57 | mriedem | no props for my 2 x 2 x 2 reference? | |
| 20:44:03 | mriedem | jaypipes: thanks. prepare for the "everything else" finale. | |
| 20:44:42 | dansmith | mriedem: too obscure for me, but: haha | |
| 20:44:52 | jaypipes | dansmith: ah, ok, sorry. for some reason I thought triple-o didn't configure rabbitmq in cluster mode.. | |
| 20:45:23 | dansmith | unless something has changed, but I don't think so :) | |
| 20:45:40 | jaypipes | dansmith: no, prolly just me badly misremembering something | |
| 21:06:21 | mriedem | man, the notes on preemptible instances are confusing | |
| 21:06:29 | mriedem | 'first phase, don't touch quota' | |
| 21:06:36 | mriedem | 'the reaper will modify quota' | |
| 21:53:31 | efried | alex_xu Should be back in a couple hours :) | |
| 21:55:28 | clarkb | trivia time, the size of the nova repo and number of changes to the nova repo and total number of comments on nova all have a direct impact on the length of the downtime :) (but don't let that stop you from using gerrit, its just annoying gerrit has upgrade scaling issues with the size of projects) | |
| 22:24:05 | jaypipes | clarkb: you know there's a full copy of OpenWRT embedded in the Nova source tree, right? happy trivia day :) | |
| 22:24:35 | clarkb | I did know that :) | |
| 22:24:42 | clarkb | if you ever need a vm image ot boot its there | |
| 22:37:32 | dims | jaypipes : seriously? wow | |
| 22:57:41 | openstackgerrit | Nicolas Simonds proposed openstack/nova master: libvirt: add support for virtio-net rx/tx queue sizes https://review.openstack.org/484997 | |
| 22:59:40 | jaypipes | dims: yup :) | |
| 23:01:22 | dansmith | mriedem_afk: daddy don't live in that new york city no more | |
| 23:45:22 | dansmith | ohai gerrit | |