| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-18 | |||
| 17:57:43 | eandersson | I see - I assumed so, but wanted to make sure | |
| 17:57:47 | melwitt | I read the help backwards and... I don't have much of an excuse | |
| 17:58:14 | efried | The only way it would be at all likely to help you is if $number_of_concurrent_deploys < $number_of_possible_hosts *and* you use a ?limit < $number_of_possible_hosts | |
| 17:58:55 | efried | but like I said, that's a pretty hacky thing to try to figure out | |
| 17:59:19 | melwitt | eandersson: the shuffle_best_same_weighed_hosts might help you, I think. numa stuff might be why it was added in the first place | |
| 17:59:35 | eandersson | Yea - sounds super promising. | |
| 18:02:02 | melwitt | +1 efried. eandersson: the randomize_allocation_candidates use case is for very large deployments where the default limit for returned candidates is < the total number of compute hosts in the deployment. think CERN with multiple cells | |
| 18:02:19 | eandersson | Just fyi we have ~1k nodes in each deployment | |
| 18:03:42 | melwitt | ack | |
| 18:04:54 | eandersson | per cell | |
| 18:06:07 | melwitt | oh, ok that is a lot | |
| 18:06:08 | eandersson | We use regions instead of cells at the moment, but might want to change that in the future | |
| 18:08:15 | melwitt | here's the background on the option https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/allocation-candidates-limit.html | |
| 18:08:57 | eandersson | We limit ourself to 1k nodes today because we don't know if neutron can handle it | |
| 18:09:20 | eandersson | (used to be nova + neutron, but doubt nova will have a problem today) | |
| 18:09:49 | eandersson | when we upgraded to rocky neutron used up like 500GB memory lol | |
| 18:10:02 | eandersson | each neutron process peaked at 8.2GB memory used | |
| 18:10:31 | melwitt | gotcha. belmiro from CERN would be the one to chat with about neutron scaling | |
| 18:10:39 | eandersson | We work with them a lot | |
| 18:10:47 | melwitt | I don't remember if they are doing something special re: neutron | |
| 18:10:49 | eandersson | They use one big Layer 2 network | |
| 18:10:50 | melwitt | ok cool | |
| 18:11:19 | melwitt | and that's... a lot of memory O.o | |
| 18:11:32 | eandersson | Yea it's nuts. I tend to ping the neutron channel but no one ever replies | |
| 18:11:38 | eandersson | Nova channel is way better :p | |
| 18:12:07 | melwitt | ah yeah, I think they might be mostly EU timezone | |
| 18:12:17 | eandersson | Some awesome people here, melwitt efried sean-k-mooney mnaser mriedem to name a few ;) | |
| 18:12:35 | melwitt | :) | |
| 18:13:10 | mnaser | \o/ | |
| 18:13:14 | mnaser | eandersson: will you be at shanghai? | |
| 18:13:22 | eandersson | I will miss shanghai :'( | |
| 18:13:32 | eandersson | Will be attending the next one for sure | |
| 18:15:38 | eandersson | Which is a bummer, especially since Train has been the largest contribution from us yet :P | |
| 18:16:05 | eandersson | 90% of that was to Designate hehe | |
| 18:17:44 | openstackgerrit | Merged openstack/nova stable/queens: Drop source node allocations if finish_resize fails https://review.opendev.org/682722 | |
| 18:20:44 | mnaser | eandersson: awesome | |
| 18:25:38 | eandersson | btw some more shameless pr, but if you know anyone in South California or Austin Texas, we are hiring OpenStack people (systems and/or software) | |
| 18:25:48 | eandersson | *that is looking for work | |
| 18:50:23 | efried | eandersson: not that I'm looking, who's "we"? | |
| 18:51:09 | efried | I'm in Austin, but have a cushy wfh gig. Wouldn't consider anything with a commute longer than eleven seconds. | |
| 18:52:23 | dansmith | agree, although the traffic on my commute is getting ridiculous | |
| 18:52:35 | dansmith | earlier this week, I had to wait for the cat to pass before I made a left at the stairs | |
| 18:52:41 | dansmith | major road rage situation | |
| 19:02:56 | mordred | efried: I believe eandersson is at blizzard | |
| 19:03:15 | efried | that makes sense | |
| 19:04:05 | efried | Are game companies still as crazy as they were in the 90s? That's the last time I paid attention. | |
| 19:05:05 | efried | people living in their office for a month at a time, kind of thing | |
| 19:06:17 | mordred | I live in my office for a month at a time | |
| 19:09:21 | efried | aaand you're crazy, what's your point? | |
| 19:09:51 | efried | oh, that things other than game companies can be crazy, got it. | |
| 19:18:05 | openstackgerrit | Eric Fried proposed openstack/nova master: ItemsMatcher: mock call list arg in any order https://review.opendev.org/689487 | |
| 19:18:06 | openstackgerrit | Eric Fried proposed openstack/nova master: Always trait the compute node RP with COMPUTE_NODE https://review.opendev.org/688979 | |
| 19:23:52 | mordred | efried: yeah. I mostly meant, you know - like, my house, which is also my office | |
| 19:24:04 | mordred | efried: I'm maybe not as funny or cleve as I sometimes think | |
| 19:24:18 | efried | Oh, you are, but probably not for the *reasons* you think. | |
| 19:25:40 | efried | See, my funniness is inversely proportional to how much I'm trying. There's no surprise funny from me. | |
| 20:50:42 | eandersson | efried, yea unfortunately still all about that office | |
| 20:53:34 | eandersson | https://careers.blizzard.com/en-us/openings/o6vmbfwS (Austin) https://careers.blizzard.com/en-us/openings/ohdYafwm (Irvine) | |
| 21:38:07 | mriedem | gmann: remember how we thought host_status would be returned for GET /servers/detail with servers from down cells? i found that won't happen | |
| 21:38:35 | mriedem | because of this filtering https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L590 | |
| 21:39:08 | mriedem | efried: sorry about the late reply, i've been not around so much today - i took your comments and threw them into my patch so i can process them on monday | |
| 21:39:23 | efried | saw dat, get some rest and feel better. | |
| 21:53:57 | efried | (How) does nova (via oslo.config?) vet permissions of nova.conf? | |
| 21:54:17 | efried | or is that completely the responsibility of the admin? | |
| 22:00:13 | sean-k-mooney | efried: i think its up to the admin/installer | |
| 22:01:06 | sean-k-mooney | e.g. we dont check that the config is 600 or 660 as far as i am aware | |
| 22:01:22 | efried | so if /etc/nova and contents were 5777 and owned by $random we would truck happily along? | |
| 22:01:31 | mriedem | gmann: melwitt: a couple of questions in the host_status_unknown patch - mostly for gmann about policy rule naming and the docs on the policy rule since 2.75+ https://review.opendev.org/#/c/679181/ | |
| 22:01:48 | sean-k-mooney | efried: i think so | |
| 22:02:09 | efried | o...kay. | |
| 22:02:24 | sean-k-mooney | nova does not create the file and we never write to it | |
| 22:02:45 | mriedem | efried: likely a question for bnemec | |
| 22:02:48 | sean-k-mooney | so its really up to the install to make sure that files with password in them are not world readable | |
| 22:03:29 | mriedem | i thought there was some talk of some how vaultifying that stuff but i might be dreaming that up | |
| 22:03:30 | efried | sean-k-mooney: well, that and if they're writable in any way a malicious user could make nova do horrible things to the system | |
| 22:03:44 | efried | mriedem: I think sdk does something like that. | |
| 22:03:49 | efried | or at least supports it. | |
| 22:03:57 | efried | for clouds.yaml | |
| 22:04:02 | efried | mordred: ^ | |
| 22:04:11 | efried | not that that's relevant here | |
| 22:04:39 | efried | I'll ML this sucker. | |
| 22:04:40 | sean-k-mooney | efried: i mean programs like ssh refuse to work if you dont set the right permisions in your ssh .config or key | |
| 22:04:45 | sean-k-mooney | so its not unresonable | |
| 22:04:59 | mordred | aroo? | |
| 22:05:03 | efried | right, swhat I'm sayin, it wouldn't be unreasonable for oslo and/or nova to enforce something like that. | |
| 22:05:27 | efried | but that doesn't mean we do | |
| 22:05:30 | mordred | oh - yeah - so - sdk supports clouds.yaml being split into clouds.yaml and secure.yaml | |
| 22:05:54 | mordred | we do not enforce that secure.yaml is actually secure - although I agree, it would be a reasonably sane thing to do | |
| 22:06:32 | efried | mordred: before I blast the ML, do you know off the top whether oslo.config does any such validation either? | |
| 22:06:35 | mriedem | https://specs.openstack.org/openstack/oslo-specs/specs/stein/secret-management-store.html ? | |
| 22:07:34 | sean-k-mooney | it looks like kolla atleast makes it 660 https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/tasks/config.yml#L57 | |
| 22:07:49 | mordred | efried: I do not know anything about oslo.config | |
| 22:07:57 | sean-k-mooney | they will be own by the kolla:kolla user and group by default | |
| 22:08:04 | sean-k-mooney | well on the host | |
| 22:08:11 | sean-k-mooney | they will be own by nova in the containers | |
| 22:08:15 | mriedem | efried: that spec looks pretty close but i don't see any code ever showed up for it | |
| 22:09:15 | efried | even if it was coded per the spec, though, it would only mean that you *could* secure your configs; it'd still be n/a for anyone using nova.conf | |
| 22:09:16 | sean-k-mooney | mriedem: i do think your recalection is right we did talk about vaultifing this at some point | |
| 22:10:03 | sean-k-mooney | im not sure it was that oslo spec specificaly but seam likely that was related | |
| 22:13:17 | mriedem | anywho, i'm going to drop now, have a good weekend o/ | |