| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-09 | |||
| 14:06:47 | mriedem | *know | |
| 14:06:51 | edleafe | efried: one reason for the force option was that admins wanted to be able to say "I know what I'm doing, dammit!". It wasn't about code efficiency | |
| 14:07:17 | efried | edleafe: But IIUC, the destination host is observed regardless. | |
| 14:07:32 | mriedem | observed? | |
| 14:07:41 | efried | meaning you either get on the suggested host or you die | |
| 14:07:50 | mriedem | correct | |
| 14:07:51 | efried | I'm not sure how we would "fix" the oversubscribe thing at this point, without adding a placement feature to allow it. | |
| 14:08:22 | efried | which I doubt we want to do | |
| 14:08:22 | mriedem | i don't think adding features to support shooting yourself is something we want to do at this point | |
| 14:08:40 | efried | "Hey placement, you know that one thing you're supposed to be designed to do? Yeah, don't do that." | |
| 14:08:42 | edleafe | efried: it isn't just oversubscription that would cause the scheduler to reject it. Things like affinity, etc., would also cause it to fail | |
| 14:08:52 | mriedem | note that even with forced live migration, conductor still runs some checks that could cause us to reject the host | |
| 14:09:05 | mriedem | edleafe: nope | |
| 14:09:12 | mriedem | we don't do any affinity checks outside of the scheduler for live migration | |
| 14:09:12 | efried | you mean besides the allocation claim? | |
| 14:09:30 | edleafe | mriedem: without the force option, we do | |
| 14:09:36 | mriedem | in the scheduler | |
| 14:10:01 | mriedem | but that's more to my point - force is bad | |
| 14:10:15 | efried | I think what I'm getting at is, force isn't so much bad as... obsolete? | |
| 14:10:20 | mriedem | you could screw up AZs too | |
| 14:10:22 | efried | Like, it doesn't do anything useful anymore. | |
| 14:10:24 | edleafe | well, yeah, but admins wanted to be able to override | |
| 14:10:28 | mriedem | efried: agree | |
| 14:10:35 | edleafe | I'm not saying it's right | |
| 14:10:38 | mriedem | efried: it made a bit more sense pre-placement | |
| 14:10:42 | edleafe | Just that that was the pushback at the time | |
| 14:10:46 | efried | because oversubscribe | |
| 14:11:11 | efried | Okay, thanks, this helps validate my response to gibi's thread. | |
| 14:13:47 | openstackgerrit | Merged openstack/os-vif master: Remove IPTools deprecated implementation https://review.openstack.org/605422 | |
| 14:15:51 | openstackgerrit | Jan Gutter proposed openstack/os-vif master: Extend port profiles with datapath offload type https://review.openstack.org/572081 | |
| 14:17:43 | belmorei_ | hi. I'm having an issue with Ironic allocations in placement | |
| 14:18:58 | belmorei_ | What I see is that allocations are only done for the resource class, and not vcpus, ram, disk. I imagine this intentional? | |
| 14:18:58 | belmorei_ | We define the Ironic flavors with resources:VCPU=0, ... as documented in "https://docs.openstack.org/ironic/queens/install/configure-nova-flavors.html". | |
| 14:20:10 | efried | what do you mean, only done for the resource class? | |
| 14:20:43 | belmorei_ | the allocation for the resource_provider is only the ironic resource_class | |
| 14:20:54 | dansmith | belmorei_: yes, expected | |
| 14:21:22 | belmorei_ | great. Let me describe the issue | |
| 14:24:13 | belmorei_ | The problem is that we enable the ironic flavors per project (these projects are only for baremetal) (projects are mapped to a cell that only has the baremetal nodes) | |
| 14:24:59 | belmorei_ | However, users also have access to the "default" flavors that are for VMs in these projects (we can't remove public flavors) | |
| 14:26:37 | belmorei_ | If a user makes the mistake to use a default flavor in these projects (flavor for VMs) placement can return already in use baremetal nodes because they have cpu, ram, ... | |
| 14:27:30 | dansmith | belmorei_: the baremetal nodes should be exposing no cpu,ram,etc inventory | |
| 14:27:40 | dansmith | belmorei_: they should expose one inventory item of the baremetal resource class and nothing else | |
| 14:28:43 | belmorei_ | dansmith: Good to know I would expect that, but it's not happening. Maybe a conf issue in my side | |
| 14:29:10 | dansmith | belmorei_: yeah, I'm not sure how that could be happening anymore.. jroll dtantsur ? | |
| 14:29:34 | mriedem | https://github.com/openstack/nova/blob/stable/queens/nova/virt/ironic/driver.py#L790 | |
| 14:29:47 | mriedem | we still reported cpu/ram/disk inventory for ironic nodes in queens | |
| 14:30:26 | mriedem | https://github.com/openstack/nova/commit/a985e34cdeef777fe7ff943e363a5f1be6d991b7#diff-1e4547e2c3b36b8f836d8f851f85fde7 removed that in stein | |
| 14:31:33 | dansmith | mriedem: um, we should have had a cutover so we're not exposing both right? | |
| 14:31:39 | dansmith | I thought that was like pike | |
| 14:31:57 | dansmith | comment there says zero in pike | |
| 14:31:58 | jroll | dansmith: we didn't do the cutover so people could migrate their flavor | |
| 14:32:08 | dansmith | jroll: right, but in queens? | |
| 14:32:19 | jroll | this is the first I've heard of this, fwiw, though it makes sense | |
| 14:32:24 | jroll | dansmith: people were scared to remove it | |
| 14:32:50 | dansmith | hmm | |
| 14:33:01 | jroll | a quick workaround would likely be to set 0 for cpu/ram in ironic node.properties | |
| 14:33:14 | dansmith | I thought this was long since sorted | |
| 14:33:14 | jroll | and then it will report 0 | |
| 14:33:24 | jroll | yeah, I thought it just worked as well | |
| 14:33:39 | dansmith | what was the plan during the overlap to avoid exposing the old values once everything was migrated? | |
| 14:33:47 | dansmith | override properties in ironic? | |
| 14:33:53 | dansmith | s/everything/flavors/ | |
| 14:33:57 | jroll | I don't remember, sorry | |
| 14:34:15 | belmorei_ | jroll: that means updating all ironic nodes... can't do it. | |
| 14:34:43 | belmorei_ | How about if I remove the "resources:VCPU=0", ... from the flavors? | |
| 14:34:57 | mriedem | your bm flavors aren't the problem | |
| 14:35:11 | mriedem | the problem is the bm flavors are getting scheduled to the ironic nodes right? | |
| 14:35:13 | mriedem | *vm flavors | |
| 14:35:15 | dansmith | right | |
| 14:35:24 | jroll | that would prevent VMs from landing on an active ironic node, but not an inactive one | |
| 14:35:28 | mriedem | and that's because the bm nodes are reporting ram/cpu/disk inventory | |
| 14:35:47 | belmorei_ | jroll: true | |
| 14:39:22 | belmorei_ | ok, so for me the best is to backport the commit mentioned by mriedem | |
| 14:39:26 | mriedem | belmorei_: you might be able to just patch https://github.com/openstack/nova/blob/stable/queens/nova/virt/ironic/driver.py#L797 to be 0 | |
| 14:39:37 | belmorei_ | yeah, thanks | |
| 14:39:38 | mriedem | belmorei_: i'm guessing that's not going to backport cleanly to queens | |
| 14:39:47 | bauzas | gibi: thanks for the email recap | |
| 14:39:54 | belmorei_ | any plan to still include this in rocky? | |
| 14:40:10 | bauzas | gibi: I now understand the problem :) | |
| 14:40:12 | belmorei_ | because for me is a bug | |
| 14:40:24 | gibi | bauzas: :) | |
| 14:40:44 | mriedem | belmorei_: i'm not sure how others would feel about a stable/queens / rocky only config option to disable reporting vcpu/ram/disk inventory for ironic | |
| 14:40:52 | mriedem | a workarounds option | |
| 14:41:01 | gibi | mriedem, efried, edleafe: force flag means I want to move the server so desperately that I don't care about any safeties | |
| 14:41:10 | gibi | at least for me it means that | |
| 14:41:11 | mriedem | default to disabled, but if enabled, report 0 total vcpu/ram/disk inventory for ironic nodes | |
| 14:41:37 | mriedem | dansmith: ^ how do you feel about a config option backdoor for belmorei_'s case? | |
| 14:42:20 | mriedem | we're not going to backport https://github.com/openstack/nova/commit/a985e34cdeef777fe7ff943e363a5f1be6d991b7 | |
| 14:42:29 | dansmith | mriedem: I'm for it because I'm not sure how we're not effing people over with this right now | |
| 14:42:31 | belmorei_ | mriedem: this is easy for me to patch. Having a conf backdoor option doesn't seem good | |
| 14:42:39 | dansmith | I'm guessing tripleo people don't care because undercloud/overcloud | |
| 14:42:58 | mriedem | belmorei_: a workarounds config option provides a generic solution for *everyone* with this problem | |
| 14:43:22 | belmorei_ | mriedem: true | |
| 14:43:25 | mriedem | essentially it means enabling it says you've done your ironic instance flavor migration and you're good to go | |
| 14:43:35 | dansmith | right | |
| 14:43:37 | mriedem | and we have a nova-status check for that as well | |
| 14:43:57 | mriedem | belmorei_: how about you report a bug to start and we can go from there? | |
| 14:44:15 | mriedem | jroll: btw i do remember something breaking after we removed that code in stein, but i can't remember what off the top of my head | |
| 14:44:34 | mriedem | which is why i wanted to hold off on removing it right before the rocky GA | |
| 14:46:09 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Don't emit warning when ironic properties are zero https://review.openstack.org/608573 | |