| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-23 | |||
| 13:25:45 | hshiina | is there anything to fix here? | |
| 13:33:32 | leakypipes | hshiina: one sec, reading :) | |
| 13:35:38 | leakypipes | hshiina: no, that looks to be a different scenario than the one described in that comment. lemme dig some more into the log there. | |
| 13:37:30 | hshiina | leakypipes: sure, thanks. | |
| 13:37:45 | leakypipes | hshiina: if I had to guess, what is happening in your case is the following: | |
| 13:37:57 | leakypipes | 1) nova boots an Ironic flavor | |
| 13:38:37 | leakypipes | 2) nova-scheduler finds an available Ironic compute node that has CUSTOM_BAREMETAL inventory of 1, used of 0. | |
| 13:38:52 | leakypipes | 3) nova-compute calls the Ironic virt driver to launch the instance | |
| 13:39:09 | leakypipes | 4) Ironic virt driver launches the instance. All good. | |
| 13:39:24 | leakypipes | 5) At some point, the Ironic node goes into an "unavailable" status | |
| 13:39:36 | leakypipes | 6) The resource tracker in nova-compute runs update_available_resource() | |
| 13:39:56 | leakypipes | 7) the Ironic virt driver returns an inventory total of 0 CUSTOM_BAREMETAL for the unavailable node | |
| 13:40:12 | leakypipes | 8) The nova-compute resource tracker dutifully tries to remove the inventory for that node | |
| 13:40:31 | leakypipes | 9) the Placement API returns an InventoryInUse exception because there is an instance running on that Ironic node | |
| 13:41:09 | leakypipes | hshiina: so what I would check in your job logs is whether that Ironic node that the instance ended up on was set to unavailable at some point between 11:17:05 and 11:18:13 | |
| 13:41:57 | leakypipes | hshiina: from what I can tell, the resource tracker and placement API are doing "the right thing" and preventing inventory from being deleted when there is an active instance on that Ironic node. | |
| 13:43:30 | hshiina | leakypipes: thank you. i will dig it later. | |
| 13:43:50 | hshiina | leakypipes: i am interested in. but, i have to leave office shortly. | |
| 13:43:59 | leakypipes | np | |
| 13:50:58 | srf | i'll try to install devstack in my computer but always error in nova http://paste.openstack.org/show/682946/ anybody can help me, ? | |
| 13:54:03 | openstack | Launchpad bug 1726260 in devstack "devstack fails while running ./stack.sh in master/queens in ubuntu 16.04 while starting n-cpu" [Undecided,New] | |
| 13:54:03 | Spazmotic | srf: Quick google revealed this.. did you try to search around a bit? https://bugs.launchpad.net/devstack/+bug/1726260 | |
| 13:54:26 | Spazmotic | Give that a try :) | |
| 13:58:14 | srf | Spazmotic : I've tried to adding ENABLED_SERVICES=placement-api but still error | |
| 14:05:33 | Spazmotic | Not too sure then, but i'm sure someone may have seen it before.. also again you may wish to ask in #openstack as that's more designed for deployment support | |
| 14:24:20 | Guest70 | hello | |
| 14:44:17 | openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Network interface attach/detach https://review.openstack.org/546813 | |
| 14:45:08 | openstackgerrit | Theodoros Tsioutsias proposed openstack/nova master: WIP: Add the reaper entry point https://review.openstack.org/547450 | |
| 14:53:14 | bauwser | if folks have specs, I'm hungry to eat them | |
| 14:53:44 | bauwser | like, you're going to the PTG and would like to discuss about your spec, I'm your man | |
| 14:54:16 | figleaf | bauwser: after you eat them, you know what they turn into :) | |
| 14:54:44 | bauwser | if they are made of cheese, I'm +2 | |
| 14:55:16 | bauwser | even if https://www.thelocal.fr/20180223/french-cheese-wars-are-the-days-of-the-real-normandy-camembert-numbered | |
| 14:57:20 | figleaf | guess it will be like taking a microbrew beer and then mass-producing it | |
| 15:04:39 | dansmith | leakypipes: I'm really not the right person to answer detailed questions about that | |
| 15:15:33 | bauwser | dansmith: I don't know your current TZ, but it's Friday here :) | |
| 15:15:54 | dansmith | bauwser: I don't know my current TZ either, but I'm pretty sure it's not friday in my body | |
| 15:16:04 | bauwser | still on the air ? | |
| 15:16:16 | dansmith | no, you caught me, I know my current TZ | |
| 15:16:19 | dansmith | but my body doesn't | |
| 15:16:22 | bauwser | heh | |
| 15:19:11 | mriedem | why do we have https://github.com/openstack/nova/blob/ef4000a0d326deb004843ee51d18030224c5630f/nova/db/sqlalchemy/api_models.py#L409 ? | |
| 15:19:24 | mriedem | seems silly to have an entire table to hold one string | |
| 15:19:40 | mriedem | maybe it goes back to the idea that server groups would have >1 policy | |
| 15:19:42 | mriedem | but the API doesn't allow that | |
| 15:20:26 | mriedem | https://github.com/openstack/nova/blob/ef4000a0d326deb004843ee51d18030224c5630f/nova/api/openstack/compute/schemas/server_groups.py#L29-L39 | |
| 15:21:42 | dansmith | mriedem: yeah it's 1:N I think | |
| 15:21:47 | dansmith | not that we can do that, I don't think | |
| 15:22:09 | mriedem | the api doesn't allow it | |
| 15:22:14 | mriedem | but that's the legacy reason i guess, | |
| 15:22:23 | mriedem | the schema makes you pass a single-item list for the policy | |
| 15:22:44 | dansmith | but even still, none of the stuff we do to honor the policy considers multiple options right? | |
| 15:22:47 | mriedem | and it's called "policies" but doesn't allow >1 | |
| 15:22:54 | mriedem | correct | |
| 15:22:56 | mriedem | it's just weird | |
| 15:23:06 | leakypipes | cfriesen: am I gonna have to get out the beating stick for you? :P | |
| 15:23:26 | mriedem | dansmith: context is https://review.openstack.org/#/c/546925 which is tied to an old blueprint from sgordon | |
| 15:23:45 | leakypipes | cfriesen: well, I suppose we could rent a pair of those fake Sumo wrestler costumes and go at it. | |
| 15:25:02 | dansmith | mriedem: ah yes, affrinity | |
| 15:25:18 | mriedem | i'm currently in the process of ripping up the spec | |
| 15:26:00 | dansmith | mriedem: ugh, that max_number_per_host thing is just silly.. that's never going to do something useful, IMHO | |
| 15:27:58 | mriedem | i can see the use, | |
| 15:28:03 | mriedem | it makes the soft policy less soft | |
| 15:28:08 | mriedem | jello-like if you will | |
| 15:28:19 | dansmith | what's the point in that? | |
| 15:28:41 | mriedem | there is some nfv use case in the spec | |
| 15:29:04 | dansmith | I think from a modeling point of view, it'd be hard to reason about that unless you knew "I'm going to start ten of the exact same thing for throughput and I want those spread across three nodes" | |
| 15:29:20 | mriedem | anti-affinity to a point, so they don't need all VMs in the group on separate hosts, but they can only tolerate up to a limit of VMs on the same host | |
| 15:30:14 | mriedem | otherwise today if you have 10 VMs in a soft-anti-affinity group, and only 2 available hosts, you could get like 1 on 1 host and 9 on the other | |
| 15:30:19 | dansmith | right, so a more useful thing would be to say "instances in this group should spread across N hosts before considering doubling up" | |
| 15:30:31 | dansmith | that's an easier to reason-about model from the outside I think | |
| 15:30:44 | dansmith | and it doesn't depend on the size of the group | |
| 15:31:02 | mriedem | i can't say if that's easier to reason about from the outside or not, | |
| 15:31:04 | dansmith | it focuses on the amount of redundancy you need regardless | |
| 15:31:10 | mriedem | since i've had this in my head the way it's proposed | |
| 15:31:21 | mriedem | dansmith: mention it in the spec? | |
| 15:31:36 | mriedem | either way i need to bring it up at the ptg b/c of some questions about how to model the thing in the api and db | |
| 15:32:21 | dansmith | I really hate the idea of adding extra_specs to anything, but especially groups | |
| 15:32:44 | mriedem | i said that on the spec | |
| 15:34:48 | dansmith | I guess their use-case actually specifically requires this as the model | |
| 15:37:12 | mriedem | is sgordon still a man about town? | |
| 15:37:18 | mriedem | isn't he in the UK/Ireland area? | |
| 15:37:40 | sgordon | mriedem, nah canadia | |
| 15:37:44 | mriedem | drats | |
| 15:37:52 | mriedem | sgordon: can you take a look at that spec regardless of your physical location? | |
| 15:38:09 | mriedem | gibi might also have some input here since ericsson added the soft affinity stuff | |
| 15:38:09 | sgordon | mriedem, yeah i just got the email notification and opened it up | |
| 15:39:14 | sgordon | mriedem, i have to admit i havent had anyone clamouring for this in a fair while, but that may just mean folks are working around it using host aggregates and higher level orchestration | |
| 15:43:44 | bauwser | mriedem: dansmith: we have a clear distinction between filters which hard-check possibilities and weighers which soft-check | |
| 15:44:15 | bauwser | having some middleground is something more than just for affinity | |
| 15:44:22 | bauwser | tbc, I'm -2 on the spexc | |
| 15:45:26 | bauwser | mriedem: dansmith: but johnthetubaguy had an idea to deprecate filters/weighers and have a different semantic for saying "here I want to hard-stop or fail-safe" | |
| 15:46:09 | johnthetubaguy | bauwser: got a pointer to the new one | |
| 15:46:22 | johnthetubaguy | might be better than my previous re-write the world plan | |
| 15:46:24 | dansmith | bauwser: this is harder than soft | |
| 15:46:38 | bauwser | soft is weighting | |
| 15:46:43 | bauwser | hard is filtering | |
| 15:46:54 | bauwser | here they want to have a soft-but-hard | |
| 15:46:57 | dansmith | bauwser: right, this spec specifies something hard, which is filtering, | |
| 15:47:08 | bauwser | bref | |