| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-28 | |||
| 10:27:08 | bauzas | johnthetubaguy: then your instance will be constrainted forever to that AZ | |
| 10:27:20 | bauzas | johnthetubaguy: yeah, I'm about to review that patch | |
| 10:27:27 | johnthetubaguy | yep, anywhere you land its meant to stay there forever, I thought | |
| 10:27:40 | frickler | so let me just add our deployment scenario for reference: we have three AZs of compute nodes with local storage and do maintainance on them in turn for a single AZ at a time. | |
| 10:27:50 | bauzas | johnthetubaguy: yup, because instance.az should never be modified | |
| 10:28:03 | johnthetubaguy | right, so its just a scheduling decision | |
| 10:28:05 | frickler | customers that want HA for their service will need to deploy three instances, one per AZ | |
| 10:28:30 | johnthetubaguy | frickler: totally, that makes sense | |
| 10:28:40 | bauzas | johnthetubaguy: in terms of placement, you're right | |
| 10:28:49 | openstackgerrit | Gao Yuan proposed openstack/nova master: Stop using mox stubs in nova.tests.unit.api.openstack.compute.test_serversV21 https://review.openstack.org/438873 | |
| 10:29:04 | bauzas | johnthetubaguy: that conf opt default instances to be placed on a specific AZ, that's what it means | |
| 10:29:22 | johnthetubaguy | frickler: the server groups stuff, once its fixed up, is how you can do that in a better way, but we got loads of placement work before we get there | |
| 10:29:41 | johnthetubaguy | bauzas: yep, I thats what I was trying to say, anyways | |
| 10:29:43 | bauzas | (but there are ways to migrate your server out of that AZ :D) | |
| 10:29:59 | frickler | now we still would like to have "don't care" instances scheduled to any of the AZs. which seems difficult, at least in the dashboard | |
| 10:30:28 | johnthetubaguy | frickler: ah, that should be the default, the dashboard may have broken that | |
| 10:30:43 | frickler | any my idea was to set default_schedule_zone differently on three API servers that are accessed round robin | |
| 10:30:48 | johnthetubaguy | frickler: like "nova boot myserver image_7" should do that | |
| 10:30:51 | bauzas | frickler: that's a separate usecase | |
| 10:31:26 | johnthetubaguy | frickler: so I have a feeling we spoke about this a few months back, and I got confused then too | |
| 10:31:27 | bauzas | frickler: setting default_schedule_zone to None (well, to that ugly empty string rather) means "I don't care about AZs" | |
| 10:31:32 | jsheeren | hi all, i looking at curl calls and filtering results on listing the servers | |
| 10:31:33 | johnthetubaguy | frickler: I thought default_schedule_zone = None does what you want | |
| 10:31:48 | bauzas | but it doesn't force the instance to stick with that AZ | |
| 10:32:03 | bauzas | frickler: for your usecase, you could just use a scheduling filter tho | |
| 10:32:03 | johnthetubaguy | bauzas: it should do I thought | |
| 10:32:38 | jsheeren | with a curl to "https:// |
|
| 10:32:53 | jsheeren | if i use "https:// |
|
| 10:33:07 | bauzas | johnthetubaguy: nope, if you wanna RR between AZs, you need to set default_schedule_zone differently between your API workers | |
| 10:33:12 | jsheeren | but when i try to filter on metadata, it just returns all the servers instead of filtering on the metadata | |
| 10:33:22 | johnthetubaguy | jsheeren: loads of details on here: https://developer.openstack.org/api-ref/compute/ | |
| 10:33:49 | johnthetubaguy | jsheeren: I don't think you can filter by everything (its a performance thing), and in some cases its a policy thing | |
| 10:34:13 | johnthetubaguy | bauzas: whaaaat? | |
| 10:34:15 | bauzas | frickler: see what I said above ^ | |
| 10:34:27 | jsheeren | filter on metadata using: "https:// |
|
| 10:34:35 | bauzas | johnthetubaguy: yup | |
| 10:34:41 | bauzas | johnthetubaguy: lemme clarify | |
| 10:35:11 | jsheeren | johnthetubaguy: aha,gonna check the policy first; i read through the developer api-ref doc a few times now :) | |
| 10:35:29 | bauzas | johnthetubaguy: again, default_schedule_zone is just a terrible thing for saying what should be the default if the user hasn't specific --az in their boot command | |
| 10:36:17 | johnthetubaguy | jsheeren: all the available filters are listed in the docs now (thats a recent change): https://developer.openstack.org/api-ref/compute/?expanded=list-servers-detail#list-servers | |
| 10:36:21 | jsheeren | compute policy: "compute:get_instance_metadata": "rule:admin_or_owner" i would assume i can see the metadata, which i can, cause it's in the full detail | |
| 10:36:26 | bauzas | the semantics of the empty string meaning the exact same thing as if the user hasn't specify --az : ie. the instance won't be AZ-constrained | |
| 10:36:40 | johnthetubaguy | jsheeren: you can see it, but not query based on it | |
| 10:36:51 | johnthetubaguy | jsheeren: not 100% sure, you can query based on server tags though | |
| 10:37:16 | frickler | bauzas: I will look into writing a custom filter, thx | |
| 10:37:25 | johnthetubaguy | bauzas: right, but None means just pick any, which appears to be the request? | |
| 10:37:32 | johnthetubaguy | bauzas: I am clearly missing something | |
| 10:37:47 | bauzas | johnthetubaguy: None means "leave that field unset" | |
| 10:38:02 | johnthetubaguy | bauzas: right, but doesn't it get set when it lands on a compute node | |
| 10:38:12 | bauzas | johnthetubaguy: I mean, None means "leave the Instance object having the AZ field unset" | |
| 10:38:19 | bauzas | exactly | |
| 10:38:51 | bauzas | if the Instance object doesn't have an AZ information, it will never stick to the AZ the host where it was placed belongs to | |
| 10:38:54 | jsheeren | johnthetubaguy: https://github.com/openstack/nova/blob/stable/mitaka/nova/compute/api.py#L2112 | |
| 10:39:03 | jsheeren | metadata is listed in the filter_mapping | |
| 10:39:54 | bauzas | there are 2 colocality informations : one for the instance and one for the host | |
| 10:40:04 | bauzas | an host can belong to an AZ | |
| 10:40:15 | bauzas | and one instance can belong to an host | |
| 10:40:31 | laci | hi! | |
| 10:40:36 | bauzas | so, this is transitive | |
| 10:41:00 | bauzas | one instance can be in one AZ by the transitivity of the relationship AZ -> host -> instance | |
| 10:41:29 | bauzas | that's the default | |
| 10:42:00 | bauzas | but you can give a specific colocality information on one instance if you specify which AZ it should belong to | |
| 10:42:28 | bauzas | in that case, we would force to honor the transitivity by making sure we pick hosts only belonging to the same AZ | |
| 10:43:04 | johnthetubaguy | bauzas: was looking in the code, I was expecting AZ to get set to the host value when we set the host on the instance, somewhere near here: https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L579 | |
| 10:43:12 | johnthetubaguy | bauzas: I can't find the right place though right now | |
| 10:43:25 | bauzas | johnthetubaguy: I would violently -1 it if I was seeing that code | |
| 10:43:41 | johnthetubaguy | bauzas: I guess I don't get why | |
| 10:43:49 | bauzas | johnthetubaguy: because the AZ information attached to the instance is what the user expressed | |
| 10:43:57 | johnthetubaguy | thats in the build request | |
| 10:44:09 | bauzas | not the result of the placement decision we made | |
| 10:44:17 | bauzas | well | |
| 10:44:25 | johnthetubaguy | yeah, that sounds wrong to me | |
| 10:44:30 | bauzas | first, build request is only temporary | |
| 10:44:37 | johnthetubaguy | yeah, I was meaning the other one | |
| 10:44:42 | bauzas | secondly, I agree, it's rather RequestSpec | |
| 10:44:48 | johnthetubaguy | yeah, request spec, my bad | |
| 10:44:55 | bauzas | but I wrote that object later | |
| 10:44:57 | bauzas | so | |
| 10:45:09 | bauzas | a move could be to totally remove the instance AZfield | |
| 10:45:12 | bauzas | AZ field | |
| 10:45:32 | bauzas | I mean, deprecate it at the object level, and wait for a major version bump to remove it | |
| 10:45:46 | johnthetubaguy | instance.host tells you the current host, instance.az feels like it should be similar | |
| 10:45:49 | johnthetubaguy | but thats just me | |
| 10:46:00 | johnthetubaguy | anyways, I think I get enough for the docs patch now | |
| 10:46:01 | jsheeren | johnthetubaguy: tags look like an option, but we're still on mitaka .. :( | |
| 10:46:09 | bauzas | johnthetubaguy: no, that's then duplicate information | |
| 10:46:39 | bauzas | johnthetubaguy: I by far prefer to only keep instance.host and leave the transitivity to determine an instance AZ | |
| 10:46:51 | johnthetubaguy | bauzas: it is duplicating host, but its kinda useful, its the AZ cached at the time the instance landed on the host | |
| 10:46:57 | johnthetubaguy | just like flavor | |
| 10:47:12 | bauzas | there are a full list of bugs related to the fact we badly manage that relationship | |
| 10:47:18 | johnthetubaguy | but anyways, I clearly see this differently, needs a spec to sort that out | |
| 10:47:43 | bauzas | well, you're kinda busy this cycle, so I could give an hand | |
| 10:47:47 | johnthetubaguy | we should record, where we first landed, and what the user requested, I believe | |
| 10:48:06 | johnthetubaguy | bauzas: I have no time to sort that out this cycle, if you have cycles feel free | |
| 10:48:12 | bauzas | but honestly, touching AZs is like trying to unfuse a bomb | |
| 10:48:40 | bauzas | certainly not a thing for people willing to contribute at first hand, unfortunately :( | |
| 10:48:43 | johnthetubaguy | bauzas: maybe a backlog spec to capture the thoughts? | |
| 10:48:49 | bauzas | sorting all that out took me like 5 cycles | |
| 10:49:02 | bauzas | I could do that | |
| 10:49:15 | bauzas | I'm pretty free this cycle since I really wanted to make reviews | |