| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-10 | |||
| 18:53:15 | mriedem | which is actually a regression | |
| 18:53:16 | jmccarthy | mriedem: You about ? Trying to get 'test_cleanup_resize_not_same_host' to pass locally with tox, I don't really know this area - any hints how to resolve this conflict ? https://paste.fedoraproject.org/paste/ntZxhXGlWmzNiL--RZU~Ww | |
| 18:53:30 | mriedem | jmccarthy: see my backports? | |
| 18:53:40 | mriedem | https://review.openstack.org/#/q/Ib10081150e125961cba19cfa821bddfac4614408 | |
| 18:54:23 | jmccarthy | mriedem: Oh ! Ok cool lemme check that out | |
| 18:54:59 | mriedem | arvindn05: i could be wrong but i don't think that actually gets stored in the RequestSpec.image field | |
| 18:55:38 | jmccarthy | mreidem: Perfect - Thanks again ! Quite what to do there was giving me a headache lol | |
| 18:55:59 | arvindn05 | from what i saw...the volume meta data is treated exactly the same as image meta data,...the requestSpec.image has volume meta data in the case | |
| 18:56:36 | mriedem | arvindn05: i'm not so sure https://github.com/openstack/nova/commit/0a7427dc58eeb271646a962604106e0e84b870b6 | |
| 18:56:36 | arvindn05 | the only difference is that for volume backed instance the "id" field in the RequestSpec.image is not present... | |
| 18:56:40 | mriedem | but i don't know what you're seeing | |
| 18:56:54 | mriedem | oh that might be true | |
| 18:57:40 | arvindn05 | yup...the code there looks at the id field as well to determine...so looks like its consistent | |
| 18:57:45 | mriedem | oh i guess we use boot_meta here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L882 | |
| 18:58:28 | mriedem | https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L132 | |
| 18:58:30 | arvindn05 | now the only question is should we honor that metadata for volume boot? | |
| 18:59:42 | mriedem | idk | |
| 19:00:00 | arvindn05 | https://blueprints.launchpad.net/python-cinderclient/+spec/support-modify-volume-image-metadata an old blueprint discussion | |
| 19:00:10 | mriedem | i'm not sure how much precedent we have for honoring image meta during scheduling that comes from a volume | |
| 19:00:11 | arvindn05 | 20:32:54 <DuncanT> fmaldonado_: volume glance meta is for all of the same things glance properties are for | |
| 19:01:58 | arvindn05 | it seems that within the code, the volume meta data is treated exactly the same as image meta data...so i am leaning towards leaving it this way and adding few more unit and functional test cases | |
| 19:04:07 | clutchbeyers | mriedem: thats great to hear on the cross-az migration, we handle that in newton by use of the nova filters and extra specs that dont allow things to cross. But our specific use case is in the resize. If one spawns an instance without giving it an az, a resize will cold migrate (shutdown) and treat it like a new vm spin up and select another az with new flavors. Are you saying this behavior also does not allow for cross- | |
| 19:05:22 | mriedem | clutchbeyers: the change in pike would apply to any move operation through the scheduler, | |
| 19:05:32 | mriedem | so resize, live migrate, unshelve and evacuate | |
| 19:09:04 | clutchbeyers | mriedem: thank you for the clarification! | |
| 19:10:40 | mriedem | clutchbeyers: note that we could consider this a regression in pike. it's in your favor i guess, but depending on how we handle this bug we might change it back. | |
| 19:11:02 | mriedem | dansmith: maybe something we need to put in your forum session on cells... | |
| 19:13:08 | dansmith | mriedem: I'm not sure what you're saying is a regression | |
| 19:13:11 | dansmith | or what it has to do with cells | |
| 19:13:31 | dansmith | that you can't migrate across az boundaries/ | |
| 19:13:38 | mriedem | dansmith: https://bugs.launchpad.net/nova/+bug/1768876 | |
| 19:13:38 | openstack | Launchpad bug 1768876 in OpenStack Compute (nova) queens "metadata-api fails to get availability zone for instances created before pike" [High,Triaged] | |
| 19:14:02 | clutchbeyers | mriedem: it my honest opinion, if a tenant does not specify an az on vm provision, then they arent too concnered about controlling their fault domain. So when they do decide to resize and it moves to another az, they theoretically shouldn't care | |
| 19:14:33 | clutchbeyers | But for those that specifically call out an az, they have a fault domain architecture in mind and wouldnt want to disrupt that | |
| 19:14:56 | dansmith | mriedem: okay, I think we talked about that in dublin, about potentially just forcing instance.az to whatever we chose if you didn't ask for one, keeping you in whatever az you happened to land in | |
| 19:15:26 | melwitt | I think that's what the original behavior was -- only allows migrating across AZ if none was selected deliberately | |
| 19:15:47 | dansmith | melwitt: you mean only keeps you in your az if you asked for one initially | |
| 19:15:53 | clutchbeyers | there is some assumptions being made there with how the tenant is designing their app/fault tolerance | |
| 19:16:07 | melwitt | dansmith: yeah | |
| 19:16:22 | dansmith | clutchbeyers: I totally agree that if you asked for one, you shouldn't be migrated out of it | |
| 19:17:18 | dansmith | I get that mriedem's metadata thing is a regression, | |
| 19:17:28 | dansmith | I'm missing what, if any, scheduler behavior regressed | |
| 19:17:29 | mriedem | dansmith: i don't remember talking about this specifically in dublin, | |
| 19:17:41 | mriedem | the only AZ thing I remember talking about wrt cells was the cross_az_attach volume upcall thing | |
| 19:18:06 | mriedem | dansmith: what i'm pointing out in this comment https://bugs.launchpad.net/nova/+bug/1768876/comments/4 | |
| 19:18:06 | openstack | Launchpad bug 1768876 in OpenStack Compute (nova) queens "metadata-api fails to get availability zone for instances created before pike" [High,Triaged] | |
| 19:18:12 | dansmith | I'm pretty sure it was dublin, with bauzas and I | |
| 19:18:12 | mriedem | i need to test it still to confirm it is a regression | |
| 19:19:47 | dansmith | okay, I'm confused because I thought the second part of your comment (that an instance would be confined to whatever AZ it landed in) wasn't yet the case, but we considered doing that | |
| 19:20:01 | dansmith | in the dublin discussion where we were talking about how to handle instance.az when az wasn't requested | |
| 19:21:19 | mriedem | maybe we were talking about that in the context of volume cross_az_attach? | |
| 19:21:53 | dansmith | I dunno, I thought not, but I don't see it on the etherpad | |
| 19:22:08 | dansmith | I _know_ I've had a convo with bauzas about this in the not too distant past | |
| 19:22:22 | dansmith | maybe the az discussion there is merging together in my head | |
| 19:22:45 | mriedem | maybe related to the ops list thread on not allowing deleting/renaming an AZ that has instances in it? | |
| 19:23:09 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: always create consumer records https://review.openstack.org/567678 | |
| 19:23:17 | dansmith | this was face-to-face or face-to-screen | |
| 19:23:19 | dansmith | with bazuas | |
| 19:24:10 | dansmith | mriedem: what change in pike do you think is making us keep things in the same az? | |
| 19:24:30 | mriedem | https://review.openstack.org/#/c/446053/ | |
| 19:25:04 | dansmith | ah, yeah | |
| 19:25:13 | mriedem | if the instance.availability_zone was set based on CONF.default_availability_zone when the user didn't request one specifically, before that change, and that defaults to None, i believe that means the instance could freely move between AZs | |
| 19:25:33 | mriedem | anyway, i can write a simple functional test to try this once i get this 'remove fping' patch up | |
| 19:25:49 | dansmith | yeah, so, this is the change I was talking about, | |
| 19:25:59 | dansmith | which is probably when I was talking to bauzas about it (before writing this) | |
| 19:26:06 | dansmith | I just forgot that we had made it | |
| 19:26:42 | mriedem | heh, time flies :) | |
| 19:26:48 | dansmith | apparently | |
| 19:26:50 | mriedem | i don't think i was aware of this change either until yesterday | |
| 19:26:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove support for /os-fping REST API https://review.openstack.org/567682 | |
| 19:26:59 | melwitt | I just checked and the default is 'nova', not None. not sure if that makes a difference | |
| 19:27:13 | mriedem | melwitt: not for CONF.default_availability_zone | |
| 19:27:20 | mriedem | there are 2 "default" AZ options | |
| 19:27:22 | dansmith | so, going back to my "up for debate" comment above, | |
| 19:27:27 | mriedem | one for the user's instance and one for the scheduler | |
| 19:27:32 | melwitt | that's what I'm looking at? | |
| 19:27:39 | dansmith | what's the actual problem with keeping it where it is? | |
| 19:27:46 | melwitt | default_schedule_zone is the None one | |
| 19:27:58 | melwitt | (has no default) | |
| 19:28:29 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: always create consumer records https://review.openstack.org/567678 | |
| 19:28:37 | mriedem | i always have to read this again https://docs.openstack.org/nova/latest/user/aggregates.html#availability-zones-azs | |
| 19:29:13 | mriedem | melwitt: oh that must have changed, | |
| 19:29:15 | mriedem | see pike | |
| 19:29:19 | mriedem | https://docs.openstack.org/nova/pike/configuration/config.html | |
| 19:29:32 | mriedem | oh nvm it didn't change https://docs.openstack.org/nova/pike/configuration/config.html#DEFAULT.default_availability_zone | |
| 19:29:35 | mriedem | wtf my eyes | |
| 19:30:23 | melwitt | tbh the only reason I remember the 'nova' availability zone being default is from using devstack | |
| 19:30:33 | dansmith | [12:27:34] <dansmith>what's the actual problem with keeping it where it is? | |
| 19:30:43 | mriedem | oh my comment in the bug is wrong | |
| 19:30:45 | mriedem | default_schedule_zone is for instances | |
| 19:30:50 | mriedem | default_availability_zone is for compute services | |
| 19:31:10 | mriedem | dansmith: i don't know that there is, | |
| 19:31:14 | mriedem | except it's a behavior change | |
| 19:31:17 | mriedem | but, | |
| 19:31:26 | mriedem | (2:11:06 PM) mriedem: dansmith: maybe something we need to put in your forum session on cells... | |
| 19:31:26 | melwitt | dansmith: the concern is if that broke the original behavior of letting an instance move across AZs if someone didn't specify an AZ to begin with, if they don't care about AZ | |
| 19:31:27 | mriedem | :) | |
| 19:31:47 | melwitt | but we're not 100% sure yet if that behavior changed, mriedem is gonna test it out in a func test | |
| 19:31:47 | mriedem | maybe it's just an FYI to operators at this point, idk | |
| 19:31:56 | dansmith | no I think it did change | |