| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-10 | |||
| 19:27:13 | mriedem | melwitt: not for CONF.default_availability_zone | |
| 19:27:20 | mriedem | there are 2 "default" AZ options | |
| 19:27:22 | dansmith | so, going back to my "up for debate" comment above, | |
| 19:27:27 | mriedem | one for the user's instance and one for the scheduler | |
| 19:27:32 | melwitt | that's what I'm looking at? | |
| 19:27:39 | dansmith | what's the actual problem with keeping it where it is? | |
| 19:27:46 | melwitt | default_schedule_zone is the None one | |
| 19:27:58 | melwitt | (has no default) | |
| 19:28:29 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: always create consumer records https://review.openstack.org/567678 | |
| 19:28:37 | mriedem | i always have to read this again https://docs.openstack.org/nova/latest/user/aggregates.html#availability-zones-azs | |
| 19:29:13 | mriedem | melwitt: oh that must have changed, | |
| 19:29:15 | mriedem | see pike | |
| 19:29:19 | mriedem | https://docs.openstack.org/nova/pike/configuration/config.html | |
| 19:29:32 | mriedem | oh nvm it didn't change https://docs.openstack.org/nova/pike/configuration/config.html#DEFAULT.default_availability_zone | |
| 19:29:35 | mriedem | wtf my eyes | |
| 19:30:23 | melwitt | tbh the only reason I remember the 'nova' availability zone being default is from using devstack | |
| 19:30:33 | dansmith | [12:27:34] <dansmith>what's the actual problem with keeping it where it is? | |
| 19:30:43 | mriedem | oh my comment in the bug is wrong | |
| 19:30:45 | mriedem | default_schedule_zone is for instances | |
| 19:30:50 | mriedem | default_availability_zone is for compute services | |
| 19:31:10 | mriedem | dansmith: i don't know that there is, | |
| 19:31:14 | mriedem | except it's a behavior change | |
| 19:31:17 | mriedem | but, | |
| 19:31:26 | mriedem | (2:11:06 PM) mriedem: dansmith: maybe something we need to put in your forum session on cells... | |
| 19:31:26 | melwitt | dansmith: the concern is if that broke the original behavior of letting an instance move across AZs if someone didn't specify an AZ to begin with, if they don't care about AZ | |
| 19:31:27 | mriedem | :) | |
| 19:31:47 | melwitt | but we're not 100% sure yet if that behavior changed, mriedem is gonna test it out in a func test | |
| 19:31:47 | mriedem | maybe it's just an FYI to operators at this point, idk | |
| 19:31:56 | dansmith | no I think it did change | |
| 19:32:01 | dansmith | but Idon't think it's user-visible | |
| 19:32:06 | dansmith | what I do think makes more sense about the new behavior, | |
| 19:32:20 | melwitt | it only would be if all the compute hosts in the same AZ are at capacity and there are free ones in another AZ, right? | |
| 19:32:24 | dansmith | is that if I boot an instance, ssh in, check the AZ that it's in, and then boot an instance in another az to make sure I have separation, | |
| 19:32:34 | dansmith | then something moves my instance into the same az and I'm surprised | |
| 19:32:39 | melwitt | and they're stuck because it won't let the AZ be crossed | |
| 19:32:41 | dansmith | which won't happen now | |
| 19:33:00 | dansmith | melwitt: yeah but they have no idea if there is space in the cloud, or in their az | |
| 19:33:59 | mriedem | one fear i have is operators will rely on the force flag for live migrate or evacuate to bypass the scheduler now to move the instance into another compute just because that other compute is in another AZ, | |
| 19:34:01 | mriedem | but idk | |
| 19:34:02 | melwitt | yeah, not until they contact support I guess, and admin can see the migration should be able to work and it doesn't | |
| 19:34:38 | mriedem | if only we could force a cold migration to bypass the scheduler.... :) | |
| 19:34:40 | dansmith | mriedem: fair point, but not super strong, IMHO | |
| 19:34:46 | mriedem | dansmith: agree it's not strong | |
| 19:35:04 | mriedem | if you have 10 computes in AZ1 and 10 in AZ2, then just pick one of the ones that's already in the same AZ | |
| 19:35:38 | mriedem | also, kablam https://review.openstack.org/567682 | |
| 19:38:19 | openstackgerrit | Arvind Nadendla proposed openstack/nova master: update scheduler to use image-traits https://review.openstack.org/560596 | |
| 19:40:36 | clutchbeyers | mriedem: cold migrations bypassing the scheduler was a joke right? :) | |
| 19:42:31 | mriedem | clutchbeyers: L645 https://etherpad.openstack.org/p/nova-ptg-rocky | |
| 19:43:01 | mriedem | oh the -5 didn't make it into the notes | |
| 19:50:48 | melwitt | dansmith: I was just reading the ML thread about the map_instances marker thing, and I think what takashin was suggesting is to print the next marker before returning 1, and then let that be passed as a starting point when the command is run again https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L1193 | |
| 19:51:03 | melwitt | would that not work for some reason? | |
| 19:57:53 | edleafe | melwitt: if the batches were all done in a row, it might. But there could be long gaps of time between batches, so the marker needs to be stored | |
| 19:58:23 | melwitt | I see | |
| 20:07:08 | openstackgerrit | Arvind Nadendla proposed openstack/nova master: update scheduler to use image-traits https://review.openstack.org/560596 | |
| 20:07:38 | dansmith | melwitt: yeah I know he was talking about that and I agree with edleafe that it's not a great solution | |
| 20:07:54 | dansmith | melwitt: the bit I'm not sure about was the "it could be exposed accidentally through the api" part | |
| 20:09:20 | melwitt | okay, I think I missed the "exposed accidentally through the api" part | |
| 20:09:51 | melwitt | I didn't see anyone mention something like that | |
| 20:10:04 | dansmith | he said "in some compute rest apis, it returns the marker" | |
| 20:10:26 | melwitt | I think he means like in instance list, it returns the marker if there's additional pages available | |
| 20:11:06 | melwitt | like if you have > 1000 (api max limit) instances in the list, it will give you a "next marker" so you can query again and pass the marker to get the next page | |
| 20:11:32 | dansmith | you mean you think he's offering the REST API's pagination as an example that could be used for the cli to be user-supplied-marker behavior? | |
| 20:12:03 | melwitt | that's what I think he was saying, yeah. if doing it similar to that could avoid storing the marker | |
| 20:12:32 | dansmith | I guess I can see it being read like that, but to me it sounded like he was saying it's bad to store it in-band because there's a way you can see it externally | |
| 20:12:41 | melwitt | if map_instances hands you back the next marker, says "I did a batch but there's more" and you can call again and give the marker and it does the next batch | |
| 20:14:37 | dansmith | I understand how the user could supply a marker, yeah | |
| 20:15:00 | dansmith | I just think that's fairly terrible UX for a CLI, especially a chatty one that has all kinds of other crap going to stdio like python warnings | |
| 20:15:39 | melwitt | oh, hm. yeah, I guess if I re-read it and connect the 3rd paragraph to the 2nd, it could sound that way. but I thought he was stating it's not ideal to store the marker and then next gave a suggestion on how we could avoid storing it | |
| 20:16:07 | dansmith | I'll send another clarification mail | |
| 20:16:38 | arvindn05 | efried: gibi: mriedem: updated the code review for https://review.openstack.org/560596 - "update scheduler to use image-traits" | |
| 20:16:43 | melwitt | yeah, maybe it doesn't fit in so well for the CLI, but it's an interesting idea. probably the only way to avoid having to store a marker record | |
| 20:17:01 | efried | arvindn05: ack | |
| 20:19:08 | arvindn05 | efried: ty...hopefully the final PS for this change :) | |
| 20:19:18 | efried | arvindn05: Hah! | |
| 20:19:37 | efried | arvindn05: You've just karmically guaranteed that won't be the case! | |
| 20:19:52 | dansmith | melwitt: it's what we had before we replaced it with this, because it was annoying | |
| 20:20:04 | dansmith | https://review.openstack.org/#/c/301916/ | |
| 20:20:05 | melwitt | ah, okay | |
| 20:20:24 | mriedem | fwiw, | |
| 20:20:29 | arvindn05 | efried: lol...well to job security then...hehe | |
| 20:20:37 | efried | :) | |
| 20:20:58 | mriedem | the new heal_allocations CLI i added has a note in there about taking a marker from the user, and dumping out the last instance processed so the next run can start from the marker rather than process a bunch of instances already healed | |
| 20:22:09 | dansmith | mriedem: we have all these other instances where we're able to keep track of things for the admin, it feels pretty sucky to go back to "you must have a mouse and a scratch pad to use this thing" experience | |
| 20:22:16 | dansmith | discovery hosts, instance mapping, reqspec fixup | |
| 20:22:26 | melwitt | that's a really old change too, I somehow stayed blissfully unaware of how the map_instances CLI worked historically | |
| 20:22:35 | dansmith | all keep track of it for you because we're talking about potentially an ass-ton of 50-item iterations | |
| 20:23:17 | mriedem | i didn't realize we had a marker for the reqspec data migration | |
| 20:23:21 | dansmith | yup | |
| 20:23:39 | dansmith | it wasn't as controversial because it wasn't fighting a UC field | |
| 20:23:43 | dansmith | but it worked basically the same way | |
| 20:24:12 | mriedem | i'm not even sure where i'd store the marker in my case | |
| 20:24:22 | mriedem | i'm iterating the instances table per cell | |
| 20:24:30 | dansmith | yeah, I understand yours is a little less obvious, but these all have reasonable places | |
| 20:24:39 | mriedem | and we can't put a fake instance in the instaces db | |
| 20:24:42 | mriedem | *table | |
| 20:25:21 | dansmith | we could have a scratch pad table we use for this kind of thing.. just "varchar(255) key, text value" | |
| 20:25:32 | dansmith | no index, never expect to have much data in there, | |
| 20:25:38 | dansmith | just some context for things like this | |
| 20:25:54 | mriedem | well, i thought about system_metadata, | |
| 20:26:20 | mriedem | because i could query for a specific system_metadata with a marker key, get that instance, delete the marker entry and continue, but it's not great | |