| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-06-28 | |||
| 12:48:36 | opendevreview | sean mooney proposed openstack/nova master: fix sr-iov support on Cavium ThunderX hosts. https://review.opendev.org/c/openstack/nova/+/777679 | |
| 13:00:08 | bauzas | stephenfin: when you're around, we can discuss on https://review.opendev.org/c/openstack/nova/+/798145 if you wish | |
| 13:00:45 | bauzas | tl;dr: problem is that we don't verify the AZs if you don't use the AZfilter | |
| 13:01:07 | bauzas | so we can't just look at them by the API service unless we know that the AZFilter is used | |
| 13:01:53 | sean-k-mooney | our down stream customer could avoid the issue they had if they just enabled the placemnt preilter | |
| 13:02:12 | sean-k-mooney | that would enforece the AZ existance check | |
| 13:02:20 | sean-k-mooney | but they could still select the host using the hack | |
| 13:03:03 | sean-k-mooney | bauzas: i do agree though that we should remove that in a new microversion now that we have teh new way to do it | |
| 13:03:08 | bauzas | sean-k-mooney: my thought is that we should just not using the az hack after a new microversion | |
| 13:03:18 | sean-k-mooney | yep | |
| 13:03:31 | sean-k-mooney | i was expecting that to have been done in the one that added --host | |
| 13:03:39 | bauzas | for sure, it wouldn't fix the issue of a requested AZ not good but... | |
| 13:04:14 | sean-k-mooney | i also agree with our assement tha the az in the request spec and instance are not always intended to match | |
| 13:04:34 | sean-k-mooney | classic example being request spec is none but instance has a value set | |
| 13:04:59 | sean-k-mooney | in princiapl i think that is the only ligitimat case where they should disagree | |
| 13:05:12 | sean-k-mooney | if the request spec is non None then they should agree | |
| 13:05:21 | sean-k-mooney | if they dont you forced a live migration | |
| 13:05:22 | stephenfin | bauzas: we don't currently, but I'm adding that | |
| 13:05:40 | stephenfin | and the AZFilter is no use to us if we're bypassing the scheduler by forcing a host | |
| 13:05:41 | sean-k-mooney | stephenfin: right but im not conviced you should | |
| 13:05:53 | sean-k-mooney | stephenfin: that is not how that works | |
| 13:06:03 | sean-k-mooney | we check that the az exists | |
| 13:06:04 | bauzas | stephenfin: what sean-k-mooney said | |
| 13:06:18 | stephenfin | requesting zone:host makes no sense if $host is not in $zone | |
| 13:06:18 | sean-k-mooney | and only proceed if it does when you use the az hack | |
| 13:06:35 | bauzas | stephenfin: it's an hack, we should just remove it | |
| 13:06:46 | stephenfin | we can't remove it for the older APIs | |
| 13:06:50 | bauzas | surely | |
| 13:06:51 | sean-k-mooney | stephenfin: that is something we could check potentally but im not sure the api is the right place | |
| 13:06:55 | stephenfin | so people will keep hitting this | |
| 13:07:04 | bauzas | stephenfin: it's an hack, right? | |
| 13:07:12 | sean-k-mooney | well its was a supported feature | |
| 13:07:13 | bauzas | and you need to be an operator | |
| 13:07:21 | bauzas | soooo | |
| 13:07:22 | sean-k-mooney | but yes | |
| 13:07:33 | bauzas | the az hack can't be used by an end user | |
| 13:07:42 | sean-k-mooney | yes it can | |
| 13:07:51 | bauzas | not by default | |
| 13:07:55 | sean-k-mooney | they just need to use an older microverion | |
| 13:08:12 | bauzas | the default policy is admin | |
| 13:08:20 | sean-k-mooney | bauzas: is it? | |
| 13:08:26 | bauzas | for the az hack ? yes | |
| 13:08:41 | sean-k-mooney | i tought we did not have a sepreate policy for it | |
| 13:08:42 | stephenfin | I must admit I don't understand the issue | |
| 13:08:43 | bauzas | (and fortunately) | |
| 13:08:45 | sean-k-mooney | just the az one | |
| 13:08:56 | stephenfin | why wouldn't a simple "does this host belong to this AZ" check make sense? | |
| 13:09:03 | stephenfin | it's not too expensive fwict | |
| 13:09:09 | gibi | bauzas: do you suggest to keep allowing calling --availability_zone my-az:host-not-in-my-az and succeed in old microversions? | |
| 13:09:13 | bauzas | sean-k-mooney: I'm 100% sure about the different policy | |
| 13:09:14 | stephenfin | a simple lookup in the API DB | |
| 13:09:53 | bauzas | gibi: we *could* fix this for old versions only, but then I have another concern | |
| 13:10:12 | sean-k-mooney | stephenfin: my main issue is that you are doing it in a different location to the other az check | |
| 13:10:15 | bauzas | gibi: my other concern is that I know some environments that don't use the AZfilter | |
| 13:10:19 | sean-k-mooney | stephenfin: whihc is doen in the schduler i belive | |
| 13:11:03 | bauzas | gibi: and previously, you were able to use the az hack without the AZFilter | |
| 13:11:22 | gibi | bauzas: yes, but that hack resulted in an inconsistent system | |
| 13:11:28 | gibi | as described in the bug | |
| 13:11:35 | bauzas | gibi: not if you don't use the filter | |
| 13:11:49 | bauzas | see the problem ? | |
| 13:12:19 | sean-k-mooney | gibi: well its basically forcing the host | |
| 13:12:26 | sean-k-mooney | same as a forced migration | |
| 13:12:33 | gibi | so if you dont use teh AzFilter then no AZ recorded in the instance or in the request_spec? | |
| 13:12:38 | bauzas | gibi: and again, I remember ourselves saying A LOT 'well, if you force a host, then meh" | |
| 13:12:56 | sean-k-mooney | gibi: the az should be recoreded in the isntacne regaradless of the filter | |
| 13:13:20 | bauzas | gibi: now, we become super picky about forcing hosts and we want to verify them | |
| 13:13:37 | gibi | I think the base issue is that if you use the hack then you end up having inconsistent az recorded in the instance and in the request_spec | |
| 13:13:39 | bauzas | but again, you *SHOULDN'T* force a destination | |
| 13:13:52 | gibi | we cannot remove the hack from old microversions | |
| 13:13:56 | bauzas | that's why we added 2.74 version | |
| 13:14:12 | bauzas | to have a way to propose a target without forcing it | |
| 13:14:19 | sean-k-mooney | gibi: you wont always | |
| 13:14:29 | bauzas | and we said as a consensus that we should stop supporting to force move | |
| 13:15:08 | bauzas | so, if operators wanna move (because again, you need to be ADMIN in order to use the AZ hack), then your dog | |
| 13:15:13 | sean-k-mooney | bauzas: this is the policy yes https://github.com/openstack/nova/blob/master/nova/policies/servers.py#L204-L225 | |
| 13:15:35 | bauzas | again, I was 100% sure about it | |
| 13:15:39 | sean-k-mooney | oh no that the new one | |
| 13:15:40 | gibi | bauzas: even if we fix the bug in the hack the admin still can move to any host just need to specy the proper az name of the host | |
| 13:15:48 | gibi | bauzas: so no functionality is lost | |
| 13:15:53 | sean-k-mooney | this is the old one https://github.com/openstack/nova/blob/master/nova/policies/servers.py#L177-L196 | |
| 13:16:00 | bauzas | gibi: admins can move to bad targets anyway | |
| 13:16:15 | sean-k-mooney | gibi: what would the fix be | |
| 13:16:28 | bauzas | gibi: admins can force migrate to hosts without verifying other attributes of the host | |
| 13:16:33 | gibi | bauzas: it is not a bad target, the host is valid, nova just record a wrong az name during the move as it trustes admin input | |
| 13:16:43 | sean-k-mooney | gibi: just include the AZ and not the host in the request spec? | |
| 13:17:07 | gibi | sean-k-mooney: the fix is to make sure admin provide an az name for the host that is valid for the host, then nova will record a valid az name | |
| 13:17:12 | bauzas | again, we're breaking existing behaviours if we change things | |
| 13:17:23 | gibi | bauzas: we are fixing a bug | |
| 13:17:29 | bauzas | because again, some operators opt-out the AZfilter | |
| 13:17:31 | gibi | and such we can break old buggy behaviro | |
| 13:17:45 | bauzas | gibi: it's not a bug, it's a 40x | |
| 13:17:56 | gibi | bauzas: the db inconsistency is the bug | |
| 13:17:59 | bauzas | no | |
| 13:18:00 | sean-k-mooney | gibi: stephenfin if we add this check we shoudl also move the az exits check to the api also | |
| 13:18:09 | bauzas | you asked for a target you can't succeed | |
| 13:18:23 | sean-k-mooney | stephenfin: you added that to your check but did you remove the check later | |
| 13:18:29 | bauzas | gibi: you can create an instance on AZ1, then force migrate to AZ2 | |
| 13:18:30 | gibi | bauzas: I still think that if nova creates an inconsistent db record then we should fix that | |
| 13:18:41 | bauzas | gibi: and then, good luck with resizing the instance | |
| 13:19:08 | sean-k-mooney | bauzas: well a resize in that case will resize back to AZ1 | |