| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-10-01 | |||
| 15:32:52 | dansmith | but, the behavior you notice is not the version, it's whether or not the field sticks, which is what I was focusing on, and I think it will, even though that doesn't match the version | |
| 15:32:54 | dansmith | meaningm | |
| 15:33:11 | artom | dansmith, wouldn't upleveling not be a problem because we have code to make it compatible to old versions? | |
| 15:33:25 | dansmith | if you store the object with the new field, it will serialize the new field even if the version is wrong, and a newer node will still pull it out because it doesn't know which version the field belongs to | |
| 15:33:30 | artom | So if the conductor uplevels something, then an old compute loads, it will automatically downlevel for that compute? | |
| 15:34:04 | dansmith | artom: well, I picked conductor because it was easy, but imagine two computes talking | |
| 15:35:43 | dansmith | so I'm not actually sure I can explain priteau's observation of the new versions | |
| 15:36:02 | dansmith | that's how I recalled it working, but I can't back that up with references to the code purely in the deserialization/serialization stuff | |
| 15:36:12 | priteau | These instances were live migrated after the Victoria upgrade, could it explain it? | |
| 15:36:22 | dansmith | do we recreate the numa objects as part of another operation? yeah, like live migration | |
| 15:38:17 | dansmith | this is the test that proves that we don't automatically uplevel it: https://github.com/openstack/oslo.versionedobjects/blob/e7b6d52aa4b1b40e68a21a122c09b968d5959b0e/oslo_versionedobjects/tests/test_objects.py#L778-L785 | |
| 15:38:31 | dansmith | which isn't how I remembered it, but re-reading that test jogs my memory about why | |
| 15:40:33 | dansmith | so I think technically that operation should do something like object.VERSION = ObjectClass.VERSION when it goes to save it out | |
| 15:46:47 | artom | So the upleveling happens at serialization then? | |
| 15:46:58 | artom | Or some other operation like live migration, as you've said | |
| 15:47:00 | dansmith | it doesn't | |
| 15:47:04 | artom | But not at de-serialization, at any rate | |
| 15:47:20 | dansmith | I'm thinking live migrations create new numa stuff for the new host and that's how those got updated | |
| 15:48:18 | dansmith | in the past, we've done things like "load and save the object to update it" which would run routines to do updates, but they are probably saving the objects back with the old version but including the updates :/ | |
| 15:48:28 | dansmith | I can't find any VERSION updating going on in the tree right now | |
| 15:48:42 | dansmith | like for online data migrations I mean | |
| 15:49:08 | dansmith | technically it probably hasn't mattered, because who (besides artom and priteau) would notice that the version was wrong if the data was right? :) | |
| 15:49:54 | artom | Yeah, maybe it's an academic debate - if it does the right thing... | |
| 15:50:00 | artom | Which it appears to do | |
| 15:50:22 | priteau | I just wanted to check it wouldn't break anything if the version number was not updated | |
| 15:50:23 | dansmith | well, we shouldn't store the version incorrectly just because that's going to be confusing, but yeah the object would still load with the data we wanted so in practice it hasn't mattered | |
| 15:50:41 | dansmith | priteau: right, but we should fix it to update the version though | |
| 15:52:02 | priteau | I will leave that to you if you don't mind :D | |
| 15:52:42 | dansmith | artom: do you want me to do that or will you? | |
| 15:53:28 | artom | dansmith, do what? What priteau said, "fix it to update the version though"? | |
| 15:53:38 | artom | Wouldn't that be within priteau own's patch? | |
| 15:53:53 | artom | Well, maybe not, since the on-load migration is pre-existing... | |
| 15:53:54 | dansmith | artom: I said it, but no, the code that claims to migrate is already in the tree | |
| 15:53:55 | artom | He's just fixin it | |
| 15:53:55 | dansmith | right | |
| 15:53:58 | dansmith | right | |
| 15:54:21 | dansmith | looks like there are several _migrate_* cases in instance_numa.py | |
| 15:54:28 | artom | And probably elsewhere... | |
| 15:54:33 | dansmith | maybe, not sure | |
| 15:54:57 | dansmith | if you do it I can review/+2 but ... :) | |
| 15:55:46 | artom | I mean, the hint of a +2 from dansmith on a PS1 is pretty alluring... | |
| 15:55:58 | dansmith | well, if you get it right in PS1.. :P | |
| 15:56:13 | artom | Crazier things have happened | |
| 15:56:14 | dansmith | let me write you a diff to start, hang on | |
| 15:58:13 | dansmith | I think this should be unified instead of done in multiple places like it is, so something like this: https://termbin.com/iawkf | |
| 15:59:12 | dansmith | oh, you know what, | |
| 15:59:22 | dansmith | the other migrate routine is for non-ovo migrations, | |
| 15:59:40 | dansmith | which is different, so probably no need to break out that helper function anyway | |
| 16:00:07 | dansmith | so just the one line VERSION thing plus an assert in whatever tests this and you're probably good | |
| 16:02:13 | artom | Oh I see (I think), your initial thought was "have any on-load migration call to a helper that updates version to latest" | |
| 16:02:49 | dansmith | yeah, because I thought there were at least two, | |
| 16:02:53 | dansmith | but that's because I didn't read | |
| 16:04:19 | artom | Yeah makes sense | |
| 16:04:34 | artom | My stomach is requiring food, I think lunch first, patch after | |
| 16:04:53 | dansmith | I +2 that ordering | |
| 16:05:32 | artom | \o/ | |
| 16:48:52 | melwitt | gmann: no I was not aware of the novnc fail, I will look at it | |
| 16:52:32 | gmann | melwitt: thanks. | |
| 20:00:38 | melwitt | gmann: just to update you, I know what is wrong but I don't find how it's happening yet. tl;dr is the "path" in the vnc proxy is wrong and doesn't contain the token (so it's not even validating a token). according to all of the config files I looked at, it should be working. so there is something I'm missing so far | |
| 20:01:01 | melwitt | this is an example of how the path is supposed to look: https://zuul.opendev.org/t/openstack/build/2a183ce6e15f4ab08d31190d4d286a7b/log/controller/logs/screen-n-novnc-cell1.txt#11 | |
| 20:01:14 | melwitt | and this is the bad one that's failing: https://zuul.opendev.org/t/openstack/build/9234f33acf1247cdb995416ca9389240/log/controller/logs/screen-n-novnc-cell1.txt#13 | |
| 20:04:14 | melwitt | this is where it would pick up the "path" variable https://github.com/novnc/noVNC/blob/v0.4/vnc_auto.html#L106 | |
| 22:45:16 | opendevreview | melanie witt proposed openstack/nova master: Add logic to enforce local api and db limits https://review.opendev.org/c/openstack/nova/+/712139 | |
| 22:45:17 | opendevreview | melanie witt proposed openstack/nova master: Update quota_class APIs for db and api limits https://review.opendev.org/c/openstack/nova/+/712143 | |
| 22:45:17 | opendevreview | melanie witt proposed openstack/nova master: Enforce api and db limits https://review.opendev.org/c/openstack/nova/+/712142 | |
| 22:45:18 | opendevreview | melanie witt proposed openstack/nova master: Update quota sets APIs https://review.opendev.org/c/openstack/nova/+/712749 | |
| 22:45:18 | opendevreview | melanie witt proposed openstack/nova master: Update limit APIs https://review.opendev.org/c/openstack/nova/+/712707 | |
| 22:45:19 | opendevreview | melanie witt proposed openstack/nova master: Enforce resource limits using oslo.limit https://review.opendev.org/c/openstack/nova/+/615180 | |
| 22:45:19 | opendevreview | melanie witt proposed openstack/nova master: Tell oslo.limit how to count nova resources https://review.opendev.org/c/openstack/nova/+/713301 | |
| 22:45:20 | opendevreview | melanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits https://review.opendev.org/c/openstack/nova/+/713498 | |
| 22:45:21 | opendevreview | melanie witt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/c/openstack/nova/+/715271 | |
| 22:45:21 | opendevreview | melanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/c/openstack/nova/+/713499 | |
| 22:45:23 | opendevreview | melanie witt proposed openstack/nova master: DNM Run against unmerged oslo.limit changes https://review.opendev.org/c/openstack/nova/+/812236 | |
| #openstack-nova - 2021-10-02 | |||
| 01:19:27 | opendevreview | melanie witt proposed openstack/nova master: Add logic to enforce local api and db limits https://review.opendev.org/c/openstack/nova/+/712139 | |
| 01:19:27 | opendevreview | melanie witt proposed openstack/nova master: DNM Run against unmerged oslo.limit changes https://review.opendev.org/c/openstack/nova/+/812236 | |
| 01:19:28 | opendevreview | melanie witt proposed openstack/nova master: Update quota_class APIs for db and api limits https://review.opendev.org/c/openstack/nova/+/712143 | |
| 01:19:28 | opendevreview | melanie witt proposed openstack/nova master: Enforce api and db limits https://review.opendev.org/c/openstack/nova/+/712142 | |
| 01:19:29 | opendevreview | melanie witt proposed openstack/nova master: Update quota sets APIs https://review.opendev.org/c/openstack/nova/+/712749 | |
| 01:19:29 | opendevreview | melanie witt proposed openstack/nova master: Update limit APIs https://review.opendev.org/c/openstack/nova/+/712707 | |
| 01:19:30 | opendevreview | melanie witt proposed openstack/nova master: Enforce resource limits using oslo.limit https://review.opendev.org/c/openstack/nova/+/615180 | |
| 01:19:30 | opendevreview | melanie witt proposed openstack/nova master: Tell oslo.limit how to count nova resources https://review.opendev.org/c/openstack/nova/+/713301 | |
| 01:19:31 | opendevreview | melanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/c/openstack/nova/+/713499 | |
| 01:19:31 | opendevreview | melanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits https://review.opendev.org/c/openstack/nova/+/713498 | |
| 01:19:33 | opendevreview | melanie witt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/c/openstack/nova/+/715271 | |
| 02:14:52 | opendevreview | melanie witt proposed openstack/nova master: DNM Run against unmerged oslo.limit changes https://review.opendev.org/c/openstack/nova/+/812236 | |
| 03:59:21 | opendevreview | melanie witt proposed openstack/nova master: DNM Run against unmerged oslo.limit changes https://review.opendev.org/c/openstack/nova/+/812236 | |
| 04:09:23 | opendevreview | Wenping Song proposed openstack/nova master: vGPU support by Cyborg https://review.opendev.org/c/openstack/nova/+/812247 | |
| 04:18:32 | opendevreview | melanie witt proposed openstack/nova master: DNM Run against unmerged oslo.limit changes https://review.opendev.org/c/openstack/nova/+/812236 | |
| 05:19:35 | opendevreview | melanie witt proposed openstack/nova master: DNM Run against unmerged oslo.limit changes https://review.opendev.org/c/openstack/nova/+/812236 | |
| 06:20:00 | opendevreview | melanie witt proposed openstack/nova master: Enforce api and db limits https://review.opendev.org/c/openstack/nova/+/712142 | |
| 06:20:00 | opendevreview | melanie witt proposed openstack/nova master: Add logic to enforce local api and db limits https://review.opendev.org/c/openstack/nova/+/712139 | |
| 06:20:01 | opendevreview | melanie witt proposed openstack/nova master: Update limit APIs https://review.opendev.org/c/openstack/nova/+/712707 | |
| 06:20:01 | opendevreview | melanie witt proposed openstack/nova master: Update quota_class APIs for db and api limits https://review.opendev.org/c/openstack/nova/+/712143 | |
| 06:20:02 | opendevreview | melanie witt proposed openstack/nova master: Tell oslo.limit how to count nova resources https://review.opendev.org/c/openstack/nova/+/713301 | |
| 06:20:02 | opendevreview | melanie witt proposed openstack/nova master: Update quota sets APIs https://review.opendev.org/c/openstack/nova/+/712749 | |
| 06:20:03 | opendevreview | melanie witt proposed openstack/nova master: Enforce resource limits using oslo.limit https://review.opendev.org/c/openstack/nova/+/615180 | |
| 06:20:04 | opendevreview | melanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/c/openstack/nova/+/713499 | |
| 06:20:04 | opendevreview | melanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits https://review.opendev.org/c/openstack/nova/+/713498 | |
| 06:20:06 | opendevreview | melanie witt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/c/openstack/nova/+/715271 | |
| 06:27:50 | opendevreview | Brin Zhang proposed openstack/nova-specs master: Move the implemented specs for the xena release https://review.opendev.org/c/openstack/nova-specs/+/812248 | |