| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-04-06 | |||
| 14:07:00 | stephenfin | bauzas: Think you could hit this up https://review.opendev.org/c/openstack/nova/+/872429/ ? | |
| 14:22:29 | opendevreview | Dan Smith proposed openstack/nova-specs master: Add compute-object-ids spec for 2023.2 https://review.opendev.org/c/openstack/nova-specs/+/877291 | |
| 14:22:44 | dansmith | sean-k-mooney: added a bunch more meat to this ^ around migration handling | |
| 14:23:08 | dansmith | as I'm working through the actual implementation I'm finding that we throw node names around more than I was really thinking, because of the migrations | |
| 14:46:12 | dansmith | sean-k-mooney: also, see my comment here: | |
| 14:46:13 | dansmith | https://review.opendev.org/c/openstack/nova/+/878238 | |
| 14:46:36 | dansmith | I don't want to recheck that and make it merge if you disagree, but.. I'm pretty sure that's not related since that same failure has been happening a lot | |
| 16:26:30 | opendevreview | Dan Smith proposed openstack/nova-specs master: Add compute-object-ids spec for 2023.2 https://review.opendev.org/c/openstack/nova-specs/+/877291 | |
| 16:49:52 | opendevreview | Merged openstack/nova stable/yoga: Reproduce live migration rollback w/o multi port bindings error https://review.opendev.org/c/openstack/nova/+/839921 | |
| 17:13:16 | sean-k-mooney | dansmith: ill be aroudn for a litle bit but before i finsih for the long weekend i wante to checkback with you for the laze load patch | |
| 17:13:20 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/878238 | |
| 17:13:46 | sean-k-mooney | you mentioned you see the same failure in other runs | |
| 17:14:33 | sean-k-mooney | given that its proably unrelated just looks suspcious given what you were updateing | |
| 17:14:46 | sean-k-mooney | i cna read +2 and recheck it | |
| 17:15:01 | sean-k-mooney | but i was wondering if you had filed a gate bz or anything for this | |
| 17:15:10 | sean-k-mooney | *gate bug | |
| 17:15:26 | dansmith | no, I haven't noticed it before this, I just went looking to see if it was really new here | |
| 17:15:41 | dansmith | I'm not sure why we'd see it for scheduler and not conductor, which already added the singleton infra | |
| 17:15:44 | sean-k-mooney | so i think i have seen this before but not consitently | |
| 17:16:07 | sean-k-mooney | well the base class is also alredy resting the singelton | |
| 17:16:17 | sean-k-mooney | all you changed is doing it via the property | |
| 17:16:28 | sean-k-mooney | which is even more strange | |
| 17:16:48 | sean-k-mooney | like i dont know why calling the singelton later would chagne the behavior | |
| 17:16:52 | dansmith | ...right | |
| 17:19:44 | sean-k-mooney | so the hits you found seam to be for 878763, 875621, 879500 and now 878238 | |
| 17:30:00 | sean-k-mooney | dansmith: so other then the fact that two of these were submited by you id dont see anything quicly that woudl make me think this is specific to your change | |
| 17:30:19 | sean-k-mooney | so i think this is an exting issue so we can proceed with the lazy loading | |
| 17:31:02 | sean-k-mooney | the first hit in the lst 30 days was Mar 28, 2023 @ 12:58:01.786 | |
| 17:31:11 | sean-k-mooney | so maybe we could see what merged just before that | |
| 17:31:49 | dansmith | but we're also not hitting them very often, so I'm not sure looking at temporal merges is likely to be very fruitful | |
| 17:32:02 | opendevreview | Dan Smith proposed openstack/nova master: Add compute_id to Instance object https://review.opendev.org/c/openstack/nova/+/879500 | |
| 17:32:34 | dansmith | sean-k-mooney: based on this I'm assuming you didn't see my ping about it above, which means you likely didn't see my ping before that about the spec? | |
| 17:32:34 | sean-k-mooney | fair i guess we can just keep an eye out and see if this starts happening more often | |
| 17:33:11 | sean-k-mooney | correct but scrolling up i see them | |
| 17:33:22 | dansmith | no rush, so don't worry about it today | |
| 17:33:37 | dansmith | just fyi for tomorrow or next week | |
| 17:35:09 | sean-k-mooney | ya proably next week | |
| 17:35:24 | sean-k-mooney | skiming it i did not think about migration being affect by this durign the ptg | |
| 17:35:39 | dansmith | me either, but we update host and node from migration a LOT | |
| 17:35:39 | sean-k-mooney | but ya it would make sesne if we are addign the srive_id to the instace | |
| 17:36:04 | sean-k-mooney | before the ptg i was not expectign to update the instnace in this cycle | |
| 17:36:16 | dansmith | my patch actually makes us do that less, updating it to be "this node here" instead of "whatever the migration said" in several places | |
| 17:36:23 | sean-k-mooney | i assumed it was just going ot be the compute but there is value in doing both | |
| 17:36:39 | sean-k-mooney | ack | |
| 17:37:18 | sean-k-mooney | ya so jira has eaten all my brin power so im not digesting the spec reading it | |
| 17:37:34 | sean-k-mooney | so ya ill take a look on tueday and just assume your right :) | |
| 17:38:01 | sean-k-mooney | i did see you had started the work on it | |
| 17:38:04 | dansmith | yep, np | |
| 17:38:21 | dansmith | as noted in the ptg, | |
| 17:38:36 | dansmith | I think we should do more developing the code and spec in parallel, after initial sign-off on the idea | |
| 17:38:45 | dansmith | so I'm trying to keep my spec updated with the wrinkles I'm finding | |
| 17:38:59 | dansmith | and just hope that bauzas will approve me before the spec lands :) | |
| 17:42:20 | sean-k-mooney | ill reply on artom patch | |
| 17:42:57 | sean-k-mooney | but if i recall corectly its because this is invoked in the perodic task asn we are trying to avoid loading all of the instnace numa toplogy blob form the data base | |
| 17:43:28 | dansmith | but we're iterating it already no? | |
| 17:43:30 | sean-k-mooney | this is invoked in the update_avaiable_resouces periodic | |
| 17:43:51 | dansmith | it seems like we're just creating a new topology by iterating the old one, then doing the math | |
| 17:44:01 | dansmith | like, instead of obj_clone() (if we need to be doing it on a copy) | |
| 17:45:04 | sean-k-mooney | i also dont have this fully loaded in cache but i think we are actully iterating over the domains intially rather then having a fully set of instance objects | |
| 17:45:30 | dansmith | we're iterating over host topology, AFAICT | |
| 17:45:41 | dansmith | but not important right now, I was just asking | |
| 17:45:45 | dansmith | (as noted in my reply) | |
| 17:47:05 | sean-k-mooney | so im not saying this cant be recactored but if i recall correctly its becasue of this https://github.com/openstack/nova/blob/49aa40394a4857a06191b05ea3b15913f328a8d0/nova/compute/resource_tracker.py#L1334-L1371 code path | |
| 17:48:33 | dansmith | right, it's making a copy, modifying it and returning | |
| 17:48:52 | dansmith | obj_clone() is the way to do that to avoid silly omissions like this bug | |
| 17:49:22 | sean-k-mooney | ya | |
| 17:49:25 | dansmith | anyway, as I said, I'm not saying it needs to be refactored right now, I was just wondering why we were doing it the most laborious and error-prone way possible | |
| 17:55:47 | sean-k-mooney | so this comes form the initall commit it seams | |
| 17:55:49 | sean-k-mooney | https://github.com/openstack/nova/commit/b11dbfa4902cdd74bad3745db177d80b1c8b07c6 | |
| 17:57:32 | sean-k-mooney | https://github.com/openstack/nova/blob/b11dbfa4902cdd74bad3745db177d80b1c8b07c6/nova/virt/hardware.py#L1121-L1151 | |
| 17:58:14 | sean-k-mooney | that predates OVOs being split out of nova i think i wonder if it just the clone did not exist then | |
| 17:59:08 | dansmith | no, it did | |
| 17:59:49 | dansmith | perhaps just an omission | |
| 17:59:58 | sean-k-mooney | https://github.com/openstack/nova/blob/b11dbfa4902cdd74bad3745db177d80b1c8b07c6/nova/objects/base.py#L370-L372 | |
| 18:00:04 | sean-k-mooney | so ya it did | |
| 18:00:11 | sean-k-mooney | and since it worked we just didnt clean it up | |
| 18:00:38 | sean-k-mooney | anyway i woudl prefer to proceed with the small fix and maybe swap to that on master in a followup | |
| 18:01:05 | sean-k-mooney | we would ahve to see if there are any field that should not be cloned | |
| 18:01:27 | sean-k-mooney | liek we want to reset the usage filed | |
| 18:01:28 | dansmith | ...that's what I said | |
| 18:02:11 | sean-k-mooney | oh i know im just confirming that looking at the git history its not obvious why that approch was nto taken orginally | |
| 18:02:34 | dansmith | okay | |
| 18:03:44 | sean-k-mooney | ok im going to finish for today. if i dont get to your spec on tuesday feel free to remind me | |
| 18:03:51 | sean-k-mooney | enjoy your weekend o/ | |
| 18:04:05 | opendevreview | Merged openstack/nova stable/xena: Add debug log for scheduler weight calculation https://review.opendev.org/c/openstack/nova/+/879404 | |
| 18:04:18 | dansmith | ack, thanks, later | |
| #openstack-nova - 2023-04-07 | |||
| 07:39:28 | Uggla | bauzas, can we merge https://review.opendev.org/c/openstack/nova/+/868089, as all fixes from Amit have been merged ? | |
| 07:40:16 | bauzas | Uggla: I can take a look | |
| 07:40:58 | Uggla | bauzas, sure last time you told me that it was ok. But you would like to merge Amit patches first. | |
| 07:41:31 | bauzas | -ETOOMANYBUGFIXESINMYBRAIN | |
| 07:43:21 | bauzas | I wish I could just change some internal SDcard | |
| 07:43:32 | Uggla | bauzas, no hurries. I just want to not forget about it. | |
| 07:44:10 | Uggla | I have another one for review but I will submit it to you next week. I don't want to ruin your Friday. ;) | |
| 07:44:48 | bauzas | well | |
| 07:44:55 | bauzas | my Friday is just a regular Friday | |
| 07:44:58 | bauzas | paperworks and so. | |
| 07:45:15 | bauzas | Gerrit was updated, yet again UI \o/ | |
| 08:49:37 | kashyap | bauzas: "Bugs increase to fill the available time" ;-) | |
| 08:50:14 | kashyap | Original quote: "Work expands to fill the available time" | |
| 11:12:30 | opendevreview | Alexey Stupnikov proposed openstack/nova stable/victoria: Cleanup old resize instances dir before resize https://review.opendev.org/c/openstack/nova/+/864730 | |