| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-08-30 | |||
| 11:12:21 | opendevreview | Merged openstack/python-novaclient master: support get and list server's metadata https://review.opendev.org/c/openstack/python-novaclient/+/915977 | |
| 11:18:41 | gibi | Uggla: on the other hand I found the reason why the instance.update notification had shares=null and it was a config issue on my side as that notification coming from other than the compute service and I only enabled shares in the notification in the compute config. So that works as expected and shares are filled when I set the config in the relevant / all services | |
| 11:26:06 | gibi | elodilles: could you check this backport https://review.opendev.org/q/I9a2a7614313297f11a55d99fb94916d3583a9504 , please? :) | |
| 12:04:09 | manuvakery1 | Hey.. All of sudden lot of vms shutoff automatically | |
| 12:04:15 | manuvakery1 | "libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: /usr/include/c++/11/bits/random.tcc:2667: void std::discrete_distribution<_IntType>::param_type::_M_initialize() [with _IntType = int]: Assertion '__sum > 0' failed" | |
| 12:04:27 | manuvakery1 | I see this error in the logs | |
| 12:05:10 | manuvakery1 | qemu instance log also has same error | |
| 12:05:19 | manuvakery1 | https://www.irccloud.com/pastebin/edKaN6J0/ | |
| 12:05:46 | manuvakery1 | running openstack wallaby | |
| 12:06:02 | manuvakery1 | any hints on the root cause | |
| 12:06:39 | auniyal | it says reason=crashed, so nova may be not aware of it ! | |
| 12:06:58 | auniyal | anything in nova-compute logs ? | |
| 12:07:31 | manuvakery1 | compute log also has same error | |
| 12:07:32 | manuvakery1 | 2024-08-30 11:56:08.942 22938 ERROR nova.compute.manager [instance: 9dea6796-4103-4f55-8caa-16dc6848bbf9] libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: /usr/include/c++/11/bits/random.tcc:2667: void std::discrete_distribution<_IntType>::param_type::_M_initialize() [with _IntType = int]: Assertion '__sum > 0' failed. | |
| 12:08:20 | manuvakery1 | https://www.irccloud.com/pastebin/5H4M8DvL/ | |
| 12:09:34 | auniyal | so this is during new VM creation | |
| 12:09:39 | manuvakery1 | yup | |
| 12:14:46 | elodilles | gibi: ACK, will do o/ | |
| 12:15:24 | elodilles | bauzas auniyal : if you have some time to review the python-novaclient release patch, that would be awesome: https://review.opendev.org/c/openstack/releases/+/927136 | |
| 12:25:08 | Uggla | gibi, sorry I have just noticed you comments. Good that the instance.update was due to a misconfiguration because I was not confident on that one. | |
| 12:26:07 | Uggla | gibi, I need to fix some stuff on the client patches, so I will bump to 2.97. I will tell you when it will be ready. | |
| 12:28:31 | auniyal | elodilles, done | |
| 12:30:21 | elodilles | auniyal: thanks \o/ | |
| 12:54:38 | sean-k-mooney | bauzas: gibi can we proceed with https://review.opendev.org/c/openstack/nova/+/922492 | |
| 12:55:46 | sean-k-mooney | the depend on is already meged | |
| 12:58:39 | opendevreview | Fabian Wiesel proposed openstack/nova master: Vmware: Remove uuid parameter from get_vmdk_info call https://review.opendev.org/c/openstack/nova/+/910627 | |
| 12:59:13 | gibi | sean-k-mooney: so 3.12 is a non mandatory test target for 2024.2 and ubuntu 24.04 not on the tested runtimes list. How stable this job is with 3.12? Do we risk a gate block? | |
| 13:01:49 | bauzas | I left a +2 for saying I'm OK but I agree with gibi, we need to discuss that point before merging | |
| 13:02:24 | sean-k-mooney | https://zuul.openstack.org/builds?job_name=devstack-platform-ubuntu-noble&skip=0 it looks resanobale to me on the devstack side https://zuul.openstack.org/builds?job_name=devstack-platform-ubuntu-noble&skip=0 we also moved whitebox to it a while ago | |
| 13:03:31 | sean-k-mooney | it will be a required testing runtime for 2025.1 so we will need to start moving all the jobs shortly | |
| 13:03:56 | sean-k-mooney | so i want to have this in dalmaiton too if we can | |
| 13:04:19 | gibi | sean-k-mooney: OK those job results looks acceptable to me. I think we can merge it and if we see an uptick of nova-next failures we can always revert it. | |
| 13:04:46 | sean-k-mooney | +1 to reverting if it proves unstable | |
| 13:05:16 | gibi | bauzas: any objection before I +A it? | |
| 13:06:04 | sean-k-mooney | i think the python 3.12 coverage is very valuable as its also a proxy for rhel 10 | |
| 13:06:10 | bauzas | I don't find any | |
| 13:06:11 | sean-k-mooney | until we get centos 10 enabled | |
| 13:06:32 | sean-k-mooney | to od that i or someone need to update disk image built to supprot it but that not that hard | |
| 13:06:33 | bauzas | we could revert the bump if needed | |
| 13:07:15 | gibi | approved | |
| 13:07:46 | sean-k-mooney | thanks :) im going to grab a coffee brb | |
| 13:15:13 | gibi | have a nice one | |
| 13:16:20 | opendevreview | Fabian Wiesel proposed openstack/nova master: Vmware: Remove uuid parameter from get_vmdk_info call https://review.opendev.org/c/openstack/nova/+/910627 | |
| 13:20:44 | sean-k-mooney | its some cold brew i made at the start of the week. i have not been sleeping weel the last few days so (4-5) horus most night beign awake until after 5 in the morning | |
| 13:21:03 | opendevreview | Fabian Wiesel proposed openstack/nova master: Vmware: Remove uuid parameter from get_vmdk_info call https://review.opendev.org/c/openstack/nova/+/910627 | |
| 13:21:27 | sean-k-mooney | so this is my first coffe of the day with a big dash of salted carammel, i normlaly drink it black | |
| 13:22:05 | sean-k-mooney | idd blame the lack fo sleep on the cofee but i normally only have one cup a day and i skip it 2 days thsi week | |
| 13:22:35 | gibi | sean-k-mooney: this year I did not brew cold. I try to limit my coffein intake as it disturbs my sleep and the cold one has more kick than a lightly packed mokka | |
| 13:23:33 | sean-k-mooney | ya i got an expresso machien about 3-4 years ago so i went form 2-3 nesspreo pods a day to one ammericano | |
| 13:23:49 | sean-k-mooney | turns out to be about the same amount of cafeine | |
| 13:54:06 | artom | Where's our opensearch/elasticsearch instance again? | |
| 13:55:28 | artom | https://opensearch.logs.openstack.org/_dashboards/app/ needs a login... | |
| 14:01:23 | dansmith | artom: openstack/openstack | |
| 14:01:31 | sean-k-mooney | ya that | |
| 14:01:32 | dansmith | apparently the new one can't be unauthenticated | |
| 14:03:38 | sean-k-mooney | artom: are you looking into my comment https://review.opendev.org/c/openstack/nova/+/926083/comments/ad6713dc_e6542156 | |
| 14:04:37 | artom | sean-k-mooney, the shelve thing? No, I got an openstacksdk error in the latest results | |
| 14:04:45 | sean-k-mooney | ah | |
| 14:04:55 | sean-k-mooney | ya i saw that job fail in one of my other patches | |
| 14:05:07 | sean-k-mooney | i have not reeckec but it passed on the next patch in my series | |
| 14:05:10 | artom | Really trivial fix, I was just looking how often it happened | |
| 14:05:13 | sean-k-mooney | so i assume there s an itermint failure | |
| 14:05:34 | artom | Yeah, cloud.get returns None, and we unconditionally try to [ ] that | |
| 14:06:00 | artom | Seems like just skipping the None case with an if and retrying the loop iteration would help... | |
| 14:07:14 | sean-k-mooney | manuvakery1: that feels like something i have heard fo before but its obviously a qemu bug | |
| 14:07:56 | sean-k-mooney | i guess it could be in libvirt but its defnintly not in nova | |
| 14:08:47 | sean-k-mooney | well cloud.get is presumable reading form the clouds.yaml which should nto be none | |
| 14:09:49 | artom | I think it's an API call... it's polling for stack events | |
| 14:10:17 | sean-k-mooney | i didnt look at the actual errror yet | |
| 14:10:19 | artom | Sorry, cloud.get_stack() :) | |
| 14:10:34 | artom | Which I assume calls Heat | |
| 14:10:44 | sean-k-mooney | presumeable | |
| 15:11:07 | opendevreview | Fabian Wiesel proposed openstack/nova master: Vmware: Remove uuid parameter from get_vmdk_info call https://review.opendev.org/c/openstack/nova/+/910627 | |
| 16:01:41 | dansmith | sean-k-mooney: +2 now ^ | |
| 17:04:35 | sean-k-mooney | dansmith: im out of energy now but i reveiwed that and approved. if you are going to request a FFE for the nova enablement of multiple locations support, given its size i think i woudl support that but if just want to leave that to next cycle it think thats also fine. it really depends n when the tempest stuff get resolved i guess so ill deffer to your judgement there. | |
| 17:05:10 | dansmith | well, tbh, I wasn't even going to consider it a feature | |
| 17:05:30 | dansmith | it's tiny and it's just using the glance api the right way, | |
| 17:05:39 | dansmith | but if you think it needs to be considered as such, then that's fine | |
| 17:05:52 | dansmith | not sure what that really means with bauzas gone next week though | |
| 17:06:09 | sean-k-mooney | well more i was suggeting its small enouch that we may want to consier not procedurlly blocking it | |
| 17:06:25 | sean-k-mooney | if the tempet issue can be resolved by the next team meeting | |
| 17:06:38 | sean-k-mooney | well FFE dont requrie PTL input | |
| 17:06:50 | sean-k-mooney | historically it only requried 3 cores to say yes | |
| 17:07:23 | sean-k-mooney | i only consider https://review.opendev.org/c/openstack/nova/+/891036 a feature in that we would not back port it | |
| 17:08:04 | sean-k-mooney | mainly because of the requirements bump but i agree otherwise its just using glances api the secure way | |
| 17:08:04 | dansmith | there's no point in backporting it | |
| 17:08:22 | sean-k-mooney | right because there is no supprot in older glance | |
| 17:08:27 | dansmith | I meant determining whether it was a feature or not, for the PTL buy-in.. I think that's bauzas' decision to make if it's unclear | |
| 17:08:58 | dansmith | anyway, we'll keep pushing on the tempest patch and can revisit on monday maybe | |
| 17:09:24 | sean-k-mooney | ack, either way i thin kthis could merge after rc1 if we decied to wait | |
| 17:09:29 | dansmith | the tempest patch isn't required, AFAIK, it just reduces a race that is already there just not hit because we don't actually use it yet, but it has passed without it | |
| 17:09:32 | sean-k-mooney | having it 2025.1 is the main thing | |
| 17:10:10 | sean-k-mooney | ah ok so its not strictly a dep but we would have some instablity without it | |
| 17:10:34 | dansmith | right | |
| 17:11:01 | sean-k-mooney | ok well im going to checkout for the weekend | |
| 17:11:07 | sean-k-mooney | catch you monday o/ | |
| 17:11:18 | dansmith | o/ | |
| 18:39:29 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Factor out metadata and send to ironic https://review.opendev.org/c/openstack/nova/+/927639 | |
| 19:38:14 | opendevreview | Merged openstack/nova stable/2023.1: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/912511 | |
| 19:39:16 | bauzas | I'm still there, I have some slides to work on | |