| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-02-01 | |||
| 09:39:12 | sean-k-mooney[m] | oh you need https://github.com/openstack/nova/blob/master/.zuul.yaml#L186-L189 as well | |
| 09:39:27 | bauzas | oh wait, I have ovn installed | |
| 09:39:33 | sean-k-mooney[m] | yes | |
| 09:39:43 | sean-k-mooney[m] | its the neturon metadata agent for ovn that failed | |
| 09:39:52 | sean-k-mooney[m] | not the ovn install | |
| 09:40:00 | bauzas | yeah | |
| 09:40:14 | bauzas | I can try to dig into the logs of that agent, if I'm able to find them | |
| 09:40:27 | sean-k-mooney[m] | they are in the journal like all the rest | |
| 09:40:41 | sean-k-mooney[m] | so just journalctl -u devstack@whatever | |
| 09:41:31 | sean-k-mooney[m] | im stepping away for 5-10 minutes to grab coffee but when im back if you want me to ssh in and take a look with you i can | |
| 09:41:50 | bauzas | yeah | |
| 09:41:55 | bauzas | and thanks | |
| 09:48:58 | bauzas | ahah, found the culprit https://paste.opendev.org/show/bFigNVupZWRpjlmuzXZ9/ | |
| 09:50:21 | sahid | o/ regarding live migrations, we are agree that we can live-migrate vm from compute node that are N-1 to N ? | |
| 09:50:53 | sahid | in a situation of upgrading version of openstack | |
| 09:53:49 | bauzas | sure, not the other way | |
| 09:58:10 | sahid | ack thanks | |
| 10:12:56 | sean-k-mooney | bauzas: ah the privsep helper | |
| 10:13:36 | sean-k-mooney | bauzas: so tht issue is that its not on your path | |
| 10:13:59 | sean-k-mooney | bauzas: i put a path fix in devstack to fix that at one point | |
| 10:14:41 | bauzas | /usr/bin/which: no privsep-helper in (/opt/stack/.local/bin:/opt/stack/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin) | |
| 10:14:41 | bauzas | [stack@lenovo-sr655-01 devstack]$ which privsep-helper | |
| 10:14:47 | bauzas | hmmmm | |
| 10:16:09 | sean-k-mooney | it will be in the venv bin dir | |
| 10:16:24 | sean-k-mooney | if you have it disabled | |
| 10:16:46 | bauzas | I disabled the GLOBAL venv indeed | |
| 10:16:50 | sean-k-mooney | then it should be in /usr/local/bin but apprently not | |
| 10:17:14 | bauzas | I have oslo.privsep | |
| 10:17:21 | bauzas | lemme see where it's landed | |
| 10:17:49 | sean-k-mooney | https://github.com/openstack/devstack/blob/5c1736b78256f5da86a91c4489f43f8ba1bce224/stack.sh#L838 this is the rwrokaround for usign the venv | |
| 10:17:59 | sean-k-mooney | so you just need to do the same with where ever it is | |
| 10:18:33 | bauzas | I suspect that privsep got installed in the venv as I first started to stack.sh with this flag | |
| 10:18:57 | sean-k-mooney | ah ya so there are a bunch of things you have to manually clean up | |
| 10:18:59 | sean-k-mooney | if you did that | |
| 10:19:14 | sean-k-mooney | and removing those symlinks form /usr/local/bin is one of them | |
| 10:19:18 | bauzas | and after that, even when asking devstack to reclone, I tihnk it got a problem | |
| 10:19:28 | sean-k-mooney | reclone is not enaouch | |
| 10:19:37 | sean-k-mooney | so you need to delete the venv dir | |
| 10:19:55 | sean-k-mooney | and then remove all the broken symlinks form /usr/local/bin | |
| 10:20:14 | bauzas | yeah | |
| 10:20:16 | bauzas | [stack@lenovo-sr655-01 devstack]$ ll /usr/local/bin/privsep-helper | |
| 10:20:18 | bauzas | lrwxrwxrwx. 1 root root 39 Jan 31 11:10 /usr/local/bin/privsep-helper -> /opt/stack/data/venv/bin/privsep-helper | |
| 10:20:30 | sean-k-mooney | yep | |
| 10:20:46 | sean-k-mooney | but neutron is proably not in the venv | |
| 10:20:48 | bauzas | okay, I'll reinstall the package | |
| 10:21:07 | bauzas | and I'll look at other symlinks in /usr/local/bin | |
| 10:21:20 | sean-k-mooney | as i said nuke the venv first then remove any broken links | |
| 10:22:28 | sean-k-mooney | i didnt document the process but it took me about an hour to do that on centos properly but its doable to make work | |
| 10:23:48 | bauzas | yeah | |
| 10:23:54 | bauzas | thanks for the help | |
| 10:24:21 | sean-k-mooney | note i had to remove some egg link files too | |
| 10:24:38 | sean-k-mooney | what i basically did ws unstack.sh; clean.sh | |
| 10:24:48 | bauzas | yup, https://github.com/openstack/devstack/blob/master/tools/fixup_stuff.sh says it | |
| 10:24:54 | sean-k-mooney | remove all openstack related thigns form /usr/local/bin | |
| 10:25:26 | sean-k-mooney | and remove all openstack packages via pip and or egglink files | |
| 10:26:26 | sean-k-mooney | after that i cloud jsut stack aagain without the venv and it was fine | |
| 10:27:03 | sean-k-mooney | the venv is a huge improvement but goign between each mode is a pain | |
| 10:27:22 | bauzas | yeah and clean.sh doesn't do all the cleanup :) | |
| 10:27:45 | bauzas | I recommend everyone stacking a new env every quarter :) | |
| 10:27:47 | sean-k-mooney | it appreanly does not install ovs and ovn which i coudl have bet money on it doing in the past | |
| 10:27:54 | sean-k-mooney | that bit me a few days ago | |
| 10:28:32 | sean-k-mooney | i almost added that to clean but i had other things to do | |
| 10:28:54 | bauzas | sean-k-mooney: I wonder, I guess we can't easily run the zuul jobs on our local machines ? | |
| 10:29:16 | sean-k-mooney | well... i can kind of | |
| 10:29:37 | bauzas | it would be simplier for a new developer to just hook their dev machines into some zuul registery so they would just run a specific zuul job for installing devstack and its knobs | |
| 10:29:52 | sean-k-mooney | i wrote an ansibel molecule senairo | |
| 10:29:59 | bauzas | yeah I remember that | |
| 10:30:00 | sean-k-mooney | that uses the upstream zuul job roles | |
| 10:30:02 | sean-k-mooney | https://github.com/SeanMooney/ard/blob/master/molecule/default/molecule.yml | |
| 10:30:15 | bauzas | ideally, I'd just use the .zuul.yaml definitions | |
| 10:30:28 | sean-k-mooney | ya that is not possibel today | |
| 10:30:46 | sean-k-mooney | there was a propoal to do that with a thing called zuul-runner | |
| 10:30:59 | sean-k-mooney | clarkb: ^ that never actully happened right | |
| 10:31:01 | bauzas | interesting | |
| 10:31:09 | bauzas | I mean, don't get me wrong | |
| 10:31:50 | sean-k-mooney | bauzas: the idea was you would point it at zuul adn give it the proejct name ectra and give it some resouce to use and well it would run the zuul executor logic locally and run the job agaist those resoucs | |
| 10:32:09 | bauzas | the usecase I see here is that if dev contributors start using zuul roles for deploying their devstacks, they could get less effort in configuring devstack and installing it, but they could also give the money back by also contributing to the roles definitions | |
| 10:32:41 | sean-k-mooney | bauzas: if you have never done it zuul is actully pretty simple to run on your laptop with a singel docker-compoes up | |
| 10:32:50 | bauzas | for a newcomer, trying to untangle devstack (if their install failed) is somewhat tricky | |
| 10:33:15 | sean-k-mooney | it is but i still think its an exersise tehy should do to understand whot things work | |
| 10:33:21 | sean-k-mooney | but thats why i create the ard stuff | |
| 10:33:30 | bauzas | the more they'd stick to 'standard validated upstream roles', the better chances they have to get their nodes ready | |
| 10:33:32 | sean-k-mooney | did you know that devstack used to have a vagrant file | |
| 10:33:36 | bauzas | yup | |
| 10:33:42 | bauzas | I remember it | |
| 10:33:55 | sean-k-mooney | it might still exist | |
| 10:34:02 | sean-k-mooney | i never got it to work for what it wanted | |
| 10:34:58 | sean-k-mooney | so the stuff i have in the ard repo can be pointed at any host and it will give you a mulit node devstack | |
| 10:35:21 | sean-k-mooney | i used it to provisin some host in china form ireland for gibi when he first joined redhat | |
| 10:36:01 | sean-k-mooney | i was looking at a zuul job yesterday when the time stamps suddely went 5 hours into the past | |
| 10:36:21 | sean-k-mooney | when i realised zuul was running in the us east coast and the vms were in europe | |
| 10:36:55 | sean-k-mooney | i was pretty churffed that zuul/ansible works that well across an ocean | |
| 12:58:46 | opendevreview | sean mooney proposed openstack/nova master: [WIP] add libvirt connection healtcheck https://review.opendev.org/c/openstack/nova/+/907424 | |
| 13:02:04 | opendevreview | Merged openstack/nova stable/2023.2: Revert "[pwmgmt]ignore missin governor when cpu_state used" https://review.opendev.org/c/openstack/nova/+/905672 | |
| 13:02:11 | opendevreview | Merged openstack/nova stable/2023.2: cpu: make governors to be optional https://review.opendev.org/c/openstack/nova/+/905673 | |
| 13:59:34 | bauzas | sean-k-mooney: fwiw, straight devstack install with GLOBAL_VENV=False worked like a charm with Centos 9 Stream | |
| 13:59:46 | bauzas | (ovn and all the likes) | |
| 13:59:55 | bauzas | no need to tweak anything | |
| 14:00:08 | sean-k-mooney | yep | |
| 14:00:15 | sean-k-mooney | centos 9 stream is a supproted distro | |