| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-03-31 | |||
| 15:08:08 | gibi | I'm | |
| 15:11:04 | dansmith | certainly for this they're plenty adequate I think | |
| 16:19:10 | dansmith | sean-k-mooney: didn't you have a recipe for using the upstream ansible playbooks to deploy multinode devstack locally? | |
| 16:54:23 | ivveh | hey, im looking to be pointed in the correct direction. i want to nova to build my custom xmls. like hooks or some other method that is supported by nova and preferably has a framework for that. i'd like to avoid modifying the libvirt driver as much as possible | |
| 17:05:32 | sean-k-mooney | dansmith: sorry yes i have | |
| 17:07:47 | sean-k-mooney | dansmith: https://github.com/SeanMooney/ard/blob/master/ansible/deploy_multinode_devstack.yaml is the playbook that does it that was driven by molecule with vagrant_libvirt to provdie the vms https://github.com/SeanMooney/ard/blob/master/molecule/default/molecule.yml | |
| 17:08:11 | dansmith | ah with vagrant | |
| 17:08:24 | sean-k-mooney | you didnt need vagrant | |
| 17:08:41 | sean-k-mooney | the playbooks could work with any servers once you hadd ssh access | |
| 17:09:28 | sean-k-mooney | so like i provisioned some server internally for vdpa testing and just poited the palybooks at it with an inventory file | |
| 17:09:45 | sean-k-mooney | https://github.com/SeanMooney/ard/tree/master/examples/vdpa | |
| 17:11:13 | sean-k-mooney | i wrote 3 roles devstack_{common,compute,controller} that wrappted the upstream devstack roles form teh gate jobs | |
| 17:11:17 | sean-k-mooney | https://github.com/SeanMooney/ard/tree/master/ansible/roles | |
| 17:12:49 | sean-k-mooney | dansmith: its been like 3+ years since i used any of this | |
| 17:12:59 | sean-k-mooney | it proably still wroks but i havent been maintining it | |
| 17:13:15 | sean-k-mooney | dansmith: was there a specific reason you asked | |
| 17:14:22 | dansmith | sean-k-mooney: I was struggling through re-re-remembering the multinode devstack rain dance | |
| 17:14:28 | dansmith | but I got it figured out whilst waiting | |
| 17:14:37 | sean-k-mooney | ah ok | |
| 17:14:44 | dansmith | sean-k-mooney: gibi: can confirm that just that placement change fixes the nova migrate-while-over-capacity bug | |
| 17:14:52 | sean-k-mooney | i have some downstream docs of the process with bash | |
| 17:15:09 | dansmith | I was able to repro, then literally swapped placement to that commit, restarted, retried the same migration and it went, confirmed, all good | |
| 17:15:18 | sean-k-mooney | and one of my team recently wrote an ansible playbook to deploy multi node devstack on our internal cloud i can ping you does internally | |
| 17:15:33 | sean-k-mooney | dansmith: oh nice | |
| 17:16:24 | dansmith | sean-k-mooney: thanks, I need to formalize my own notes, because I have some of my own scripting and making things work with those is most ideal, | |
| 17:16:31 | dansmith | I just suck at persisting the notes after I get it workin, | |
| 17:16:39 | dansmith | but I'm going to do that now | |
| 17:18:11 | ivveh | as more context to my previous question is that i want to include iothread stanza into the xml (for now) | |
| 17:19:58 | sean-k-mooney | dansmith: the main parts of multi node are syncing the tls/ceph data, exchantign the ssh keys/known_hosts, ntp sync and updating /etc/hosts before deploying the comptues. then adding a discover host at the end | |
| 17:20:28 | dansmith | yeah, it was just the CA stuff I forgot this time | |
| 17:20:56 | dansmith | as soon as it failed, I knew the problem, but then had to remember to copy the CA dir *and* the bundle, lest it would nuke the latter on stacking the subnode | |
| 17:21:01 | sean-k-mooney | did you know that tls is appreantly not on by default in devstack. i always used the ci jobs as a basis for mine | |
| 17:21:14 | sean-k-mooney | but if you use devstack by default it does not enabel the tls-proxy | |
| 17:21:31 | dansmith | yeah, I have it on my base node though, so.. | |
| 17:21:43 | gibi | dansmith: nice test results | |
| 17:22:12 | dansmith | gibi: also works for the case where reserved is changed to cover the node, so that covers the OTU case too | |
| 17:22:17 | sean-k-mooney | ivveh: we do not have any hook point in nova to allow you to modify the xml. that is a very intetional decsions | |
| 17:23:05 | sean-k-mooney | ivveh: if you want to modify the xml you are required to modify the virt driver because we do not want to supprot external customisation fo the domain thoguh any kind of hooks | |
| 17:24:32 | sean-k-mooney | ivveh: what is it you are actully trying to do to the xml | |
| 17:25:40 | gibi | dansmith: cool | |
| 17:29:47 | dansmith | I was hoping to have tested the migration allocation thing for real last week but ran out of time. Luckily my monday morning meeting was canceled I could get a jump on it | |
| 17:46:34 | ivveh | sean-k-mooney: add <iothreads/> | |
| 17:48:06 | ivveh | (as i couldn't find it anywhere in the extraspecs or other methods of modifying the xml) but feel free to point me in the correct direcion if iothreads do exist, i need to point them out and pin on them, maybe some other stuff too | |
| 17:49:01 | sean-k-mooney | its not currently supproted. we had a blueprint to add it a year ago but the person working on it did actuly push any code | |
| 17:49:30 | ivveh | gotcha, would it be as an extraspec or some other method? | |
| 17:50:03 | sean-k-mooney | what i was suggeting was to allway allcoate 1 and make it float over the cpu_share_set | |
| 17:50:37 | sean-k-mooney | we discussed if we shoudl supprot more then one but the advice form our virt team was no | |
| 17:51:23 | sean-k-mooney | they suggested that in the future 1 per cinder volume might make sense but the high level degisn we had dicussed in teh past was 1 keep it simple and start with all vms having 1 iothread | |
| 17:51:38 | sean-k-mooney | 2 in the future add a flavor extra spec to opt into one per cinder volum eif needed | |
| 17:52:09 | sean-k-mooney | 3 possible allow this to be set on the image or config instead (TBD if/when we add the 1 per cinder volume feature) | |
| 17:52:49 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/939254 is the most recent attempt | |
| 17:53:03 | sean-k-mooney | https://blueprints.launchpad.net/nova/+spec/iothreads-for-instances | |
| 17:53:19 | ivveh | thanks, ill have a look | |
| 17:53:37 | sean-k-mooney | this version is adding a config option to contol the behvior | |
| 17:54:31 | sean-k-mooney | ah the blueprint has the link to the ptg dicssion form last cyle https://etherpad.opendev.org/p/nova-2025.1-ptg#L686 | |
| 17:55:20 | ivveh | yeah, i wouldn't know how to do it at this point, i was thinking of having a pool of cores for this use and somehow allocate to those cores via vms (no matter how many volumes they had). i also need to do the same thing for virtiofsd (a different more complex story as i also have to figure out how to force it to use fifo or change some code) | |
| 17:56:19 | sean-k-mooney | ivveh: so this level of low level customstion of the domain is onlyt possibe via modifying the in tree driver | |
| 17:56:41 | sean-k-mooney | upstream we consier any out of band modification of the dmoain to make the vm unsupported | |
| 17:56:49 | ivveh | yeah, thats what i thought. i was hoping to avoid it, but.. | |
| 17:57:23 | sean-k-mooney | ivveh: we only added the iniall use of virtiofs this cycle | |
| 17:57:43 | sean-k-mooney | so we have not looekd at any kind of affinity for vritio fs deamon | |
| 17:57:44 | ivveh | my usecase is very specific as i know how many and how large my vms can be and i wanna assign them a certain amount of cores | |
| 17:58:16 | sean-k-mooney | i see. that not really a usecase that is very compatible with cloud computeing :) | |
| 17:58:37 | ivveh | depends on what flavors you rent ;) | |
| 17:58:50 | ivveh | but i get your point | |
| 17:59:18 | sean-k-mooney | waht i mean is we allow you to express constratis like "i want this vm to have dedicated pinned cpus" and obviosly you can create falvors with X amount of cpus | |
| 17:59:32 | sean-k-mooney | but we intentioally do not allow you to do direct maping of logcial cpus to host cpus | |
| 17:59:46 | ivveh | yeah no, i understand openstack needs to have its constraints, its normal and acceptable | |
| 17:59:49 | sean-k-mooney | nova can do that internlly but its not exposed directly in the flavor | |
| 18:00:18 | ivveh | for shenenings you have to go custom | |
| 18:00:41 | sean-k-mooney | we talked about having a pool for iothread. but we didnt want to have to schdule on them | |
| 18:01:32 | ivveh | i think either way, it would be great for nova to have a iothread pool that nova/cinder could take resources from (as these days defaults aren't enough) | |
| 18:03:00 | ivveh | would be great to have like flavors with iothread spec extraspec for example, to be able to utilize those resources, or not | |
| 18:03:21 | sean-k-mooney | when we dicussed it the idea was for it to just be an addtion set of core, taht woould default to cpu_shared_set if not defiend in the config | |
| 18:04:38 | sean-k-mooney | i.e. add a iothread_set to nova. those cpu could overlap with cpu_shared_set but now cpu_dedicated_set. we kind of felt that was too complex to start with | |
| 18:04:44 | sean-k-mooney | that does not mean we could not add that later | |
| 18:06:12 | ivveh | yea if you use cpu_shared_set and vms are pinned then they would always be free for stuff like iothreads or virtiofsd | |
| 18:06:54 | sean-k-mooney | so we already supprot moving the emulator thread to cpu_shared_set for pinned vms | |
| 18:07:11 | sean-k-mooney | so we were effectivly going to treat the extra iothread the same | |
| 18:07:49 | ivveh | or cpu_isolation if you wanna let things do whatever, i guess | |
| #openstack-nova - 2025-04-01 | |||
| 01:56:08 | opendevreview | Yaguang Tang proposed openstack/nova stable/2023.2: Fix device type when booting from ISO image https://review.opendev.org/c/openstack/nova/+/945903 | |
| 06:52:10 | opendevreview | Yaguang Tang proposed openstack/nova stable/2024.1: Fix device type when booting from ISO image https://review.opendev.org/c/openstack/nova/+/945816 | |
| 09:28:55 | sahid | o/ | |
| 09:29:26 | sahid | any chance to have review regarding that one, et the serie behind? | |
| 09:29:27 | sahid | https://review.opendev.org/c/openstack/neutron/+/940983/6 | |
| 09:29:35 | sahid | it's waiting for a while now :-) | |
| 09:31:54 | sean-k-mooney | i assume you ment ot put that in the neutron channel | |
| 09:32:12 | stephenfin | sean-k-mooney: trivial README change here if you've 3.5 seconds https://review.opendev.org/c/openstack/nova/+/944994 | |
| 09:32:25 | sean-k-mooney | sahid: but i would not recommend creating a theread pool dynmaiclly in a plugin | |
| 09:32:46 | sean-k-mooney | sahid: if your going to use thread pools you shoudl create one globaly for neutron to use and reuse it | |
| 09:32:50 | sean-k-mooney | like we do in nova | |
| 09:34:00 | sahid | sean-k-mooney: oh yes... my mistake | |
| 09:34:22 | sahid | sean-k-mooney: that is an interesting point yes | |
| 09:34:41 | sean-k-mooney | sahid: https://review.opendev.org/c/openstack/nova/+/922497 | |
| 09:35:03 | sahid | i will discuss that with neutron, i think ralonsoh may also need it from it's change to the l3 agent | |
| 09:35:09 | sahid | its | |
| 09:35:09 | sean-k-mooney | actully that not quite the patch i wanted | |
| 09:35:36 | sean-k-mooney | well no that uses futureist | |
| 09:36:19 | sean-k-mooney | oh im not adding eventlet tpool there im just moving the import | |
| 09:36:26 | sean-k-mooney | thats what was confusing me | |