| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-02-19 | |||
| 17:27:09 | sean-k-mooney | if we are not crating any addtional fake nodes then we can likely do it with just a config overried to change the virt driver to ironic and populate the nova.conf with the ironic details | |
| 17:27:11 | dansmith | ironic has no multinode jobs? | |
| 17:27:44 | TheJulia | We have one, adding further complexity to it is not ideal | |
| 17:27:49 | Uggla | gibi, thx, I will have a look. | |
| 17:28:00 | dansmith | eesh | |
| 17:28:14 | JayF | oh, we do > ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode https://zuul.opendev.org/t/openstack/build/7a8be5f2987745f390f6078e8ac87d82 : SUCCESS in 1h 14m 03s (non-voting) | |
| 17:28:16 | JayF | non-voting | |
| 17:28:18 | TheJulia | JayF: to the orignal ask, that is how the job operates now | |
| 17:28:41 | JayF | Ack, so as a second step, I can look into making that job also test sharding | |
| 17:28:57 | TheJulia | I think it just would if set, fwiw | |
| 17:29:08 | dansmith | I guess I'm not sure what configuration is more deserving of a multinode layout than the configuration that arranges for more-than-trivial nodes | |
| 17:29:39 | JayF | I honestly don't know what that job tests; it'll be on my plate to look into it | |
| 17:30:12 | JayF | if I can just flip sharding on in that job and use it to test, that'd be even simpler (although I still think there's value in doing the "check resources" type job, because it'll fail more cleanly if we break things) | |
| 17:31:03 | sean-k-mooney | there is some logic in devstack to run multiple fake computes on a single node | |
| 17:31:56 | sean-k-mooney | so we could (next cycle) explore doign that for ironic. but honestly i think all that is need to enable the comptue service on a secnd node and set the virt driver to ironci and set the other config options | |
| 17:31:59 | TheJulia | JayF: multiple ironic-conductors and nova-computes if I'm recalling correctly | |
| 17:32:15 | TheJulia | It has just been a while since I've done any work on it | |
| 17:32:30 | JayF | oooh, so is that the job I dind't know existed that does coverage for conductor_groups | |
| 17:32:33 | JayF | that would make a lot of sense, actually | |
| 17:33:57 | sean-k-mooney | thats one of the two jobs we have in our expermiental pipeline | |
| 17:33:59 | sean-k-mooney | https://github.com/openstack/nova/blob/master/.zuul.yaml#L981-L990 | |
| 17:34:15 | sean-k-mooney | we have ironic-tempest-bfv and ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode | |
| 17:35:04 | JayF | I think that is the whole mystery laid out there then | |
| 17:35:12 | JayF | looks like I have a template job, and need to just shardify it | |
| 17:35:56 | sean-k-mooney | ya so this is how hte second node is configured https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L692-L733 | |
| 17:36:36 | JayF | those OVS config bits where the parts I was worried about figuring out, honestly | |
| 17:37:00 | sean-k-mooney | well thats common with the libvirt jobs | |
| 17:37:09 | sean-k-mooney | for the most part anyway | |
| 17:37:34 | JayF | My Ironic knowledge is concentrated closer to the endpoint :) I'm working to spread it up the stack more and more over time but networking is a big blind spot | |
| 17:39:00 | sean-k-mooney | JayF: what i would suggest is focus on the testing you had planed before | |
| 17:39:32 | sean-k-mooney | if we can speratelly add teh config seeting to tag the ironic nodes in that other job with a shard key | |
| 17:39:57 | sean-k-mooney | and configure the relevnet nova-computes on ech host to look at only the shard key for the mv on the relevent host | |
| 17:40:05 | sean-k-mooney | that will cover us for the multi node job | |
| 17:40:24 | JayF | Well, except for the "you should | |
| 17:40:32 | JayF | do the other one first" that's what I had in mind | |
| 17:41:14 | sean-k-mooney | well if you want to do the multi node one first that fine | |
| 17:41:15 | JayF | I'll take your advice and still keep on the path I added to the spec -- even though you did an ace job of making the multinode seem easier :D | |
| 17:41:30 | sean-k-mooney | i was just hoping you could provide an exmaple of configuring it | |
| 17:41:39 | sean-k-mooney | then we could help prot it to the multi node one | |
| 17:41:56 | opendevreview | Fabian Wiesel proposed openstack/nova master: Do not close returned image-chunk iterator https://review.opendev.org/c/openstack/nova/+/909474 | |
| 17:43:12 | sean-k-mooney | JayF: you can set teh shard key in the nova.conf trivially like this https://github.com/openstack/nova/blob/master/.zuul.yaml#L459-L485 | |
| 17:43:57 | sean-k-mooney | im not sure how you go about tagging the relevent fake ironics with a shard key | |
| 17:44:46 | sean-k-mooney | if really needed to hack thing quickly i would use a local.sh file created by a pre playbook | |
| 17:45:10 | sean-k-mooney | like this https://review.opendev.org/c/openstack/nova/+/679656/13/playbooks/nfv/nfv.yaml | |
| 17:45:24 | sean-k-mooney | but i assume you have a nicer way to do that in teh ironic plugin | |
| 17:45:37 | sean-k-mooney | ideally via a local.conf parmater | |
| 17:46:40 | sean-k-mooney | JayF: if your not familar with /opt/stack/devstack/local.sh if it exist it run right before devstack finishes so if a job creates on eit will be run after devstack has otherwise run fully but before any tests are run | |
| 17:47:31 | sean-k-mooney | so its a way to hack in funcitonaltiy quickly via a zuul job that hsould eventually be in a devstack plugin | |
| 17:50:27 | JayF | honestly that's as nifty for local dev setup as it is for a CI trick, ty | |
| 17:50:58 | sean-k-mooney | yes so the reason ci jobs should not use it hevialy | |
| 17:51:05 | sean-k-mooney | is because its ment to be reserved for local dev | |
| 17:51:12 | sean-k-mooney | but they can use it | |
| 17:51:28 | sean-k-mooney | plugins just should not require it to have stuff to work | |
| 17:51:46 | sean-k-mooney | since its for the user not the "framework/plugins" to use | |
| 17:52:32 | sean-k-mooney | so form a ci point of view its perfectly valid to use | |
| 17:52:46 | sean-k-mooney | but if its complex it proably shoudl be a feature in the plugin/devstack instead | |
| 17:54:00 | JayF | I mean, yeah, it's kinda obvious it'd be hacky to land a CI job using that | |
| 17:56:58 | sean-k-mooney | JayF: i was epecting to see something in enroll_nodes https://github.com/openstack/ironic/blob/245505dc04da6e401e1810812809fef3f3757f04/devstack/lib/ironic#L2431 | |
| 17:57:09 | sean-k-mooney | to set the shard key on the node | |
| 17:57:12 | sean-k-mooney | did i miss that | |
| 17:57:49 | JayF | shard key is optional; you only need it when setting things up | |
| 17:58:07 | JayF | and it's unused upstream until the nova change lands | |
| 17:58:20 | JayF | so there's nothing to miss (yet) | |
| 17:58:29 | sean-k-mooney | ah ok so the devstack pluging change has not happened yet | |
| 17:58:47 | sean-k-mooney | i was epectign to see a new Settign added to the plugin | |
| 17:59:03 | sean-k-mooney | and hten have it append the shared key into the node options if its defined | |
| 17:59:17 | JayF | sean-k-mooney: https://opendev.org/openstack/ironic-tempest-plugin/commit/f2b9c74c9cb8459a979d9002aae3c1a41737c77a is the testing that exists r/n in tempest, no scenario tests yet | |
| 17:59:23 | JayF | this conversation is me kicking off that work for this week | |
| 17:59:54 | sean-k-mooney | right but that patch is in the tempest plugin not the devstack plugin | |
| 18:00:24 | JayF | yep; that's what I'm saying, nothing scenario is done, I have the old patch from last cycle which is basically useless now and never landed (but did what you're lookign for) | |
| 18:00:33 | sean-k-mooney | ah ok | |
| 18:01:29 | sean-k-mooney | well its not useless in that it allows you to deploy locally with ironic fake nodes via devstack | |
| 18:01:36 | sean-k-mooney | with shards | |
| 18:01:54 | JayF | Yeah, but it's useless in terms of testing coverage | |
| 18:02:07 | JayF | because it was passing even when the client wasn't even passing ?shard=blah to the Ironic PAI | |
| 18:02:11 | JayF | that's what I mean by useless :D | |
| 18:18:39 | sean-k-mooney | it depends on what you were trying to test. but i agree it alone was not suffienct | |
| 18:20:27 | sean-k-mooney | dansmith: melwitt nova-lvm passed with https://review.opendev.org/c/openstack/nova/+/909469 by the way https://zuul.opendev.org/t/openstack/build/5069b706178e4430ba48bdd52234b065 so do we want to just merge that for now | |
| 18:21:15 | sean-k-mooney | memory_tracker low_point: 358852 | |
| 18:21:51 | sean-k-mooney | SwapFree: 5804540 kB | |
| 18:22:06 | sean-k-mooney | so we are not ussing most of the addtional swap | |
| 18:22:37 | dansmith | I'm cool with it yeah :) | |
| 18:22:52 | sean-k-mooney | but that does not mean it was not needed and just not clolelcted i nthe memory tracker | |
| 18:23:11 | dansmith | um, parse error | |
| 18:23:26 | dansmith | but yeah, I'm in favor of 8G swap for any of these that are on the border | |
| 18:23:47 | sean-k-mooney | i mean the memory tracker log runs perodically adn the extra swap could have been needed breifly in between runs | |
| 18:26:32 | melwitt | sean-k-mooney: noice, +W | |
| 20:43:53 | opendevreview | Merged openstack/nova stable/2023.1: libvirt: stop enabling hyperv feature reenlightenment https://review.opendev.org/c/openstack/nova/+/909229 | |
| 21:11:31 | opendevreview | Merged openstack/nova master: bump nova-lvm to use 8G of swap from 4G https://review.opendev.org/c/openstack/nova/+/909469 | |
| #openstack-nova - 2024-02-20 | |||
| 06:26:12 | opendevreview | Merged openstack/nova stable/2023.1: libvirt: Stop unconditionally enabling evmcs https://review.opendev.org/c/openstack/nova/+/909228 | |
| 07:01:12 | opendevreview | Merged openstack/nova master: HyperV: Remove RDP console API https://review.opendev.org/c/openstack/nova/+/906809 | |
| 08:56:23 | opendevreview | Fabian Wiesel proposed openstack/nova master: Do not close returned image-chunk iterator https://review.opendev.org/c/openstack/nova/+/909474 | |
| 09:05:25 | opendevreview | Fabian Wiesel proposed openstack/nova master: Do not close returned image-chunk iterator https://review.opendev.org/c/openstack/nova/+/909474 | |
| 11:51:14 | opendevreview | Mohammed Naser proposed openstack/nova stable/zed: libvirt: Stop unconditionally enabling evmcs https://review.opendev.org/c/openstack/nova/+/909541 | |
| 11:52:03 | opendevreview | Mohammed Naser proposed openstack/nova stable/zed: libvirt: stop enabling hyperv feature reenlightenment https://review.opendev.org/c/openstack/nova/+/909098 | |
| 12:04:01 | opendevreview | Fabian Wiesel proposed openstack/nova master: Do not close returned image-chunk iterator https://review.opendev.org/c/openstack/nova/+/909474 | |
| 12:05:43 | opendevreview | Fabian Wiesel proposed openstack/nova master: Do not close returned image-chunk iterator https://review.opendev.org/c/openstack/nova/+/909474 | |
| 12:53:53 | frickler | if someone would want to look into nova testing failures when upgrading jsonschema, that would be great https://review.opendev.org/c/openstack/requirements/+/909452 | |
| 13:00:31 | sean-k-mooney | looks like there is only one failing test | |
| 13:02:24 | sean-k-mooney | this might be an issue in placment rather then nova | |