| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-03-19 | |||
| 16:27:04 | elodilles | for this time | |
| 16:27:11 | elodilles | #info stable gates seem to be OK | |
| 16:27:45 | bauzas | cool | |
| 16:27:56 | elodilles | (as we discussed, there are jobs that fail intermittently :/) | |
| 16:28:05 | elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | |
| 16:28:33 | elodilles | that's it from me about stable branches | |
| 16:29:38 | bauzas | cool thanks | |
| 16:29:45 | bauzas | #topic vmwareapi 3rd-party CI efforts Highlights | |
| 16:29:47 | bauzas | fwiesel: ? | |
| 16:29:57 | fwiesel | #info CI comments on openstack/nova CR confirmed to be working. | |
| 16:30:06 | fwiesel | Best maybe seen here: | |
| 16:30:08 | fwiesel | #link https://review.opendev.org/dashboard/36571 | |
| 16:30:32 | fwiesel | #info Fixed FIPs instability. Tests quite reproducible. Only one error left with all proposed patches applied. | |
| 16:30:54 | fwiesel | The issue and fix were in retrospect easy... we have now a working pipeline. | |
| 16:31:01 | bauzas | \o/ | |
| 16:31:28 | fwiesel | Just with the new branches came in some instability, as it expects all service to actually have the branch. | |
| 16:32:00 | fwiesel | I.e. it fails now to check out stable/2024.1 for requirements, but we have it in nova. Not sure, if I should do something about that one. | |
| 16:32:14 | fwiesel | Missed that one: | |
| 16:32:15 | fwiesel | #link http://openstack-ci-logs.global.cloud.sap/sapcc-nova--nmllh/tempest.html | |
| 16:32:29 | fwiesel | The tempest test with the last open problem. | |
| 16:32:30 | elodilles | now ~everything should have branched (well, requirements, devstack and grenade will be created around end of week) | |
| 16:32:38 | bauzas | are you sure you use the requirements from the stable releases ? | |
| 16:32:55 | bauzas | fwiesel: awesome about where you are | |
| 16:33:32 | fwiesel | Missing now are hopefully only cleaning up the rough edges... A nicer UI with linkable lines we still are missing. | |
| 16:34:45 | bauzas | okay, anything else ? | |
| 16:34:52 | fwiesel | No, not from my side. | |
| 16:35:00 | fwiesel | Questions? | |
| 16:35:41 | fwiesel | One remark: A test run with fixes in place takes roughly 20minutes in total. | |
| 16:35:42 | bauzas | nope | |
| 16:35:51 | bauzas | wow | |
| 16:36:27 | fwiesel | bauzas: Back to you then | |
| 16:36:43 | bauzas | cool, that will be short | |
| 16:36:51 | bauzas | #topic Open discussion | |
| 16:36:55 | bauzas | anyuthing anyone ? | |
| 16:37:04 | dansmith | a tempest run takes 20 minutes? that would be shockingly fasty | |
| 16:37:52 | fwiesel | Well, it is a hefty machine, and I run it as parallel as I can. | |
| 16:38:19 | bauzas | ok guys, I think I can close the meeting | |
| 16:38:21 | bauzas | thanks all | |
| 16:38:23 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2024/nova.2024-03-19-16.00.log.html | |
| 16:38:23 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2024/nova.2024-03-19-16.00.txt | |
| 16:38:23 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2024/nova.2024-03-19-16.00.html | |
| 16:38:23 | opendevmeet | Meeting ended Tue Mar 19 16:38:23 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:38:23 | bauzas | #endmeeting | |
| 16:38:28 | fwiesel | Thank you too | |
| 16:38:37 | sean-k-mooney | well i guess it depend on how many tests you run. devstack can stack in about 8 mins if its not io bound | |
| 16:39:03 | elodilles | thanks o/ | |
| 16:39:40 | fwiesel | The tempest run takes 10 minutes, by the way :) | |
| 16:40:19 | sean-k-mooney | yep but your also unly runnign 525 ish tests | |
| 16:40:31 | sean-k-mooney | at least based on | |
| 16:40:33 | sean-k-mooney | http://openstack-ci-logs.global.cloud.sap/openstack-nova-909474-f54wv/tempest.html | |
| 16:40:33 | fwiesel | Sure, are there tests I am missing? | |
| 16:40:57 | sean-k-mooney | fwiesel: well you only runing the compute api tests | |
| 16:41:16 | sean-k-mooney | your not runng the senario tests ro any fo teh test fo other services like cinder/nuetorn extra | |
| 16:41:25 | sean-k-mooney | we dont need you to run all of those | |
| 16:41:40 | sean-k-mooney | but i was just providing context for others | |
| 16:41:47 | sean-k-mooney | i.e. its not tempest-full | |
| 16:42:45 | fwiesel | No, indeed it isn't. | |
| 16:43:26 | fwiesel | I am not even running all compute.api-tests, only those that are implemented for the vmware api driver. | |
| 16:43:51 | sean-k-mooney | ack that kind of makes sense | |
| 16:44:11 | sean-k-mooney | for the hardware based cis they took a simialr approch | |
| 16:44:38 | sean-k-mooney | i.e. the old intel ci just ran the cpu pinning/sriov/numa related subset | |
| 16:46:22 | fwiesel | Possibly it would make sense to check what code-coverage the tempest test generates on the driver implementation, and adjust the tests accordingly. | |
| 17:55:03 | sean-k-mooney | dansmith: correct me if im wrong but we do not have the ablity to ask placement for a host that does not have a resouce type right like a forbiden trait, we would have to do that in a scheduler filter | |
| 17:56:42 | sean-k-mooney | related to our converation about non numa instances | |
| 17:56:49 | dansmith | sean-k-mooney: I really don't know currently, but I'm pretty sure that was one of the early limitations.. we could ask for inclusion but not exclusion | |
| 17:57:27 | sean-k-mooney | ack i think its still that way. i belive we coudl have a new filter that does the following check | |
| 17:58:26 | sean-k-mooney | for a given instance look at the guest falvor/image to see if it is requesting numa aware memory i.e. hw:mem_page_size | |
| 17:58:54 | sean-k-mooney | if it is not check the allcoation candates for the host to see if it has either the file backed memory trait or an inventory of PCPU | |
| 17:59:02 | sean-k-mooney | if it has either reject the host | |
| 18:00:05 | sean-k-mooney | that would allow operators to opt in/out by not using the filter | |
| 18:00:39 | sean-k-mooney | we can add it to the default filters to enforce the correct behavior. | |
| 18:00:54 | dansmith | well, that feels more formal of a feature opt-out than a workaround named I_like_to_oom_my_instances=True | |
| 18:01:12 | dansmith | but it would mean an abort in scheduling instead of compute, which would be better | |
| 18:02:07 | sean-k-mooney | well we cloud perhaps not make it a filter but a config option that woudl work like a filter | |
| 18:02:19 | sean-k-mooney | i.e. have a workaround to remove this mandatory filter | |
| 18:02:21 | dansmith | sean-k-mooney: btw, I am still interested in the answer to my question here: https://review.opendev.org/c/openstack/nova/+/877773/9/nova/virt/libvirt/migration.py | |
| 18:02:44 | dansmith | because it seems like it should always fail and I don't know why it's not | |
| 18:03:41 | sean-k-mooney | reading that now ill need to check but if i rememebr correctly we pop the value form the dict before it ever get there | |
| 18:04:00 | dansmith | that's a bad solution IMHO if so because it would break computes that don't have this code yet, | |
| 18:04:05 | dansmith | but I also couldn't find any such pop | |
| 18:04:14 | sean-k-mooney | your asertion is calling int(a string containing a list of ints) | |
| 18:04:35 | dansmith | sean-k-mooney: no it's calling int() on the fake key | |
| 18:04:36 | sean-k-mooney | should always explode so either we are not hitting that path in the test or there is somethign else going on here | |
| 18:04:58 | dansmith | that dict is supposed to be keys of '1', '2', etc and so it int()'s the keys, | |
| 18:05:08 | dansmith | but now there's a key of "dst_cpu_whatever" in the dict | |
| 18:05:20 | dansmith | and I don't know how it's not failing to int() that when it iterates the items | |
| 18:05:35 | dansmith | and I'm wondering if it's because we don't have test coverage that was hitting that code path with the three conditions being true | |
| 18:07:58 | sean-k-mooney | oh right "guest_id, host_ids in info.cpu_pins.items()" so guest_id would be the key | |
| 18:08:04 | dansmith | right | |
| 18:08:09 | dansmith | so I don't know how that's not breaking | |
| 19:23:13 | sean-k-mooney | artom: https://review.opendev.org/c/openstack/nova/+/631053 and https://bugs.launchpad.net/nova/+bug/1811886 are related to our converstaion about the issue relating to not requesting a page size. | |
| 19:23:52 | sean-k-mooney | its not the example i was giving when we talks but its a similar equally wrong case. | |
| 19:42:02 | opendevreview | Merged openstack/nova master: Remove nova.wsgi module https://review.opendev.org/c/openstack/nova/+/902686 | |
| 19:42:13 | opendevreview | Merged openstack/nova master: Add new nova.wsgi module https://review.opendev.org/c/openstack/nova/+/902687 | |
| 19:48:25 | sean-k-mooney | dansmith: i looked brifly but i dont know why that is not brekaing things but my best guess is we are somehow mocking this and we dont have cpu_shared_set defiend in any of the jobs | |
| 19:49:05 | sean-k-mooney | there is no reason why we cant have cpu share set in one of our live migration jobs so i might try and enable that | |
| 19:50:27 | sean-k-mooney | i.e. 0-5 on one node and 2-7 on the other | |
| 19:51:00 | sean-k-mooney | that pretty trivial to do so ill add a patch on top or renes serise to do that | |
| 19:52:13 | sean-k-mooney | in theory rene's code should be runnign even if that is not devied but we should be able to test this in the gate anyway | |
| 19:59:42 | opendevreview | Stephen Finucane proposed openstack/nova master: Make overcommit check for pinned instance pagesize aware https://review.opendev.org/c/openstack/nova/+/631053 | |