| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-01-16 | |||
| 17:34:26 | JayF | sean-k-mooney: /me points an ironic change at 890693 for science | |
| 17:34:27 | sean-k-mooney | its off by default locally but on by default in ci currently | |
| 17:34:30 | dansmith | sean-k-mooney: if you disable it by default I'll +2 now, but if not, I want a read from gmann at least | |
| 17:35:09 | sean-k-mooney | ack ill do that i need to deploy a devstack vm for my healthcheck stuff anyway so ill upated it then | |
| 17:35:24 | clarkb | sean-k-mooney: the memory usage varies by the service talking to the privsep instance (which implies to me that workload probably impacts buffer sizes). In total I think it was about half a gig for privsep | |
| 17:35:58 | sean-k-mooney | clarkb: i think tis more related to the size of our oslo.config global + the code that is imported | |
| 17:36:54 | clarkb | sean-k-mooney: oh ya I guess everything is a regex? Those shouldn't be terribly large but if you have enough of them... might also be worth checking if libre2 is more efficient maybe | |
| 17:37:09 | sean-k-mooney | we have some chaced module level varables for example that might be related to this | |
| 17:37:42 | clarkb | sean-k-mooney: for zswap I notice you set swappiness to 100. I think devstack sets swappiness elsewhere you might need to reconcile the two to ensure they don't fight each other | |
| 17:38:25 | clarkb | sean-k-mooney: https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/roles/configure-swap/tasks/main.yaml#L46-L61 this is the code that does it and I think devstack jobs call that first so you would override which I guess is fine | |
| 17:38:41 | sean-k-mooney | clarkb: i was thinking fo https://github.com/openstack/nova/blob/a72f7eaac78927892b937d451cbacb24a83c05ac/nova/api/validation/parameter_types.py#L106 | |
| 17:38:54 | sean-k-mooney | although privsep shoudl not be using the api stuff | |
| 17:39:00 | sean-k-mooney | so hopefully that wont matter | |
| 17:39:19 | clarkb | I don't have any other comments on the zswap change. Seems like it may be worth experimenting with | |
| 17:39:32 | dansmith | yep | |
| 17:40:15 | sean-k-mooney | i know people have seen a lot of success with desktop responsiveness on rassbery pi with it | |
| 17:40:23 | sean-k-mooney | so i was hoping it woudl help us in a simialr way | |
| 17:40:38 | sean-k-mooney | ok ill go remove the default change in teh patch | |
| 17:40:47 | sean-k-mooney | and we can then enabel it on a per job basises in nova | |
| 17:44:18 | sean-k-mooney | JayF: did you see https://review.opendev.org/c/openstack/nova/+/905406 by the way | |
| 17:44:41 | sean-k-mooney | JayF: that extract the common instnace metadata code up to the driver level and refactors libvirt to use it | |
| 17:44:56 | JayF | https://review.opendev.org/c/openstack/nova/+/900831 I really want this to land before I push up more patches | |
| 17:45:09 | JayF | I've been burned in the past by stacking a bunch of patches and being in rebase hell | |
| 17:45:21 | sean-k-mooney | ah yes well want that to land to make backport simpler | |
| 17:45:23 | JayF | So I have that in my back pocket as a "next thing" but waiting for 900831 to land first | |
| 17:48:43 | clarkb | JayF: this may or may not be useful but the way I try to deal with those is to "squash back" I make all of my edits on the tip of my dev branch and then commit will be something like "squash fix for commit foo" then I git rebase -i HEAD~X where X is the number of commits I need to go back to and squash the new commit back into the old one. Then git review the whole thing. I | |
| 17:48:44 | clarkb | find it simplifies things because I can focus on the necessary fixes and then focus on the accounting and usually I only need a single rebase to update a whole stack with any number of fixes | |
| 17:49:12 | JayF | clarkb: it's more that I have enough things on my list that I can action and will move more quickly so I just prioritize those over things which will languish for longer | |
| 17:49:34 | bauzas | git reflog FTW | |
| 17:49:40 | JayF | I can't remember the last time I was just like "there's nothing to do today" :D | |
| 17:55:45 | jrosser | clarkb: a colleague of mine demoed this recently https://github.com/tummychow/git-absorb | |
| 17:56:02 | jrosser | which is some kind of magic | |
| 17:56:44 | clarkb | that looks similar to what I do with the pairing of changes to squash done automatically based on file heuristics | |
| 17:57:13 | clarkb | There is also git restack which corvus wrote so that you don't have to manually determine the value of X when rebasing | |
| 19:05:23 | sean-k-mooney | johnthetubaguy: are you able to be the second core to review https://review.opendev.org/c/openstack/nova/+/900831 its JayF fix for https://launchpad.net/bugs/2043036 | |
| 19:05:39 | sean-k-mooney | otherwizse melwitt dansmith ^ perhasp ye can review | |
| 19:33:09 | opendevreview | Merged openstack/osc-placement master: Bump hacking https://review.opendev.org/c/openstack/osc-placement/+/905715 | |
| 20:19:36 | opendevreview | Sylvain Bauza proposed openstack/nova master: Reserve mdevs to return to the source https://review.opendev.org/c/openstack/nova/+/904209 | |
| 20:19:36 | opendevreview | Sylvain Bauza proposed openstack/nova master: Check if destination can support the src mdev types https://review.opendev.org/c/openstack/nova/+/904177 | |
| 20:19:37 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP(docs): Modify the mdevs in the migrate XML https://review.opendev.org/c/openstack/nova/+/904258 | |
| 20:38:01 | opendevreview | sean mooney proposed openstack/nova master: enable zswap in nova ci jobs. https://review.opendev.org/c/openstack/nova/+/905791 | |
| 20:44:41 | opendevreview | sean mooney proposed openstack/nova master: enable zswap in nova ci jobs. https://review.opendev.org/c/openstack/nova/+/905791 | |
| 20:51:58 | opendevreview | Merged openstack/python-novaclient master: Bump hacking https://review.opendev.org/c/openstack/python-novaclient/+/905707 | |
| 21:06:20 | opendevreview | Merged openstack/placement master: Bump hacking https://review.opendev.org/c/openstack/placement/+/905706 | |
| #openstack-nova - 2024-01-17 | |||
| 02:00:49 | opendevreview | sean mooney proposed openstack/nova master: enable zswap in nova ci jobs. https://review.opendev.org/c/openstack/nova/+/905791 | |
| 02:56:31 | opendevreview | melanie witt proposed openstack/nova master: WIP libvirt: make |
|
| 02:56:31 | opendevreview | melanie witt proposed openstack/nova master: WIP Suppport migration with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/905512 | |
| 02:56:32 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 03:19:58 | songwenping | hi team, the latest nova code raise exception for 'nova list' with the error info TypeError: __init__() missing 2 required positional arguments: 'params' and 'orig', what's wrong? | |
| 04:48:56 | opendevreview | Amit Uniyal proposed openstack/nova master: Updates glance fixture for create image https://review.opendev.org/c/openstack/nova/+/905684 | |
| 04:48:57 | opendevreview | Amit Uniyal proposed openstack/nova master: Fixes bug 2048184 https://review.opendev.org/c/openstack/nova/+/904817 | |
| 05:20:27 | opendevreview | Amit Uniyal proposed openstack/nova master: Fixes bug 2048184 https://review.opendev.org/c/openstack/nova/+/904817 | |
| 06:49:17 | opendevreview | Amit Uniyal proposed openstack/nova master: enforce remote console shutdown https://review.opendev.org/c/openstack/nova/+/901824 | |
| 10:27:34 | bauzas | humpf, upgraded to F39 and now tox and python3.12 are creating me problems :( | |
| 10:27:59 | bauzas | tox==4.12.1 | |
| 10:30:11 | bauzas | good news is that I can still run functional tests if I target by the python version (like -e functional-py39) | |
| 10:42:04 | bauzas | ok, behavioral change with tox, no worriezsq | |
| 11:15:06 | kashyap | Good to know; /me is still to ugprade to F39. Was planning to do this week | |
| 12:53:33 | opendevreview | Fabian Wiesel proposed openstack/nova master: Add more password generation options https://review.opendev.org/c/openstack/nova/+/865669 | |
| 14:47:41 | opendevreview | Sylvain Bauza proposed openstack/nova master: Check if destination can support the src mdev types https://review.opendev.org/c/openstack/nova/+/904177 | |
| 14:47:41 | opendevreview | Sylvain Bauza proposed openstack/nova master: check both source and dest compute libvirt versions for mdev lv https://review.opendev.org/c/openstack/nova/+/904176 | |
| 14:47:42 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP(docs): Modify the mdevs in the migrate XML https://review.opendev.org/c/openstack/nova/+/904258 | |
| 14:47:42 | opendevreview | Sylvain Bauza proposed openstack/nova master: Reserve mdevs to return to the source https://review.opendev.org/c/openstack/nova/+/904209 | |
| 14:55:18 | opendevreview | Sylvain Bauza proposed openstack/nova master: check both source and dest compute libvirt versions for mdev lv https://review.opendev.org/c/openstack/nova/+/904176 | |
| 14:55:18 | opendevreview | Sylvain Bauza proposed openstack/nova master: Augment the LibvirtLiveMigrateData object https://review.opendev.org/c/openstack/nova/+/904175 | |
| 14:55:19 | opendevreview | Sylvain Bauza proposed openstack/nova master: Reserve mdevs to return to the source https://review.opendev.org/c/openstack/nova/+/904209 | |
| 14:55:19 | opendevreview | Sylvain Bauza proposed openstack/nova master: Check if destination can support the src mdev types https://review.opendev.org/c/openstack/nova/+/904177 | |
| 14:55:20 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP(docs): Modify the mdevs in the migrate XML https://review.opendev.org/c/openstack/nova/+/904258 | |
| 15:17:15 | opendevreview | Hervé Beraud proposed openstack/nova master: bump eventlet to latest version that support python 3.12 https://review.opendev.org/c/openstack/nova/+/905928 | |
| 15:17:15 | opendevreview | Hervé Beraud proposed openstack/nova master: bump eventlet to latest version that support python 3.12 https://review.opendev.org/c/openstack/nova/+/905928 | |
| 17:35:15 | bauzas | argh, I'm hitting py312 issues on my laptop and I can't generate a relnote as we don't support venv target with py versions | |
| 18:09:29 | opendevreview | Sylvain Bauza proposed openstack/nova master: Modify the mdevs in the migrate XML https://review.opendev.org/c/openstack/nova/+/904258 | |
| 18:09:29 | opendevreview | Sylvain Bauza proposed openstack/nova master: Reserve mdevs to return to the source https://review.opendev.org/c/openstack/nova/+/904209 | |
| 18:13:12 | _colby | Hey Guys. We upgraded to oenstack yoga recently and now all our vGPU stuff does not work any more. We had to patch Wallaby, but Xena worked without issues. Now on Yoga we can spin up a single vGPU on a host (has 2 x A40 gpu (SR-IOV with Virtual Functions). But no more than 1 vgpu instance. We are specifying enabled_vgpu_types and device_Addresses for each type. Are we doing this incorrectly now with yoga? | |
| 18:14:45 | bauzas | _colby: I need to leave litterally now, but what's the stack when you ask for more | |
| 18:14:47 | bauzas | ? | |
| 18:15:24 | bauzas | I guess you know all the quirks with SRIOV GPUs | |
| 18:15:24 | _colby | bauzas: it says not valid host | |
| 18:15:47 | bauzas | you may have residues that lead to wrong placement decisions | |
| 18:15:48 | _colby | yes just wondering what changed between xena and yoga. Things worked fine before. | |
| 18:16:05 | bauzas | first thing is to doublecheck your inventories and allocations | |
| 18:16:21 | bauzas | and compare them with the existing mdevs and instances | |
| 18:17:27 | bauzas | particularly if you rebooted the host, you may have mediated devices using the wrong GPUs | |
| 18:17:30 | bauzas | or VFs (rather) | |
| 18:20:10 | _colby | yea we have a script that runs and recreates all the VFs on boot. They are the same. Should we still be specifying the device_addresses for the types? | |
| 18:20:25 | _colby | mdevctl types shows all the correct addresses and types we are using | |
| 18:50:51 | _colby | bauzas: I got some more info. My coworker had tested before. I did some tests of my own and Im seeing a different error. One host wont spin up a vgpu at all. Im seeing: Insufficient compute resources: mdev-capable resource is not available. The VFs are all there and correct, the mdectl types lists all the correct addresses. How do I check the placement inventory to verify it matches correctly? | |
| 18:55:25 | bauzas | _colby: sorry, I'm on and off by now (7:55pm here) | |
| 18:55:51 | bauzas | but you can use openstack resource provider list and then openstack resource provider inventory list for knowing | |
| 18:56:26 | bauzas | _colby: see https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#checking-allocations-and-inventories-for-virtual-gpus | |
| 18:56:52 | bauzas | (instead of seeing the GPUs, you would see the VFs) | |
| 18:57:06 | _colby | yea that looks correct too unfortunatly | |
| 18:57:26 | bauzas | _colby: we also have a bug report for SRIOV GPUs, see https://bugs.launchpad.net/nova/+bug/2041519 | |
| 18:57:34 | bauzas | I have a bugfix series for it | |
| 18:58:48 | bauzas | _colby: with SRIOV VFs, you should make sure that all your mdevs are created | |
| 18:59:00 | bauzas | and then you should verify the sysfs | |
| 18:59:19 | _colby | ok Ill read through this bug report. Thanks | |
| 18:59:20 | bauzas | calculate the number of mdevs, look at which VFs they use | |
| 18:59:40 | bauzas | then, look at the available_instances value for the type | |
| 18:59:45 | bauzas | for each of the VFs | |