| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-11-02 | |||
| 13:04:16 | sean-k-mooney | https://github.com/openstack/python-openstackclient/blob/master/openstackclient/compute/v2/server.py#L1714-L1725 | |
| 13:04:36 | sean-k-mooney | at least i think so | |
| 13:05:29 | sean-k-mooney | oh no its https://github.com/openstack/python-openstackclient/blob/master/openstackclient/compute/v2/server.py#L1762-L1769 | |
| 13:06:22 | sean-k-mooney | greatgatsby: so i think what is missing is you likely dont have boot_index=0 set in the the --block-device argumetns? | |
| 13:06:54 | greatgatsby | if I do `openstack server create --block-device source_type=image,uuid=<uuid>,destination_type=volume,volume_size=10,delete_on_termination=true I still get an error about one of those other flags being required, OCS 5.8.0 | |
| 13:07:09 | greatgatsby | I tried with boot_index as well | |
| 13:08:16 | sean-k-mooney | can you past the output of the command somewhere so we can review | |
| 13:09:14 | sean-k-mooney | since that does not have boot_index that should raise but i want to see if i can find the exact error message in code | |
| 13:09:15 | greatgatsby | I'm on two different networks (openstack for work, home network for IRC) but I'll pastebin what I'm entering... couple seconds to type it out. Really appreciate the help. | |
| 13:09:51 | greatgatsby | if you just want the output, it's literally just the openstack server create help followed by: | |
| 13:10:14 | greatgatsby | openstack server create: error: one of the arguments --image --image-property --volume --snapshot is required | |
| 13:10:30 | greatgatsby | my command is: | |
| 13:10:55 | sean-k-mooney | ok let me see if i can find that as the code i linked to has a slight diffent message so its failign somewhere else | |
| 13:11:40 | greatgatsby | openstack --os-cloud foo server create --flavor medium --block-device source_type=image,uuid=<image-uuid>,destination_type=volume,volume_size=10,delete_on_termination=true,boot_index=0 test-vm | |
| 13:12:12 | sean-k-mooney | ya that looks valid to me | |
| 13:12:17 | greatgatsby | OCS 5.8.0 | |
| 13:14:01 | sean-k-mooney | https://github.com/openstack/python-openstackclient/commit/91277e7e51849d197554b633a579c92116a5afc4 | |
| 13:15:11 | sean-k-mooney | greatgatsby: so ^ is the fix frickler was refering too and its in 6.1.0+ | |
| 13:15:47 | greatgatsby | nice, thanks a lot. Hopefully we'll be upgrading off yoga soon and will also upgrade our recommended OCS version. | |
| 13:16:14 | sean-k-mooney | well in general i recommend pepoel to install osc in a python venv form pip | |
| 13:16:17 | greatgatsby | we try to go by the version matrix for all the python utils and dependencies | |
| 13:16:25 | sean-k-mooney | so that that they can use the latest indepent of the cloud | |
| 13:16:25 | greatgatsby | that's exactly what we do | |
| 13:16:49 | greatgatsby | we still use the versions according to the docs... maybe we should reconsider that? | |
| 13:17:12 | sean-k-mooney | ya so in general osc shoudl work with any release | |
| 13:17:22 | sean-k-mooney | so i generally suggest using the latest version | |
| 13:17:50 | sean-k-mooney | the cli should be stable and generally only have additive changes | |
| 13:18:09 | sean-k-mooney | but if you have a lot of internal docs using a fixed verion might make sense | |
| 13:18:45 | greatgatsby | https://releases.openstack.org/yoga/index.html#service-client-projects | |
| 13:19:07 | sean-k-mooney | that just documenting the version that was released in yoga | |
| 13:19:28 | sean-k-mooney | its not a recomendation to use that version to interact with a yoga cloud | |
| 13:19:54 | greatgatsby | oh, that's great to know, thanks, I'll pass that on! | |
| 13:20:23 | greatgatsby | again, really appreciate getting help in here, can be tough banging ones head against the desk for too long | |
| 13:20:25 | sean-k-mooney | the reason tha tis good to refence is for co installablity | |
| 13:20:32 | sean-k-mooney | if you stay to the version in that list | |
| 13:20:44 | sean-k-mooney | then we are commiting to keepign them co installable | |
| 13:20:56 | sean-k-mooney | if you deviate form that you might have compitbality issues | |
| 13:21:08 | sean-k-mooney | but i f you are using osc on your laptop in its own venv | |
| 13:21:14 | sean-k-mooney | you dont need to really consider that | |
| 13:21:33 | greatgatsby | sorry what do you mean by co installability? Just want to pass this info on and not get pushback | |
| 13:21:34 | sean-k-mooney | so we maintain that list mainly for distro packagers | |
| 13:21:57 | sean-k-mooney | the ablity to install all the release artifacts in the same python envionment | |
| 13:22:15 | greatgatsby | I see, ok, thanks again | |
| 13:22:40 | opendevreview | Merged openstack/nova master: Fix rebuild compute RPC API exception for rolling-upgrades https://review.opendev.org/c/openstack/nova/+/899235 | |
| 13:23:07 | sean-k-mooney | we test for co-installablity in our ci system and partly enforce that with upper-constraints.txt in the requirements repo for external deps | |
| 13:32:34 | opendevreview | sean mooney proposed openstack/nova master: Use centos-9-stream for functional-py39 job https://review.opendev.org/c/openstack/nova/+/899959 | |
| 13:32:34 | opendevreview | sean mooney proposed openstack/nova master: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/nova/+/899753 | |
| 13:34:13 | opendevreview | sean mooney proposed openstack/nova master: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/nova/+/899753 | |
| 13:36:41 | dansmith | bauzas: are you around to explain something to me about the RP patch | |
| 13:36:42 | dansmith | ? | |
| 13:36:49 | bauzas | dansmith: yup, I'm here now | |
| 13:36:58 | bauzas | thanks for having started to look at it, btw. | |
| 13:37:12 | bauzas | (me also needs to look at your spec) | |
| 13:37:20 | dansmith | yeah sorry I forgot you were off yesterday until after I pinged | |
| 13:37:21 | bauzas | so, shoot your questions | |
| 13:37:31 | bauzas | have you seen the bug report btw.? | |
| 13:37:43 | bauzas | I tried to summarize the problem in there | |
| 13:37:48 | dansmith | my questions are actually *about* the stuff in the bug :) | |
| 13:37:52 | bauzas | https://bugs.launchpad.net/nova/+bug/2041519 | |
| 13:37:58 | bauzas | okay | |
| 13:38:10 | bauzas | and why I did that, I guess ? | |
| 13:38:23 | dansmith | so I get the inventory before, then after one allocation, one RP goes to zero | |
| 13:38:37 | bauzas | more than one RP but yeah | |
| 13:38:48 | dansmith | no, only one according to your report | |
| 13:38:57 | bauzas | and this is not exacly *always* after | |
| 13:39:23 | bauzas | the fact is that we automatically create a mdev if none of them are not used yet | |
| 13:39:25 | dansmith | but what i don't get is why after the second allocation they *all* go to zero.. that would imply we're using inventory wrong | |
| 13:39:43 | bauzas | okay, gonna clarify | |
| 13:39:56 | bauzas | we are not counting mdevs for inventories | |
| 13:40:31 | bauzas | what we count as VGPU resources are *either unused mdevs *or* available_instances* | |
| 13:40:44 | dansmith | oh, hang on | |
| 13:40:46 | bauzas | which means something you can allocate to a guest | |
| 13:41:08 | dansmith | re-reading this with fresh eyes, I think I was confusing the very similar sysfs and placement dumps | |
| 13:41:25 | dansmith | so it's the mdev availability that goes off a cliff | |
| 13:42:29 | bauzas | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L8090-L8133 | |
| 13:42:38 | bauzas | that's where we calculate inventories | |
| 13:42:53 | dansmith | yep | |
| 13:43:08 | bauzas | available_instances is a value that the kernel driver gives | |
| 13:43:12 | bauzas | but, | |
| 13:43:13 | dansmith | right | |
| 13:43:24 | bauzas | once you create a mdev, the value decreases by 1 | |
| 13:43:33 | bauzas | that doesn't mean that the mdev will be used by a guest | |
| 13:43:53 | sean-k-mooney | bauzas: yep shich is why you are ment to be internally tracking which devices the mdev are created form | |
| 13:43:56 | bauzas | if you basically precreate all mdev resources, then available_instances is 0 for all the gpus | |
| 13:44:11 | sean-k-mooney | and using that to compute the inventoy total as (aviablie_instnace + mdevs form that device) | |
| 13:44:16 | dansmith | does nvidia-472 imply that more than 1/total share of the card is used? | |
| 13:44:28 | bauzas | but the inventory still tracks that you can bind N vGPUs | |
| 13:44:49 | bauzas | dansmith: that's the crux of the problem | |
| 13:45:01 | bauzas | the number of vGPUs can differ per type | |
| 13:45:04 | sean-k-mooney | bauzas: that problem is sovled by enforcing that the device adress | |
| 13:45:10 | dansmith | that feels like we're using placement incorrectly to have inventory of unequal resources | |
| 13:45:15 | bauzas | nvidia-472 can create up to two vGPUs | |
| 13:45:32 | bauzas | so there are three possibilities | |
| 13:45:41 | bauzas | either available_instances = 2 | |
| 13:45:56 | bauzas | or, available_instances = 1 and one mdev exists that'not used by a guest | |
| 13:46:06 | bauzas | or, there are two mdevs available, which are both unused | |
| 13:46:17 | bauzas | with SR-IOV, things change | |
| 13:46:19 | sean-k-mooney | bauzas: the decreasing inventory behavoir is a nvidia specirci issue with vfs form teh same card | |
| 13:46:54 | bauzas | now, with SR-IOV, instead of having one single pci device that have available_instances = 2 (and then only one RP), | |
| 13:47:11 | bauzas | then we have *two* VFs, each of them having available_instances = 1 | |