| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-05-16 | |||
| 13:46:22 | bauzas | dansmith: glance-api is absolutely blind on the req-id, so how could I see any logs from glance apart from devstack@g-api | |
| 13:46:55 | dansmith | is it/ | |
| 13:47:08 | dansmith | but gateway exception usually means apache not glance | |
| 13:51:16 | dansmith | unless glance is 500'ing or something | |
| 13:53:51 | bauzas | okay, I can look at httpd's logs | |
| 13:58:20 | bauzas | dansmith: can you help me for a sec ? I want to backup my instance but then it's queued on the glance side for a while and eventually I no longer see the image | |
| 13:58:40 | dansmith | about to start a meeting, but can try in parallel | |
| 13:58:44 | dansmith | using backup or snapshot? | |
| 13:58:45 | bauzas | nothing is told on the glance side, only the server event record in nova gives me the exception | |
| 13:58:49 | bauzas | server backup | |
| 13:59:15 | dansmith | because you want to test our backup stuff, or because you want a snapshot of your image? | |
| 13:59:44 | dansmith | I don't know what the plumbing through our server backup stuff looks like in detail, which is why I'm asking | |
| 14:00:07 | dansmith | so you see an image created on the glance side (POST /images) .. do you see PUT/POST /images/$uuid/data ? | |
| 14:02:26 | bauzas | dansmith: yep | |
| 14:02:31 | bauzas | POST /images | |
| 14:02:50 | bauzas | I have an instance, I just want a new image from it | |
| 14:03:23 | dansmith | okay, I'd be using snapshot not backup.. backup is a bit of a different thing.. still results in an image, but snapshot is more straightfortward | |
| 14:04:04 | sean-k-mooney | if using osc i think you shoudl b eusing "openstack server create image" | |
| 14:04:08 | sean-k-mooney | to take the snapshot | |
| 14:04:25 | sean-k-mooney | but ya backup is for snapshot roation really | |
| 14:04:32 | dansmith | sean-k-mooney: exactly | |
| 14:04:50 | sean-k-mooney | also backup does not work for BFV as an fyi | |
| 14:05:07 | sean-k-mooney | not sure why but it does not | |
| 14:05:49 | sean-k-mooney | we call it out in the api ref in anycase https://docs.openstack.org/api-ref/compute/#create-server-back-up-createbackup-action | |
| 14:05:56 | sean-k-mooney | so its a know limitaiton not a bug | |
| 14:06:36 | bauzas | okay, I'll try snapshot then | |
| 14:20:04 | bauzas | doh | |
| 14:20:06 | bauzas | eq-67d7b42c-c6a6-46cc-a86b-7eb66afabb79 demo admin] hit limit for project: [Resource image_size_total is over limit of 1000 due to current usage 5257 and delta 0] {{(pid=2376125) enforce_limits /usr/local/lib/python3.9/site-packages/oslo> | |
| 14:25:00 | sean-k-mooney | is that a 1TB limit ? | |
| 14:25:12 | sean-k-mooney | with a 5.2TB image | |
| 14:25:36 | dansmith | no I think it's KiB | |
| 14:25:38 | sean-k-mooney | image size is in GB right | |
| 14:25:52 | dansmith | or maybe MiB | |
| 14:25:59 | dansmith | I don't remember :P | |
| 14:26:14 | sean-k-mooney | we have local_MB for our storage class | |
| 14:26:19 | opendevreview | Merged openstack/nova master: Fix notification object hashes for python 3.12 https://review.opendev.org/c/openstack/nova/+/919763 | |
| 14:26:26 | sean-k-mooney | but the flavor and glance i tought was in GBs | |
| 14:26:28 | opendevreview | Merged openstack/nova master: Fix hacking test with syntax error https://review.opendev.org/c/openstack/nova/+/919764 | |
| 14:26:38 | dansmith | https://docs.openstack.org/glance/latest/admin/quotas.html | |
| 14:27:05 | dansmith | MiB | |
| 14:27:08 | sean-k-mooney | ok so the quota is also in MiB | |
| 14:28:10 | dansmith | I wrote that feature, but they already had some limits in MiB so that's why it uses the same units they chose | |
| 14:28:21 | dansmith | most of the image stuff is MiB, volumes in GiB | |
| 14:28:28 | bauzas | yep, saw it | |
| 14:28:33 | bauzas | 1000 for me | |
| 14:28:43 | bauzas | so, only images for less than 1GB | |
| 14:28:51 | bauzas | hence why | |
| 14:29:34 | sean-k-mooney | dansmith: ok the galnce api is kind of incositent. the size field on an image is in bytes, the min_disk is in GB and virtual_size is undefined but presumable also bytes | |
| 14:30:38 | dansmith | bytes yeah | |
| 14:33:17 | dansmith | sean-k-mooney: what min_disk are you talking about? | |
| 14:34:02 | sean-k-mooney | on the image | |
| 14:34:20 | sean-k-mooney | you can spcify the min disk for an instance or voluem | |
| 14:34:29 | sean-k-mooney | that is defiend in the api ref as in GB | |
| 14:34:52 | sean-k-mooney | which is why i was confused the quota woudl be in MiB when the api either uese bytes or GB elsehwere | |
| 14:34:56 | sean-k-mooney | anyway not imporant | |
| 14:36:16 | dansmith | isn't that a nova thing? | |
| 14:36:25 | dansmith | or nova/cinder anyway | |
| 14:38:20 | sean-k-mooney | https://docs.openstack.org/api-ref/image/v2/index.html#create-image | |
| 14:38:41 | sean-k-mooney | its a filed in the respocne | |
| 14:39:02 | sean-k-mooney | oh and the request | |
| 14:39:06 | dansmith | yeah those aren't core glance properties, I think those were added for nova/cinder and probably inherited the units | |
| 14:40:16 | sean-k-mooney | they have been there since i worked on openstack and have never been called out as any diffent then name | |
| 14:40:37 | dansmith | yeah I'm just sayin' they're not glance-specific :) | |
| 14:47:28 | zigo | Hi there! One quick question: how can I decrypt the output of "nova get-password <server>" knowing that I don't have access to my private key (it's stored in my yubikey...) ? | |
| 14:48:38 | sean-k-mooney | dansmith: but it considerd a base property by glance "https://github.com/openstack/glance/blob/705b145ab83defb3cb2f8d3243e1ce900fe337b9/glance/api/v2/images.py#L1121" proably becasue of how old and wid spread it is | |
| 14:49:02 | sean-k-mooney | zigo: its encypted with the serverers keypair | |
| 14:49:19 | zigo | sean-k-mooney: Yeah, I know, but with what command do I decrypt it? | |
| 14:49:21 | sean-k-mooney | zigo: so you can use your ssh private key | |
| 14:49:32 | zigo | It's in my yubikey ... | |
| 14:49:38 | zigo | I don't have it as a file. | |
| 14:49:53 | dansmith | sean-k-mooney: in how it's stored, because of age yeah | |
| 14:51:47 | sean-k-mooney | zigo: i guess you cant unless you can use ssh to do the decyption somehow | |
| 14:52:56 | zigo | sean-k-mooney: novaclient uses in novaclient/crypto.py: cmd = ['openssl', 'rsautl', '-decrypt', '-inkey', private_key] | |
| 14:52:56 | zigo | Though rsautl is deprecated ... :/ | |
| 14:53:23 | sean-k-mooney | yep and im not sure we ever supproted it in osc | |
| 14:53:32 | sean-k-mooney | because we dont want to main that | |
| 14:53:48 | bauzas | huzzah it worked | |
| 14:53:59 | bauzas | upping the registered limit, I mean | |
| 14:54:35 | bauzas | size | 25571885056 | |
| 14:54:38 | bauzas | voila the reason | |
| 14:54:54 | bauzas | 25GB | |
| 15:26:32 | clarkb | zigo: https://docs.yubico.com/hardware/yubihsm-2/hsm-2-user-guide/hsm2-openssl-libp11.html this is for their hsm product but maybe similar for regular yubikeys | |
| 15:27:22 | clarkb | alternatively ssh using the key then change the password and don't bother involving nova at all? | |
| 15:36:42 | zigo | clarkb: There's no such thing as sshd in our windows instances, just cloud-base-init. I'm trying to figure out what format "nova get-password" is giving out, and if I can unencrypt it with my yubikey... | |
| 15:37:44 | clarkb | zigo: got it (I think you can run an sshd on windows now fwiw, but that makes sense). Maybe those yubikey hsm docs are the clue you need for using the hardware to decrypt | |
| 15:38:05 | zigo | Thanks. I'm reading it. | |
| 15:38:06 | dansmith | yep sshd on windows is trivial now | |
| 15:38:10 | dansmith | built-in almost | |
| 15:42:40 | opendevreview | Merged openstack/nova master: Make python 3.12 unit and functional voting https://review.opendev.org/c/openstack/nova/+/919767 | |
| 16:54:39 | bauzas | melwitt: I want to demonstrate VGPU quota usage on my demo next week | |
| 16:55:02 | bauzas | when setting unifiedlimits driver in nova.conf, which services are using it ? n-sch and n-api right? | |
| 16:55:29 | bauzas | none from the conductor or the compute ? | |
| 16:56:01 | melwitt | bauzas: I think only n-api. I'll double check | |
| 16:56:24 | bauzas | I quickly looked at the code, I only found the scheduler and the api were importing nova.quota | |
| 16:58:44 | melwitt | bauzas: there's a "recheck" in nova-conductor that enforces "strict quota" like races | |
| 16:59:06 | bauzas | then I need to also update nova_cell1.conf | |
| 17:00:41 | bauzas | Unable to initialize OpenStackSDK session: An auth plugin is required to determine endpoint URL: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin | |
| 17:00:48 | bauzas | humpf, I probably need to set something else | |
| 17:01:29 | bauzas | melwitt: do you know what I missed ? ^ | |
| 17:01:56 | bauzas | (that's the API exception I get when wanting to create a new instance) | |