| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-17 | |||
| 10:07:24 | gibi | I have no clue who has the time to fix it | |
| 10:07:33 | gibi | but still the feedback is appreciated | |
| 10:08:09 | stephenfin | It could be something for future students to work on. I'll mention it to diablo_rojo | |
| 10:09:27 | sean-k-mooney[m] | im really not sure why we have a volume attaments api in nova iteslf. to me the atacment is not something we shoudl be exposing to users we should just have the volume resource associated with the server | |
| 10:10:04 | sean-k-mooney[m] | we dont expose port bindign in our api. they exist in neutorn but not in nova | |
| 10:10:19 | stephenfin | sean-k-mooney[m]: how would propose modifying e.g. the delete on termination behaviour for an attached volume in that scenario | |
| 10:10:24 | stephenfin | or doing volume swaps | |
| 10:10:31 | stephenfin | *how would you | |
| 10:10:43 | sean-k-mooney[m] | it would be an atibute on the volume | |
| 10:10:54 | sean-k-mooney[m] | and or set on the cinder resouce | |
| 10:10:55 | stephenfin | what about multi-attach volumes? | |
| 10:10:56 | sean-k-mooney[m] | not on novas | |
| 10:11:16 | sean-k-mooney[m] | you can have volume attaments they shoould just not be part fo the nova api | |
| 10:11:24 | stephenfin | this isn't a cinder thing, right? nova decides whether to delete $thing or not | |
| 10:11:33 | sean-k-mooney[m] | nova should have /server/uuid/volumes | |
| 10:11:51 | stephenfin | that's what we have :) | |
| 10:12:09 | sean-k-mooney[m] | that should just list the volume not the attacments | |
| 10:12:16 | stephenfin | /server/{server_id}/os-volumes | |
| 10:12:22 | sean-k-mooney[m] | all other data should be in cinder | |
| 10:13:06 | stephenfin | actually, no, tell a lie: the API is '/servers/{server_id}/os-volume_attachments' | |
| 10:13:10 | sean-k-mooney[m] | conceptually if you are working wiht the nova api you should not be thinking in terms of volume attachment but server and volume reosuces | |
| 10:13:26 | stephenfin | https://docs.openstack.org/api-ref/compute/?expanded=list-volume-attachments-for-an-instance-detail#list-volume-attachments-for-an-instance | |
| 10:15:20 | sean-k-mooney[m] | really all nova should be tracking in its api is the volume uuid is associated with the server | |
| 10:15:39 | sean-k-mooney[m] | perhaps the status of the volume too | |
| 10:15:45 | stephenfin | the mount point? | |
| 10:16:05 | sean-k-mooney[m] | not the mount point the tag yes since that is a nova concept | |
| 10:16:24 | sean-k-mooney[m] | we dont actully guarentee the mount point and in libvirt it is jut not correct | |
| 10:17:09 | sean-k-mooney[m] | if you specifiy the mount point name there is no way for libvit to enforce it | |
| 10:17:15 | stephenfin | oh, I didn't know that | |
| 10:17:41 | stephenfin | we should _probably_ mention that in the API ref | |
| 10:17:50 | lyarwood | it is | |
| 10:18:09 | lyarwood | https://docs.openstack.org/api-ref/compute/?expanded=attach-a-volume-to-an-instance-detail#attach-a-volume-to-an-instance | |
| 10:18:15 | lyarwood | Name of the device such as, /dev/vdb. Omit or set this parameter to null for auto-assignment, if supported. If you specify this parameter, the device must not exist in the guest operating system. Note that as of the 12.0.0 Liberty release, the Nova libvirt driver no longer honors a user-supplied device name. This is the same behavior as if the device name parameter is not supplied on the request. | |
| 10:18:50 | stephenfin | lyarwood++ Sweet. I was looking at https://docs.openstack.org/api-ref/compute/?expanded=list-volume-attachments-for-an-instance-detail#list-volume-attachments-for-an-instance and thought it would be mentioned there also | |
| 10:18:55 | stephenfin | once place is good enough though, for sure | |
| 10:19:02 | lyarwood | and I've wanted to remove it entirely from the response but that's going to take reworking the entire attach flow between the API and compute to drop some useless RPC stuff | |
| 10:19:28 | lyarwood | tbh we could also list it in the GET docs | |
| 10:20:25 | sean-k-mooney[m] | stephenfin: so ya i dont think we should have the volume attaments api we have currently and i dont think we should mirror that for the manilla shares going forward | |
| 10:22:55 | lyarwood | sean-k-mooney: I've updated the manila spec FWIW | |
| 10:23:08 | sean-k-mooney[m] | just opened it | |
| 10:23:17 | sean-k-mooney[m] | ill review it this morning | |
| 10:23:51 | sean-k-mooney[m] | i have a doctors appointment in an hour so i might loop back with you later | |
| 10:24:04 | sean-k-mooney[m] | did you see my comment regarding the vm memory | |
| 10:25:05 | sean-k-mooney[m] | oh you going to require file backed memory i almost feel like -2 for that | |
| 10:25:11 | lyarwood | yeah I've suggested going with the simple option for now and queuing the image property work for later on | |
| 10:25:21 | lyarwood | sure go ahead | |
| 10:25:29 | sean-k-mooney[m] | requireing hugepages i could live with | |
| 10:25:46 | sean-k-mooney[m] | file backed memory is not somethign we can schdule on today | |
| 10:26:28 | sean-k-mooney[m] | so there is no way to enforce it so the vm will just not be able to access the shares if it lands on a host without it | |
| 10:27:01 | lyarwood | well the compute would fail the request at that point | |
| 10:27:03 | lyarwood | late on but still | |
| 10:27:12 | lyarwood | we wouldn't have the attachment | |
| 10:27:33 | sean-k-mooney[m] | your expecting a build failure. if its like normal vhost user | |
| 10:27:41 | sean-k-mooney[m] | it will boot but the connectivy wont work | |
| 10:27:46 | lyarwood | well no | |
| 10:28:01 | lyarwood | file backed memory is a configurable on the compute right? | |
| 10:28:05 | sean-k-mooney[m] | yes | |
| 10:28:11 | lyarwood | and with this spec we are only talking about a basic attach share flow | |
| 10:28:42 | lyarwood | if we support shelved it would make this harder to assert but either way | |
| 10:28:58 | lyarwood | during the attach or boot we'd be able to tell if the compute supported file backed or not | |
| 10:29:39 | sean-k-mooney[m] | i guess since this is not boot its not as bad | |
| 10:30:00 | lyarwood | if we support shelved it would be that's more awkward yeah | |
| 10:30:40 | sean-k-mooney[m] | well the other issue is | |
| 10:31:17 | sean-k-mooney[m] | as a normal user you cant tell if file backed memory is used | |
| 10:31:37 | sean-k-mooney[m] | so you dont know if it will work | |
| 10:32:23 | lyarwood | Yeah it's awkward for end users, admins would need file backed host aggregates for this to work I guess | |
| 10:32:37 | lyarwood | but without the image property stuff this is the best we can do in the short term tbh | |
| 10:32:56 | lyarwood | so it's either deliver something this cycle or back it up behind a pile of other work | |
| 10:33:00 | sean-k-mooney[m] | yes but the vm when it was booted did not request “must be able to attached shares” in any way | |
| 10:33:38 | sean-k-mooney[m] | well lets just say use file backed memory or hugepages | |
| 10:34:00 | sean-k-mooney[m] | and i can look at creating the new imge/flavor extra spec in a sperate spec | |
| 10:34:20 | sean-k-mooney[m] | i think it would be a good addtion outside of this feature | |
| 10:35:08 | sean-k-mooney[m] | lyarwood: hugepages also solve the requirement for vhost-user and are user requestable today via flavor or image | |
| 10:36:31 | sean-k-mooney[m] | kashyap by the way do you recall i mention that file backed memory seamed to be not allocating memory form the file | |
| 10:36:48 | kashyap | sean-k-mooney[m]: Very vaguely :) | |
| 10:37:01 | kashyap | sean-k-mooney[m]: Can you refresh my memory, please? Is there a ticket/bug for this? | |
| 10:37:04 | sean-k-mooney[m] | kashyap i have been wondering the last day or two could that be related to tb-cache or something similar | |
| 10:37:21 | sean-k-mooney[m] | no i just deployed it at home to test it | |
| 10:37:40 | kashyap | Interesting. Can you share your guest XML + QEMU command-line to see if I can reproduce it | |
| 10:37:43 | sean-k-mooney[m] | then booted vms and could not over subscibe my ram with OOM | |
| 10:38:05 | sean-k-mooney[m] | well ill have to repoduce it my self in a test envionent | |
| 10:38:23 | sean-k-mooney[m] | but if i do i can share it with you | |
| 10:38:50 | sean-k-mooney[m] | lyarwood: are you setting up a deployment with file backed memory for your manila dev? | |
| 10:39:02 | lyarwood | yeah I plan to | |
| 10:39:18 | sean-k-mooney[m] | ok can you try to reverify the behavior | |
| 10:39:49 | lyarwood | yeah sure | |
| 10:40:01 | kashyap | lyarwood: Hope you're hale and hearty now | |
| 10:40:14 | kashyap | sean-k-mooney[m]: Yeah, that behaviour does sound like tb-cache thing | |
| 10:40:30 | lyarwood | kashyap: yup back to normal now thanks | |
| 10:40:30 | gibi | can I get a second core on this bugfix (bauzas and sean-k-mooney[m] are already positive on it) https://review.opendev.org/c/openstack/nova/+/813419 ? | |
| 10:40:50 | lyarwood | gibi: queued | |
| 10:41:00 | gibi | lyarwood: thanks! I'm glad you are back! | |
| 10:41:54 | sean-k-mooney[m] | basiclly i just tried to boot 6 8G vms on a host with 48G of ram and the 6th one triggered OOM | |
| 10:46:10 | sean-k-mooney[m] | gibi: my +1 dissapeared at some point but its back on it. the config help text is now better then some of our dedicated docs :) | |
| 10:46:44 | gibi | sean-k-mooney[m]: thanks. bauzas pushed me to have proper config doc and even config value validation | |
| 10:47:25 | sean-k-mooney[m] | i proably would have skipped the validation because its easy to miss updating that if we add a new type | |
| 10:47:44 | sean-k-mooney[m] | but we will proably rememeber | |
| 10:48:20 | sean-k-mooney[m] | on the other hand i am seeing a lot of issue related to edgecases with the network vif plugged events | |
| 10:48:56 | sean-k-mooney[m] | which makes me think we need a systematic soluntion to this problem sooner rather then later | |
| 10:49:46 | sean-k-mooney[m] | i might see if i can revie the work to pass the driver form neutron to nova this cycle | |