| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-10-18 | |||
| 14:19:50 | sean-k-mooney | or if you have hostdev devices | |
| 14:19:55 | bauzas | dansmith: good question, I know that we test some move ops | |
| 14:20:20 | sean-k-mooney | so you think the best approch woudl be to start creating new geneic mdev tests in tempest right | |
| 14:20:32 | bauzas | and yeah I have a 2-node env with one mtty, can try to migrate my test instance | |
| 14:20:42 | sean-k-mooney | cool | |
| 14:20:56 | dansmith | sean-k-mooney: depends on what they look like really, but.. yeah why not? | |
| 14:20:59 | sean-k-mooney | cold migrate should work live migrate will likely fail or we/libvirt have a bug | |
| 14:21:01 | bauzas | back to dansmith's point about status checks | |
| 14:21:06 | dansmith | this is clearly a gap | |
| 14:21:37 | sean-k-mooney | dansmith: no reason not to just wanted to confirm that is your prefernce | |
| 14:22:04 | bauzas | dansmith: so, about status checks, I'd say revert it | |
| 14:22:05 | sean-k-mooney | so we will need a new compute feature config option in tempest for it and perhasp a cofnig option to identify how to detect the device in the vm | |
| 14:22:16 | dansmith | sean-k-mooney: yeah | |
| 14:22:23 | bauzas | no real gain in having a warning imho | |
| 14:22:31 | dansmith | bauzas: okay that was sean-k-mooney's opinion too | |
| 14:22:46 | dansmith | bauzas: so revert in master, backport that, and then maybe add it back later when we start depending on it? | |
| 14:22:54 | bauzas | wfm | |
| 14:23:06 | dansmith | aight | |
| 14:23:23 | sean-k-mooney | ya so i think that would make sense | |
| 14:23:32 | sean-k-mooney | fell like pushing the revert patch | |
| 14:23:50 | sean-k-mooney | and we can review it quickly since we are talking about it | |
| 14:24:42 | dansmith | I'm doing it | |
| 14:24:45 | dansmith | I need to find that bug tho | |
| 14:25:50 | opendevreview | Dan Smith proposed openstack/nova master: Revert "Add upgrade check for compute-object-ids linkage" https://review.opendev.org/c/openstack/nova/+/898741 | |
| 14:28:57 | bauzas | hmmm, looking more at the mdev parent being 'computer' in the device XML, there may be some confusions when creating inventories if we were about asking two different virtual devices, like say mtty and mdpy | |
| 14:29:14 | bauzas | that's a potential bug, so I'm pdb'ing it on my devstack | |
| 14:29:49 | bauzas | another slight difference from libvirt that may need to change on their side | |
| 14:51:54 | sean-k-mooney | well we can have mulitpel inventories of mdevs for diffent types | |
| 14:52:31 | sean-k-mooney | so it will be a slightly didfdfent toplogy i.e. 1 rp for all virtual devs rahter then one per pareent devce | |
| 14:52:34 | sean-k-mooney | but i think that is fine | |
| 14:52:55 | bauzas | no, because we don't support heterogenous VGPUs (yet) | |
| 14:53:13 | bauzas | a single device can only support one type | |
| 14:53:27 | sean-k-mooney | i didnt think that was enforecd there i tought that was done as part of the config | |
| 14:53:29 | bauzas | that's how we account those | |
| 14:53:30 | sean-k-mooney | parsing | |
| 14:53:44 | bauzas | there is a config parser that checks that yes | |
| 14:53:54 | bauzas | but our internal dicts rely on that too | |
| 14:53:59 | sean-k-mooney | ok well that another reason to use the device_adress | |
| 14:54:03 | sean-k-mooney | for virtrual stuff | |
| 14:54:14 | bauzas | yeah, I filed a bug against libvirt fwiw | |
| 14:54:24 | bauzas | the vfio guys are aware of the bug | |
| 14:54:30 | bauzas | and we'll just adapt | |
| 14:55:20 | bauzas | it occurs to me that OpenStack is probably the only high-level project that's about to integrate mtty for testing ;:) | |
| 14:55:35 | bauzas | mtty was probably just a toy project | |
| 14:55:54 | bauzas | but if that helps others's testing, that's a net win | |
| 15:43:13 | bauzas | netcalol | |
| 16:13:26 | bauzas | sean-k-mooney: fwiw, read_sys only reads files :) https://github.com/openstack/nova/blob/master/nova/filesystem.py#L28 | |
| 16:13:42 | bauzas | sean-k-mooney: and /sys/devices/virtual/mtty is a dir :) | |
| 16:14:27 | bauzas | I can provide some helper method that would wrap os.path.isdir() but would you nack if I was just calling os.path.isdir() straight in the driver ? | |
| 16:14:39 | bauzas | or .exists() | |
| 16:21:36 | colby__ | noonedeadpunk: Thanks Ill check on that. Did not realize the tunnel was dropped. Was this a change on xena to yoga? Everything worked fine on xena. | |
| 16:23:05 | sean-k-mooney | bauzas: good point on it being a dir i guess you can just add a new helper ya in the same module | |
| 16:23:29 | sean-k-mooney | or just call os.path.isdir() but it would be ok ot put in the filesystem module | |
| 16:23:45 | sean-k-mooney | an exist check is enough | |
| 16:23:53 | bauzas | sean-k-mooney: well, if the helper method only wraps isdir, that sounds to me a bit unnecessary | |
| 16:24:10 | sean-k-mooney | so i would go with exist because it might actully be a simlink | |
| 16:24:13 | colby__ | I have another question. It appears that the amount of local disk available changed in yoga. It appears to be only considering the actual local disk now (without ceph). But instances that use a ceph backed volume to boot are still counting as local disk usage. Was this intentional? Its causing no resources available issues for me. Im working on changing the disk allocation ratio to get around it. | |
| 16:24:27 | bauzas | yeah, agreed on exists(), semantically better | |
| 16:24:41 | bauzas | but if I'm just returning the boolean, meh | |
| 16:25:00 | bauzas | I can pretend to return a FileNotFound exception if exists says no | |
| 16:25:12 | bauzas | at least that would justify the helper | |
| 16:25:46 | sean-k-mooney | colby__: boot form volume instance shoudl use flavor with disk_gb = 0 | |
| 16:26:26 | sean-k-mooney | we allow boot form volue instnace with flavor that dont set it to 0 but it is only for legacy/upgrade reasons | |
| 16:26:53 | sean-k-mooney | if you mena ceph volume via image_type=rbd | |
| 16:26:59 | sean-k-mooney | then use flavor as normal | |
| 16:27:08 | sean-k-mooney | with disk_gb set to the correct size | |
| 16:27:52 | sean-k-mooney | in yoga and master for that mater we report local_gb based on the rbd pool size for every instnace as far as i remember | |
| 16:31:06 | colby__ | yea unfortunately our flavors have disk size set even though they all use ceph boot volumes. Everythign we use is in ceph (images/ephemeral/volumes). But seems there is no way around the already set flavor disk size then. From the hypervisor screen its seems to be only showing the hypervisor boot disk volume for local disk | |
| 16:31:51 | colby__ | Im in the process of upgrading yoga and trying to live migrate instances to the already upgraded machines and running into this issue | |
| 16:32:07 | colby__ | is there a way to override the disk size of a running instance flavor? | |
| 16:32:34 | sean-k-mooney | colby__: are you using images_type=rbd | |
| 16:32:38 | sean-k-mooney | or cinder volumes | |
| 16:33:24 | colby__ | cinder volumes | |
| 16:33:47 | sean-k-mooney | adn your using images_type=qcow | |
| 16:33:52 | sean-k-mooney | or raw or flat | |
| 16:34:14 | colby__ | we did not set it so its currently default | |
| 16:34:32 | sean-k-mooney | ok that woudl be defcault which for libvirt is qcow | |
| 16:35:09 | sean-k-mooney | imporanly in that config then the expect bevioru is to report local_disk_gb as the free space on the compute node partion where /var/lib/nova is | |
| 16:35:20 | sean-k-mooney | and that was the ame behvior in older release | |
| 16:35:33 | colby__ | ok so set images_type=rbd then? | |
| 16:35:37 | sean-k-mooney | no | |
| 16:35:51 | sean-k-mooney | that means use cpeh for root disk | |
| 16:35:56 | sean-k-mooney | even with out ceinder | |
| 16:36:14 | colby__ | ah sorry I misunderstood. so raw then? | |
| 16:36:21 | sean-k-mooney | no dont change it | |
| 16:36:35 | sean-k-mooney | nova is working as it should form a compute config point of view | |
| 16:36:47 | colby__ | ok | |
| 16:37:01 | sean-k-mooney | i need to check somehting but i alo need to join a 1:1 with my manager | |
| 16:37:12 | colby__ | ok no problem. Thanks for the help | |
| 16:37:42 | sean-k-mooney | did you used to set allcoation raitio by aggreates prehaps? | |
| 16:39:56 | dansmith | ugh, feels like we're back to gate suckage.. some trivial patches are in recheck hell | |
| 16:41:03 | colby__ | sean-k-mooney: we never set anything for disk allocation | |
| 17:03:28 | colby__ | sean-k-mooney: I was able to increase the disk allocation ratio but now seeing this: nova.exception.InvalidSharedStorage: <host> is not on shared storage: Shared storage live-migration requires either shared storage or boot-from-volume with no local disks. This is a boot from volume ceph instance | |
| 17:04:04 | colby__ | is this all due to the disk_gb of the flavors? | |
| 17:08:19 | colby__ | Im migrating from a xena hypervisor to a yoga and seeing this issue | |
| 17:21:38 | sean-k-mooney | colby__: so partly. but you shoudl have had the same behavior on xena and yoga i think | |
| 17:22:01 | sean-k-mooney | colby__: so you should not be passing --shared-srotage wehn live migrating | |
| 17:22:32 | colby__ | I was just using the GUI for the live migration but can switch to the cli | |
| 17:22:36 | sean-k-mooney | you should be using a newer microvstion where that defautl to auto | |
| 17:23:10 | sean-k-mooney | so for boot form volume guest you should not tick shared storage | |