| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-30 | |||
| 09:42:53 | lyarwood | tobias-urdin: or did you mean call the compute rpc_api from the libvirt driver instead of ssh? | |
| 09:43:07 | lyarwood | tobias-urdin: I don't think we've allowed that in the past | |
| 09:43:23 | lyarwood | tobias-urdin: but we could look into it again in W | |
| 09:44:50 | tobias-urdin | i see, either way just wanted to confirm since we were hitting issues with resizing between nodes, getting rid of ssh connection would be preferrable somehow | |
| 09:45:24 | frickler | bauzas: just play back at 32 fps instead of 25? ;) | |
| 09:46:03 | tobias-urdin | i assume that would mean abstracting away the action so that the manager does a rpc call to the destination host, which as you said would mean a change for all drivers to implement some new method to prepare the dest node (or not implement it at all if not needed) | |
| 09:46:17 | tobias-urdin | lyarwood: appreciate it! not gonna steal more of your time :) | |
| 09:46:53 | lyarwood | yeah something like that, it would be easier to call from the virt driver but like I said there's no prior art for doing that and I think that's by design | |
| 09:47:01 | lyarwood | tobias-urdin: anytime! | |
| 09:47:41 | lyarwood | gmann / stephenfin: https://review.opendev.org/#/c/754684/ - would you mind hitting this btw? | |
| 09:52:10 | lyarwood | thanks | |
| 09:55:42 | openstackgerrit | Wenping Song proposed openstack/nova master: Add instance project_id for cyborg arq https://review.opendev.org/738428 | |
| 10:33:09 | sean-k-mooney | tobias-urdin: you dont have to usee ssh for resize | |
| 10:33:18 | sean-k-mooney | tobias-urdin: you can use rsync instead | |
| 10:34:04 | sean-k-mooney | we can also use an rpc instead fo the current test we use for detecting shared storage but that would require a spec/blueprint | |
| 10:35:25 | sean-k-mooney | tobias-urdin: lyarwood alternitivly we could finally stop supporting implict shared storgage | |
| 10:35:48 | lyarwood | sean-k-mooney: doubt that would fly with operators | |
| 10:35:55 | sean-k-mooney | e.g. if you put the nova state dir on shared storage you must set a config flag instead of detecting it | |
| 10:36:07 | lyarwood | sean-k-mooney: oh, well yeah that would be nicer | |
| 10:36:17 | sean-k-mooney | yep an ooo or whatever could set that | |
| 10:36:18 | lyarwood | also TIL about [libvirt]/remote_filesystem_transport | |
| 10:36:30 | lyarwood | I didn't think rsync would support that | |
| 10:36:43 | sean-k-mooney | rsync can be used to copy the disk | |
| 10:37:02 | sean-k-mooney | i am not sure about the detection of the shared filesystem | |
| 10:37:46 | sean-k-mooney | but ya https://github.com/openstack/nova/blob/cff7382fb3de4eef7aeddc34b6ae3409cbe50ea0/nova/virt/libvirt/volume/remotefs.py#L201 | |
| 10:38:02 | lyarwood | yeah looks like it can also do that | |
| 10:38:27 | sean-k-mooney | oh cause it can create the file on the dest and see if it shows up on the source | |
| 10:38:34 | lyarwood | yeah | |
| 10:38:46 | lyarwood | haha we create the dir and rsync it across, of course. | |
| 10:40:23 | sean-k-mooney | tobias-urdin: what was you issue by the way i did not read all of scrollback | |
| 10:41:06 | lyarwood | I don't think there was an issue, he just wanted to drop the use of ssh | |
| 10:41:21 | sean-k-mooney | ah ok | |
| 10:41:40 | sean-k-mooney | mnaser ask about the same about 2 weeks ago. e.g. could you drop ssh | |
| 10:42:14 | sean-k-mooney | i suggested an rpc at that time too although the config value occured to me after | |
| 10:44:06 | tobias-urdin | sean-k-mooney: my deployment assumed that no ssh access was required and since our resize operations never changed the destination host in the past i never hit the ssh issue | |
| 10:44:23 | tobias-urdin | and I was assuming ssh access was only needed when supporting ephemeral disks | |
| 10:44:54 | tobias-urdin | (we run soley on cinder volumes and images_type=rbd backend) | |
| 10:46:24 | sean-k-mooney | i see | |
| 10:46:35 | sean-k-mooney | images_type=rbd should not need this path | |
| 10:46:39 | sean-k-mooney | we know that is shared | |
| 10:46:55 | sean-k-mooney | so for that case we do not need to check | |
| 10:47:18 | tobias-urdin | which is what i thought as well, apparantely not since resize fails trying to ssh to the destination host | |
| 10:47:21 | sean-k-mooney | we dont support resizing across images_type | |
| 10:47:42 | tobias-urdin | i've been trying to follow the code path | |
| 10:47:44 | sean-k-mooney | tobias-urdin: no i mean we can file a bug for that specific case | |
| 10:47:53 | sean-k-mooney | and we can fix that it should not be needed for rbd | |
| 10:48:25 | sean-k-mooney | we can just add an if around the call and check if the images type is knonw to be shared | |
| 10:50:18 | lyarwood | sean-k-mooney: just reading back, no we still need to check even with images_type=rbd | |
| 10:50:22 | tobias-urdin | that would be great, how can we determine if it's shared? | |
| 10:50:30 | sean-k-mooney | lyarwood: why would we | |
| 10:50:43 | lyarwood | sean-k-mooney: the instance directory and instance storage being shared are two different things | |
| 10:50:45 | sean-k-mooney | lyarwood: we have never supported resizing between host with different backends | |
| 10:50:55 | sean-k-mooney | lyarwood: oh ya your right | |
| 10:51:13 | sean-k-mooney | lyarwood: actully didnt we skip this and revert it for rbd before | |
| 10:51:19 | lyarwood | yeah | |
| 10:51:22 | lyarwood | it's checking again now | |
| 10:51:33 | lyarwood | as the skip broke when the instance directory wasn't shared | |
| 10:51:43 | sean-k-mooney | ya | |
| 10:52:02 | sean-k-mooney | is there anything in the instance directory we care about by the way | |
| 10:52:16 | sean-k-mooney | config drive maybe? | |
| 10:52:30 | lyarwood | yeah that the console log | |
| 10:52:35 | sean-k-mooney | the console log not really | |
| 10:52:36 | lyarwood | and maybe other things, I forget | |
| 10:52:45 | sean-k-mooney | it will be cleared when the vm is started again | |
| 10:52:58 | lyarwood | right but this path is also used during LM | |
| 10:53:12 | sean-k-mooney | ah ok | |
| 10:53:36 | sean-k-mooney | ya so config option of rpc are the only things we can do then | |
| 10:54:23 | sean-k-mooney | well in the rbd case there is one other thing we could do but its a bit of a hack | |
| 10:54:41 | sean-k-mooney | mount the instance dir in an rbd volumn | |
| 10:55:31 | sean-k-mooney | or on cephfs but nova doing that is a bit of a hack | |
| 10:55:40 | sean-k-mooney | its really something the installer should do | |
| 10:59:21 | lyarwood | sean-k-mooney: yeah this is all just debt from the early days of Nova | |
| 10:59:34 | lyarwood | sean-k-mooney: we could rework this now or just live with it | |
| 11:07:16 | noonedeadpunk | hey everyone! question: is it possible that during resize siolated aggregates are just ignored? | |
| 11:08:02 | noonedeadpunk | as I have a host which is in the isolated agrregate, and vm's are not created on it. but, during vm resize it catched it... | |
| 11:08:49 | noonedeadpunk | it's on train though | |
| 11:09:29 | ralonsoh | frickler, sorry again. What do you mean with "not in ovs"? | |
| 11:09:48 | ralonsoh | related to http://paste.openstack.org/show/798558/ | |
| 11:10:15 | ralonsoh | maybe I'm wrong but this command is executed from nova-compute | |
| 11:10:47 | ralonsoh | I checked oslo_concurrency but this is just a library and fix should be in nova too | |
| 11:11:05 | sean-k-mooney | noonedeadpunk: i dont think it should change the behavior | |
| 11:11:19 | sean-k-mooney | noonedeadpunk: which form of isolated aggreates are you using | |
| 11:11:35 | sean-k-mooney | noonedeadpunk: tenant isolation or the placement/traits based isolated aggreates | |
| 11:12:11 | noonedeadpunk | placement traits | |
| 11:12:20 | noonedeadpunk | but once you said that... | |
| 11:12:26 | sean-k-mooney | we shoudl be using the traits request form the new flavor | |
| 11:12:30 | noonedeadpunk | how can I make tenant isolation?:) | |
| 11:13:09 | noonedeadpunk | so right now aggregate looks like this in nova http://paste.openstack.org/show/798568/ | |
| 11:13:23 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_multitenancy_isolation.py | |
| 11:13:25 | noonedeadpunk | and no flavors or images have this trait set.. | |
| 11:13:58 | sean-k-mooney | ralonsoh: that traceback i think is an eventlet issue | |
| 11:14:22 | ralonsoh | sean-k-mooney, yes, related to https://bugs.launchpad.net/nova/+bug/1863021 | |
| 11:14:22 | openstack | Launchpad bug 1863021 in OpenStack Object Storage (swift) "[SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError" [Undecided,In progress] - Assigned to Chris MacNaughton (chris.macnaughton) | |
| 11:14:30 | ralonsoh | but that should be already fixed in Nova | |
| 11:14:39 | noonedeadpunk | sean-k-mooney: I guess tenant aggregation will allow other tenants to create instances in this aggregate? | |
| 11:14:45 | sean-k-mooney | ralonsoh: ya so the approch they are takign to fix it is a hack | |
| 11:14:51 | noonedeadpunk | *tenant isolation | |
| 11:14:58 | sean-k-mooney | ralonsoh: the real fix is to fix eventlets | |
| 11:15:03 | ralonsoh | sean-k-mooney, I know, but works for now | |
| 11:15:32 | ralonsoh | sean-k-mooney, what I don't understand is why this is failing when calling processutils.execute | |