Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-01-06
19:09:25 darkhorse Hi team, I would like to resize an shelved_offloaded instance. The use case is that when an instance with pci device is shelved_offloaded and the device is broken, it fails to unshelve. However, as a user, I would like to recover my data in the instance so I would like to change the flavor with a new one that does not have pci cards.
19:09:49 darkhorse Is there a quick workaround to this?
19:10:05 darkhorse Thank you in advance for any help!
19:19:35 opendevreview Danylo Vodopianov proposed openstack/nova master: Napatech SmartNIC support https://review.opendev.org/c/openstack/nova/+/859577
19:24:52 dansmith melwitt: around?
19:35:01 melwitt dansmith: o/
19:35:17 dansmith hey so,
19:35:32 dansmith I don't really know what I was going to say
19:35:46 dansmith part of it was that now that I've implemented compute undelete later in the series,
19:35:51 dansmith I'm failing a couple more regression tests,
19:36:09 dansmith but around ironic because of all the hash ring rebalance weirdness
19:36:24 dansmith unfortunately I think those are going to have to change a bit as well, which makes me nervous,
19:36:56 dansmith but they're asserting things like "this shouldn't create another thing, but it does, so assert that it happens, and then assert that it goes away later" sort of stuff
19:37:12 dansmith which is kinda expected and kinda why we're doing this, and the ironic hash-ring-ectomy thing
19:37:42 dansmith but I dunno, I guess I just want to say... I hope you're going to check all my work :P
19:40:09 melwitt dansmith: ack, that's the intent (to check everything) :) thanks for the heads up
19:40:48 dansmith I know
19:41:10 dansmith but this rabbit hole is deep, the tea tastes funny, and everyone is wearing strange hats
19:42:50 melwitt haha, I hear you (and am not surprised)
19:49:58 opendevreview Danylo Vodopianov proposed openstack/nova master: Napatech SmartNIC support https://review.opendev.org/c/openstack/nova/+/859577
19:52:51 sean-k-mooney darkhorse: no quick workaround. resize is currently not supproted while shelve_offloaded but we have discussed that it could be supported in the future
19:53:24 sean-k-mooney darkhorse: i think artom has already fixed the issue with pci device shelve however
19:53:37 sean-k-mooney so i dont think that happens any more
20:06:13 darkhorse sean-k-mooney: do you have a link to the patchset? when you say the issue is fixed, does that mean you can unshelve instances even if pci card is broken or unavailable?
20:12:52 opendevreview Danylo Vodopianov proposed openstack/nova master: Napatech SmartNIC support https://review.opendev.org/c/openstack/nova/+/859577
20:14:05 artom sean-k-mooney, IIUC darkhorse wants to resize a shelved_offloaded instance
20:14:10 artom Which IIUC is... not a thingÉ
20:14:10 artom Which IIUC is... not a thingÉ
20:14:12 artom ?
20:14:18 artom As in, you have to unshelve first, and then resize?
20:14:34 artom And yeah, unshelve with PCI has been backported to... I want to say Ussuri?
20:14:49 artom Or maybe wallaby
20:14:54 darkhorse sean-k-mooney: If you can share the link of the discussion of the resize support for shelved instances, it would be helpful. I will take a look and work on it.
20:15:44 darkhorse artom: Do you mean you can unshelve instance when pci card is broken/unavailable in Ussuri or Wallaby?
20:16:56 artom darkhorse, https://review.opendev.org/q/Icfa8c1d6e84eab758af6223a2870078685584aaa
20:16:57 artom wallaby
20:19:03 darkhorse artom: We are operating on xena. So if I understood you correct, all I need to do to allow users to unshelve pci instance even if card is broken/unavailable is to backport this patch to xena, is that correct?
20:19:51 artom darkhorse, no, you should be set. Xena is after wallaby :)
20:20:00 artom When the master patch merged, master was xena
20:20:26 artom darkhorse, hold on though - define "card is broken/unavailable"?
20:20:52 artom The unselve will attempt to find a PCI card that fits the port (if it's Neutron SRIOV)/flavor
20:21:30 artom But... if no such cards are available, then it will (legitimately) fails to schedule
20:21:39 darkhorse artom: not neutron SRIOV but fpga device.
20:21:47 artom So flavor PCI passthrough...
20:21:59 darkhorse right!
20:22:16 artom That should just... work. Off the top of my head I don't recall any issues with PCI and unshelve
20:23:01 darkhorse artom: no it fails to unshelve because the pci device is unavailable.
20:23:35 artom Unavailable how? It got pulled from the server? :)
20:23:45 darkhorse in that case, i would like to either snapshot or resize the instance so that I don't lose the data inside it.
20:24:21 darkhorse either because the card is occupied by another instances or physically broken
20:32:33 darkhorse artom: did i answer your question?
20:34:18 artom darkhorse, ah, I think I see. If you can't unshelve the instance because the cloud lacks the resources the instance needs (in this case, a PCI card), you'd like to be able to boot it regardless with its disk intact, just without the PCI device
20:34:39 artom So a shelved_offloaded instance lives as an image in Glance
20:34:55 artom IIRC you should just be able to boot a new instance from that image?
20:35:12 artom If keeping the same UUID is important to you though, you're out of luck I believe :(
20:36:32 darkhorse artom: the point is i want to recover the data inside the instance. if i boot a new instance, i think i am not able to get the data?
20:37:00 artom If it's been shelved offloaded, its disk has been uploaded to Glance as an image.
20:37:27 artom But... if you want data to persist, the "real" solution is to use volumes
20:38:38 darkhorse artom: will you elaborate? i was thinking of snapshotting or resizing with a new flavor that does not have pci so that i can unshelve.
20:39:38 artom darkhorse, elaborate on which aspect? Volumes, or booting from the Glance image?
20:40:22 darkhorse 1. if booting from glance image will save the data 2. volumes
20:41:02 darkhorse artom:1. if booting from glance image will save the data 2. volumes
20:42:07 artom darkhorse, it's been a while since I've done this, but a shelved_offloaded image will have its disk uploaded as image in Glance
20:42:28 artom I believe you can just boot from that image with `openstack server create --image <image uuid> <etc>`
20:42:59 artom And for volumes... you create a volume, attach it to your instance
20:43:26 artom Inside the guest you mount it as /data or whatever
20:43:49 artom And then anything in /data will live on the volume, so even if the instance is deleted, that volume persists and can be attached to other instances
20:44:43 darkhorse artom: the point is to recover the data in the instance. what should i do when instance is not able to get unshelved?
20:45:47 artom I'm not sure how much more clear I can be
20:45:53 artom <artom> I believe you can just boot from that image with `openstack server create --image <image uuid> <etc>`
20:45:53 artom <artom> darkhorse, it's been a while since I've done this, but a shelved_offloaded image will have its disk uploaded as image in Glance
20:46:52 darkhorse artom: ok thank you! let me try that.
21:01:44 opendevreview Danylo Vodopianov proposed openstack/os-vif master: MTU support for DPDK port added https://review.opendev.org/c/openstack/os-vif/+/859574
23:57:32 opendevreview Ghanshyam Mann proposed openstack/python-novaclient master: DNM: test tox<4 pinning in stable branches https://review.opendev.org/c/openstack/python-novaclient/+/869516
23:59:57 opendevreview Ghanshyam Mann proposed openstack/osc-placement stable/zed: DNM: test tox<4 pinning in stable branches https://review.opendev.org/c/openstack/osc-placement/+/869517
#openstack-nova - 2023-01-07
00:00:05 opendevreview Ghanshyam Mann proposed openstack/placement stable/zed: DNM: test tox<4 pinning in stable branches https://review.opendev.org/c/openstack/placement/+/869518
02:11:08 opendevreview sean mooney proposed openstack/os-vif master: add CAP_DAC_OVERRIDE to test privsep contexts https://review.opendev.org/c/openstack/os-vif/+/869500
02:24:15 opendevreview Ghanshyam Mann proposed openstack/osc-placement stable/yoga: DNM: test tox<4 pinning in stable branches https://review.opendev.org/c/openstack/osc-placement/+/869451
02:26:28 opendevreview Ghanshyam Mann proposed openstack/python-novaclient stable/zed: DNM: test tox<4 pinning in stable branches https://review.opendev.org/c/openstack/python-novaclient/+/869452
04:07:28 opendevreview Ghanshyam Mann proposed openstack/python-novaclient stable/zed: DNM: test tox<4 pinning in stable branches https://review.opendev.org/c/openstack/python-novaclient/+/869452
20:05:41 opendevreview Ghanshyam Mann proposed openstack/python-novaclient master: [stable-only] Pin tox <4 https://review.opendev.org/c/openstack/python-novaclient/+/869524
#openstack-nova - 2023-01-08
00:31:24 opendevreview Ghanshyam Mann proposed openstack/nova master: Enable new defaults and scope checks by default https://review.opendev.org/c/openstack/nova/+/866218
00:53:04 opendevreview Ghanshyam Mann proposed openstack/nova master: Enable new defaults and scope checks by default https://review.opendev.org/c/openstack/nova/+/866218
05:06:56 opendevreview Ghanshyam Mann proposed openstack/placement master: DNM: testing rbac new defaults https://review.opendev.org/c/openstack/placement/+/869145
06:17:21 opendevreview Ghanshyam Mann proposed openstack/placement master: DNM: testing rbac new defaults https://review.opendev.org/c/openstack/placement/+/869145
07:10:19 opendevreview Ghanshyam Mann proposed openstack/placement master: Avoid rbac defaults conflict in functional tests https://review.opendev.org/c/openstack/placement/+/869525
07:14:06 opendevreview Ghanshyam Mann proposed openstack/nova master: Enable new defaults and scope checks by default https://review.opendev.org/c/openstack/nova/+/866218
07:15:41 opendevreview Ghanshyam Mann proposed openstack/placement master: DNM: testing rbac new defaults https://review.opendev.org/c/openstack/placement/+/869145
20:09:53 opendevreview Ghanshyam Mann proposed openstack/placement master: DNM: testing rbac new defaults https://review.opendev.org/c/openstack/placement/+/869145
20:10:39 opendevreview Ghanshyam Mann proposed openstack/placement master: Avoid rbac defaults conflict in functional tests https://review.opendev.org/c/openstack/placement/+/869525
20:15:05 opendevreview Ghanshyam Mann proposed openstack/python-novaclient stable/zed: [stable-only] Pin tox <4 https://review.opendev.org/c/openstack/python-novaclient/+/869527
20:17:30 gmann seems we are hitting missing interpreter again? https://zuul.opendev.org/t/openstack/build/3dbb7278e8b94e77aa25d207286bfcba/log/job-output.txt
20:17:35 gmann stephenfin: ^^
20:36:23 gmann seeing this in placement master patches also
#openstack-nova - 2023-01-09
03:54:29 opendevreview David Sedgmen proposed openstack/nova master: Nova: Add workaround to mask mpx on compareCPU() https://review.opendev.org/c/openstack/nova/+/869536
08:55:34 gibi gmann, stephenfin: it is strange I see the periodic passed during the weekend but now I also see failing jobs with missing interpreter: here is a fresh one https://1729c20500306078d1cb-f18cd0734c7002742f0baf633d0573f2.ssl.cf2.rackcdn.com/854615/11/check/nova-tox-functional-py310/548baa7/job-output.txt
09:05:10 gibi it seems that nova functional jobs are hit but unit test jobs arent
09:06:51 songwenping_ passwd
09:19:27 gibi both job using tox 4.2.6 which supposed to contain the fix of missing interpreters
09:20:35 bauzas gibi: sorry, this vgpu downstream reproducer vamps me

Earlier   Later