Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-04-09
14:59:41 sean-k-mooney which i think is what stephenfin ment by make it an error too right
14:59:44 bauzas sean-k-mooney: stephenfin: lemme think about it
15:00:02 bauzas I'm honestly not sure we'd have to hardstop but I could be wrong
15:01:19 bauzas stephenfin: worth accepting a FUP for the doc nit ?
15:01:22 stephenfin bauzas: What would happen in that situation? You wouldn't be able to restart the instance using the mdev because it wasn't recreated?
15:02:16 stephenfin bauzas: Yeah, I'm not sure if we can hard fail for that case (missing '[vgpu_{name}]' sections) after all
15:02:37 stephenfin We already allowed people to list multiple values in 'enabled_vgpu_types', yeah?
15:02:43 stephenfin Even though they did nothing
15:03:18 stephenfin So it's possible there are nova.conf files in the wild with e.g. '[devices] enabled_vgpu_types = nvidia-35, nvidia-36'
15:03:21 sean-k-mooney bauzas: we could not start instance vm at that point right
15:03:45 stephenfin and if they upgrade to Ussuri, they suddenly won't be able to start nova
15:03:49 stephenfin *nova-compute
15:03:58 bauzas stephenfin: I considered this point (having set multiple types while we weren't supporting it)
15:04:22 bauzas stephenfin: I think the hardstop is a reasonable failure
15:04:53 bauzas stephenfin: for operators having set multiple types, they would see that it wasn't really supported before and now they have some action to do
15:05:08 bauzas stephenfin: instead of blindly thinking it was working
15:05:19 stephenfin right, but it was allowed and didn't break anything
15:05:28 bauzas stephenfin: we only refuse to start
15:05:38 bauzas stephenfin: people can just adapt their confs if so
15:05:52 bauzas again, we made it on purpose
15:06:11 bauzas I see this as "we won't impact you if you made things the right way"
15:06:26 bauzas 'but if you made things the wrong way, it's time to change your config to make it work correctly"
15:06:30 stephenfin idk, I thought it was a given that nova.conf from N should continue to work in N+1
15:06:34 stephenfin with warnings, sure
15:06:38 bauzas stephenfin: it will
15:07:03 bauzas stephenfin: but I can honestly revert to PS10
15:07:51 bauzas stephenfin: tbc, there is zero upgrade impact for single-typed configurations
15:08:13 bauzas it's debatable tho and I don't want to hold
15:08:18 sean-k-mooney bauzas: any chace you could add a nova-staus command for this?
15:08:32 bauzas sean-k-mooney: certainly not for Train :)
15:08:41 bauzas which is the case we're discussing about
15:08:46 sean-k-mooney ah ok hehe
15:09:05 sean-k-mooney im also not sure how useful it would be given you would have eto run it on the compute node
15:09:29 stephenfin Yeah, I think what you had before was better. "Your nova.conf is wrong and needs to be fixed. We're reverting to the old Train behaviour until you do that"
15:09:46 bauzas stephenfin: okay here is a trade-off, I can write an upgrade note saying that multiple-typed confs have to adapt their confs
15:10:06 bauzas stephenfin: okay, let's revert to PS10 then
15:10:14 sean-k-mooney that still does not adress https://review.opendev.org/#/c/715490/11/nova/virt/libvirt/driver.py@815
15:10:32 bauzas sean-k-mooney: yup, it will be PS10 with recreate hard stopping
15:10:37 bauzas we all okay ?
15:10:49 sean-k-mooney in that case you have a direct conflict between an instance type used by a guest on the current host and and the config
15:10:52 stephenfin Yeah, change where we do the hard stop and I'm happy
15:10:55 sean-k-mooney bauzas: yep
15:10:57 sean-k-mooney that works for me
15:11:06 bauzas cool, on it
15:11:13 stephenfin that's a straight up misconfiguration and should be a hard stop
15:11:35 stephenfin Might even be worth backporting a variant of that to stable/train
15:11:54 sean-k-mooney the warning
15:11:59 sean-k-mooney maybe
15:21:33 bauzas stephenfin: you're talking of the recreate issue you raised ?
15:21:57 bauzas stephenfin: apologies btw. I fucked up with your point, I thought you were arguing about what I fixed in PS11
15:22:11 bauzas looks like multitasking with kids raises bugs
15:22:27 bauzas -ETOOMANYTHINGS
15:23:53 stephenfin bauzas: Um, I am :) I'm saying I think you need to hard fail at https://review.opendev.org/#/c/715490/11/nova/virt/libvirt/driver.py@815 and *not* hard fail at https://review.opendev.org/#/c/715490/11/nova/virt/libvirt/driver.py@6531
15:24:07 bauzas yup, I finally understood
15:27:57 bauzas stephenfin: worth saying, do you think we should return a InvalidLibvirtGPUConfig within https://review.opendev.org/#/c/715490/11/nova/virt/libvirt/driver.py@815 ?
15:28:02 bauzas or another exception ?
15:28:19 stephenfin It's invalid config, so that makes sense IMO
15:28:21 bauzas I honestly think the operator messed up their config if so
15:28:26 bauzas yup, ok
15:30:29 bauzas stephenfin: to answer your last comment on https://review.opendev.org/#/c/715490/11/nova/virt/libvirt/driver.py@815 ,
15:31:05 bauzas if the operator messes up their config and self._get_vgpu_type_per_pgpu(parent) can't find the right vgpu type because $mess
15:31:25 bauzas then we get a None value and the conditional statement fails
15:32:31 bauzas stephenfin: but unless the operator did provided a section for each of the vGPU types and added devices, you're right, we fall back to only supporting one type, like we did previously
15:34:46 stephenfin cool. gtk I was reading that correctly
15:47:28 openstackgerrit Sylvain Bauza proposed openstack/nova master: Support different vGPU types per pGPU https://review.opendev.org/715490
15:47:28 openstackgerrit Sylvain Bauza proposed openstack/nova master: Functional test with pGPUs https://review.opendev.org/717975
15:47:55 bauzas stephenfin: sean-k-mooney: gibi ^
15:49:23 sean-k-mooney i like how gerrit leaves the filtes ticked if they have not changed form the last revision
15:49:27 sean-k-mooney looking now
15:50:49 gibi bauzas: ack
15:50:57 gibi will look shortly
15:51:21 gibi nova meeting starts in 9 minutes on #openstack-meeting-3
15:52:04 bauzas gibi: sean-k-mooney: shit, forgot to remove two things, will provide a FUP if you don't disagree
15:52:33 stephenfin bauzas: https://review.opendev.org/#/c/715490/12/nova/virt/libvirt/driver.py@817
15:52:37 stephenfin ah, guess that's one of them
15:52:57 stephenfin just respin it now? not like it's even in the queue yet
15:52:58 sean-k-mooney the continue
15:52:59 bauzas stephenfin: yup
15:53:06 bauzas stephenfin: okay, just doing
15:53:08 sean-k-mooney i have a commnet i was going to make
15:53:20 sean-k-mooney its just dead code so it wont break anythying but ya
15:54:45 bauzas uploading a new rev as of I speak
15:55:01 openstackgerrit Sylvain Bauza proposed openstack/nova master: Support different vGPU types per pGPU https://review.opendev.org/715490
15:55:01 openstackgerrit Sylvain Bauza proposed openstack/nova master: Functional test with pGPUs https://review.opendev.org/717975
15:55:05 bauzas (live my life, DSL with 1Mbps up)
15:55:21 bauzas stephenfin: sean-k-mooney: gibi: sorry, last rev ^
15:55:32 sean-k-mooney big claime :P
15:55:33 gibi last? are you sure? ;) (just kidding)
15:55:41 bauzas latest*
15:56:21 sean-k-mooney bauzas: so out of scope for this cycle but is there any reason in victora we could not auto report custom triats for the vgpu providres
15:57:07 bauzas sean-k-mooney: it's within the spec, said as "planned"
15:57:17 sean-k-mooney bauzas: so we can skip https://review.opendev.org/#/c/715490/13/doc/source/admin/virtual-gpu.rst@290
15:57:21 sean-k-mooney ok cool
15:57:36 sean-k-mooney doing it manually for now is fine by the way i was just wondering
15:58:07 gibi bauzas: Is there proper 4G coverage where you live? that would be a lot more than 1Mbps
15:58:42 sean-k-mooney gibi: bauzas was ment to be getting fiber a few months ago but there were issues
15:59:24 bauzas sean-k-mooney: I did not implemented it on purpose since mdev types are passed directly from the kernel driver without any kind of abstractional outcome
15:59:49 sean-k-mooney bauzas: sure i dont really thing that is a proablem
15:59:56 bauzas from my position, it is
16:00:11 sean-k-mooney we dont really have an abstration for cpu flags

Earlier   Later