Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-12-12
17:17:26 mriedem your change is merged,
17:17:29 mriedem which is why i'm in rebase hell
17:18:04 mriedem i'll add it when i move onto reworking one of these
17:18:16 mriedem https://review.opendev.org/#/q/path:%255Enova/tests/functional/.*+status:open+owner:self+branch:master :(
17:18:24 mriedem i guess that won't work for others...
17:18:35 mriedem there we go https://review.opendev.org/#/q/path:%255Enova/tests/functional/.*+status:open+owner:mriedem+branch:master
17:19:16 stephenfin /o\ sorry
17:19:20 stephenfin you're gonna hate https://review.opendev.org/#/c/697537/ so
17:19:29 stephenfin I can respin those for you, if you want
17:19:32 stephenfin seeing as I broke em
17:19:39 mriedem i'll do it
17:19:52 mriedem i realize to make an omelette you have to break a few eggs
17:20:06 mriedem but these are my faberge eggs
17:26:54 pmatulis how do i know what command will provide a known attribute, such as 'OS-EXT-SRV-ATTR:kernel_id'? the 'server show' command does not show this
17:27:53 mriedem you're likely not using a high enough microversion
17:28:04 mriedem you need >= 2.3 for that
17:28:12 mriedem openstack --os-compute-api-version 2.3 server show <server>
17:28:20 mriedem https://docs.openstack.org/api-ref/compute/?expanded=show-server-details-detail#show-server-details
17:28:32 mriedem tells you the response params and required min versions
17:30:09 pmatulis mriedem, wow ok, i've never encountered that option before. thank you
17:30:49 melwitt TheJulia: sorry, what did you mean about the cpu_arch? it
17:32:56 melwitt TheJulia: *it's the opposite, we take the cpu_arch from the ironic node properties to use as an advertised capability of a compute, for scheduling. what is the correct thing for us to do if an ironic node has _no_ cpu_arch in it's node properties? today we are advertising no cpu_arch in that case, which means any glance image with arch specified will not match an arch-less host
17:34:05 sean-k-mooney melwitt: do you need a filter to get that behavior?
17:34:26 melwitt sean-k-mooney: yeah it's the ImagePropertiesFilter
17:34:42 sean-k-mooney oh i assumeed the computecapablitys filter
17:34:43 melwitt which is enabled by default
17:35:00 sean-k-mooney the reason i was asking is that shoudl move to placemnt at some point
17:35:20 sean-k-mooney cpu architure is the perfect thing to model as a trait
17:35:22 melwitt I'm having a really hard time finding out what we should do in the case when an ironic node has no cpu_arch specified in it
17:36:00 sean-k-mooney well if the value is not set in the hoststate object we could just allow the host
17:36:01 melwitt we've been doing the "no match" thing forever, since the ironic driver code was first added. but recently it seems to be a problem
17:36:29 sean-k-mooney it may fail but if we dont have the info we cant really do the right thing either way
17:36:31 melwitt yeah, that's what I was wondering if we should use arch.ALL if there's nothing in the ironic node
17:36:59 sean-k-mooney well maybe a .ANY would be better
17:37:33 sean-k-mooney its a arch is a feild object so .ALL would be the list of all arches
17:37:43 melwitt I thought that's how we would get an "any" behavior is by saying "this node supports all archs"
17:38:29 sean-k-mooney yes but .ALL is a list not a singel value
17:38:36 sean-k-mooney so the types would not match
17:39:20 sean-k-mooney well maybe not https://github.com/openstack/nova/blob/master/nova/objects/fields.py#L172
17:40:13 melwitt I was thinking that based on this https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L48 but maybe I'm missing something
17:40:16 sean-k-mooney look like . all is https://github.com/openstack/nova/blob/master/nova/virt/arch.py#L57
17:42:40 sean-k-mooney yes so that function just returns a single value
17:43:01 sean-k-mooney i think
17:44:13 sean-k-mooney oh no you are right
17:44:31 sean-k-mooney its arch.ALL https://github.com/openstack/nova/blob/master/nova/conf/scheduler.py#L552-L566
17:44:43 melwitt no, I think you're right. it's choices=arch.ALL so you have to pick one
17:44:52 sean-k-mooney oh yes
17:44:56 sean-k-mooney its choice not default
17:45:04 melwitt you have to set the default, there's no default. which confused me xD
17:45:46 sean-k-mooney ya me too, so looking at how its used https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L52-L54
17:46:21 sean-k-mooney if you dont set the config option and its not in the image image_arch will be teh empty stirng or None so it will be Falsy
17:46:26 melwitt the thing that's bugging me is, the glance images must have arch set in this NoValidHost case. and I was thinking, if you want anything to match, shouldn't you just not put arch on the glance image props? but I don't know that much about glance images and whether or not that is thing that could make sense
17:46:37 sean-k-mooney and we would bailout on line 64
17:47:19 sean-k-mooney melwitt: although perhaps that is being done automatically by inspection of the image
17:47:45 sean-k-mooney i think with the new image import workflow in glance you can have hooks to add metadta automatically
17:47:50 sean-k-mooney so maybe that is what is happening
17:47:54 melwitt ohhh.. hm
17:48:15 sean-k-mooney i dont know if that is one of the things that can be set by the way
17:48:21 sean-k-mooney its just a guess
17:48:43 melwitt downstream they were saying that in queens this same scenario worked fine and starting in rocky it fails
17:49:15 melwitt and I don't find anything changed in the code in like 5 years. so it feels like it has to be a difference in the image props
17:49:16 stephenfin lyarwood: It's late now, but think you could hit [1] tomorrow? [1] https://review.opendev.org/#/q/topic:bp/amd-sev-libvirt-support+(status:open+OR+status:merged)+branch:stable/train
17:49:44 lyarwood stephenfin: yeah, I might even hit it tonight while watching the results come in.
17:50:15 sean-k-mooney melwitt: ya do you know what code path they are hitting is it the return False on line 75 or 104
17:50:41 sean-k-mooney melwitt: i assume 75?
17:51:41 melwitt sean-k-mooney: no I don't. I'd have to see if the sosreport is still around to look
17:52:19 melwitt (if debug logging is enabled)
17:52:31 sean-k-mooney well from inspection if the image had an arch request and the ironci node did not have detail i think it would fail as written
17:53:00 sean-k-mooney removing the metadata form the image should work if they do not set the config option
17:53:34 sean-k-mooney but im kind of wondering if https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L75 shoudl be configurable
17:53:43 melwitt that was my conclusion as well. but I don't have an answer yet about what is going on with the images they're using
17:55:18 melwitt configurable ... I don't like the sound of that.
17:56:07 sean-k-mooney melwitt: i was thinking a workaround config option for backports only
17:56:43 sean-k-mooney incase they need the arch for some other reason to be set in the image
17:57:41 sean-k-mooney but i think the correct thing to do would be for them to update the ironic nodes with the arch in that case
17:58:24 sean-k-mooney an arm image wont work on an x86 host after all so really we should always know the arch of a compute node
17:58:40 dansmith stephenfin: I might have gotten out of sync, but .. is the qga thing just "also broken" in addition to the scsi device one?
17:59:39 melwitt sean-k-mooney: yeah ... I don't really get the use case. it's a scsi deploy and they say it's single arch so for simplicity they don't want to specify the arch
18:00:06 melwitt and apparently they used to be able to prior to rocky, which I don't know how that's possible unless the images had no arch metadata before
18:00:42 sean-k-mooney ya i think that is the most likely reason
18:00:59 dansmith okay I see the qga was in the original note about being broken too, I had just always thought it was the virtio-scsi thing that was the big deal
18:01:02 sean-k-mooney im just going to check my local openstack install
18:01:32 sean-k-mooney virtio scisi virtio gpu and qga
18:02:14 stephenfin dansmith: Yeah, it was multiple things
18:02:15 sean-k-mooney but yes all virtio devices just need the iommu driver opt and they orginally missed a few
18:02:28 sean-k-mooney those patches all meged this moring right
18:02:28 dansmith stephenfin: yeah I see now
18:02:36 stephenfin cool
18:03:36 sean-k-mooney melwitt: i just checked my home deployment and by default it looks like the arch does not get added
18:03:56 sean-k-mooney melwitt: glance is added os_hash_value and os_hash_algo by default now
18:04:27 melwitt sean-k-mooney: arch does not get added to images automatically you're saying?
18:04:51 sean-k-mooney correct, so its doing some things on import by default but adding the arch is not one of them
18:05:06 sean-k-mooney you might be able to confgiure it but that does not seem to be the defualt upstream behavior
18:05:09 melwitt gah, I wonder what happened then
18:06:11 melwitt the bug was opened several months ago but was already marked Triaged so we didn't see it until recently. gonna be hard to get answers this far later
18:07:51 sean-k-mooney so i was assuming that it might be somethin like https://docs.openstack.org/glance/latest/admin/interoperable-image-import.html#the-image-property-injection-plugin
18:08:45 sean-k-mooney but they would have had to enable it and specyif the arch
18:09:16 sean-k-mooney i dont see one that use libguestfs or soemthing else to inspec the image and add things like teh arch but it might exists
18:09:56 sean-k-mooney the plugin interfce was added so operators could do things like that or scan for vulnerablity on import ectra
18:10:19 sean-k-mooney but its much simpler to assume a human did it

Earlier   Later