| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-12-12 | |||
| 16:07:16 | sean-k-mooney | so when the compute agent reconnect it does not always recreate it queue | |
| 16:07:29 | sean-k-mooney | you can fix it by restarting the compute agent | |
| 16:07:47 | dansmith | sean-k-mooney: I'm all for fixing bugs in oslo.messaging, but that has nothing to do with us trying to hack around a stuck instance state | |
| 16:07:56 | sean-k-mooney | the mandaotry flag woudl allow the conductor to detect that the queue was not present and we could make the compute node as down | |
| 16:08:22 | dansmith | jesus lord, pelase no | |
| 16:08:24 | sean-k-mooney | dansmith: https://bugs.launchpad.net/oslo.messaging/+bug/1661510 | |
| 16:08:24 | openstack | Launchpad bug 1661510 in oslo.messaging "topic_send may loss messages if the queue not exists" [Medium,In progress] - Assigned to Gabriele Santomaggio (gsantomaggio) | |
| 16:08:47 | sean-k-mooney | dansmith: well we dont have to mark it as down but it would be nice to at least log the fact | |
| 16:08:47 | dansmith | rabbit has split brain, one conductor thinks the compute queue is gone, and starts marking computes down? | |
| 16:09:12 | sean-k-mooney | ya marking as down is proably an over reaction | |
| 16:09:19 | dansmith | log what we know, not what we guess.. so sure, "hmm, seems like there should be a compute queue here..." | |
| 16:11:06 | sean-k-mooney | anyway the oslo change have merged but we are not using them in nova yet | |
| 16:27:59 | mriedem | oof https://review.opendev.org/#/c/697694/ ruins my life | |
| 16:29:29 | gibi | there is no progress without pain ;) | |
| 16:30:00 | mriedem | yeah yeah | |
| 16:30:27 | gibi | I will feel the pain when I have to backport functional tests | |
| 16:30:40 | mriedem | this is the cross cell thing, so no pain for you there | |
| 16:30:49 | mriedem | but i've got a lot of patches to sift through and update | |
| 16:31:10 | gibi | yeah I figured out that you now have a nice big merge conflict | |
| 16:31:13 | efried | dansmith: we finally got a grenade fail here https://review.opendev.org/692402 | |
| 16:31:18 | efried | It was my magic touch | |
| 16:31:43 | mriedem | gibi: the merge conflict is trivial, it's the behavior/interface change | |
| 16:31:43 | efried | (hopefully it was for the issue we care about) | |
| 16:33:59 | melwitt | sean-k-mooney: I read the backscroll about the rabbit thing and I'm wondering, was there not a way that oslo.messaging could have recreated the missing queue? I wonder why restarting our service is the only way? | |
| 16:34:54 | mriedem | gibi: after https://review.opendev.org/#/c/695905/ isn't there supposed to be a patch to enable live migration with qos ports in the api? | |
| 16:35:01 | mriedem | with a release note and docs update and such? | |
| 16:35:36 | mriedem | oh nvm i see, "subsequent patches will add support for migration with target host and other edge case like reschedule." | |
| 16:35:38 | gibi | mriedem: that patch only supporst the happy path. I'm working on the re-schedule cases | |
| 16:35:43 | mriedem | ack | |
| 16:35:46 | gibi | mriedem: yeah | |
| 16:35:51 | gibi | stay tuned :) | |
| 16:36:18 | sean-k-mooney | melwitt: it proably could although we coudl also proably try to do it if we got the error that it was not deliverable | |
| 16:36:58 | sean-k-mooney | melwitt: i think they would prefer if we as the client of oslo.messaging did that since they dont want to assume it is correct | |
| 16:37:55 | dansmith | efried: okay, did you look for any juiciness? | |
| 16:38:06 | dansmith | efried: I can try to look through there later, but I barely even remember what the deal was | |
| 16:38:32 | efried | dansmith: I didn't look at all, just continue to be very interested in making that f'in bug go away. | |
| 16:38:51 | efried | but I never really understood the original issue, so not sure it would do anybody much good for me to dig. | |
| 16:52:17 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: Spec: Ussuri: Encrypted Emulated Virtual TPM https://review.opendev.org/686804 | |
| 16:53:06 | efried | stephenfin, gibi: That's ready for your (hopefully final) nod ^ | |
| 16:53:15 | gibi | efried: ack | |
| 16:53:19 | stephenfin | cool | |
| 16:53:21 | efried | This iteration is simpler in terms of weirdnesses | |
| 16:53:33 | efried | and fleshes out some of the interplay necessary for object store | |
| 16:53:39 | efried | jroll: ^ | |
| 16:53:59 | efried | TxGirlGeek: Spec: Ussuri: Encrypted Emulated Virtual TPM https://review.opendev.org/686804 | |
| 17:00:58 | TxGirlGeek | @efried Check | |
| 17:13:18 | mriedem | stephenfin: so your change to _build_minimal_create_server_request | |
| 17:13:24 | mriedem | to remove the API that's passed in, | |
| 17:13:33 | mriedem | that's a problem for any testing doing multi-tenant testing | |
| 17:13:38 | mriedem | like https://review.opendev.org/#/c/697331/4/nova/tests/functional/test_scheduler.py | |
| 17:13:44 | mriedem | *any test | |
| 17:14:14 | mriedem | so to avoid temporarily mutating the self.api in these types of tests we're going to have to add an api kwarg or something | |
| 17:14:33 | stephenfin | I thought I did, no? | |
| 17:14:38 | stephenfin | looking | |
| 17:15:17 | mriedem | nope https://review.opendev.org/#/c/697694/1/nova/tests/functional/integrated_helpers.py@a98 | |
| 17:16:20 | stephenfin | okay, that's a good point. Can you -1 that and I'll respin with the api parameter | |
| 17:16:20 | stephenfin | I had it for some of the helpers to deal with that but clearly missed this one | |
| 17:16:20 | stephenfin | *missed some | |
| 17:17:13 | mriedem | same https://review.opendev.org/#/c/694179/2/nova/tests/functional/test_external_networks.py | |
| 17:17:26 | mriedem | your change is merged, | |
| 17:17:29 | mriedem | which is why i'm in rebase hell | |
| 17:18:04 | mriedem | i'll add it when i move onto reworking one of these | |
| 17:18:16 | mriedem | https://review.opendev.org/#/q/path:%255Enova/tests/functional/.*+status:open+owner:self+branch:master :( | |
| 17:18:24 | mriedem | i guess that won't work for others... | |
| 17:18:35 | mriedem | there we go https://review.opendev.org/#/q/path:%255Enova/tests/functional/.*+status:open+owner:mriedem+branch:master | |
| 17:19:16 | stephenfin | /o\ sorry | |
| 17:19:20 | stephenfin | you're gonna hate https://review.opendev.org/#/c/697537/ so | |
| 17:19:29 | stephenfin | I can respin those for you, if you want | |
| 17:19:32 | stephenfin | seeing as I broke em | |
| 17:19:39 | mriedem | i'll do it | |
| 17:19:52 | mriedem | i realize to make an omelette you have to break a few eggs | |
| 17:20:06 | mriedem | but these are my faberge eggs | |
| 17:26:54 | pmatulis | how do i know what command will provide a known attribute, such as 'OS-EXT-SRV-ATTR:kernel_id'? the 'server show' command does not show this | |
| 17:27:53 | mriedem | you're likely not using a high enough microversion | |
| 17:28:04 | mriedem | you need >= 2.3 for that | |
| 17:28:12 | mriedem | openstack --os-compute-api-version 2.3 server show <server> | |
| 17:28:20 | mriedem | https://docs.openstack.org/api-ref/compute/?expanded=show-server-details-detail#show-server-details | |
| 17:28:32 | mriedem | tells you the response params and required min versions | |
| 17:30:09 | pmatulis | mriedem, wow ok, i've never encountered that option before. thank you | |
| 17:30:49 | melwitt | TheJulia: sorry, what did you mean about the cpu_arch? it | |
| 17:32:56 | melwitt | TheJulia: *it's the opposite, we take the cpu_arch from the ironic node properties to use as an advertised capability of a compute, for scheduling. what is the correct thing for us to do if an ironic node has _no_ cpu_arch in it's node properties? today we are advertising no cpu_arch in that case, which means any glance image with arch specified will not match an arch-less host | |
| 17:34:05 | sean-k-mooney | melwitt: do you need a filter to get that behavior? | |
| 17:34:26 | melwitt | sean-k-mooney: yeah it's the ImagePropertiesFilter | |
| 17:34:42 | sean-k-mooney | oh i assumeed the computecapablitys filter | |
| 17:34:43 | melwitt | which is enabled by default | |
| 17:35:00 | sean-k-mooney | the reason i was asking is that shoudl move to placemnt at some point | |
| 17:35:20 | sean-k-mooney | cpu architure is the perfect thing to model as a trait | |
| 17:35:22 | melwitt | I'm having a really hard time finding out what we should do in the case when an ironic node has no cpu_arch specified in it | |
| 17:36:00 | sean-k-mooney | well if the value is not set in the hoststate object we could just allow the host | |
| 17:36:01 | melwitt | we've been doing the "no match" thing forever, since the ironic driver code was first added. but recently it seems to be a problem | |
| 17:36:29 | sean-k-mooney | it may fail but if we dont have the info we cant really do the right thing either way | |
| 17:36:31 | melwitt | yeah, that's what I was wondering if we should use arch.ALL if there's nothing in the ironic node | |
| 17:36:59 | sean-k-mooney | well maybe a .ANY would be better | |
| 17:37:33 | sean-k-mooney | its a arch is a feild object so .ALL would be the list of all arches | |
| 17:37:43 | melwitt | I thought that's how we would get an "any" behavior is by saying "this node supports all archs" | |
| 17:38:29 | sean-k-mooney | yes but .ALL is a list not a singel value | |
| 17:38:36 | sean-k-mooney | so the types would not match | |
| 17:39:20 | sean-k-mooney | well maybe not https://github.com/openstack/nova/blob/master/nova/objects/fields.py#L172 | |
| 17:40:13 | melwitt | I was thinking that based on this https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L48 but maybe I'm missing something | |
| 17:40:16 | sean-k-mooney | look like . all is https://github.com/openstack/nova/blob/master/nova/virt/arch.py#L57 | |