| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-07-14 | |||
| 16:00:54 | NobodyCam | ahh Thank you melwitt !! | |
| 16:01:09 | NobodyCam | been looking at nova db. haven't even looked at the api side ;p | |
| 16:06:25 | melwitt | was this bug https://bugs.launchpad.net/nova/+bug/1784093 | |
| 16:06:25 | melwitt | NobodyCam: yeah.. you likely don't have any trace of the instance in the nova db. usually the reason you can see the instance in the 'server list' but not the 'server show' is because 'server list' will count build_requests as instances as well in order to preserve the behavior where you can see an instance immediately after you request to boot it. but if there's no instance_mappings record, nova will think it can't find anything. it | |
| 16:06:37 | melwitt | are you running a version older than queens? | |
| 16:09:16 | NobodyCam | actually in this region it is Queens, it's in process of migrating to ussuri but not complete yet | |
| 16:10:35 | melwitt | hm, ok. there might be another corner case but ^ was the most common one | |
| 16:11:22 | melwitt | released in 17.0.11 | |
| 16:20:31 | NobodyCam | Interesting not in build_requests but did find reference to the instance_id in instance_mappings table in nova_api DB | |
| 16:24:00 | melwitt | oh, so the inverse? hm | |
| 16:25:21 | NobodyCam | I should not this region had a network outage, caused some strange things, I am attempting to recover | |
| 16:25:32 | NobodyCam | snot/note/ | |
| 16:26:11 | melwitt | I need to go back through and look at all those again, there's some places where if one db write succeeds while another fails, you get these bad states | |
| 16:26:28 | NobodyCam | Safe to just nuke the instance_mapping record? | |
| 16:27:25 | melwitt | yeah, I'd make sure it's not in nova.instances and then delete nova_api.instance_mappings, nova_api.request_specs for the instance uuid | |
| 16:27:58 | NobodyCam | : thumbs_up : | |
| 16:42:19 | NobodyCam | Thank you melwitt I think this will get things back on track for us! | |
| 16:43:22 | melwitt | np, good luck | |
| 17:18:27 | gmann | gibi: dansmith replied on these comment for 'project admin getting hypervisor uuid spec' https://review.opendev.org/c/openstack/nova-specs/+/793011/3/specs/xena/approved/allow-project-admin-list-hypervisors.rst#43 | |
| 17:18:29 | gmann | gibi: dansmith https://review.opendev.org/c/openstack/nova-specs/+/793011/3/specs/xena/approved/allow-project-admin-list-hypervisors.rst#49 | |
| 17:18:54 | gmann | can you please confirm/reply for those, accordingly I will update spec | |
| 17:20:07 | gmann | sorry for responding late on this | |
| 17:31:27 | opendevreview | melanie witt proposed openstack/nova stable/train: [CI] Fix gate by using zuulv3 live migration and grenade jobs https://review.opendev.org/c/openstack/nova/+/795435 | |
| 17:33:09 | opendevreview | Stephen Finucane proposed openstack/nova master: nova-manage: Introduce bdm show, refresh, get_connector commands https://review.opendev.org/c/openstack/nova/+/800634 | |
| 18:25:38 | opendevreview | Ghanshyam proposed openstack/nova-specs master: Allow project admin to list hypervisors https://review.opendev.org/c/openstack/nova-specs/+/793011 | |
| 18:26:05 | gmann | dansmith: ^^ updated | |
| 18:28:31 | dansmith | gmann: okay just to be clear, you're planning to accept the uuid as the hostname, in the same field, and without a microversion for that? | |
| 18:28:55 | gmann | dansmith: yeah. | |
| 18:29:47 | dansmith | okay I guess I dunno about the legitimacy of that.. because someone won't be able to know whether a nova is new enough to accept that behavior. it's not a structural change in the request/response, but it is a different behavior that they won't know if they can use or not, right? | |
| 18:31:23 | gmann | dansmith: humm, behavior wise yes it is changed.. | |
| 18:32:42 | dansmith | yeah, so I dunno, probably best to get someone else's opinion on the matter, but it sure seems like that should go hand-in-hand with the microversion to expose it | |
| 18:34:25 | gmann | issue with microversion bump is then it would not be aligned with policy change which are without microversion. | |
| 18:34:48 | sean-k-mooney | why are we over loading the filed | |
| 18:35:00 | gmann | I think booting server with uuid will end up with error ? | |
| 18:35:06 | sean-k-mooney | i can kid of understand at the osc level allowign --host | |
| 18:35:12 | gmann | currently | |
| 18:35:13 | sean-k-mooney | to be the uuid or hostname | |
| 18:35:25 | gmann | humm | |
| 18:35:25 | sean-k-mooney | but at the api that feels weird to me | |
| 18:35:55 | gmann | testing here https://review.opendev.org/c/openstack/tempest/+/793632 | |
| 18:36:06 | sean-k-mooney | we have instance of this i think for instnace show or flavor show where you can pass the name or uuid | |
| 18:37:22 | sean-k-mooney | but i kind of assume we were goning to add a new filed for this | |
| 18:38:18 | sean-k-mooney | its that not we ment by https://review.opendev.org/c/openstack/nova-specs/+/793011/4/specs/xena/approved/allow-project-admin-list-hypervisors.rst#97 | |
| 18:38:36 | gmann | sean-k-mooney: for this case, we are thinking to allow in same field and that is why interop issue | |
| 18:38:56 | sean-k-mooney | well we said we would accpet hypervisor-uuid now | |
| 18:39:09 | sean-k-mooney | so that is a new field no? | |
| 18:39:17 | sean-k-mooney | i have not been following this closely sorry | |
| 18:39:42 | gmann | sean-k-mooney: no, in same field like in 'availability_zone' az:noda:host host as uuid | |
| 18:40:00 | gmann | if new field then sure we need microversion bump | |
| 18:41:10 | sean-k-mooney | i kind of feel like this should be a micoverion bump | |
| 18:41:48 | sean-k-mooney | i mean its unlikely you are suing uuids for your hypervior host names but it would have been allowed before | |
| 18:41:57 | sean-k-mooney | actully dont we do that for ironic | |
| 18:42:17 | sean-k-mooney | the hypervior_hostname is the ironic node uuid | |
| 18:42:56 | dansmith | but that is the actual hostname we record, | |
| 18:42:58 | dansmith | so it's different | |
| 18:43:05 | sean-k-mooney | so if we reuse the same filed dont we have the possibliyt of a uuid colission even thought that wont happen in reality | |
| 18:43:15 | opendevreview | Merged openstack/nova master: Use neutronclient's port binding APIs https://review.opendev.org/c/openstack/nova/+/706295 | |
| 18:43:48 | sean-k-mooney | dansmith: well for ironic server the hypervior hostname for each server is the uuid right | |
| 18:44:18 | dansmith | right, that's my point.. so it's not like there's prior art for making them interchangeable, it's that we actually record the uuid *as* the hostname in that case | |
| 18:44:38 | sean-k-mooney | yes | |
| 18:44:57 | sean-k-mooney | do we also make the compute node uuid match the host name for ironic? | |
| 18:46:12 | dansmith | yeah, I think that changed at some point and now we do | |
| 18:46:21 | sean-k-mooney | yes https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L354-L355 | |
| 18:46:33 | sean-k-mooney | ya i dont think it always was either but it is now | |
| 18:46:51 | sean-k-mooney | https://github.com/openstack/nova/commit/9f28727eb75e05e07bad51b6eecce667d09dfb65 | |
| 18:47:12 | sean-k-mooney | to fix https://bugs.launchpad.net/nova/+bug/1771806 | |
| 18:48:50 | sean-k-mooney | gmann: that tempest test https://review.opendev.org/c/openstack/tempest/+/793632/7/tempest/scenario/test_server_multinode.py is still using the old way to select hosts | |
| 18:48:58 | sean-k-mooney | using the az hack no? | |
| 18:49:43 | gmann | sean-k-mooney: adding host uuid there https://review.opendev.org/c/openstack/tempest/+/793632/7/tempest/scenario/test_server_multinode.py#61 | |
| 18:50:17 | sean-k-mooney | but you are using host_name | |
| 18:51:02 | sean-k-mooney | not host or hypervisor_hostname | |
| 18:51:32 | sean-k-mooney | its still computing the az | |
| 18:51:34 | sean-k-mooney | https://review.opendev.org/c/openstack/tempest/+/793632/7/tempest/scenario/test_server_multinode.py#82 | |
| 18:52:21 | sean-k-mooney | its not useing https://specs.openstack.org/openstack/nova-specs/specs/train/implemented/add-host-and-hypervisor-hostname-flag-to-create-server.html | |
| 18:52:45 | gmann | host_name is what i changed to add uuid in L61 | |
| 18:53:00 | gmann | host['host_name'] is uuid | |
| 18:53:24 | gmann | I remember it failed with uuid but double checking | |
| 18:53:58 | sean-k-mooney | so using the az as this test is is more or less deprecated | |
| 18:54:12 | sean-k-mooney | we can test this but we shoudl be testing with 5.74 | |
| 18:54:18 | sean-k-mooney | and not seeting the az at all | |
| 18:54:20 | gmann | sean-k-mooney: I am testing with AZ case for force host. 'host', 'hostname' which use sch I am not testing currently | |
| 18:54:49 | gmann | AZ way is not deprecated right? | |
| 18:55:00 | sean-k-mooney | its stognly discuaged | |
| 18:55:01 | gmann | create server support both | |
| 18:55:13 | sean-k-mooney | its not deperecated and i dont think we want to extend its usage | |
| 18:55:26 | gmann | yeah discuaged but not deprecated | |
| 18:55:30 | sean-k-mooney | i would like to deprecated it | |
| 18:55:51 | gmann | I am just checking compatibility of uuid case | |
| 18:55:58 | sean-k-mooney | its replacement has been avaiable since train | |
| 18:57:18 | sean-k-mooney | gmann: well i gues my point is i dont think we should be supporting this the old way | |
| 18:57:55 | sean-k-mooney | im fine with project admins requesting a host using the info form the hypervior api to get the uuid | |
| 18:58:09 | sean-k-mooney | but im not sure they shoudl be able to force the host and bypass the schudler filters | |
| 18:59:17 | gmann | sean-k-mooney: sure. but in current proposed change I am checking uuid compatibility. But if we want to disallow the bypass the schudler filters that is separate behavior change or we can say 'bypass the schudler filters stop working with new rbac' | |
| 19:00:04 | sean-k-mooney | no i dont think that is really valid. | |
| 19:00:14 | sean-k-mooney | its an iterup issue | |
| 19:00:22 | sean-k-mooney | i cant tell if you are using the new rbac or not | |
| 19:00:34 | gmann | I think we can discuss this separately to deprecate/stop the 'bypass the schudler filters' case instead of mixing two | |
| 19:00:44 | sean-k-mooney | which is why i have argrued that any policy change should be a micorover bump | |
| 19:01:13 | sean-k-mooney | the old az format can work without any nova code changes right | |
| 19:01:19 | sean-k-mooney | with the uuid becaue it accpeted both | |