| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-08-30 | |||
| 18:10:57 | dansmith | I mean it doesn't catch that case, but should | |
| 18:11:54 | sean-k-mooney | so the bit im conflicted by is Verify if volume exists and cinder has an attachment ID else delete its BDMS data from nova DB and vice versa. | |
| 18:12:18 | sean-k-mooney | the "and cinder has an attachment ID" part | |
| 18:13:10 | dansmith | what about it? | |
| 18:13:35 | sean-k-mooney | i was not expecting use to delete the BDM if the volume existiend but the atachment was gone | |
| 18:13:57 | dansmith | why not? that means it's been detached underneath us | |
| 18:14:10 | dansmith | (/me will go poof in a few minutes without warning) | |
| 18:14:10 | sean-k-mooney | whihc we dont allow so we shoudl repair it? | |
| 18:14:20 | sean-k-mooney | or error if you attached it to somethign else | |
| 18:14:36 | dansmith | we should heal ourselves and not require admin intervention | |
| 18:14:39 | dansmith | which is what is needed right now | |
| 18:15:20 | sean-k-mooney | in the case the volume is deleted i agree | |
| 18:15:35 | sean-k-mooney | im unconfrotabel with just the case wehre the attachment is deleted | |
| 18:15:44 | sean-k-mooney | perhaps that is valid | |
| 18:15:45 | dansmith | why? from our perspective it's the same | |
| 18:16:13 | dansmith | the state of the volume has been mucked with undereath us, so bail | |
| 18:16:31 | dansmith | don't try to hook it back up or recover in any other way than just avoiding our own problems in the future | |
| 18:16:40 | sean-k-mooney | so i woudl error in the attachment delete case personslly and say no you srewed with thsi you need ot manuall fix it | |
| 18:16:52 | dansmith | that's exactly what we're trying to avoid | |
| 18:17:30 | dansmith | because the user can't manually fix it | |
| 18:17:58 | dansmith | just like when we go to delete something and it's already deleted, we say "okay fair enough" and move on | |
| 18:18:16 | dansmith | these are distributed systems.. nova's DB could have been restored from ten minutes ago, etc | |
| 18:18:20 | sean-k-mooney | well the admin can with the attachemtn refesh command. i was debating if we shoudl do the refresh in the case the voluem is aviabale but i get you dont want that complexity | |
| 18:19:05 | sean-k-mooney | i guess the fact the attachment api needs service user | |
| 18:19:09 | sean-k-mooney | tokens now | |
| 18:19:26 | sean-k-mooney | fixes most of the concerns i have | |
| 18:19:50 | sean-k-mooney | as long as we are still very clear that if you detach a volume form cidner or a prot form neutorn your isntance is tainted | |
| 18:20:13 | sean-k-mooney | and all we are doing is minimal efffort to boot again | |
| 18:20:16 | sean-k-mooney | i guess that is ok | |
| 18:25:57 | sean-k-mooney | ill look at this again tomorooww but without the service user chagne i dont this woudl have been the correct approch to take. | |
| 18:35:11 | dansmith | sean-k-mooney: right the point in this is very specifically to *not* require the admin to intervene when the only thing they can reasonably do is run a command to do the same thing, which is eliminate the BDM from nova's side | |
| 18:35:22 | dansmith | we wouldn't want the admin to "fix" it (IMHO) | |
| 18:36:04 | dansmith | and especially with the cinder policy change, the only real thing that could cause this workflow-wise would be a database restore from checkpoint such that the two sides are out of sync, or an admin operation on the cinder side that just then needs to be mirrored with admin operations on the nova side | |
| 18:36:30 | dansmith | it's not to _enable_ any sort of bad behavior by the user, it's to avoid the user being stuck and unable to get their instance back | |
| 18:36:55 | dansmith | for self-service reasons, I'd much prefer my volume to just become detached and me still be able to work on my instance without waiting in an admin's queue | |
| 18:46:12 | opendevreview | Merged openstack/nova master: Add a new NumInstancesWeigher https://review.opendev.org/c/openstack/nova/+/886232 | |
| 19:23:52 | opendevreview | melanie witt proposed openstack/nova master: Add documentation for unified limits https://review.opendev.org/c/openstack/nova/+/893127 | |
| 19:23:52 | opendevreview | melanie witt proposed openstack/nova master: nova-manage: Add 'limits migrate_to_unified_limits' https://review.opendev.org/c/openstack/nova/+/891646 | |
| 19:34:05 | opendevreview | melanie witt proposed openstack/nova master: Add documentation for unified limits https://review.opendev.org/c/openstack/nova/+/893127 | |
| 19:36:11 | sean-k-mooney | dansmith: ok i can bye that. i kind of feel like in addtion to this we show have a external event form cidner saying its detach or something like that to make it properly automatic but again with the service token change that shoudl not be reuqired either. | |
| 19:36:46 | dansmith | to be clear, I do not think we should _ever_ tolerate detach via cinder, thus no need for an event | |
| 19:36:58 | dansmith | this is a corruption recovery mechanism and nothing else | |
| 19:36:59 | sean-k-mooney | so ill remove my -1 on the repoducer. i still have not looked at the actual final patch however | |
| 19:37:14 | sean-k-mooney | ack | |
| #openstack-nova - 2023-08-31 | |||
| 02:00:15 | opendevreview | melanie witt proposed openstack/nova master: Enable use of native threads in scatter_gather_cells https://review.opendev.org/c/openstack/nova/+/650172 | |
| 02:47:02 | opendevreview | Merged openstack/nova master: libvirt: Add 'COMPUTE_ADDRESS_SPACE_*' traits support https://review.opendev.org/c/openstack/nova/+/873221 | |
| 04:58:50 | opendevreview | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.opendev.org/c/openstack/nova/+/891648 | |
| 06:11:37 | opendevreview | Amit Uniyal proposed openstack/nova master: Reproducer for dangling bdms https://review.opendev.org/c/openstack/nova/+/889947 | |
| 06:11:38 | opendevreview | Amit Uniyal proposed openstack/nova master: Delete dangling bdms https://review.opendev.org/c/openstack/nova/+/882284 | |
| 06:50:23 | auniyal | thanks sean-k-mooney for the reviews | |
| 07:21:34 | opendevreview | Merged openstack/nova master: Remove deprecated AZ filter. https://review.opendev.org/c/openstack/nova/+/886779 | |
| 08:08:05 | opendevreview | Amit Uniyal proposed openstack/nova master: Delete dangling bdms https://review.opendev.org/c/openstack/nova/+/882284 | |
| 08:55:57 | opendevreview | John Garbutt proposed openstack/nova master: Deprecate ironic.peer_list https://review.opendev.org/c/openstack/nova/+/883346 | |
| 08:56:11 | opendevreview | John Garbutt proposed openstack/nova master: Deprecate ironic.peer_list https://review.opendev.org/c/openstack/nova/+/883346 | |
| 11:21:59 | auniyal | Hi sean-k-mooney, tat for +W this https://review.opendev.org/c/openstack/nova/+/889947/7, | |
| 11:22:19 | auniyal | but the dependent patch is not +W yet, its has all +2's https://review.opendev.org/c/openstack/nova/+/881457/35 | |
| 11:34:45 | opendevreview | sean mooney proposed openstack/nova master: increase job timeout for ocational slow nodes https://review.opendev.org/c/openstack/nova/+/893360 | |
| 11:41:14 | opendevreview | sean mooney proposed openstack/nova master: increase job timeout for ocational slow nodes https://review.opendev.org/c/openstack/nova/+/893360 | |
| 11:41:43 | sean-k-mooney | auniyal: thats intentionaly we will +w the previsou patch after we are happy with the final one | |
| 11:41:57 | sean-k-mooney | and merge the seriese together | |
| 11:42:05 | auniyal | ack, | |
| 11:43:46 | auniyal | but it won't fail merging the reproducer one (something like merge conflict) - because reproducer is over that one | |
| 11:46:19 | sean-k-mooney | no it should not | |
| 12:42:00 | bauzas | auniyal: have you seen the -1 from sean-k-mooney ? | |
| 12:42:05 | bauzas | on the top change ? | |
| 12:42:13 | dvo-plv | sean-k-mooney, Hello. Are you here ? | |
| 12:48:05 | Uggla | hello, I have some troubles with the service token, my assumption is cinder.py, neutron.py, ... were modified to use service token by calling service_auth.get_auth_plugin method. But I think this was not done for the services using the sdk. Can you confirm this assumption ? | |
| 12:52:30 | sean-k-mooney | bauzas: if your around can you take a look at https://review.opendev.org/c/openstack/nova/+/893360 i have seen 1 or two timeouts so this might be good to land | |
| 12:54:29 | auniyal | bauzas, yes, updating | |
| 12:54:36 | auniyal | was afk so late reply, | |
| 12:55:03 | bauzas | sean-k-mooney: well, today, I'm litterrally sitting in front of Gerrit | |
| 12:55:26 | bauzas | I still have melwitt's series on unified-limits nova-manage to look at | |
| 12:57:18 | sean-k-mooney | i can respin it now it will take 30 seconds | |
| 12:59:11 | bauzas | cool | |
| 12:59:22 | opendevreview | sean mooney proposed openstack/nova master: increase job timeout for occasional slow nodes https://review.opendev.org/c/openstack/nova/+/893360 | |
| 13:00:32 | sean-k-mooney | ok i need to swap to somehting else. just an fyi your num instance change and my change to remove the az filter have merged since yesterday | |
| 13:01:09 | sean-k-mooney | bauzas: kasyaps min libivrt bump is still in the queue. it hit the rbac job timeout the last time | |
| 13:01:22 | bauzas | yup saw | |
| 13:01:32 | bauzas | I need to CC the min libvirt bump | |
| 13:01:39 | bauzas | but I'll track it anyway for RC1 | |
| 13:01:49 | bauzas | (not impacted by FF) | |
| 13:02:04 | Uggla | sean-k-mooney, bauzas any advices about my question above ? | |
| 13:02:15 | sean-k-mooney | sorry didnt see it | |
| 13:02:52 | sean-k-mooney | is manial requireign service tokens | |
| 13:03:32 | sean-k-mooney | when using the sdk ya you would have to handel that yourself am stephenfin might be able to point you in the right direction | |
| 13:03:37 | sean-k-mooney | although i belive he is off today | |
| 13:03:59 | sean-k-mooney | let me check his ironic series and see if he has done that convertion | |
| 13:06:09 | sean-k-mooney | Uggla: so for ironic stpehin is using | |
| 13:06:11 | sean-k-mooney | self._ironic_connection = utils.get_sdk_adapter( | |
| 13:06:13 | sean-k-mooney | 'baremetal', check_service=True) | |
| 13:06:47 | sean-k-mooney | which is https://github.com/openstack/nova/blob/master/nova/utils.py#L969-L996 | |
| 13:07:29 | Uggla | yep, I use the same "helper". But I think it does not manage service token. | |
| 13:08:36 | sean-k-mooney | so its calling ks_loading.load_session_from_conf_options | |
| 13:09:51 | Uggla | yes | |
| 13:11:01 | sean-k-mooney | so no that does not look like it load the service user section | |
| 13:11:20 | sean-k-mooney | it ends up calling https://opendev.org/openstack/keystoneauth/src/commit/8d24892f9dc2742d9bd837b3a71fce260a5ec326/keystoneauth1/loading/session.py#L235-L258 | |
| 13:13:18 | Uggla | sean-k-mooney, yep so no service token right ? | |
| 13:13:49 | sean-k-mooney | no so you will need to figure out how to do htat | |
| 13:14:00 | sean-k-mooney | im looking at there docs quickly to see if they have an example | |
| 13:14:08 | sean-k-mooney | i think its part of the session but that just a guess | |