| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-01-07 | |||
| 14:12:54 | opendevreview | Rajesh Tailor proposed openstack/nova master: Add support for showing requested az in output https://review.opendev.org/c/openstack/nova/+/904568 | |
| 15:54:50 | opendevreview | Rajesh Tailor proposed openstack/nova master: Add support for showing requested az in output https://review.opendev.org/c/openstack/nova/+/904568 | |
| #openstack-nova - 2024-01-08 | |||
| 08:00:26 | opendevreview | Amit Uniyal proposed openstack/nova master: Fixes bug 2048184 https://review.opendev.org/c/openstack/nova/+/904817 | |
| 08:40:52 | bauzas_pto | oh my manners | |
| 08:41:17 | bauzas | hey folks, happy new year ! | |
| 09:14:53 | tobias-urdin | o/ happy new year! | |
| 10:40:59 | opendevreview | Amit Uniyal proposed openstack/nova master: Fixes bug 2048184 https://review.opendev.org/c/openstack/nova/+/904817 | |
| 12:14:02 | sean-k-mooney | o/ | |
| #openstack-nova - 2024-01-09 | |||
| 08:20:29 | LarsErikP | Happy new year everyone! Any updates on nova zed 26.2.1? https://review.opendev.org/c/openstack/releases/+/899604 | |
| 09:06:03 | bauzas | LarsErikP: we need to merge https://review.opendev.org/c/openstack/nova/+/900307/3 first | |
| 09:06:12 | bauzas | sean-k-mooney: could you please +2 this backport ? | |
| 09:06:14 | bauzas | https://review.opendev.org/c/openstack/nova/+/900307/3 | |
| 09:43:06 | tobias-urdin | had a issue in december where a nova-compute was installed with same hostname and nova-compute service happily started even though placement RP creation got a conflict | |
| 09:43:21 | tobias-urdin | imo nova-compute should fail on startup if placement RP create gets conflict | |
| 09:43:23 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 09:43:53 | tobias-urdin | so this: https://review.opendev.org/c/openstack/nova/+/904381 – but that changes all exceptions, see the child patch on that for just catching ResourceProviderCreationFailed | |
| 09:43:53 | tobias-urdin | so this: https://review.opendev.org/c/openstack/nova/+/904381 – but that changes all exceptions, see the child patch on that for just catching ResourceProviderCreationFailed | |
| 09:44:04 | tobias-urdin | which approach is acceptable? | |
| 10:05:14 | opendevreview | Amit Uniyal proposed openstack/nova stable/yoga: Fix the PCI device capability dict creation https://review.opendev.org/c/openstack/nova/+/899252 | |
| 10:05:15 | opendevreview | Amit Uniyal proposed openstack/nova stable/yoga: Store pf_mac_address and vf_num in extra_info https://review.opendev.org/c/openstack/nova/+/899253 | |
| 10:05:16 | opendevreview | Amit Uniyal proposed openstack/nova stable/yoga: Translate VF network capabilities to port binding https://review.opendev.org/c/openstack/nova/+/899254 | |
| 10:36:12 | zigo | stephenfin: Hi there! Do you agree with the new version of my patch? https://review.opendev.org/c/openstack/keystoneauth/+/903860 | |
| 11:41:38 | stephenfin | zigo: wrong channel, but comment left on same | |
| 12:24:00 | sean-k-mooney | zigo: for what its worht the current version of the patch looks ok to me | |
| 12:25:01 | sean-k-mooney | bauzas: ill review it now | |
| 12:28:02 | sean-k-mooney | bauzas: done | |
| 12:33:49 | tobias-urdin | sean-k-mooney: can u check above – would like your feedback on it and i'll whip up a correct patch based on that | |
| 12:34:33 | sean-k-mooney | you started 2 nova compute services withthe same host name | |
| 12:34:59 | sean-k-mooney | and the placement rp failed to be created because it was using a diffent service uuid | |
| 12:35:22 | sean-k-mooney | that should only be possibel if they are in diffent cells today | |
| 12:36:32 | sean-k-mooney | the first one to start will register its self as a service, and write the uuid to /var/lib/nova/compute_node | |
| 12:36:56 | sean-k-mooney | the second one will find the service and not the file containing the uuid and will fail to start | |
| 12:40:58 | tobias-urdin | kind of, the original nova-compute was already removed, new node was added with same hostname before it was removed, but nova-compute reported up even though RP creation failed | |
| 12:41:14 | tobias-urdin | i would expect nova-compute to fail if RP could not be created i.e wrong uuid = conflict | |
| 12:41:40 | tobias-urdin | s/removed/stopped/g | |
| 12:41:52 | sean-k-mooney | the current expection is that it will eventually be crated | |
| 12:42:01 | sean-k-mooney | but it soudn like you incorrectly replafced the node | |
| 12:42:14 | sean-k-mooney | if you are replacing the node and keeping the same identity/hostname | |
| 12:42:27 | sean-k-mooney | then you shoudl not do a compute service delete | |
| 12:42:31 | sean-k-mooney | as part of the replacement | |
| 12:43:01 | sean-k-mooney | and you shoudl take the old compute_node uuid and put it in the compute node file | |
| 12:43:06 | tobias-urdin | yes automation/manual steps failed for removing the nova service record (and thus RP) before installing new node, but issue would be caught if nova-compute failed start so that it reported as down | |
| 12:43:28 | sean-k-mooney | right but what im saying is the logic is wrong | |
| 12:43:44 | sean-k-mooney | there are 2 ways to do that replacement | |
| 12:44:13 | sean-k-mooney | you can scale in (fully deleting the node in nova/neutron/placemetn) and then scale out | |
| 12:44:23 | sean-k-mooney | if you do that there is a strict ordering requirement | |
| 12:44:34 | sean-k-mooney | the name cannot be reused until its fully deleted | |
| 12:44:49 | sean-k-mooney | or you can replace the node without deleting the compute/neutron agent services | |
| 12:45:14 | sean-k-mooney | which is how we normally recommend you replace a node when maintaining its identity | |
| 12:45:40 | tobias-urdin | normally we just delete everything but this time somebody forgot the compute service delete step, i would like to protect operators from having a nova-compute reporting up when it's not working | |
| 12:46:28 | sean-k-mooney | ... | |
| 12:46:34 | tobias-urdin | fail on nova-compute startup if RP creation failed | |
| 12:46:50 | tobias-urdin | like neutron-ovs-agent is looping forever and not reporting up if br-int or local_ip is wrong | |
| 12:47:07 | tobias-urdin | safety first ¯\_(ツ)_/¯ | |
| 12:47:14 | sean-k-mooney | so the problem is that we expect to be able to start the compute agents before placement is up | |
| 12:47:29 | sean-k-mooney | and for them to eventually register when it becomes aviableable | |
| 12:47:55 | sean-k-mooney | that woudl still be true if you have something liek systemd | |
| 12:48:00 | sean-k-mooney | that will restart nova-compute externally | |
| 12:48:13 | tobias-urdin | but if we explicitly fail on placement RP conflict then? would that work | |
| 12:48:14 | sean-k-mooney | so your change might be ok | |
| 12:48:42 | sean-k-mooney | it woudl only work if you have an external service restarting it | |
| 12:49:20 | sean-k-mooney | im not nessisarly agaisnt this change | |
| 12:50:10 | sean-k-mooney | im a little wary of just doing it for any exception | |
| 12:51:27 | sean-k-mooney | tobias-urdin: the up status of a compute service is primarly ment to signal that its able to comunicate via rpc with the hypervior | |
| 12:51:48 | sean-k-mooney | i say it that way as we have specific logic to mark it down if the connection to libivrt is lost | |
| 12:52:06 | sean-k-mooney | its technially not ment to be a healtcheck | |
| 12:52:37 | sean-k-mooney | so im a little concerned about scope creap | |
| 12:53:26 | sean-k-mooney | with that said you are stoping the agent if it fails | |
| 12:53:43 | sean-k-mooney | so the meaning of up technically has not been changed by this patch | |
| 12:54:37 | tobias-urdin | yeah i understand, for us basically since we don't verify anything in placement specifically with monitoring, the nova-compute was up but it never scheduled any workload there | |
| 12:55:06 | tobias-urdin | it's a messy edge case but if we get ResourceProviderCreationFailed and it's a conflict we could perhaps just fail on startup to not report up | |
| 12:55:18 | sean-k-mooney | for what its worht this is why im working on adding a per process healt check endpoint | |
| 12:55:26 | tobias-urdin | so took us some day or two before we noticed it | |
| 12:55:44 | tobias-urdin | yea | |
| 12:55:51 | sean-k-mooney | so i think i would be happier if you added a new except for ResourceProviderCreationFailed | |
| 12:55:58 | sean-k-mooney | rather then doing this for just Exception | |
| 12:56:16 | sean-k-mooney | the other thing im thinking about is | |
| 12:56:23 | sean-k-mooney | how does your chage work with ironic | |
| 12:57:05 | sean-k-mooney | im not expictly aware of an issue with ironic but each ironic agent is managing multiple compute nodes (1 per ironic node) each with there own RP | |
| 12:58:26 | sean-k-mooney | in its current form if there was an error for any of the ironic RPs it would cause the ironic compute agent to exit | |
| 12:58:43 | sean-k-mooney | so we may want to skip this if its ironic | |
| 13:01:30 | sean-k-mooney | tobias-urdin: what release are you runing by the ay | |
| 13:01:34 | sean-k-mooney | *way | |
| 13:05:11 | tobias-urdin | mix of yoga and zed, but nova is yoga | |
| 13:05:28 | sean-k-mooney | ack so ya you dont have the stabel uuid feature | |
| 13:05:34 | sean-k-mooney | which would catch this. | |
| 13:06:03 | tobias-urdin | didn't think about the ironic use-case | |
| 13:06:18 | tobias-urdin | would nova-compute service be down in that case? | |
| 13:07:11 | sean-k-mooney | so usign the normal meaning of up/down it should be up if it cna comunicate with teh ironic rest api and it accessabel over the rpc bus | |
| 13:07:32 | sean-k-mooney | even if one of the ironic computes is not usabel | |
| 13:07:42 | sean-k-mooney | comptue nodes do not have an up or down state | |
| 13:07:47 | sean-k-mooney | only comptue services do | |
| 13:08:01 | tobias-urdin | sorry, with stable uuid feature | |
| 13:08:18 | sean-k-mooney | oh for stable uuid we explcitly dont use the fature with ironic | |
| 13:08:40 | sean-k-mooney | we put if checks in the code to skip it if its ironic because ironic is special | |
| 13:08:56 | sean-k-mooney | it will be less special once the peer_list and hashring is removed | |
| 13:08:56 | tobias-urdin | i mean if we were on newer release with stable uuid feature would nova-compute service fail to start and not report itself as up if there was a conflict like this? | |
| 13:09:08 | sean-k-mooney | yes | |
| 13:09:24 | sean-k-mooney | in antelope it shoudl not start for libvirt | |
| 13:09:59 | sean-k-mooney | to reuse the same hostname you now need to create /var/lib/nova/comptue_node with the old comptue node uuid | |