| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-09-28 | |||
| 14:30:07 | auniyal | how this works - https://opendev.org/openstack/nova/src/commit/aad31e6ba489f720f5bdc765c132fd0f059a0329/nova/context.py#L396 | |
| 14:55:54 | noonedeadpunk | gibi: just to express my pain as operator. https://paste.openstack.org/show/bpA2Iq28mHEfo8r4sH2W/ consumes about 3 seconds to execute. And as you see - overrides microversion <2.88. | |
| 14:56:13 | noonedeadpunk | If I am to use palcement to get the same data - I will need code like that https://paste.openstack.org/show/bOsJwmnMMpftgZ01St0o/ and it takes... 42 seconds to execute against exact same cluster | |
| 14:57:28 | noonedeadpunk | And not saying about load on APIs as for each resource provider I will need to make 2 calls to placement | |
| 14:59:41 | noonedeadpunk | so for users deprecation of providing resource statistics from hypervisors is quite serious regression in performance. And if I want to monitor that regulary - it's really waste of time. | |
| 15:00:02 | gibi | noonedeadpunk: interesting. so there is some heavy inefficiencies somewhere as it just 2x the number of calls but it takes more than 10 times the time to execute | |
| 15:00:49 | noonedeadpunk | fwiw - I was testing remote cloud (not from inside of it) | |
| 15:03:28 | noonedeadpunk | just to show that I'm not exaggerating https://paste.openstack.org/show/b1irA4GLwJEdUORuLarb/ | |
| 15:04:40 | noonedeadpunk | But I'm not sure about 2x number of calls. As it feels that from hypervisor data was fetched with single api call... | |
| 15:04:47 | noonedeadpunk | Not sure though | |
| 15:05:04 | noonedeadpunk | (or at least I can't explain it otherwise) | |
| 15:06:31 | gibi | hm, you are right the hypervisor one returned all compute from the cluster | |
| 15:06:34 | gibi | /os-hypervisors/detail | |
| 15:06:46 | gibi | so then I think it would make sense to add a similar api for placement | |
| 15:06:55 | gibi | single call that returns all the usages per provider | |
| 15:07:28 | gibi | this explains the preformance differences | |
| 15:08:00 | noonedeadpunk | yeah and it will depend on amount of providers actually right now quite dramatically | |
| 15:08:57 | gibi | so I would support adding such api to placement. I cannot commit to implement it, but sure I can review the implementation | |
| 15:09:46 | noonedeadpunk | I can only put it to my backlog and hope to get to it one day... | |
| 15:10:02 | gibi | noonedeadpunk: and if you want to raise this issue then I think we have dedicated time on the coming PTG for operator feedback | |
| 15:10:22 | noonedeadpunk | that is good idea actually | |
| 15:11:43 | gibi | noonedeadpunk: https://etherpad.opendev.org/p/oct2022-ptg-operator-hour-nova | |
| 15:11:44 | noonedeadpunk | first hour does not intersect with anything, so can join:) | |
| 15:11:52 | gibi | cool | |
| 15:17:23 | jkulik | you couldâ„¢ make the calls in parallel at least ;) | |
| 15:19:07 | noonedeadpunk | isn't it still waste of resources? | |
| 15:19:24 | noonedeadpunk | and regression from operator prespective? | |
| 15:19:43 | jkulik | sure. would be helpful to get it in a single call, but at least it reduces the pain of waiting | |
| 15:20:06 | noonedeadpunk | and add some hardware to serve increased load?:) | |
| 15:20:23 | noonedeadpunk | but yeah, that's fair | |
| 15:20:35 | noonedeadpunk | *fair workaround | |
| 15:28:40 | noonedeadpunk | fwiw, even with 10 threads it's still 5 times slower then jsut ask nova | |
| 15:28:58 | noonedeadpunk | maybe I shouldn't have used joblib... | |
| 22:07:58 | rm_work | hey, i've noticed the OSC seems to sometimes hide the "fault" field on a server show command... but also sometimes it doesn't. anyone know why this is the case? | |
| 22:08:35 | rm_work | starting to dig into the code now, but hoping someone has some idea, since the clients are often a bit funky to interpret, lots of magic 😛 | |
| 22:21:00 | rm_work | yeah, didn't find anything specifically relating to "fault" in there... this is super weird, I can see the "fault" come back with `--debug` in the response from nova, it just gets filtered out or something before the results are shown | |
| 22:31:09 | rm_work | nevermind, figured it out, there's a client patch I didn't know about >_< FML | |
| 22:35:45 | clarkb | rm_work: don't leave us hanging. What causes it? | |
| 22:41:40 | rm_work | local client patch where someone did something ... ill advised | |
| 22:41:54 | rm_work | trying to figure out how to untangle it now | |
| 22:45:30 | clarkb | Oh I see what you mean by client patch now. I thought you meant you found the chagne in gerrit taht did it or something | |
| #openstack-nova - 2022-09-29 | |||
| 08:29:10 | songwenping | bauzas: hi, i have some doubt about nova zed highlight, what is the improvement for 'the possibility to rebuild a volume-backed instance'? | |
| 08:32:14 | gibi | songwenping: this is the code we merged to suppot it https://review.opendev.org/q/topic:bp/volume-backed-server-rebuild+status:merged | |
| 08:33:46 | songwenping | is this the API imporvement? | |
| 08:39:00 | songwenping | gibi: i means this feature is not only for API, but include compute modify. | |
| 08:39:33 | gibi | it includes everythin (cinder and nova impacts) that allows reimaging the root volume of a VM booted from that volume | |
| 08:40:08 | gibi | so yes, it is not just an API change | |
| 08:42:19 | songwenping | thanks, got it. and what's the feature for 'the change to only accept importing a public key but also with an extended name pattern'. | |
| 08:44:52 | gibi | that is actually two features. 1) nova stoped supporting generating ssh keys, we only support importing ssh keys. (it is in a new microversion so old microversions still allow generating ssh keys) 2) we extended the list of allowed chars set for the name of the key. Now both @ and . is allowed in the name | |
| 08:52:55 | songwenping | thanks gibi, got it. | |
| 16:01:51 | open10k8s | Hi team, Task_state=deleting VMs don't exist on hypervisors but openstack keeps those list as Active and running. Nova-compute restart solves this but it is only a temporary solution. After delete another machine, same thing happens. I cannot find any error logs from nova services. Is this known issue in a specific condition or any workaround for this? | |
| 16:03:49 | open10k8s | I am running wallaby and never experienced this issue on other clouds with same version | |
| 16:09:02 | melwitt | open10k8s: no, that's not a known issue and it shouldn't be doing that. you are seeing this happen with all delete requests? if the actual VM is gone but the instance still listed, that means that somehow the delete did not reach the DB update to remove the instance record | |
| 16:09:35 | open10k8s | yes, all deletion operation | |
| 16:10:36 | melwitt | I would expect to see some evidence of why in the nova-compute and or nova-conductor logs. are you running logs with debug=True to diagnose? if there's no message in the logs, it will be difficult to find why it's happening | |
| 16:10:48 | open10k8s | i rather tried to find any issues on oslo messaging | |
| 16:11:39 | melwitt | yeah that is a good thing to look for, also the database if there's any issue there when nova-compute attempts to delete the instance record by way of the nova-conductor | |
| 16:12:00 | melwitt | but in both of those cases usually there is an error logged | |
| 19:33:15 | opendevreview | Ghanshyam proposed openstack/os-vif stable/stein: Fix zuul config error for os-vif https://review.opendev.org/c/openstack/os-vif/+/859892 | |
| 19:38:52 | opendevreview | Ghanshyam proposed openstack/os-vif stable/rocky: Fix zuul config error for os-vif https://review.opendev.org/c/openstack/os-vif/+/859893 | |
| #openstack-nova - 2022-09-30 | |||
| 06:48:35 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds check if resized to swap zero https://review.opendev.org/c/openstack/nova/+/857339 | |
| 07:01:44 | amorin | hey nova team, if you have time someday to review this: https://review.opendev.org/c/openstack/nova/+/853682 | |
| 07:15:18 | Uggla | Poke bauzas --> https://photos.app.goo.gl/MRmeetQr749GFh3s5 ;) | |
| 07:31:11 | bauzas | danm fucking numpy | |
| 07:31:22 | bauzas | I should have prepared better | |
| 07:32:02 | bauzas | and now I have to wait until Feb 2023 IIRC | |
| 07:32:05 | bauzas | graaaah | |
| 07:34:23 | bauzas | oh no, actually after Mar 29th :( | |
| 09:15:42 | Uggla | gibi, bauzas could you have a look at https://review.opendev.org/c/openstack/nova/+/854355 | |
| 09:17:21 | bauzas | ouch. | |
| 09:17:50 | bauzas | Uggla: what do you mean by "soft delete is deprecated ?" | |
| 09:18:40 | Uggla | bauzas, This is mentioned and checked in a test that we should not create soft delete objects. | |
| 09:19:27 | bauzas | I'd say this isn't recommended, true | |
| 09:19:35 | bauzas | but "deprecated" seems harsh to me | |
| 09:20:01 | bauzas | like, if you signal that instances won't be soft deleted, it would be an operators revolution | |
| 09:21:36 | Uggla | bauzas, https://opendev.org/openstack/nova/src/commit/adeea3d5e7d7337d2817dd5c46334c76c05995ef/nova/tests/unit/db/test_models.py#L21 | |
| 09:22:14 | Uggla | I have just reuse the wording L54. | |
| 09:23:13 | Uggla | but I can change the wording if you wish. | |
| 09:29:24 | gibi | Uggla: done | |
| 09:30:00 | gibi | bauzas: what we want to say that no new ovo can be created with soft delete support | |
| 09:30:16 | gibi | and this is enforced by a test case as Uggla noted above | |
| 09:31:26 | bauzas | ok, the wording seems weird to me but when I read the original commit msg, this is saying the same | |
| 09:31:31 | bauzas | anyway | |
| 09:31:36 | bauzas | I knew about new tables | |
| 09:33:23 | Uggla | gibi, thx | |
| 09:37:20 | ralonsoh | gibi, hey, do you know how to book the operator-hour? | |
| 09:37:22 | ralonsoh | I tried "#operator-hour-neutron book icehouse-FriB1" | |
| 09:37:35 | ralonsoh | but I don't want to break anything testing other commands | |
| 09:38:00 | gibi | ralonsoh: I did not tried to book it. bauzas did you booked? | |
| 09:38:08 | ralonsoh | I did for neutron | |
| 09:38:15 | ralonsoh | but not for the operator-hour | |
| 09:38:36 | bauzas | ralonsoh: gibi: yeah I did it for the nova one | |
| 09:38:52 | bauzas | ralonsoh: but you need to ask the infra folks to add you as the neutron owner | |
| 09:38:57 | bauzas | sec | |
| 09:39:03 | ralonsoh | bauzas, I am | |
| 09:39:13 | ralonsoh | I already booked the neutron slots | |
| 09:39:20 | bauzas | https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030301.html | |
| 09:39:35 | bauzas | you need to register the new track | |
| 09:39:38 | ralonsoh | ahhhhh | |
| 09:39:41 | ralonsoh | bauzas, thanks a lot | |