| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-09-02 | |||
| 16:09:22 | sean-k-mooney | ok i guess we should capture this in a bug as a possibel performance reguression between 2023.1 and 2024.1 | |
| 16:10:01 | noonedeadpunk | ok, I just wanted to check if you might be aware of that before digging deeper | |
| 16:10:59 | sean-k-mooney | unfortunetly i suspect whatever it may be does not present on devstack or other small deployments | |
| 16:11:58 | sean-k-mooney | i tought maybe it was https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id63 | |
| 16:12:33 | sean-k-mooney | since if you have down cells that might change behaivor | |
| 16:12:53 | noonedeadpunk | nah, it's a single cell and single AZ environment | |
| 16:13:10 | noonedeadpunk | as we've started upgrade from the smallest thing we had | |
| 16:13:47 | sean-k-mooney | ya so get /servers at the nova api level should be very quick | |
| 16:13:51 | sean-k-mooney | https://docs.openstack.org/api-ref/compute/#id9 | |
| 16:14:01 | sean-k-mooney | as it only retuns the minimal info in the api db | |
| 16:14:08 | sean-k-mooney | but openstack server list | |
| 16:14:13 | sean-k-mooney | is not the same thing as that | |
| 16:14:39 | sean-k-mooney | i think it does more behind the scenes | |
| 16:15:15 | sean-k-mooney | noonedeadpunk: around that time osc started replacing using python-novaclient with the sdk as well | |
| 16:15:22 | noonedeadpunk | client also does pass bunch of filters: /v2.1/servers/detail?flavor=None&image=None&name=None&project_id=None&reservation_id=None&status=None&user_id=None&deleted=False&host=None&ip=None&ip6=None&changes-since=None&changes-before=None&all_tenants=True | |
| 16:15:34 | noonedeadpunk | so I wonder if that might be client that I've upgraded | |
| 16:15:56 | sean-k-mooney | you could try doing it with nova client or installing an older osc in a venv | |
| 16:16:09 | sean-k-mooney | that woudl at least tell us if its server or client side | |
| 16:16:12 | noonedeadpunk | or just do REST call | |
| 16:16:17 | sean-k-mooney | yep or that | |
| 16:16:40 | sean-k-mooney | you can do it with curl if you have a keyston token but i always have to look up the syntax | |
| 16:19:42 | noonedeadpunk | so curl takes half of the time, as I assume osc does fetch more info to render uuids | |
| 16:20:04 | noonedeadpunk | still 1m24s while haproxy used to be set to 60s timeout | |
| 16:20:37 | noonedeadpunk | and request was `time curl -g -i --cacert "/etc/ssl/certs/ca-certificates.crt" -X GET "http://cloud.com:8774/v2.1/servers/detail?all_tenants=True" -H "Accept: application/json" -H "X-Auth-Token: ${token}" -H "X-OpenStack-Nova-API-Version: 2.92"` | |
| 16:26:44 | sean-k-mooney | if you have the request id it would be good to check how long the request taks on the nova-api side | |
| 16:27:12 | sean-k-mooney | /server/detail is not the same as /server | |
| 16:27:33 | sean-k-mooney | so if you dont want all the addtional info avoid /server/detail | |
| 16:28:26 | noonedeadpunk | well, I'm not sure about detail part, but assume it was preffered to limit amount of following requests. | |
| 16:28:40 | noonedeadpunk | but also interesting to find | |
| 16:29:12 | sean-k-mooney | well /detail is not really advisabel on large cloud if using --all-tenatns | |
| 16:29:35 | sean-k-mooney | it can inovled calls to neutron or other sirvices for info in some cases | |
| 16:29:59 | sean-k-mooney | we mostly try to cache some of that info but we dont cache all of it | |
| 16:34:27 | sean-k-mooney | i am thinking of https://bugs.launchpad.net/nova/+bug/1923560 | |
| 16:34:43 | sean-k-mooney | without https://review.opendev.org/c/openstack/nova/+/786348 we need to do a call to neutron for every server | |
| 16:34:53 | sean-k-mooney | if you use /server/detail | |
| 17:41:02 | opendevreview | sean mooney proposed openstack/nova master: enable numa live migration in the ceph job https://review.opendev.org/c/openstack/nova/+/913842 | |
| 17:41:36 | opendevreview | sean mooney proposed openstack/nova master: enable numa live migration in the ceph job https://review.opendev.org/c/openstack/nova/+/913842 | |
| 22:53:43 | opendevreview | sean mooney proposed openstack/nova master: [DNM] testing rdo experimental jobs https://review.opendev.org/c/openstack/nova/+/927762 | |
| #openstack-nova - 2024-09-03 | |||
| 00:56:17 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Followup fixes around flavor ids https://review.opendev.org/c/openstack/nova/+/927764 | |
| 01:39:54 | stan | Guys can I set a limit of how many VMs can be created simultaneously? Let's say if the user wants to create 20 VMs at a time, is there a mechanism to create 5vms in the first batch, and put the rest of the vm request in queue? | |
| 01:45:21 | bauzas | I officially called out the FeatureFreeze | |
| 01:45:40 | bauzas | all the 2024.2 blueprints are now either Deferred or Implemented | |
| 01:45:53 | bauzas | numbers aren't great, tbh. | |
| 01:46:47 | bauzas | https://blueprints.launchpad.net/nova/2024.2 | |
| 06:33:08 | opendevreview | Takashi Kajinami proposed openstack/nova master: Add hw_mem_encryption_model image property https://review.opendev.org/c/openstack/nova/+/927706 | |
| 06:33:09 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | |
| 07:34:25 | opendevreview | Viktor Křivák proposed openstack/nova master: Ensure claims to placement are freed after evacuate complete Improve resource_tracker delete_allocation_for_evacuated_instance to handle non-cached compute hosts Fix test to accept this change https://review.opendev.org/c/openstack/nova/+/926292 | |
| 09:05:40 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | |
| 09:05:40 | opendevreview | Takashi Kajinami proposed openstack/nova master: Add hw_mem_encryption_model image property https://review.opendev.org/c/openstack/nova/+/927706 | |
| 10:43:59 | sean-k-mooney | stan: not on a per user basis or at teh api level | |
| 10:44:08 | sean-k-mooney | stan: you can set a limit per compute node | |
| 10:44:37 | sean-k-mooney | it defaults to 10 https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.max_concurrent_builds | |
| 10:59:37 | stan | sean-k-mooney Thanks for the information. :) | |
| 11:04:08 | stan | I assume I can make changes to this in combination with scheduler timeout to achieve our use case. | |
| 11:04:30 | sean-k-mooney | note really | |
| 11:04:52 | sean-k-mooney | the compute node config only takes effect after the instance is schduled and just delays it spawning | |
| 11:17:38 | stan | sean-k-mooney Alright. I will test this in our lab 👍 | |
| 11:30:02 | stan | sean-k-mooney So nova timeout will effectively come into play after its schedule or after the VM starts to spawn? | |
| 11:30:46 | sean-k-mooney | no | |
| 11:31:36 | sean-k-mooney | there are a few tiemouts at play. there is an api request time out for the intiall http respocne, then there is the rpc timeout between the conductor schduelr adn computes | |
| 11:32:04 | sean-k-mooney | and then there is an instance build timeout https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.instance_build_timeout | |
| 11:32:23 | sean-k-mooney | which defaults ot 0 meaning no limit | |
| 12:16:56 | sean-k-mooney | dansmith: looks like we have another test that should be skiped without qemu-img | |
| 12:16:58 | sean-k-mooney | https://paste.opendev.org/show/bOFlCycmdpOV1shHsHQX/ | |
| 12:17:36 | sean-k-mooney | did we plan to merge your patch to use the oslo version? i lost track of where we were with that | |
| 12:29:51 | jlejeune_ | sean-k-mooney: hello, indeed, that's why I've proposed: https://review.opendev.org/c/openstack/nova/+/925208 | |
| 12:30:16 | sean-k-mooney | ah yes i tought this was familar | |
| 12:30:23 | jlejeune | sean-k-mooney: :) | |
| 12:31:12 | sean-k-mooney | i am usign distobox to create a clean dev env on my laptop to workaround some packaging issues so didnt have it installed | |
| 12:31:48 | jlejeune | like on our CI | |
| 12:32:27 | sean-k-mooney | well in the first party ci we do have qemu-img isntalled | |
| 12:32:35 | sean-k-mooney | but in general it should not be requried | |
| 12:33:22 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/925208/comment/2b6ac8e7_286f92b1/ is still an open question that needs input form others | |
| 12:34:47 | sean-k-mooney | im ok with the stub as i dont think the existin unit test should be executing that code but we still need others to weigh in | |
| 12:35:19 | sean-k-mooney | jlejeune: to be honest i had tought we merged that already | |
| 12:36:09 | jlejeune | I know I have some work to do to tka in consideration your comment but I didn't have time to work on | |
| 12:36:28 | sean-k-mooney | thats fine we didnt have time to review it | |
| 12:46:13 | jlejeune | ok so I don't change my code and let someone else review it based on your opinion ? | |
| 12:55:24 | opendevreview | Takashi Kajinami proposed openstack/nova master: Replace deprecated glanceclient exception https://review.opendev.org/c/openstack/nova/+/914761 | |
| 13:04:16 | sean-k-mooney | jlejeune: dan has a stonger prefernce then i do. i was ok with the reductionin code execution because i dont think a unit test should be as large as that one is and the coverage shoudl eb provided by smaller more targed test or or functional test suite | |
| 13:32:47 | dansmith | sean-k-mooney: yes, merged, and that trace isn't in my code | |
| 13:33:08 | sean-k-mooney | dansmith: i know its just a missing mock | |
| 13:36:13 | dansmith | okay, I'm not sure what you're asking | |
| 13:37:34 | sean-k-mooney | i was just letting you knwo we still have once ase where its not mocked and jlejeune remindiend me that we had dicussed this on tehre patch previously https://review.opendev.org/c/openstack/nova/+/925208 | |
| 13:56:44 | jlejeune | sean-k-mooney: dansmith: not sure what I have to do about it now :) | |
| 13:59:08 | dansmith | tbh, I don't hate the very first rev of this patch, and I think it's better than the current one | |
| 13:59:22 | dansmith | I would str(e) and make sure there's a qemu-img in there so we only skip for that case | |
| 13:59:33 | dansmith | but that leaves us all the coverage we have today on systems that can test it, | |
| 13:59:42 | dansmith | and just skips it for limited environments | |
| 13:59:53 | dansmith | just like our opportunistic DB testing | |
| 14:07:34 | dansmith | jlejeune: ^ | |
| 14:11:59 | jlejeune | dansmith: ok, you mean I had to check the str(e) of the exception to detect if it's really related to qemu-img, in my first patchset ? | |
| 14:12:11 | dansmith | jlejeune: yeah | |
| 14:12:21 | jlejeune | ok clear, doing, thanks | |
| 14:12:27 | dansmith | sounded like sean-k-mooney was not okay with that approach though, but I'm not sure why.. it keeps the most coverage | |
| 14:12:39 | dansmith | jlejeune: might want to wait for sean-k-mooney to say that's okay as well | |
| 14:12:45 | jlejeune | ok | |
| 14:13:18 | sean-k-mooney | dansmith: i not entirly against it but form my point of view the unit test is not a unit test today | |
| 14:13:43 | sean-k-mooney | unit tests should not be calling code out side of the same call liek that | |
| 14:14:03 | sean-k-mooney | if you think we have a clean way to keep the coverage and make this work that you are happy with then thats fine | |