| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-10-17 | |||
| 12:43:02 | pas-ha[m] | API ref says the same btw. if you use older that 2.30 api, the scheduler does not apply when migrating to a specific host | |
| 12:43:29 | sean-k-mooney | pas-ha[m]: every request we make to placmenet unconditon fobids host with the COMPUTE_STAUS_DISBALED trait | |
| 12:44:12 | pas-ha[m] | afaiu in older api scheduler is not involved at all I guess... | |
| 12:44:28 | pas-ha[m] | https://docs.openstack.org/api-ref/compute/#live-migrate-server-os-migratelive-action | |
| 12:44:29 | sean-k-mooney | it is but it does very minimal checks | |
| 12:44:29 | pas-ha[m] | Prior to microversion 2.30, specifying a host will bypass validation by the scheduler, which could result in failures to actually migrate the instance to the specified host, or over-subscription of the host. | |
| 12:44:40 | sean-k-mooney | but the palcment one i did nto think was skiped | |
| 12:44:54 | sean-k-mooney | basically it does nto run any fitler or weighers | |
| 12:45:00 | sean-k-mooney | but it does confirm the host extist | |
| 12:45:28 | sean-k-mooney | its possibel we have a bug and we are early outing before check with placment | |
| 12:49:21 | pas-ha[m] | this is also also something that was working before the AZFilter was removed (we had to re-instate it back in downstream 😕 ) | |
| 12:49:21 | pas-ha[m] | sean-k-mooney: and, changing to other scheduling problems, what would you think on https://bugs.launchpad.net/nova/+bug/2078669 (and https://review.opendev.org/c/openstack/nova/+/927744)? | |
| 12:50:29 | pas-ha[m] | how can I boot an instance 'not in any of those AZs that I see' ? | |
| 12:52:27 | sean-k-mooney | thats a diffent issue and strictly speakign also not supproted | |
| 12:53:02 | sean-k-mooney | pas-ha[m]: see the red warining box in https://docs.openstack.org/nova/latest/admin/availability-zones.html | |
| 12:54:20 | sean-k-mooney | i am more open to supproting the defautl nova az but bauzas is stongly against it. we have docuemented taht you should not ever request the nova az effectivly since AZs were first added | |
| 12:57:42 | bauzas | sean-k-mooney: hmmm, what is the question ? | |
| 12:57:49 | pas-ha[m] | yeah, I know that doc. Still, this is in admin part of docs, and users rarely are aware of it, and they do see 'nova' as in az list... and again, if I have hosts in az 'foo' and hosts in no explicit az ('nova'), previously it was possible to boot "not in foo" == boot in 'nova' az. Now its not possible any more 😕 | |
| 12:58:28 | pas-ha[m] | btw, the api ref says that regular users should not see 'nova' az in az list, but I am pretty sure that's still the case even on master | |
| 12:58:48 | pas-ha[m] | bauzas: https://bugs.launchpad.net/nova/+bug/2078669 and the patch to that | |
| 12:59:07 | sean-k-mooney | horizon adds it to the lsit if no azs are retruned i belive | |
| 12:59:28 | pas-ha[m] | openstack cli does show it too | |
| 13:00:30 | bauzas | pas-ha[m]: tbc, the concern is that by default 'nova' isn't an AZ | |
| 13:01:06 | bauzas | if you want, you can change the default value for instances so there you could create a specific aggregate with a 'nova' AZ | |
| 13:01:34 | bauzas | but if you set an instance with '-az nova' then we'll pin into it | |
| 13:03:17 | pas-ha[m] | I know, but it is represented as such to the users still. And the user sees az 'expensive' and az 'nova', user thinks 'i definitely do not need 'expensive', and uses 'nova'. Before, with AZFilter being available, the instances landed where user intended. Now they are not. | |
| 13:05:18 | sean-k-mooney | ubuntu@devstack-iso:~$ openstack --os-cloud=devstack --debug availability zone list --compute 2>&1| nc termbin.com 9999 | |
| 13:05:20 | sean-k-mooney | https://termbin.com/b8q9 | |
| 13:05:26 | sean-k-mooney | bauzas: we do have a bug ^ | |
| 13:06:38 | pas-ha[m] | I proposed a patch that would make az 'nova' (or whatever default az is) be treated as 'not any host that belongs to any explicit az' (if there's no actual explicit az nova of course). IMO that should be exaclty what the user intended. | |
| 13:06:53 | sean-k-mooney | pas-ha[m]: that would break things | |
| 13:07:31 | pas-ha[m] | this was effectively working like that until AZFilter was removed recently | |
| 13:07:43 | pas-ha[m] | (in bobcat or caracal) | |
| 13:07:43 | sean-k-mooney | the problem with that is we do not allow moving host betwene azs | |
| 13:08:15 | sean-k-mooney | if they have instnace on them that is | |
| 13:08:23 | pas-ha[m] | yeah, but I am not forcing any AZ here. I am only forcing placement to discard any other candidates | |
| 13:09:04 | sean-k-mooney | you cant do that effiectnly without pasically passing all az aggreatd in a not | |
| 13:09:11 | sean-k-mooney | which does not scale | |
| 13:09:19 | sean-k-mooney | and will hit the max url limit in large clouds | |
| 13:09:44 | sean-k-mooney | the only way to make this work woudl be to group all hosts not in an az into a seperate aggreate | |
| 13:09:58 | sean-k-mooney | and then explcitly request that | |
| 13:10:28 | pas-ha[m] | yeah, that's an interesting thought.. about the url length at least | |
| 13:10:52 | bauzas | not sure I understand the problem sorry | |
| 13:11:04 | bauzas | the fact that we return 'nova' per the user AZ list ? | |
| 13:11:06 | sean-k-mooney | bauzas: as a non admin if you do an az list you get nova | |
| 13:11:16 | bauzas | if so, we should *not* return it, I agree | |
| 13:11:22 | sean-k-mooney | which mean it looks valid to and end suer | |
| 13:11:33 | sean-k-mooney | because they cant no what the defautl az is | |
| 13:11:42 | sean-k-mooney | *default internal az is | |
| 13:12:03 | sean-k-mooney | actuilly no i was right the first time https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.default_availability_zone | |
| 13:12:32 | bauzas | okay, then I agree: we shouldn't return the default AZ in that list | |
| 13:12:35 | bauzas | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.default_availability_zone | |
| 13:12:48 | sean-k-mooney | you can actully create a real az callded nova and change that to something else | |
| 13:13:03 | sean-k-mooney | the defautl az can be returned if its an actual az | |
| 13:13:07 | bauzas | for defaulting to pin a specific AZ, this is https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.default_schedule_zone | |
| 13:13:28 | bauzas | we could even make default_az to None, I don't see why we need to have 'nova' | |
| 13:13:37 | sean-k-mooney | i was just thinking about that | |
| 13:13:41 | bauzas | but we could and should discuss this at the PTG | |
| 13:13:55 | sean-k-mooney | ya i dont know of the top of my head anything that would break | |
| 13:14:27 | sean-k-mooney | it would stop instance reporting an az if they are on a host that are not part of any az | |
| 13:14:34 | sean-k-mooney | but that is more correct then "nova" | |
| 13:14:36 | pas-ha[m] | as I said, api ref already promises that non-admin should not see it | |
| 13:14:37 | pas-ha[m] | https://docs.openstack.org/api-ref/compute/#create-server | |
| 13:14:37 | pas-ha[m] | > The default availability zone is named nova. This AZ is only shown when listing the availability zones as an admin. | |
| 13:15:02 | sean-k-mooney | pas-ha[m] reight beu twe have a bug because that is not what the api is doing | |
| 13:16:22 | sean-k-mooney | http://192.168.50.7:80 "GET /compute/v2.1/os-availability-zone HTTP/1.1" 200 97 | |
| 13:16:24 | sean-k-mooney | RESP: [200] Connection: close Content-Length: 97 Content-Type: application/json Date: Thu, 17 Oct 2024 13:15:54 GMT OpenStack-API-Version: compute 2.1 Server: Apache/2.4.58 (Ubuntu) Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-17af8a83-82f0-45ce-81fd-c6d54a0c0ad9 x-openstack-request-id: | |
| 13:16:26 | sean-k-mooney | req-17af8a83-82f0-45ce-81fd-c6d54a0c0ad9 | |
| 13:16:28 | sean-k-mooney | RESP BODY: {"availabilityZoneInfo": [{"zoneName": "nova", "zoneState": {"available": true}, "hosts": null}]} | |
| 13:16:56 | sean-k-mooney | ill quickly checkc but im 99% shure i dont have any hostaggrate defiend on this devstack | |
| 13:17:24 | pas-ha[m] | And I am afraid something would break if we start doing that. This would effectively force operators to always put hosts into some explicit AZ, because users who are used to pick the AZ from the list will just use one of those the list presents them, making hosts not in any AZ underused | |
| 13:18:30 | sean-k-mooney | well az are optional | |
| 13:18:42 | sean-k-mooney | i would expect most user not to sepcify one | |
| 13:18:59 | sean-k-mooney | unless its via horizon in shich case they still dont need to select one | |
| 13:24:54 | sean-k-mooney | pas-ha[m]: the alaswer ot this is AZ shoudl be replacced with a first class api object and we shoudl have some way to auto enrole comptue services in AZ | |
| 13:25:23 | sean-k-mooney | the problem is we have never found an accpatbale solution to do that but i know other are interested in that direction | |
| 13:27:08 | pas-ha[m] | just to re-iterate, for the exact use case that we had broken for us:... (full message at <https://matrix.org/oftc/media/v1/media/download/AZi6ENlpQm3FPa8euVAaf8oGaS7R6Eq6HS5W4Rwq_LvrIBUm5dYilERrZnNQr5tnKVfobGYw-LXZy-WaqFYBXT5CeS4sIl5AAG1hdHJpeC5vcmcva0RHSEVMREVleFlzVUF6SldadkRXeWlW>) | |
| 13:28:43 | sean-k-mooney | an AZ is nto the correct way to to enabel that usecase | |
| 13:28:51 | sean-k-mooney | ti can be doen with az but its error prone | |
| 13:31:06 | sean-k-mooney | pas-ha[m]: just to make sure i understand correctly do you allow hugepage flavor to be used elsewhere or rather what is the cirtia for allowing it on a dpdk host | |
| 13:33:17 | pas-ha[m] | only those hosts are configured with huge pages, so scheduler filters other hosts out. the problem is to limit flavors that not use huge pages out of those hosts. | |
| 13:34:27 | pas-ha[m] | probably adding some extra spec like forbidden=... to any other flavor would work too, but that's more cumbersome than what was working before. | |
| 13:34:38 | sean-k-mooney | you can map specific flavor to host with the AggregateTypeAffinityFilter you can use can also use AggregateInstanceExtraSpecsFilter or aggregate_image_properties_isolation to enfoce extra specs are requested in teh flavor/image or you can use the palcemnet feature ath was created explcitly for this type of grouping | |
| 13:34:40 | sean-k-mooney | https://docs.openstack.org/nova/latest/reference/isolate-aggregates.html | |
| 13:35:00 | sean-k-mooney | if you use ^ | |
| 13:35:54 | sean-k-mooney | you can make so that if a falvor does not request a custome trait it cant be shcheduled to the host | |
| 13:37:30 | sean-k-mooney | you just create a CUSTOM_DPDK trait. create a host aggate with your dpdk host and add trait:CUSTOM_DPDK=required | |
| 13:37:46 | sean-k-mooney | and request trait:CUSTOM_DPDK=required in your dpdk flavors | |
| 13:40:32 | sean-k-mooney | pas-ha[m]: that was added in train https://specs.openstack.org/openstack/nova-specs/specs/train/implemented/placement-req-filter-forbidden-aggregates.html | |
| 18:39:55 | opendevreview | Ghanshyam proposed openstack/nova master: Update gate jobs as per the 2025.1 cycle testing runtime https://review.opendev.org/c/openstack/nova/+/932648 | |
| 20:00:19 | opendevreview | Mengyang Zhang proposed openstack/nova-specs master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova-specs/+/932653 | |
| 20:25:29 | opendevreview | Mengyang Zhang proposed openstack/nova-specs master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova-specs/+/932653 | |
| #openstack-nova - 2024-10-18 | |||
| 03:21:38 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Wrap un-proxied listDevices() and listAllDevices() https://review.opendev.org/c/openstack/nova/+/932669 | |
| 04:42:18 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Wrap un-proxied listDevices() and listAllDevices() https://review.opendev.org/c/openstack/nova/+/932669 | |
| 05:26:31 | opendevreview | norman shen proposed openstack/nova master: Add timeout to ceph util's command line if necessary https://review.opendev.org/c/openstack/nova/+/932674 | |
| 08:51:12 | opendevreview | norman shen proposed openstack/nova master: Add timeout to ceph util's command line if necessary https://review.opendev.org/c/openstack/nova/+/932674 | |
| 13:12:41 | haleyb | bauzas: do you think we need a nova/neutron cross-project next week? the only thing i know about is multiple port bindings, the thing we never seem to get to :( | |
| 13:13:10 | bauzas | haleyb: I have no ask in the etherpad | |
| 13:13:59 | haleyb | bauzas: ack, if something changes we probably have some extra time slots | |
| 13:14:31 | sean-k-mooney | haleyb: the one about using service tokens or the ceanup we shoudl eb doing about shelve or makign it required for all neutron deploymeents | |