Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-02-04
19:44:53 sean-k-mooney ok here are the docs on that topic if you have not got them set on each compute host
19:44:56 sean-k-mooney https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#allocation-ratios
19:45:19 alexe9191 just checked right now and we have that on rocky
19:45:31 alexe9191 older versions are a different story but we are getting rid of those
19:45:35 sean-k-mooney ok so its in the nova.conf on each of your compute nodees
19:45:40 alexe9191 yes
19:46:26 alexe9191 docker exec -it nova_compute grep cpu /etc/nova/nova.conf
19:46:26 alexe9191 cpu_allocation_ratio=1.0
19:46:48 sean-k-mooney ok then what you basically need to do is compare the avaiable resouces in plamcnet to those reported in the hyperviors api
19:47:21 sean-k-mooney if there is a missmatch due to stales allcoation the usage wont be the same in nova view and placments
19:47:50 alexe9191 I am actually building now a json file with the resources using openstack hypervisor show
19:47:59 sean-k-mooney which would be why plamcent would have elminated the host before it got to the az filter
19:49:05 sean-k-mooney unfortunetly im more or less done for the day so i wont be able to help you contiue debuging but my best guess is placemetn and nova are out of sync
19:49:37 sean-k-mooney so placment is filtering out the host in that az first. before it gets to the schduler
19:50:00 alexe9191 You've already sat me on the good path:)  I am going to check this and check the heal command if that is the case
19:50:12 alexe9191 one more question though, can I query the resources per node? or is that per class only?
19:50:27 sean-k-mooney in placment?
19:50:31 alexe9191 yes
19:50:34 sean-k-mooney yes you can
19:50:53 sean-k-mooney you can use the resouce providers endpoint ot list all the invetores and the usage per host
19:51:10 sean-k-mooney there is an osc plugin for that too
19:51:27 sean-k-mooney called osc-placement if you dont have it installed
19:51:40 sean-k-mooney it will give you some placment command for the openstack client
19:52:16 alexe9191 installing now:)  thanks alot
19:53:00 sean-k-mooney here are the docs https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-inventory-list
19:53:32 sean-k-mooney openstack resource provider show [--allocations] <uuid> will also be useful
19:53:54 sean-k-mooney or openstack resource provider usage show <uuid>
19:54:23 alexe9191 Testing it now :)
19:54:52 alexe9191 yup, got the numbers
20:10:52 alexe9191 yup, I see differences between what nova is reporting vs what placement is reporting
20:44:19 openstackgerrit Merged openstack/nova master: functional: Add tests for mixed CPU policy https://review.opendev.org/c/openstack/nova/+/755852
21:03:37 openstackgerrit Merged openstack/nova master: Remove __unicode__() from nova unit test Exception https://review.opendev.org/c/openstack/nova/+/769894
22:39:37 openstackgerrit Merged openstack/nova stable/ussuri: Warn when starting services with older than N-1 computes https://review.opendev.org/c/openstack/nova/+/770764
22:40:19 openstackgerrit Merged openstack/nova stable/ussuri: Reproduce bug 1896463 in func env https://review.opendev.org/c/openstack/nova/+/770768
22:40:19 openstack bug 1896463 in OpenStack Compute (nova) ussuri "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 - Assigned to Balazs Gibizer (balazs-gibizer)
22:40:41 openstackgerrit Merged openstack/nova stable/ussuri: [doc]: Fix glance image_metadata link https://review.opendev.org/c/openstack/nova/+/761977
22:55:47 openstackgerrit Merged openstack/nova stable/train: Fix a hacking test https://review.opendev.org/c/openstack/nova/+/767793
#openstack-nova - 2021-02-05
00:12:26 openstackgerrit Merged openstack/nova stable/train: Use cell targeted context to query BDMs for metadata https://review.opendev.org/c/openstack/nova/+/771407
00:28:54 openstackgerrit Artom Lifshitz proposed openstack/nova master: WIP: libvirt: start tracking NUMACell.socket for hosts https://review.opendev.org/c/openstack/nova/+/766816
00:28:55 openstackgerrit Artom Lifshitz proposed openstack/nova master: WIP: extra specs/image pros: add `socket` PCI NUMA affinity https://review.opendev.org/c/openstack/nova/+/772748
00:28:55 openstackgerrit Artom Lifshitz proposed openstack/nova master: WIP: Add `socket` PCI NUMA affinity policy request prefilter https://review.opendev.org/c/openstack/nova/+/772749
00:28:56 openstackgerrit Artom Lifshitz proposed openstack/nova master: WIP: Track host NUMA topology in PCI manager https://review.opendev.org/c/openstack/nova/+/774149
00:28:56 openstackgerrit Artom Lifshitz proposed openstack/nova master: WIP: pci: implement the `socket` NUMA affinity policy https://review.opendev.org/c/openstack/nova/+/772779
01:38:05 openstackgerrit Merged openstack/nova master: hyperv: Configures chassis asset tags for VMs. https://review.opendev.org/c/openstack/nova/+/752723
01:59:04 openstackgerrit Merged openstack/nova stable/ussuri: Reproduce bug 1907522 in functional test https://review.opendev.org/c/openstack/nova/+/773930
01:59:04 openstack bug 1907522 in OpenStack Compute (nova) ussuri "test_*_with_qos_min_bw_allocation fails in the nova-multi-cell job with: nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host " [Medium,In progress] https://launchpad.net/bugs/1907522 - Assigned to Balazs Gibizer (balazs-gibizer)
04:13:07 openstackgerrit Merged openstack/nova master: Fix ForbiddenWithAccelerators to HTTPForbidden for shelve API https://review.opendev.org/c/openstack/nova/+/770007
04:14:15 openstackgerrit Merged openstack/nova master: pci: Improve testing of 'nova.pci.request' module https://review.opendev.org/c/openstack/nova/+/764446
05:42:19 openstackgerrit Merged openstack/nova stable/ussuri: Set instance host and drop migration under lock https://review.opendev.org/c/openstack/nova/+/770769
06:01:07 openstackgerrit Khomesh Thakre proposed openstack/nova master: api: Reject requests to reset the state of SHELVE_OFFLOADED servers https://review.opendev.org/c/openstack/nova/+/773238
06:40:08 openstackgerrit Wenping Song proposed openstack/nova master: Replaces tenant_id with project_id from List/Show usage APIs https://review.opendev.org/c/openstack/nova/+/768509
06:40:09 openstackgerrit Wenping Song proposed openstack/nova master: Replace tenants* with projects* of policies https://review.opendev.org/c/openstack/nova/+/765315
06:40:10 openstackgerrit Wenping Song proposed openstack/nova master: Replace os-simple-tenant-usage with os-simple-project-usage https://review.opendev.org/c/openstack/nova/+/768852
08:18:07 openstackgerrit Wenping Song proposed openstack/nova master: Nova supports password encrypted VNC https://review.opendev.org/c/openstack/nova/+/622336
08:23:32 alexe9191 Good day everyone, is there an issue anyone aware of in OpenStack rocky in the scheduler? it seems to think that some of my hosts belong to the nova AZ, while nova-api is reporting that they are in a different zone.
08:49:42 alexe9191 somehow removing one host of the aggregate and re-adding it fixed the problem for the whole aggregate. I think there is some kind of a update all mechanism when adding/removing hosts to aggregates.
08:54:25 bauzas alexe9191: have you named an aggregate with a 'nova' name for its AZ ?
08:55:15 alexe9191 no
08:55:20 alexe9191 that's the default one
08:55:25 bauzas I mean, is there an aggregate having a metadata value for AZ being "nova" ?
08:55:29 bauzas k
08:55:32 alexe9191 nop
08:55:36 bauzas https://docs.openstack.org/nova/stein/user/aggregates.html#availability-zones-azs
08:55:56 alexe9191 could be that at some certain point someone did something and just renamed it later without updating the aggregates..,,
08:56:17 bauzas hmmm
08:56:31 bauzas not sure, since the API verifies whether there are instances
08:56:47 alexe9191 the api was reporting that the host is in the correct az
08:56:54 alexe9191 nova service-list | grep host
08:56:58 alexe9191 showed the correct values
08:56:59 bauzas are you sure the API is correct ?
08:57:03 bauzas ack ok
08:57:05 alexe9191 the scheduler was not though...
08:57:16 alexe9191 also in the DB the mapping between the host and it's aggregate was also correct
08:57:27 bauzas alexe9191: how do you know the scheduler was wrong, then ?
08:57:42 alexe9191 the scheduler was saying that host X is in AZ: nova
08:57:46 alexe9191 which is wrong
08:57:49 alexe9191 cause it's not
08:57:51 bauzas when ?
08:58:06 bauzas the AZFilter, you mean ?
08:58:09 alexe9191 when trying to build a machine, the AZfilter
08:58:10 alexe9191 yes
08:58:47 alexe9191         metadata = utils.aggregate_metadata_get_by_host(
08:58:48 alexe9191                 host_state, key='availability_zone')
08:58:48 alexe9191         if 'availability_zone' in metadata:
08:58:49 alexe9191             hosts_passes = availability_zone in metadata['availability_zone']
08:58:49 alexe9191             host_az = metadata['availability_zone']
08:58:50 alexe9191         else:
08:58:50 alexe9191             hosts_passes = availability_zone == CONF.default_availability_zone
08:58:51 alexe9191             host_az = CONF.default_availability_zone
08:58:51 alexe9191         if not hosts_passes:
08:58:52 alexe9191             LOG.debug("Availability Zone '%(az)s' requested. "
08:58:52 alexe9191                       "%(host_state)s has AZs: %(host_az)s",
08:58:53 alexe9191                       {'host_state': host_state,
08:58:53 alexe9191                        'az': availability_zone,
08:58:54 alexe9191                        'host_az': host_az})
08:58:59 alexe9191 ow sorry, I should've used spwan my apologies
08:59:19 bauzas by Rocky, we now have a placement prefilter for AZs
08:59:31 alexe9191 Indeed:)  We are gonna switch to that
08:59:32 bauzas you could use it
08:59:41 bauzas and verify whether it works

Earlier   Later