| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-05 | |||
| 15:47:13 | stephenfin | claudiub: I won't have time to look at this in depth today (on RFE review/rework duty pending feature freeze) but I could help work on this the end of next week | |
| 15:47:21 | stephenfin | any chance you could open a bug? | |
| 15:47:30 | claudiub | also, I saw that the NUMATopologyFilter was populating the limits. Why is that? Couldn't that just be initialized in the HostState anyways? It's not dependent on a request anyways. | |
| 15:48:05 | claudiub | stephenfin: will do. :) | |
| 15:49:16 | claudiub | I was wondering if, on the nova-compute side, if we get a check_can_live_migrate_destination call from somewhere and we don't get any limits, couldn't we just use the CPU / mem allocation ratios in the nova.conf file as the limits? | |
| 15:50:14 | stephenfin | I assume we could. I think the reason we package the limits up is so we can check them in the scheduler (since we don't want to use the limits of the nova-scheduler service) | |
| 15:53:51 | claudiub | ok then, will open up that issue, and I'll check how it goes if we use the nova.conf cpu / mem allocation ratios as limits if no limits are given. :) | |
| 15:54:15 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Consolidate device detach error handling https://review.opendev.org/c/openstack/nova/+/778978 | |
| 15:54:20 | stephenfin | sounds good | |
| 15:54:43 | stephenfin | lyarwood: Can you look at my two replies here before I pushed the rebased series? https://review.opendev.org/c/openstack/nova/+/775689 | |
| 15:55:00 | sean-k-mooney | claudiub: the limit not being poplulated is the corrct behaivor | |
| 15:55:00 | lyarwood | stephenfin: yup can do, just need to grab a tea first | |
| 15:55:03 | stephenfin | ack | |
| 15:55:50 | sean-k-mooney | if you do not have hw:cpu_policy=dedicated or hw:mem_page_size set then we dont really expect it to be set | |
| 15:55:59 | sean-k-mooney | although there is a sperate bug fix that is relevent | |
| 15:56:41 | sean-k-mooney | claudiub: https://review.opendev.org/c/openstack/nova/+/769614 | |
| 15:57:16 | sean-k-mooney | claudiub: the limits are depended on the request at least for cpus | |
| 15:58:09 | elod | melwitt: the functional job failure seems legit in the patch: https://review.opendev.org/c/openstack/nova/+/777218/ | |
| 15:58:19 | sean-k-mooney | claudiub: none of the behviaor your are discibing is new to me | |
| 15:58:51 | sean-k-mooney | claudiub: this is why i said added hw:numa_nodes with no other numa parmateters is not valid | |
| 15:59:00 | elod | melwitt: otherwise it would be clean & OK o:) | |
| 15:59:29 | melwitt | elod: oh, sorry I completely missed that and only saw the pep8 cherry pick fail. thanks for pointing it out, I will fix :) | |
| 16:00:01 | sean-k-mooney | claudiub: the limits are not in general related to the nova.conf allocation ratios | |
| 16:00:17 | gibi | lyarwood: fyi, now I think I fixed every comment in https://review.opendev.org/c/openstack/nova/+/770246/ with two follow up patch | |
| 16:00:56 | elod | melwitt: np :) | |
| 16:03:03 | sean-k-mooney | claudiub: well there are 2 limits object https://github.com/openstack/nova/blob/db666e2118972e501637141e48164a94f9bead54/nova/objects/numa.py#L273 and https://github.com/openstack/nova/blob/db666e2118972e501637141e48164a94f9bead54/nova/objects/request_spec.py#L1076-L1087 | |
| 16:03:35 | lyarwood | gibi++ awesome | |
| 16:03:59 | gibi | lyarwood: you had very good comments about simplifying error handling | |
| 16:04:35 | claudiub | oh ok. I only saw the host_state.limits here: https://github.com/openstack/nova/blob/master/nova/scheduler/filters/numa_topology_filter.py#L112 | |
| 16:06:12 | sean-k-mooney | that being none or not has nothing to do with your stacking problem | |
| 16:06:45 | sean-k-mooney | if you dont have hw:cpu_polciy=dedicated or hw:mem_page_size | |
| 16:07:00 | sean-k-mooney | you have not told nova to claim any reslouce form the numa nodes | |
| 16:07:19 | sean-k-mooney | so it will never decrement the capsity | |
| 16:07:47 | sean-k-mooney | therefor as long as the vm can fit on the numa node e.g. does not exceed the reouce avaiable on the node it will be accpeted | |
| 16:08:30 | claudiub | sean-k-mooney: indeed, my current stacking problem is not caused by the numa topology filter, since that is skipped if you specify the destination host on live migration. Will try live-migration with instances having that flavor extra_spec to see if it still persists. | |
| 16:08:36 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move the guest.get_disk test to test_guest https://review.opendev.org/c/openstack/nova/+/777151 | |
| 16:08:50 | sean-k-mooney | claudiub: the same logic is used on the compute hosts | |
| 16:08:51 | claudiub | (with the destination host specified) | |
| 16:09:19 | sean-k-mooney | we should be blocking all flaovr that only have hw:numa_node at the api level but we can do inter extraspec validation currenlty | |
| 16:09:40 | lyarwood | stephenfin: LGTM, sorry about that, two pointless things to block that on previously | |
| 16:09:56 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Enable mypy on libvirt/guest.py https://review.opendev.org/c/openstack/nova/+/777155 | |
| 16:10:00 | lyarwood | gibi: you're welcome! :) | |
| 16:10:30 | sean-k-mooney | claudiub: well to be percise using a flaovr with hw:numa_node and no other numa prarmater with an image that also does not set themis invalid | |
| 16:10:54 | sean-k-mooney | claudiub: the fact that you have to chekc the flavor and image is why we cand do the valdiation at flavor create time | |
| 16:10:54 | gibi | lyarwood: thanks :) | |
| 16:11:15 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Follow up type hints for a634103 https://review.opendev.org/c/openstack/nova/+/777159 | |
| 16:11:54 | sean-k-mooney | claudiub: we should technically be blocking it here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L760 | |
| 16:13:11 | sean-k-mooney | by adding a check here https://github.com/openstack/nova/blob/a65e4201cc03f89d37296ddb803934c5a7977a71/nova/virt/hardware.py#L1942 | |
| 16:14:04 | sean-k-mooney | numa_get_constraints should raise an excption in this case or default the mem_page_size=any | |
| 16:24:46 | sean-k-mooney | claudiub: i was going to defaut the pagezie request to any there fyi so that it would at least blance based on memory usage | |
| 16:58:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Add firmware auto-configuration support https://review.opendev.org/c/openstack/nova/+/776678 | |
| 16:58:54 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Add tests for insecure UEFI configuration https://review.opendev.org/c/openstack/nova/+/777367 | |
| 16:58:55 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Stop passing around virt_type, caps https://review.opendev.org/c/openstack/nova/+/775689 | |
| 16:58:59 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Add missing type hints https://review.opendev.org/c/openstack/nova/+/775688 | |
| 16:59:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Clarify purpose of 'Host.supports_*' properties https://review.opendev.org/c/openstack/nova/+/778739 | |
| 16:59:06 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Start parsing 'os_secure_boot' https://review.opendev.org/c/openstack/nova/+/682628 | |
| 16:59:12 | openstackgerrit | Stephen Finucane proposed openstack/nova master: compute: Report COMPUTE_SECURITY_UEFI_SECURE_BOOT https://review.opendev.org/c/openstack/nova/+/776679 | |
| 16:59:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Translate secure boot requests to trait https://review.opendev.org/c/openstack/nova/+/776680 | |
| 16:59:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Switch to libvirt's firmware auto-selection https://review.opendev.org/c/openstack/nova/+/775691 | |
| 16:59:31 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Wire up 'os_secure_boot' property https://review.opendev.org/c/openstack/nova/+/776681 | |
| 16:59:35 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Report secure boot support to scheduler https://review.opendev.org/c/openstack/nova/+/775690 | |
| 16:59:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add functional tests for UEFI, secure boot https://review.opendev.org/c/openstack/nova/+/776682 | |
| 16:59:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Remove duplicated 'start_compute' helper https://review.opendev.org/c/openstack/nova/+/776683 | |
| 16:59:56 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Document UEFI secure boot feature https://review.opendev.org/c/openstack/nova/+/776684 | |
| 17:00:31 | stephenfin | lyarwood: That's mostly merge conflict resolution. I'm working on undoing the firmware auto-selection changes now but it should free up everything below it to merge, at least | |
| 17:00:44 | stephenfin | just FYI | |
| 17:00:51 | stephenfin | bauzas also | |
| 17:01:03 | lyarwood | stephenfin: thanks, can we chat about the auto selection stuff on Monday just so I'm sure I get the direction? | |
| 17:01:16 | stephenfin | sure | |
| 17:01:18 | lyarwood | stephenfin: I'll also hit everything up to that first thing on Monday regardless | |
| 17:01:44 | stephenfin | in short though, I'm just going to drop that patch and go back to generating everything manually | |
| 17:01:50 | stephenfin | https://that.guru/blog/uefi-secure-boot-in-libvirt/ | |
| 17:02:08 | stephenfin | that might help explain what it was supposed to do and why we can't use it (scroll to the bottom for that) | |
| 17:02:18 | lyarwood | kk I thought so, I couldn't get my head around all of the details in the bug | |
| 17:05:35 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add device event constants to fakelibvirt https://review.opendev.org/c/openstack/nova/+/778887 | |
| 17:20:01 | kashyap | stephenfin: That's quite some speed there. | |
| 17:20:11 | kashyap | I mean, in reworking the series. | |
| 17:52:48 | openstackgerrit | melanie witt proposed openstack/nova stable/ussuri: Add regression test for bug 1914777 https://review.opendev.org/c/openstack/nova/+/777218 | |
| 17:52:51 | openstack | bug 1914777 in OpenStack Compute (nova) victoria "Possible race condition between n-cpu and n-api when deleting a building instance" [High,In progress] https://launchpad.net/bugs/1914777 - Assigned to melanie witt (melwitt) | |
| 17:52:52 | openstackgerrit | melanie witt proposed openstack/nova stable/ussuri: Handle instance = None in _local_delete_cleanup https://review.opendev.org/c/openstack/nova/+/777219 | |
| 18:32:15 | admin0 | hi all openstack server show $uuid => hostId -- how do i check what hostId is which hypervisor ? | |
| 18:33:43 | sean-k-mooney | as a non admin you cant intntionally | |
| 18:33:55 | sean-k-mooney | ifyou are an admin you can check it differently | |
| 18:34:29 | sean-k-mooney | hostid is a hash of the hostname salted by your poject id | |
| 18:35:35 | sean-k-mooney | OS-EXT-SRV-ATTR:hypervisor_hostname should be avaiable as an admin | |
| 18:36:32 | sean-k-mooney | hostid: "An ID string representing the host. This is a hashed value so will not actually look like a hostname, and is hashed with data from the project_id, so the same physical host as seen by two different project_ids, will be different. It is useful when within the same project you need to determine if two instances are on the same or different physical hosts for the purposes of | |
| 18:36:34 | sean-k-mooney | availability or performance." | |
| 18:36:39 | sean-k-mooney | from the api ref ^ | |
| 18:38:22 | sean-k-mooney | admin0: hopefully that helps. its considerd a security issue if vms can determin what host they are running on directly | |
| 18:38:40 | sean-k-mooney | assuming they can use that info to target another vm to the same host | |
| 18:39:12 | sean-k-mooney | that is why the hostid is hashed and different proejct will see different values for the same host | |
| 18:39:18 | admin0 | oh .. got it | |
| 18:49:43 | openstackgerrit | sean mooney proposed openstack/nova master: support per port numa policies with sriov https://review.opendev.org/c/openstack/nova/+/773792 | |
| 19:27:44 | dansmith | has anyone noticed compute nodes getting 500 errors from placement? | |
| 19:27:45 | dansmith | https://zuul.opendev.org/t/openstack/build/19989232c4b94c848894c6e54d9fa49c/log/logs/subnode-2/screen-n-cpu.txt#9204 | |
| 19:28:25 | dansmith | oh you know, | |
| 19:28:38 | dansmith | that might be during the upgrade part where the service is down | |
| 19:35:48 | melwitt | dansmith: I haven't noticed that before but can see the corresponding request doesn't show up in the placement logs, so that seems to support what you're thinking that it's when the service is down? I didn't know how/if the service can 500 while it is maybe partially up/in a state where it would not log the request in placement. timestamps match too | |
| 19:36:21 | dansmith | melwitt: because of the TLS proxies I expect | |
| 19:36:44 | dansmith | melwitt: after that in the log, conductor is down for a while when it's trying to update its service record, which I think is the key indicating we're in the upgrade phase | |