| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-30 | |||
| 21:59:37 | mriedem | is tempest doing the 2nd delete during a cleanup? | |
| 21:59:43 | melwitt | I don't yet know why the teardown of the test is trying to delete an already deleted volume | |
| 21:59:50 | melwitt | yeah, it's the cleanup step that fails | |
| 21:59:51 | mriedem | i.e. first delete is explicit in the test, but not waiting for the volume to actually be gone | |
| 21:59:58 | melwitt | that's my guess but haven't confirmed yet | |
| 22:00:08 | mriedem | because most of tempest adds an automatic cleanup if you create the resource in one of the base test methods | |
| 22:00:17 | mriedem | so the test likely just needs to wait for the volume to be gone, | |
| 22:00:22 | mriedem | then on cleanup, it will handle the 404 and ignore it | |
| 22:00:42 | melwitt | ah, got it | |
| 22:01:34 | melwitt | hm, none of the tests seem to be doing an explicit delete in the test https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_backup.py | |
| 22:04:38 | melwitt | ohhh, wait, this is a cinder in-tree tempest test from ocata, so different than what I just linked | |
| 22:09:58 | openstackgerrit | Merged openstack/nova master: Expand tests for multiple shared resources case https://review.openstack.org/569637 | |
| 22:17:25 | melwitt | ah, indeed we have redundant cleanups because the test is adding a cleanup and the base class is also adding a cleanup during self.create_volume https://github.com/openstack/cinder/blob/stable/ocata/cinder/tests/tempest/api/volume/test_volume_backup.py#L77 | |
| 22:18:54 | eandersson | Is numa pinning exposed in any api calls? | |
| 22:21:59 | openstackgerrit | Jan Gutter proposed openstack/nova master: Convert vrouter legacy plugging to os-vif https://review.openstack.org/571325 | |
| 22:22:06 | openstackgerrit | Jan Gutter proposed openstack/os-vif master: Add multiqueue field to InstanceInfo https://review.openstack.org/570959 | |
| 22:22:42 | melwitt | eandersson: do you mean the flavor extra specs that show what numa pinning was requested? it will show that in the server detail GET call and recently will also show it via the flavors API as of this recent bp https://blueprints.launchpad.net/nova/+spec/add-extra-specs-to-flavor-list | |
| 22:24:46 | eandersson | I think case I actually wanted to see the pinning on the compute itself. | |
| 22:25:14 | eandersson | For each VM | |
| 22:26:17 | eandersson | We wrote an ansible playbook for this, and checked each host using that, but would have been nice to be able to get that kind of data from the api as well. | |
| 22:26:22 | eandersson | (as an admin at least) | |
| 22:27:11 | melwitt | I see. I have limited knowledge about it, but maybe cfriesen might know if he's still around today | |
| 22:27:30 | eandersson | In this particular case we figured out that most of our hosts had the network interrupt rx queue pinned to a specific core | |
| 22:27:46 | eandersson | (well the ones that didn't have multiqueue enabled anyway) | |
| 22:29:55 | mriedem | eandersson: there isn't anything in the server details that say where an instance is pinned | |
| 22:30:31 | mriedem | stephenfin: dansmith: went over the numa aware vswitch spec again, i think it's pretty close, just missed some cleanups and other small details https://review.openstack.org/#/c/541290/ | |
| 22:31:18 | mriedem | eandersson: there isn't anything in the os-hypervisors API either | |
| 22:31:23 | eandersson | figured as much, but wanted to double check (as the data is available in the db at least) | |
| 22:31:32 | mriedem | NUMA and CPU pinning stuff will eventually be expressed in the placement API | |
| 22:31:38 | eandersson | Nice | |
| 22:31:51 | mriedem | there used to be an os-pci API added back in juno and removed later in i think pike (or maybe older) | |
| 22:31:55 | mriedem | since it never did anything | |
| 22:32:43 | mriedem | eandersson: btw, was it you or someone else from your company that asked about the TypeAffinityFilter being removed during the nova project update session in vancouver? | |
| 22:32:48 | mriedem | or just totally coincidental? | |
| 22:33:14 | eandersson | One of my guys asked about it, but heard that someone else also asked about it | |
| 22:33:29 | eandersson | as well | |
| 22:34:12 | mriedem | i heard someone named their first born "TypeAffinityFilter" they loved it so much | |
| 22:34:17 | eandersson | haha | |
| 22:34:45 | eandersson | To be fair the version in nova stinks, especially since it's based on uuid | |
| 22:35:19 | mriedem | uuid? the one that was in tree was based on the flavor.id primary key | |
| 22:35:26 | mriedem | which is pretty much the reason it was removed | |
| 22:35:29 | eandersson | Yea | |
| 22:35:35 | eandersson | Our version was always based on name, not id | |
| 22:35:49 | mriedem | because i can create flavorid=foo with id=1, delete it and recreate it with the same flavorid but now it's id=2 | |
| 22:35:54 | mriedem | and your affinity stuff is all busted | |
| 22:36:01 | mriedem | ok | |
| 22:36:03 | mriedem | well, fyi http://lists.openstack.org/pipermail/openstack-operators/2018-May/015277.html | |
| 22:36:45 | mriedem | https://review.openstack.org/#/c/569247/ will cause a perf penalty on out of tree filters that need to access any fields on HostState.instances values | |
| 22:37:03 | mriedem | so if people (ops at large) are needing a type affinity filter, one should probably be proposed upstream | |
| 22:40:03 | eandersson | btw unrelated but was a feature to allow rebuild to move the VM to a new compute? | |
| 22:40:15 | eandersson | or even restart (similar to how Amazon does it) | |
| 22:40:24 | eandersson | ever discussed? | |
| 22:41:09 | eandersson | We are trying to find a strategy to make it easier for our customers when we need to bring a compute down for maintenance | |
| 22:42:49 | mriedem | eandersson: that's the evacuate API | |
| 22:43:14 | eandersson | Yea - I guess it's the same thing. | |
| 22:43:22 | mriedem | that's after nova-compute is down though | |
| 22:43:30 | mriedem | https://docs.openstack.org/nova/latest/admin/evacuate.html | |
| 22:44:13 | eandersson | Our ideal scenario would be that if we want to do maintenance on compute X, we mark it as "going down in 7 days", and anyone that hits rebuild on their VM on that host gets moved to a new compute | |
| 22:44:24 | eandersson | and after 7 days they would just automatically get rebuilt and moved | |
| 22:44:48 | mriedem | i think you could build that on top of existing APIs in an external service | |
| 22:45:00 | eandersson | Yea - that is our general plan of action for now | |
| 22:45:03 | mriedem | you can force a compute service down so new things can't go to it but the existing VMs continue to run | |
| 22:45:27 | mriedem | https://developer.openstack.org/api-ref/compute/#update-forced-down | |
| 22:45:46 | mriedem | so you can force it down to evacuate things off it, | |
| 22:45:50 | mriedem | or disable it so new things can't go to it | |
| 22:45:53 | eandersson | Nice | |
| 22:46:24 | mriedem | rebuild to the same host will work while the service is disabled, it won't auto-migrate it | |
| 22:46:45 | mriedem | non-admins can only move during things like shelve_offload/unshelve and resize | |
| 22:47:08 | mriedem | your external thing could mark the compute as disabled/down, and then after 7 days, trigger a cold migration | |
| 22:47:17 | mriedem | or live if you support that | |
| 22:48:13 | mriedem | actually if you disable the compute, shelve all the instances, unshelve them, you get the same thing you're asking for | |
| 22:48:38 | mriedem | with some additional glance snapshot traffic, but otherwise the same as rebuild | |
| 23:20:10 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (6) https://review.openstack.org/571330 | |
| 23:52:06 | openstackgerrit | karim proposed openstack/nova master: Handle rebuild of instances with image traits https://review.openstack.org/569498 | |
| 23:59:45 | openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Add "must" to automatic names for multiple servers https://review.openstack.org/571335 | |
| #openstack-nova - 2018-05-31 | |||
| 00:20:29 | openstackgerrit | Merged openstack/nova master: add parameter docstring for 'params' to libvirt.guest.Guest.migrate() https://review.openstack.org/571236 | |
| 00:45:42 | openstackgerrit | Merged openstack/nova master: mirror nova host aggregate members to placement https://review.openstack.org/553597 | |
| 00:45:52 | openstackgerrit | Merged openstack/nova master: Remove mox in libvirt/test_driver.py (3) https://review.openstack.org/569320 | |
| 00:46:00 | openstackgerrit | Merged openstack/nova master: libvirt: Skip fetching the virtual size of block devices https://review.openstack.org/569062 | |
| 00:50:44 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in test_compute_api.py (3) https://review.openstack.org/568184 | |
| 00:50:57 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in test_compute_api.py (4) https://review.openstack.org/568462 | |
| 01:00:17 | eandersson | mriedem_afk, we do something similar already for upgrades | |
| 01:00:40 | eandersson | we move computes into a upgrade aggregate and once there are no more vms we autoamtically upgrade os, firmware etc | |
| 01:01:24 | eandersson | (we just don't enforce a timelimit yet at least :p) | |
| 01:01:56 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Change the anti-affinity Filter to adapt to new policy https://review.openstack.org/571166 | |
| 01:36:58 | openstackgerrit | wanghongtao proposed openstack/nova master: Fix the metadata re to match the unicode https://review.openstack.org/571352 | |
| 01:36:58 | openstackgerrit | wanghongtao proposed openstack/nova master: Fix the metadata re to match the unicode https://review.openstack.org/536236 | |
| 01:44:53 | openstackgerrit | tianhui proposed openstack/nova master: Fix some wrong urls in doc https://review.openstack.org/571147 | |
| 02:16:49 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: metadata: add vf_trusted field to device metadata https://review.openstack.org/566344 | |
| 02:16:50 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: network: update pci request spec to handle trusted tags https://review.openstack.org/458820 | |
| 02:28:34 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: metadata: add vf_trusted field to device metadata https://review.openstack.org/566344 | |
| 02:28:35 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: network: update pci request spec to handle trusted tags https://review.openstack.org/458820 | |
| 03:35:55 | bhagyashris | efried; yes | |
| 03:36:03 | bhagyashris | efried: yes | |
| 03:53:00 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: metadata: add vf_trusted field to device metadata https://review.openstack.org/566344 | |
| 03:53:01 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: network: update pci request spec to handle trusted tags https://review.openstack.org/458820 | |
| 03:57:46 | openstackgerrit | tianhui proposed openstack/nova master: Fix bug to api-ref https://review.openstack.org/571375 | |
| 04:16:27 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: always create consumer records https://review.openstack.org/567678 | |
| 04:16:28 | openstackgerrit | Jay Pipes proposed openstack/nova master: add consumers generation field https://review.openstack.org/557958 | |
| 04:16:29 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: Allocation.consumer field https://review.openstack.org/565405 | |