| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-15 | |||
| 16:19:42 | mriedem | er https://developer.openstack.org/api-ref/compute/?expanded=show-hypervisor-statistics-detail#show-hypervisor-statistics | |
| 16:20:01 | mriedem | so capabilities:running_vms < 10 ? | |
| 16:20:11 | bauzas | mriedem: https://github.com/openstack/nova/blob/master/nova/tests/unit/scheduler/filters/test_compute_capabilities_filters.py#L72 | |
| 16:20:20 | bauzas | mriedem: ^ is the flavor extra specs | |
| 16:20:29 | bauzas | mriedem: but you can ask for anything in the HostState object | |
| 16:20:43 | bauzas | of course, cpu_info is one attribute of it | |
| 16:20:49 | mriedem | so the answer is, go look at the HostState code | |
| 16:20:54 | bauzas | mriedem: exactly | |
| 16:20:58 | mriedem | which is a shit answer | |
| 16:21:01 | bauzas | exactly | |
| 16:21:19 | bauzas | no, it's just something driven by hack | |
| 16:21:45 | bauzas | but until we get traits sorting, that's the only thing you can do | |
| 16:22:03 | bauzas | lovely, isn't it ? | |
| 16:22:10 | mriedem | bauzas: is the HostState fields that feed that filter finite? | |
| 16:22:15 | openstackgerrit | John Griffith proposed openstack/nova master: WIP/POC Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 16:22:16 | mriedem | s/is/are/ | |
| 16:22:22 | bauzas | mriedem: absolutely not | |
| 16:22:25 | mriedem | gdi | |
| 16:22:35 | bauzas | mriedem: because you can use stats | |
| 16:22:44 | bauzas | and voila, you can report anything you want | |
| 16:23:17 | mriedem | but what feeds the stats? | |
| 16:23:24 | bauzas | I refrained in the past a long list of changes wanting to add more things in the HostState or filtering based on any cool attributes | |
| 16:23:55 | bauzas | the only answer I was able to provide before jaypipes saved the world with ResourceProviders and Traits was "go use stats" | |
| 16:24:20 | jaypipes | bauzas: s/saved the world/caused the world to burn to a crisp/ | |
| 16:25:14 | bauzas | jaypipes: you should just feel like http://api.theweek.com/sites/default/files/styles/tw_image_9_4/public/50692_article_full.jpg?itok=X1oW13j5&resize=1260x560 | |
| 16:25:28 | jaypipes | heh | |
| 16:25:29 | bauzas | with the catchphrase | |
| 16:25:48 | bauzas | mriedem: to your question about stats, sec | |
| 16:26:07 | bauzas | I don't want to say something wrong | |
| 16:26:52 | mriedem | running_vms i guess? | |
| 16:26:54 | mriedem | oops | |
| 16:26:58 | mriedem | get_host_cpu_stats | |
| 16:27:02 | mriedem | which goes into the cpu monitor thing | |
| 16:27:19 | mriedem | and we said no more memory or nic bandwidth monitors | |
| 16:27:27 | mriedem | no more metrics or stats collection | |
| 16:28:14 | mriedem | or not, because that goes into compute_nodes.metrics | |
| 16:28:15 | bauzas | mriedem: yeah, so stats are compute monitors | |
| 16:29:13 | mriedem | what sets resources['stats'] though? https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L554 | |
| 16:29:24 | mriedem | resources comes from driver.get_available_resources() | |
| 16:29:35 | bauzas | mriedem: from compute.monitors | |
| 16:29:51 | bauzas | are you really sure you want to enter that rathole ? | |
| 16:30:01 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/474672 | |
| 16:30:50 | mriedem | i'm not making the connection between the cpu metric monitor and how that sets a "stats" key in the resources dict that the RT is using | |
| 16:32:28 | openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/467173 | |
| 16:32:48 | mriedem | i don't think anything sets this https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L554 | |
| 16:33:32 | bauzas | I have a fun KB | |
| 16:33:33 | bauzas | https://www.ibm.com/support/knowledgecenter/SS8MU9_2.3.0/Admin/tasks/integratingplugin.html | |
| 16:34:23 | mriedem | well it used to be stevedore plugin extensions i think | |
| 16:34:30 | mriedem | i think we might have killed that in newton or so | |
| 16:35:21 | mriedem | anyway, i don't even remember what i was doing 40 minutes ago when i went down this mess, and now it's lunch time | |
| 16:37:36 | bauzas | gosh, all that spaghetti code is confusing me too | |
| 16:39:25 | bauzas | hmpf, looks like I was confusing stats and metrics | |
| 16:39:54 | mriedem | i don't see how that could be possibly confusing | |
| 16:40:19 | bauzas | but anyway, we have *something* that is amending the HostState so that ComputeCapabilitiesFilter can use it thru the extra specs | |
| 16:44:01 | bauzas | oh, and I remember other thing | |
| 16:44:15 | bauzas | mriedem: virt drivers are reporting the instances they can support | |
| 16:44:25 | bauzas | mriedem: thru a supported_instances field | |
| 16:45:07 | bauzas | anyway, I need to bail out | |
| 16:45:41 | bauzas | not sure we can just summarize all of that but saying it's just hard | |
| 16:46:16 | NostawRm | lol, I thought I'd try to do what I was saying with the accessIPv4, but its not as straight forward as I'd thought just because i was confused thinking that 'instance' was an arg for get_injected_network_template() | |
| 17:01:16 | openstackgerrit | Merged openstack/python-novaclient master: Revert "client.logger.warning wrongly used in migrations" https://review.openstack.org/474600 | |
| 17:25:52 | openstackgerrit | Gábor Antal proposed openstack/nova master: Transform instance.resize_prep notification https://review.openstack.org/465081 | |
| 17:44:34 | eandersson | How do I output python logging even on successful tests? | |
| 17:45:24 | eandersson | Want to compare the output from successful and failed runs | |
| 17:45:36 | clarkb | eandersson: the logs are collected and are in the subunit, you should be able to run the subunit output through a filter or just look at the subunit log file after hte tests run (I tend to just read the data afterwards .testrepository/$testrunid | |
| 17:52:36 | cfriesen | bauzas: when running "nova-manage db online_data_migrations" in newton is it expected that the pci_requests field in the "instance_extra" table would be nulled out as the data is copied to the "request_specs" table in the nova_api db? This seems like it would prevent the instance from getting at instance.pci_requests. | |
| 17:53:45 | cfriesen | bauzas: ah, nevermind. looks like it was a different failure that was doing that, not the actual online data migration. | |
| 17:54:47 | efried | sdague Can we get a quick merge on this 'un: https://review.openstack.org/#/c/473975/ ? | |
| 17:54:54 | sdague | efried: looking | |
| 17:55:34 | sdague | efried: +2 | |
| 17:55:38 | efried | Thanks. | |
| 18:07:02 | mriedem | efried: powervm ci failed on that change http://184.172.12.213/75/473975/1/check/nova-in-tree-pvm/973064a/console.html | |
| 18:07:18 | mriedem | /opt/stack/devstack/lib/neutron: line 301: neutron_plugin_create_nova_conf: command not found | |
| 18:10:37 | efried | mriedem Yeah, it's the CI, not the change; we're working on getting it back up, allllmost there. | |
| 18:11:15 | eandersson | melwitt I think I added the dependency on the wrong patch :D 474404 depends on 390984 | |
| 18:11:18 | eandersson | Should be the other way around | |
| 18:13:25 | melwitt | jaypipes: heh. it's not wrong bc Melania is the same name, just in a different language | |
| 18:14:57 | melwitt | eandersson: yeah. looks like you're still seeing failures with it though? looking ... | |
| 18:15:22 | eandersson | Yea - it's odd | |
| 18:15:45 | melwitt | it keeps saying the "received update from unknown host" | |
| 18:16:09 | eandersson | I almost believe that the successful runs are actually the bad ones | |
| 18:16:53 | eandersson | And the fails are legit | |
| 18:19:57 | eandersson | It basically looks like the resource usage isn't registered at all, and because there are two nodes, when the host gets re-registered new vms can land on either host1 or host2 (not based on resource usage). | |
| 18:21:13 | eandersson | Maybe an issue with the new variable memory_mb_used ? https://review.openstack.org/#/c/390984/13/nova/scheduler/host_manager.py | |
| 18:21:30 | eandersson | This for sure worked when I tested it, but maybe something has changed since I made the patch... last year | |
| 18:22:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: fix hypervisor_hostname description for Ironic https://review.openstack.org/474723 | |
| 18:23:54 | eandersson | Yea - I don't think that variable is updated | |
| 18:25:23 | openstackgerrit | melanie witt proposed openstack/nova master: placement project_id, user_id in PUT /allocations https://review.openstack.org/469634 | |
| 18:25:24 | openstackgerrit | melanie witt proposed openstack/nova master: placement: Add GET /usages to placement API https://review.openstack.org/469635 | |
| 18:29:52 | eandersson | Those values aren't updated on master either, so I guess that is wai | |
| 18:32:46 | melwitt | eandersson: I'm going to pull down your patch and run the problematic test with debug logging enabled (you can do by OS_DEBUG=1 tox -efunctional ...) and see where the instances are landing. the test is trying to boot 3 servers and the third is supposed to fail to find a host with capacity, because of the affinity policy they're supposed to all land on the same host, I think | |
| 18:33:22 | eandersson | So I am pretty sure the issue is with the above variable | |
| 18:33:44 | eandersson | Unless having spent too many hours on this is making me delirious | |
| 18:33:50 | eandersson | It looks like the value is never updated | |
| 18:33:56 | melwitt | eandersson: oh, okay. sorry I didn't understand that part :) | |
| 18:36:25 | eandersson | No worries - I understand very little of this in general hah | |
| 18:38:15 | eandersson | and thaks for the help melwitt | |
| 18:40:56 | melwitt | np. I'm concerned if that test is busted | |
| 18:41:33 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: [WIP] Deactivate powervm compute until ready https://review.openstack.org/471773 | |
| 18:47:32 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: [WIP] Implement virt driver set_host_enabled https://review.openstack.org/472794 | |