| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-15 | |||
| 16:24:20 | jaypipes | bauzas: s/saved the world/caused the world to burn to a crisp/ | |
| 16:25:14 | bauzas | jaypipes: you should just feel like http://api.theweek.com/sites/default/files/styles/tw_image_9_4/public/50692_article_full.jpg?itok=X1oW13j5&resize=1260x560 | |
| 16:25:28 | jaypipes | heh | |
| 16:25:29 | bauzas | with the catchphrase | |
| 16:25:48 | bauzas | mriedem: to your question about stats, sec | |
| 16:26:07 | bauzas | I don't want to say something wrong | |
| 16:26:52 | mriedem | running_vms i guess? | |
| 16:26:54 | mriedem | oops | |
| 16:26:58 | mriedem | get_host_cpu_stats | |
| 16:27:02 | mriedem | which goes into the cpu monitor thing | |
| 16:27:19 | mriedem | and we said no more memory or nic bandwidth monitors | |
| 16:27:27 | mriedem | no more metrics or stats collection | |
| 16:28:14 | mriedem | or not, because that goes into compute_nodes.metrics | |
| 16:28:15 | bauzas | mriedem: yeah, so stats are compute monitors | |
| 16:29:13 | mriedem | what sets resources['stats'] though? https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L554 | |
| 16:29:24 | mriedem | resources comes from driver.get_available_resources() | |
| 16:29:35 | bauzas | mriedem: from compute.monitors | |
| 16:29:51 | bauzas | are you really sure you want to enter that rathole ? | |
| 16:30:01 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/474672 | |
| 16:30:50 | mriedem | i'm not making the connection between the cpu metric monitor and how that sets a "stats" key in the resources dict that the RT is using | |
| 16:32:28 | openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/467173 | |
| 16:32:48 | mriedem | i don't think anything sets this https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L554 | |
| 16:33:32 | bauzas | I have a fun KB | |
| 16:33:33 | bauzas | https://www.ibm.com/support/knowledgecenter/SS8MU9_2.3.0/Admin/tasks/integratingplugin.html | |
| 16:34:23 | mriedem | well it used to be stevedore plugin extensions i think | |
| 16:34:30 | mriedem | i think we might have killed that in newton or so | |
| 16:35:21 | mriedem | anyway, i don't even remember what i was doing 40 minutes ago when i went down this mess, and now it's lunch time | |
| 16:37:36 | bauzas | gosh, all that spaghetti code is confusing me too | |
| 16:39:25 | bauzas | hmpf, looks like I was confusing stats and metrics | |
| 16:39:54 | mriedem | i don't see how that could be possibly confusing | |
| 16:40:19 | bauzas | but anyway, we have *something* that is amending the HostState so that ComputeCapabilitiesFilter can use it thru the extra specs | |
| 16:44:01 | bauzas | oh, and I remember other thing | |
| 16:44:15 | bauzas | mriedem: virt drivers are reporting the instances they can support | |
| 16:44:25 | bauzas | mriedem: thru a supported_instances field | |
| 16:45:07 | bauzas | anyway, I need to bail out | |
| 16:45:41 | bauzas | not sure we can just summarize all of that but saying it's just hard | |
| 16:46:16 | NostawRm | lol, I thought I'd try to do what I was saying with the accessIPv4, but its not as straight forward as I'd thought just because i was confused thinking that 'instance' was an arg for get_injected_network_template() | |
| 17:01:16 | openstackgerrit | Merged openstack/python-novaclient master: Revert "client.logger.warning wrongly used in migrations" https://review.openstack.org/474600 | |
| 17:25:52 | openstackgerrit | Gábor Antal proposed openstack/nova master: Transform instance.resize_prep notification https://review.openstack.org/465081 | |
| 17:44:34 | eandersson | How do I output python logging even on successful tests? | |
| 17:45:24 | eandersson | Want to compare the output from successful and failed runs | |
| 17:45:36 | clarkb | eandersson: the logs are collected and are in the subunit, you should be able to run the subunit output through a filter or just look at the subunit log file after hte tests run (I tend to just read the data afterwards .testrepository/$testrunid | |
| 17:52:36 | cfriesen | bauzas: when running "nova-manage db online_data_migrations" in newton is it expected that the pci_requests field in the "instance_extra" table would be nulled out as the data is copied to the "request_specs" table in the nova_api db? This seems like it would prevent the instance from getting at instance.pci_requests. | |
| 17:53:45 | cfriesen | bauzas: ah, nevermind. looks like it was a different failure that was doing that, not the actual online data migration. | |
| 17:54:47 | efried | sdague Can we get a quick merge on this 'un: https://review.openstack.org/#/c/473975/ ? | |
| 17:54:54 | sdague | efried: looking | |
| 17:55:34 | sdague | efried: +2 | |
| 17:55:38 | efried | Thanks. | |
| 18:07:02 | mriedem | efried: powervm ci failed on that change http://184.172.12.213/75/473975/1/check/nova-in-tree-pvm/973064a/console.html | |
| 18:07:18 | mriedem | /opt/stack/devstack/lib/neutron: line 301: neutron_plugin_create_nova_conf: command not found | |
| 18:10:37 | efried | mriedem Yeah, it's the CI, not the change; we're working on getting it back up, allllmost there. | |
| 18:11:15 | eandersson | melwitt I think I added the dependency on the wrong patch :D 474404 depends on 390984 | |
| 18:11:18 | eandersson | Should be the other way around | |
| 18:13:25 | melwitt | jaypipes: heh. it's not wrong bc Melania is the same name, just in a different language | |
| 18:14:57 | melwitt | eandersson: yeah. looks like you're still seeing failures with it though? looking ... | |
| 18:15:22 | eandersson | Yea - it's odd | |
| 18:15:45 | melwitt | it keeps saying the "received update from unknown host" | |
| 18:16:09 | eandersson | I almost believe that the successful runs are actually the bad ones | |
| 18:16:53 | eandersson | And the fails are legit | |
| 18:19:57 | eandersson | It basically looks like the resource usage isn't registered at all, and because there are two nodes, when the host gets re-registered new vms can land on either host1 or host2 (not based on resource usage). | |
| 18:21:13 | eandersson | Maybe an issue with the new variable memory_mb_used ? https://review.openstack.org/#/c/390984/13/nova/scheduler/host_manager.py | |
| 18:21:30 | eandersson | This for sure worked when I tested it, but maybe something has changed since I made the patch... last year | |
| 18:22:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: fix hypervisor_hostname description for Ironic https://review.openstack.org/474723 | |
| 18:23:54 | eandersson | Yea - I don't think that variable is updated | |
| 18:25:23 | openstackgerrit | melanie witt proposed openstack/nova master: placement project_id, user_id in PUT /allocations https://review.openstack.org/469634 | |
| 18:25:24 | openstackgerrit | melanie witt proposed openstack/nova master: placement: Add GET /usages to placement API https://review.openstack.org/469635 | |
| 18:29:52 | eandersson | Those values aren't updated on master either, so I guess that is wai | |
| 18:32:46 | melwitt | eandersson: I'm going to pull down your patch and run the problematic test with debug logging enabled (you can do by OS_DEBUG=1 tox -efunctional ...) and see where the instances are landing. the test is trying to boot 3 servers and the third is supposed to fail to find a host with capacity, because of the affinity policy they're supposed to all land on the same host, I think | |
| 18:33:22 | eandersson | So I am pretty sure the issue is with the above variable | |
| 18:33:44 | eandersson | Unless having spent too many hours on this is making me delirious | |
| 18:33:50 | eandersson | It looks like the value is never updated | |
| 18:33:56 | melwitt | eandersson: oh, okay. sorry I didn't understand that part :) | |
| 18:36:25 | eandersson | No worries - I understand very little of this in general hah | |
| 18:38:15 | eandersson | and thaks for the help melwitt | |
| 18:40:56 | melwitt | np. I'm concerned if that test is busted | |
| 18:41:33 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: [WIP] Deactivate powervm compute until ready https://review.openstack.org/471773 | |
| 18:47:32 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: [WIP] Implement virt driver set_host_enabled https://review.openstack.org/472794 | |
| 18:49:45 | melwitt | eandersson: from what I see in the debug logging, you're right that memory_mb_used stays stuck at 512. it never increases after each instance is booted | |
| 18:51:31 | eandersson | Maybe that calculation has moved since I created that patch. | |
| 18:52:32 | eandersson | Or maybe I messed up somewhere! :D | |
| 18:52:52 | eandersson | Then again not sure how all those unit-tests could fail if that was so | |
| 18:53:12 | eandersson | *could not | |
| 18:53:19 | melwitt | hang on, I'm also seeing free_ram_mb not changing. | |
| 19:06:25 | mriedem | sdague: huzzah! | |
| 19:06:58 | mriedem | always have to remember my old thunderbird bug trick - after an update, threading is broken, and you have to select "threading with unread" to fix threading, and then do "threading > all" to get threading on read and unread | |
| 19:07:01 | mriedem | yay for bugs | |
| 19:07:51 | melwitt | my thunderbird trick is not using threading (I didn't know it could thread) | |
| 19:08:07 | mriedem | i need threading | |
| 19:08:27 | melwitt | I might need it but I don't know it yet | |
| 19:09:54 | mriedem | we already had like 10lb of brown sugar in the house but since laura couldn't find it immediately she decided that we needed another 5lbs | |
| 19:10:02 | mriedem | so now i'm swimming in gd brown sugar | |
| 19:10:16 | openstackgerrit | Merged openstack/nova master: Stop caching compute nodes in the request https://review.openstack.org/474382 | |
| 19:11:42 | melwitt | heh | |
| 19:12:12 | melwitt | you have a lot of baked goods in your future | |
| 19:13:03 | mriedem | i do | |
| 19:13:14 | mriedem | i asked for homemade oatmeal raisin cookies for fathers day | |
| 19:13:19 | mriedem | sans chocolate chips | |
| 19:13:29 | mriedem | b/c if i wanted chocolate chip cookies, i'd have asked for those | |
| 19:14:16 | melwitt | haha | |
| 19:14:24 | clarkb | mriedem: as a cookie eater I'm happy that people like you don't eat the chocolate chip cookies so that there are more for me | |