| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-27 | |||
| 11:25:02 | johnthetubaguy | I tralled through the code and logs a bit this morning, haven't seem any more clues as such, just more things that confuse me | |
| 11:25:29 | johnthetubaguy | I haven't double checked that we correctly have the min_version caching working, that was one thread I haven't double checked | |
| 11:27:11 | cdent | is here a chance that the old version can be cached, leaving the cache wrong? that is, the scheduler starts up while a node is old and caches the bad version? | |
| 11:27:18 | cdent | we could test that by turning off the caching? | |
| 11:30:16 | cdent | that does seem possible | |
| 11:30:25 | cdent | assuming the cache is local to the current process | |
| 11:34:02 | cdent | johnthetubaguy: yeah, I'm pretty sure the above is possible, because unlike the conductor, the scheduler isn't required to listen to a hup to reset the cache. This is the review that put the cache in https://review.openstack.org/#/c/265326/ | |
| 11:35:06 | johnthetubaguy | cdent: so in this case we want to ensure we get the older version | |
| 11:35:15 | johnthetubaguy | cdent: so I was expecting the scheduler to help really | |
| 11:35:24 | johnthetubaguy | I mean the cache to help | |
| 11:35:33 | cdent | oh. hmm. | |
| 11:36:09 | cdent | I guess that means we have two different problems :) | |
| 11:36:54 | cdent | because now we still need to make sure the cache becomes right after the final compute node becomes modern? | |
| 11:46:10 | johnthetubaguy | thats always going to require a restart anyways | |
| 11:46:32 | johnthetubaguy | for the compute RPC api version | |
| 11:47:21 | cdent | johnthetubaguy: yesterday we had a lot of discussions about what happens when people are "accidentally" upgrading stuff | |
| 11:48:08 | cdent | lots of test failures in in unit tests and things on that latest run of 417961, since bauzas is on a train should I get that stuff? | |
| 11:58:14 | johnthetubaguy | cdent: I think its due to a hack, those failures | |
| 11:59:49 | johnthetubaguy | bummer | |
| 11:59:50 | johnthetubaguy | ServersTestManualDisk-1134536160] Bauzas: all services: ServiceList(objects=[Service(1),Service(2),Service(3),Service(4),Service(5),Service(6),Service(7),Service(8)]) _get_all_host_states | |
| 11:59:54 | johnthetubaguy | thats not what we wanted there | |
| 12:00:12 | cdent | ugh, yeah, that's not very helpful | |
| 12:00:44 | johnthetubaguy | maybe fix up str(service) ? | |
| 12:01:00 | johnthetubaguy | then we should find out the versions its seeing | |
| 12:01:24 | johnthetubaguy | http://logs.openstack.org/61/417961/33/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/a2da154/logs/new/screen-n-sch.txt.gz#_2017-01-27_10_48_09_048 | |
| 12:01:35 | cdent | Is there some way we can model some of this without having to use the gate? | |
| 12:02:17 | johnthetubaguy | if you have an old compute node running, and a new compute node running, and something to query the DB using objects, that should give us the same thing | |
| 12:02:46 | johnthetubaguy | just at this point, I really don't get whats happening, so its hard to know what we could fake | |
| 12:03:30 | johnthetubaguy | its tempting to also log service versions in the service updates, just to be sure | |
| 12:03:53 | johnthetubaguy | I can do those changes, one sec | |
| 12:18:09 | openstackgerrit | John Garbutt proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961 | |
| 12:18:30 | cdent | I hope dong.wenjuan is having a nice vacation | |
| 12:18:36 | johnthetubaguy | cdent: I think these should help: https://review.openstack.org/#/c/417961/33..34/ | |
| 12:18:49 | johnthetubaguy | cdent: if you could check for fat fingers, that would be awesome | |
| 12:20:38 | cdent | johnthetubaguy++ | |
| 12:20:58 | johnthetubaguy | I just thought, I should add something in the service object save too | |
| 12:21:03 | johnthetubaguy | to make sure it gets across RPC OK | |
| 12:22:16 | johnthetubaguy | cdent: thanks for checking, fingers crossed on this while I get some lunch | |
| 12:22:22 | openstackgerrit | John Garbutt proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961 | |
| 12:24:00 | johnthetubaguy | heh | |
| 12:53:57 | dakmetov | greetings everyone! | |
| 13:03:40 | dakmetov | 2 mbooth: thank you for the great review! | |
| 13:04:19 | dakmetov | 2 mbooth: could you please clarify one thing: https://review.openstack.org/#/c/399679/5/nova/virt/libvirt/driver.py | |
| 13:04:46 | dakmetov | 2 mbooth: what do you mean under 'source info'? | |
| 13:05:38 | cdent | johnthetubaguy: something weird: http://logs.openstack.org/61/417961/35/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/7e52701/logs/new/screen-n-sch.txt.gz#_2017-01-27_12_55_20_946 | |
| 13:05:56 | cdent | min is showing as 16, but a compute is 15! | |
| 13:08:56 | wznoinsk | sfinucan, hi | |
| 13:09:24 | sfinucan | wznoinsk: hey | |
| 13:09:27 | sfinucan | what's up? | |
| 13:09:58 | bhagyashris | cdent: Hi, Is the create resource_provider available for the user or it's only for the internal api for service? | |
| 13:11:47 | cdent | bhagyashris: It's possible for anything with the admin role to create resource providers, but at the moment there's not a lot of reason to do so. Long term the idea is that resource providers for things like shared disk farms will exist | |
| 13:12:44 | bhagyashris | cdent: And Is it necessary that the resource_provider name should be the host name? | |
| 13:13:20 | wznoinsk | sfinucan, I saw your chat with vladikr yesterday, I'm guessing he has some hardware dependent tests, was that passthrough and alike or more of networking like sriov work? | |
| 13:14:22 | cdent | bhagyashris: the name of the resource provider is there for humans to remember and filter on as needed. The resource providers that are automatically created for compute nodes use the hostname. | |
| 13:17:48 | bhagyashris | cdent: Thank you for useful information. :) | |
| 13:18:06 | cdent | you're welcome | |
| 13:21:00 | dakmetov | guys, is there any way to get user name and tenant name using userid and tenantid inside nova? | |
| 13:21:18 | dakmetov | without external lookups to keystone, I mean | |
| 13:26:06 | johnthetubaguy | cdent: crud, I think I know the problem... | |
| 13:26:35 | cdent | johnthetubaguy: oh good, because I've been staring at that trying to figure it out to no avail. do tell? | |
| 13:27:09 | johnthetubaguy | adding a comment, one sec | |
| 13:27:15 | sfinucan | wznoinsk: So yeah, he's looking to test the exposing of VLAN tag through the metadata service | |
| 13:27:36 | sfinucan | We're thinking that will require physical hardware. Not sure if it's something the Intel PCI CI could validate or not | |
| 13:27:53 | johnthetubaguy | cdent: does my comment make sense? | |
| 13:28:00 | johnthetubaguy | going to fix up now, and push it | |
| 13:28:22 | cdent | oh hell | |
| 13:28:38 | cdent | i hate it when it is something like that | |
| 13:29:23 | cdent | I just assumed that was correct because all this service version stuff is new to me | |
| 13:29:30 | cdent | savagely educated | |
| 13:29:56 | johnthetubaguy | I read that three times and checked it | |
| 13:30:07 | openstackgerrit | John Garbutt proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961 | |
| 13:30:22 | johnthetubaguy | I like that phrase | |
| 13:30:25 | johnthetubaguy | its a bit like used in anger | |
| 13:30:30 | cdent | "bugger" is the best comment ever | |
| 13:30:43 | johnthetubaguy | its the correct british way, I feel | |
| 13:30:49 | cdent | quite | |
| 13:30:56 | johnthetubaguy | one doesn't want to offend | |
| 13:31:08 | cdent | unless one really needs to | |
| 13:31:13 | cdent | we were nearly there | |
| 13:31:13 | johnthetubaguy | heh | |
| 13:31:23 | johnthetubaguy | so I left the debugging in, just in case | |
| 13:31:44 | cdent | good choice | |
| 13:35:07 | cdent | edleafe: you surfaced yet? | |
| 13:36:38 | markus_z | dansmith: If you find time to review: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/newton+topic:%22bug+1455252%22 | |
| 13:42:52 | markus_z | ^ or any other stable core, I'm not picky ;) | |
| 13:54:26 | figleaf | cdent: I'm here | |
| 13:55:07 | cdent | figleaf: just wanted to know if you're up to date on the above | |
| 13:55:20 | figleaf | cdent: still reading... | |
| 13:55:50 | figleaf | cdent: want to give me the summary? | |
| 13:56:22 | cdent | figleaf: the only thing that real matters is: https://review.openstack.org/#/c/417961/35/nova/scheduler/filter_scheduler.py | |
| 13:56:29 | cdent | (john's discovery) | |
| 13:56:44 | figleaf | just reading the 'bugger' comment :) | |
| 13:57:39 | figleaf | looks like johnthetubaguy has taken care of it | |
| 13:58:16 | cdent | yes | |
| 13:58:21 | figleaf | and I have a merge conflict on the last of the RP series https://review.openstack.org/#/c/414769/ | |
| 13:58:55 | cdent | the debugging stuff that's right around bauzas patch is still in there for the current gate run, but if it happys will be cleared out | |
| 13:59:22 | figleaf | cool | |
| 13:59:44 | cdent | we're not trying to get nested rps in ocata, or are we? | |
| 14:00:08 | figleaf | cdent: yes, but they won't be doing very much at all | |
| 14:01:05 | cdent | then I guess my rp update 9 email is wrong, could you extend it? | |