Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-01-27
10:05:10 bauzas it just takes all the computes, and force an instance to boot against each one
10:05:22 johnthetubaguy http://logs.openstack.org/61/417961/32/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/bee8a26/logs/new/screen-n-sch.txt.gz#_2017-01-27_03_42_11_626
10:05:28 johnthetubaguy http://logs.openstack.org/61/417961/32/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/bee8a26/logs/subnode-2/old/screen-n-cpu.txt.gz#_2017-01-27_03_31_28_948
10:05:32 bauzas so it's passing on the new node, but not on the newton subnode
10:05:53 johnthetubaguy do we truncate the hostname?
10:06:10 bauzas I don't think so
10:06:32 bauzas oh wait
10:06:56 bauzas ergh, I'm almost arriving to Paris
10:07:07 bauzas I should loose some connectivity soon
10:07:19 bauzas let's add a bit of logging before my connection drops
10:13:39 openstackgerrit Sylvain Bauza proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961
10:14:21 bauzas johnthetubaguy: I'm loosing connectivity, but ^
10:14:38 bauzas johnthetubaguy: ttyl once I'm arriving at the office
10:15:02 bauzas we won't get a lot of details by that call but hopefully, we would get enough info to see if the service is found
10:15:17 johnthetubaguy +1 thats the hack I was thinking about
10:20:18 openstackgerrit Ghanshyam Mann proposed openstack/nova: Add release note and docs for filter/sort whitelist https://review.openstack.org/421760
10:21:09 gmann johnthetubaguy: alex_xu mriedem : all_tenant things are ready now - https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/add-whitelist-for-server-list-filter-sort-parameters
10:21:34 gmann johnthetubaguy: alex_xu mriedem : updated spec and new version bump with 'no 403' on new rule
10:22:46 gmann johnthetubaguy: alex_xu mriedem : this one with spec patch also - https://review.openstack.org/#/q/status:open+branch:master+topic:bp/add-whitelist-for-server-list-filter-sort-parameters
10:27:38 johnthetubaguy gmann: thanks for fixing that up, I will try get to that
10:27:47 johnthetubaguy just chasing the approved FFEs
10:28:25 gmann johnthetubaguy: thanks, ll check tomorrow on those if any comments
10:36:27 cdent I'm available to be directed to whatever matters by whomever.
10:44:10 johnthetubaguy bauzas: http://logs.openstack.org/61/417961/32/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/bee8a26/logs/new/screen-n-cpu.txt#_2017-01-27_03_42_12_029
10:44:51 johnthetubaguy cdent: I think the priorities are the custom resource patches and bauzas's call placement patch
10:45:10 johnthetubaguy seems like we get the min_service version wrong somehow
10:45:13 johnthetubaguy but the plot thickens
10:45:29 cdent eeeph
10:45:35 johnthetubaguy bauzas: then later you see this: http://logs.openstack.org/61/417961/32/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/bee8a26/logs/new/screen-n-cpu.txt#_2017-01-27_03_42_14_659
10:48:23 johnthetubaguy I mean if the min_service version stuff is broken, thats a big deal for other things
10:48:35 johnthetubaguy so it could be a good find, or a total red herring, but thats life I guess
10:55:47 openstackgerrit Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163
10:56:35 openstackgerrit Ildiko Vancsa proposed openstack/nova: Implement new attach/detach Cinder flow https://review.openstack.org/330285
10:56:38 openstackgerrit Ildiko Vancsa proposed openstack/nova: Remove check_attach https://review.openstack.org/335358
10:59:23 openstackgerrit Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163
11:12:00 cdent johnthetubaguy: It looks like edleafe has addressed mriedem's concers on https://review.openstack.org/#/c/404472/ and I just gave it read through and nothing lept out as crazy, and the voiting bits of jenkins are happy, so it might be gtg
11:12:21 johnthetubaguy cdent: ah, sweet, I should look into that one
11:18:06 johnthetubaguy cdent: edleafe: I wonder if we should add a reno note for that patch, not a blocker as it can be a follow on, but worth squaring that away
11:20:20 cdent johnthetubaguy: perhaps, but if we're looking for an excuse to why not, this change has no visible change on end behavior (you have to look in the db or placement api with specific intent to know).
11:21:39 johnthetubaguy cdent: good point
11:21:56 johnthetubaguy "hey that thing you didn't know way rubbish, we fixed it, yey us"
11:22:14 cdent heh
11:24:01 cdent bauzas, johnthetubaguy : the current strategy on the scheduler changes is to watch the debug output and from that figure out what's next?
11:24:36 johnthetubaguy cdent: yeah
11:25:02 johnthetubaguy I tralled through the code and logs a bit this morning, haven't seem any more clues as such, just more things that confuse me
11:25:29 johnthetubaguy I haven't double checked that we correctly have the min_version caching working, that was one thread I haven't double checked
11:27:11 cdent is here a chance that the old version can be cached, leaving the cache wrong? that is, the scheduler starts up while a node is old and caches the bad version?
11:27:18 cdent we could test that by turning off the caching?
11:30:16 cdent that does seem possible
11:30:25 cdent assuming the cache is local to the current process
11:34:02 cdent johnthetubaguy: yeah, I'm pretty sure the above is possible, because unlike the conductor, the scheduler isn't required to listen to a hup to reset the cache. This is the review that put the cache in https://review.openstack.org/#/c/265326/
11:35:06 johnthetubaguy cdent: so in this case we want to ensure we get the older version
11:35:15 johnthetubaguy cdent: so I was expecting the scheduler to help really
11:35:24 johnthetubaguy I mean the cache to help
11:35:33 cdent oh. hmm.
11:36:09 cdent I guess that means we have two different problems :)
11:36:54 cdent because now we still need to make sure the cache becomes right after the final compute node becomes modern?
11:46:10 johnthetubaguy thats always going to require a restart anyways
11:46:32 johnthetubaguy for the compute RPC api version
11:47:21 cdent johnthetubaguy: yesterday we had a lot of discussions about what happens when people are "accidentally" upgrading stuff
11:48:08 cdent lots of test failures in in unit tests and things on that latest run of 417961, since bauzas is on a train should I get that stuff?
11:58:14 johnthetubaguy cdent: I think its due to a hack, those failures
11:59:49 johnthetubaguy bummer
11:59:50 johnthetubaguy ServersTestManualDisk-1134536160] Bauzas: all services: ServiceList(objects=[Service(1),Service(2),Service(3),Service(4),Service(5),Service(6),Service(7),Service(8)]) _get_all_host_states
11:59:54 johnthetubaguy thats not what we wanted there
12:00:12 cdent ugh, yeah, that's not very helpful
12:00:44 johnthetubaguy maybe fix up str(service) ?
12:01:00 johnthetubaguy then we should find out the versions its seeing
12:01:24 johnthetubaguy http://logs.openstack.org/61/417961/33/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/a2da154/logs/new/screen-n-sch.txt.gz#_2017-01-27_10_48_09_048
12:01:35 cdent Is there some way we can model some of this without having to use the gate?
12:02:17 johnthetubaguy if you have an old compute node running, and a new compute node running, and something to query the DB using objects, that should give us the same thing
12:02:46 johnthetubaguy just at this point, I really don't get whats happening, so its hard to know what we could fake
12:03:30 johnthetubaguy its tempting to also log service versions in the service updates, just to be sure
12:03:53 johnthetubaguy I can do those changes, one sec
12:18:09 openstackgerrit John Garbutt proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961
12:18:30 cdent I hope dong.wenjuan is having a nice vacation
12:18:36 johnthetubaguy cdent: I think these should help: https://review.openstack.org/#/c/417961/33..34/
12:18:49 johnthetubaguy cdent: if you could check for fat fingers, that would be awesome
12:20:38 cdent johnthetubaguy++
12:20:58 johnthetubaguy I just thought, I should add something in the service object save too
12:21:03 johnthetubaguy to make sure it gets across RPC OK
12:22:16 johnthetubaguy cdent: thanks for checking, fingers crossed on this while I get some lunch
12:22:22 openstackgerrit John Garbutt proposed openstack/nova: Scheduler calling the Placement API https://review.openstack.org/417961
12:24:00 johnthetubaguy heh
12:53:57 dakmetov greetings everyone!
13:03:40 dakmetov 2 mbooth: thank you for the great review!
13:04:19 dakmetov 2 mbooth: could you please clarify one thing: https://review.openstack.org/#/c/399679/5/nova/virt/libvirt/driver.py
13:04:46 dakmetov 2 mbooth: what do you mean under 'source info'?
13:05:38 cdent johnthetubaguy: something weird: http://logs.openstack.org/61/417961/35/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/7e52701/logs/new/screen-n-sch.txt.gz#_2017-01-27_12_55_20_946
13:05:56 cdent min is showing as 16, but a compute is 15!
13:08:56 wznoinsk sfinucan, hi
13:09:24 sfinucan wznoinsk: hey
13:09:27 sfinucan what's up?
13:09:58 bhagyashris cdent: Hi, Is the create resource_provider available for the user or it's only for the internal api for service?
13:11:47 cdent bhagyashris: It's possible for anything with the admin role to create resource providers, but at the moment there's not a lot of reason to do so. Long term the idea is that resource providers for things like shared disk farms will exist
13:12:44 bhagyashris cdent: And Is it necessary that the resource_provider name should be the host name?
13:13:20 wznoinsk sfinucan, I saw your chat with vladikr yesterday, I'm guessing he has some hardware dependent tests, was that passthrough and alike or more of networking like sriov work?
13:14:22 cdent bhagyashris: the name of the resource provider is there for humans to remember and filter on as needed. The resource providers that are automatically created for compute nodes use the hostname.
13:17:48 bhagyashris cdent: Thank you for useful information. :)
13:18:06 cdent you're welcome

Earlier   Later