| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-20 | |||
| 12:02:21 | bauzas | yeah | |
| 12:02:25 | sdague | I think this biggest trick of wsgiref is going to be picking ports | |
| 12:02:40 | sdague | eventlet can bind an unused port and give it back to you | |
| 12:02:52 | sdague | which is what we use | |
| 12:02:58 | bauzas | but I have to be explicit for wsgiref? gotcha | |
| 12:03:01 | sdague | yeh | |
| 12:03:17 | bauzas | let's try to do something at least | |
| 12:04:17 | bauzas | sdague: from a upper standpoint, our functional tests were working because we still update the DB | |
| 12:04:40 | bauzas | sdague: which is unfortunate, I thought we were already mocking that placement upcall from the compute | |
| 12:04:52 | bauzas | that's why we're at risk now | |
| 12:05:00 | bauzas | I haven't identified it would require a new fixture | |
| 12:05:37 | sdague | bauzas: actually, have you tried just launching the placement app in nova.wsgi.Server? | |
| 12:05:48 | bauzas | sdague: I was tempted to | |
| 12:06:09 | sdague | because, yes, that uses eventlet to do it | |
| 12:06:22 | sdague | but ... it's not really that eventlet specific | |
| 12:06:27 | sdague | as I'm looking through the code | |
| 12:07:20 | sdague | I think the only tricky part is the paste load is a little different | |
| 12:07:31 | sdague | but, you might just be able to do that | |
| 12:07:40 | bauzas | lemme write that | |
| 12:07:44 | bauzas | my fixture is there | |
| 12:07:48 | bauzas | I just need to update it | |
| 12:08:07 | bauzas | and my change is a canary patch so I can see | |
| 12:19:32 | bauzas | sdague: that looks working | |
| 12:19:36 | bauzas | using wsgi.Server() | |
| 12:19:53 | bauzas | I now need to fake the keystone calls | |
| 12:19:59 | bauzas | and verify a few tings | |
| 12:25:34 | openstackgerrit | sahid proposed openstack/nova: libvirt: make coherent logs when reboot success https://review.openstack.org/416994 | |
| 12:25:35 | openstackgerrit | sahid proposed openstack/nova: libvirt: rework _destroy method https://review.openstack.org/416995 | |
| 12:25:35 | openstackgerrit | sahid proposed openstack/nova: libvirt: rework hard reboot https://review.openstack.org/376350 | |
| 12:30:56 | snikitin | BobBall: Hi, Bob! JFYI: I found the reason of diagnostics tempest tests fail. Sometimes (I don't know why) xen returns "NaN" instead of numbers. For example I saw such response in Xen CI job: {"last_update": "1484900660", "cpu0": "NaN", "memory": "NaN"}. | |
| 12:30:56 | snikitin | My code can't convert float('NaN') into int and this is the reason of the fails. On my local Xen env everything is ok. I saw 'NaN' values only one time and I don't know how to reproduce it. But on CI I saw normal values only one time. All other times I see 'NaN'. | |
| 12:30:56 | snikitin | I guess I need to add 'is NaN' checks into my code. Did you face with such strange behavior? | |
| 12:32:01 | BobBall | I suspect this will be because we're getting the stats from the RRDs which are updated every 5 seconds. Perhaps a pause in the tempest test for 10 seconds(?) to ensure some stats are populated would fix it? | |
| 12:35:52 | snikitin | BobBall: Thank you for advice! I will check it. P.S. Even if some stats aren't populated I expect to see something like 'None' or 'null' rather than 'NaN'. strange behavior. | |
| 12:38:37 | BobBall | snikitin: The RRDs are typically based on min, max or averages over a period (as an instant reading doesn't make much sense for things like CPU usage). My guess is that they are providing a number based on the average over X-X seconds - which will be a number divided by 0 as we don't have sufficient time periods. I get what you're saying, but if a 10 second delay fixes the issue I would understand why it is returning "Not a Number" | |
| 12:41:54 | bauzas | sdague: quick question, we deduce the endpoint from the token unless we use the endpoint_override here https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/session.py#L509-L513 | |
| 12:42:18 | bauzas | sdague: am I correct if I say that if I'm providing endpoint_override, I'd not lookup the SC ? | |
| 12:42:28 | bauzas | and directly call the placement service | |
| 12:42:58 | sdague | bauzas: looks like... however | |
| 12:43:05 | sdague | if you are really using noauth | |
| 12:44:16 | sdague | then probably you are actually going to want to build a fake client entirely, I think mocking out keystoneauth at all the right points is going to be harder | |
| 12:44:57 | sean-k-mooney | dims: jaypipes looking at https://review.openstack.org/#/c/422838 do we still need to merge this and make a release to fix the python 3.5 support today? | |
| 12:45:04 | bauzas | I don't want to mock out all the keystoneclient | |
| 12:45:18 | bauzas | sdague: just recreating a new Session with an endpoint_override | |
| 12:45:33 | snikitin | BobBall: Ok. I will check it, but I think you right. Anyway I think we should check 'NaN' values in nova, because user can get a 500 error because of it. | |
| 12:45:47 | sdague | bauzas: ok | |
| 12:45:57 | bauzas | anyway, let's try | |
| 12:46:21 | bauzas | just taking a few mins for refueling myself at $lunch, and then back quickly | |
| 12:46:26 | BobBall | snikitin: Sure - the xenapi drive could easily translate NaN to None if it makes more sense up the stack | |
| 12:51:59 | snikitin | BobBall: I checked. It works with sleep(10)! Thank you! | |
| 12:53:44 | dims | sean-k-mooney : nah. no need to rush it. py35 is just experiment anyway at the moment | |
| 12:54:09 | markus_z | sdague: We could make the testing gap for live-migration a little smaller with https://review.openstack.org/#/c/347471/ Would be nice if you could find the time for that. | |
| 12:54:26 | markus_z | mriedem: Maybe you too, I answered your questions there ^ | |
| 12:55:40 | sean-k-mooney | dims: ok ill try and get a linux bridge devstack enviornment up and running next week and we can also look into adding a linux bridge job to os-vif also going forward | |
| 12:55:53 | dims | ++ sean-k-mooney thansk! | |
| 13:08:26 | jaypipes | dims: what's the latest status on the mox3 thing? has that revert merged? | |
| 13:09:06 | dims | jaypipes : y, should be green. please rebase if you still see an issue | |
| 13:09:18 | jaypipes | dims: will do. cheers! | |
| 13:25:57 | mriedem | getting this resource providers patch in, which is pretty simple, closes out the aggregates stuff for ocata https://review.openstack.org/#/c/407309/ | |
| 13:26:01 | mriedem | needs another core ^ | |
| 13:26:20 | leakypipes | mriedem: I'm on it, captain. | |
| 13:26:28 | leakypipes | oh wait, that's mine. | |
| 13:26:29 | mriedem | leakypipes: it's yours | |
| 13:26:30 | leakypipes | never mind :) | |
| 13:26:30 | mriedem | ha | |
| 13:27:21 | mriedem | leakypipes: you could deal with the -1 on this though https://review.openstack.org/#/c/404472/ | |
| 13:27:43 | leakypipes | mriedem: ya, will do. | |
| 13:28:02 | leakypipes | mriedem: just trying to get sahid's reserved CPUs patches reviewed. | |
| 13:28:08 | mriedem | melwitt_: sdague: i got the discover_hosts running in d-g http://logs.openstack.org/05/423005/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/2704d02/logs/discover-hosts.txt.gz | |
| 13:28:50 | mriedem | needs to be fixed up for stable branches though | |
| 13:28:53 | mriedem | and grenade | |
| 13:29:22 | mriedem | http://logs.openstack.org/05/423005/2/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/d230b29/logs/discover-hosts.txt.gz | |
| 13:29:56 | openstackgerrit | Timofey Durakov proposed openstack/nova: Make all timestamps formats equal https://review.openstack.org/421981 | |
| 13:32:23 | tbachman | a little levity | |
| 13:32:26 | tbachman | is | |
| 13:32:27 | tbachman | needed | |
| 13:33:33 | leakypipes | tbachman: indeed :( | |
| 13:34:32 | mfeoktistov | Hello! Dear cores, can you please look at https://review.openstack.org/#/c/312488/ | |
| 13:36:45 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/422323 | |
| 13:37:33 | sdague | mriedem: cool | |
| 13:39:52 | sdague | mriedem: once I get some breakfast and this set of grenade results, I'll take a look | |
| 13:40:28 | mriedem | the d-g change isn't really ready yet, i need to make it conditional on branches, or put a script in devstack to run discover_hosts and then d-g would only run it if that script exists | |
| 13:40:37 | mriedem | https://review.openstack.org/#/c/423005/2/devstack-vm-gate.sh | |
| 13:41:23 | mriedem | if you have input on what's the preferred approach there that would be good | |
| 13:42:13 | leakypipes | tbachman: :) | |
| 13:43:45 | mdbooth | mriedem: NFS https://review.openstack.org/#/c/383859/ | |
| 13:43:59 | mriedem | blarg shared file systems | |
| 13:44:09 | mdbooth | I've actually been hammering on this over the last couple of days because it fixes another bug too | |
| 13:44:40 | mdbooth | If you suspend an instance with an attached volume, the volume can be unmounted while it's suspended | |
| 13:44:48 | mdbooth | So it fails when you resume it | |
| 13:44:55 | mdbooth | Confirmed this patch fixes that bug | |
| 13:45:53 | mdbooth | https://review.openstack.org/#/c/383859/ | |
| 13:46:24 | mdbooth | Except it's not a race, it's entirely deterministic in this case | |
| 13:47:20 | mriedem | ok | |
| 13:47:55 | mriedem | i'll be honest i'm going to have a very distracted day today, trying to wrap things up before flying out for vacation tomorrow | |
| 13:48:03 | BobBall | snikitin: Glad to hear it | |
| 13:48:14 | mdbooth | Ah, it's already in there | |
| 13:49:20 | openstackgerrit | ChangBo Guo(gcb) proposed openstack/nova: Set access_policy for messaging's dispatcher https://review.openstack.org/408494 | |
| 13:50:40 | mdbooth | Unrelated, can anybody check my memory for me: https://review.openstack.org/#/c/376350/ . The behaviour of reboot where is tries to fix potential local hypervisor problems is intentional, right? | |
| 13:51:21 | mdbooth | i.e. recreating backing files and rewriting the domain xml. | |