| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-13 | |||
| 13:32:50 | efried | mriedem I'm not going to be able to abandon the DNMs, cause I don't own 'em. Is that something you have the power to do? | |
| 13:32:56 | mriedem | i can do that | |
| 13:33:09 | efried | Cool cool. | |
| 13:34:59 | mriedem | sdague: want to abandon this? https://review.openstack.org/#/c/416733/ | |
| 13:35:06 | sdague | mriedem: looking | |
| 13:35:37 | sdague | sure | |
| 13:35:47 | sdague | did we run a test to make sure this was all working as expected? | |
| 13:36:01 | mriedem | sdague: it's enabled in the nova-next job | |
| 13:36:14 | sdague | mriedem: right, but unless you tune down the timeouts | |
| 13:36:20 | sdague | you aren't going to hit it in a gate job | |
| 13:36:31 | mriedem | that's what the abandoned nova dnm patches were for | |
| 13:36:40 | sdague | ok, sure | |
| 13:36:41 | mriedem | they enabled it and did a long sleep during something like live migration | |
| 13:36:53 | sdague | ok, sounds good | |
| 13:47:59 | openstackgerrit | Béla Vancsics proposed openstack/nova master: Reduced the complexity of the _init_instance method https://review.openstack.org/374661 | |
| 14:04:01 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use plain routes list for user_data instead of stevedore https://review.openstack.org/462183 | |
| 14:04:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use plain routes list for os-server-groups endpoint instead of stevedore https://review.openstack.org/462169 | |
| 14:06:53 | openstackgerrit | Mathieu Gagné proposed openstack/nova master: Add ability to signal and perform online volume size change https://review.openstack.org/454322 | |
| 14:10:14 | mriedem | bauzas: the change on master is merged for https://review.openstack.org/#/c/472687/ if you want to get that in, then we can do an ocata release | |
| 14:10:37 | bauzas | mriedem: easy peasy | |
| 14:28:08 | openstackgerrit | Eric Fried proposed openstack/nova master: Glance download: only fsync files https://review.openstack.org/451094 | |
| 14:29:44 | dansmith | sdague: so this is passing a canary from the nova side: https://review.openstack.org/#/c/436094/ | |
| 14:29:57 | dansmith | sdague: would appreciate it if you'd take another pass through that to see if we can get that landed soonlyish | |
| 14:38:03 | edmondsw | mriedem sdague I think one of you had reminded us about adding powervm to the support matrix... we've got that change up for review: https://review.openstack.org/#/c/470999/ | |
| 14:39:02 | mriedem | yeah i did, just need to get some free time | |
| 14:39:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Handle uuids in os-hypervisors API https://review.openstack.org/471899 | |
| 14:39:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle uuid in HostAPI.compute_node_get https://review.openstack.org/461532 | |
| 14:39:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Use uuid for id in Services API https://review.openstack.org/464280 | |
| 14:40:07 | sdague | dansmith: the grenade bit is already in? (i can't remember) | |
| 14:40:15 | dansmith | sdague: yar | |
| 14:40:35 | dansmith | sdague: no depends-on for that patch, and just a canary that depends-on it for testing | |
| 14:40:44 | dansmith | sdague: https://review.openstack.org/#/c/473565/ | |
| 14:40:47 | dansmith | canary ^ | |
| 14:41:34 | dansmith | sdague: if you think we need a canary against any other project let me know.. I don't really know what other sorts of complicated jobs we have in other places that might be affected | |
| 14:41:49 | sdague | I think it's probably fine | |
| 14:42:02 | dansmith | cool | |
| 14:42:27 | openstackgerrit | Béla Vancsics proposed openstack/nova master: Reduced the complexity of the _init_instance method (2) https://review.openstack.org/420619 | |
| 14:42:58 | mriedem | dansmith: i see an issue in the ironic tinyipa job in there http://logs.openstack.org/94/436094/35/check/gate-tempest-dsvm-ironic-ipa-wholedisk-bios-agent_ipmitool-tinyipa-ubuntu-xenial-nv/1df37be/logs/devstacklog.txt.gz#_2017-06-12_20_26_29_362 | |
| 14:43:03 | mriedem | but that might just already be busted | |
| 14:44:43 | mriedem | actually, | |
| 14:44:52 | mriedem | it only fails that way on that devstack change and the canary patch | |
| 14:44:58 | mriedem | so it's likely an issue from that setup | |
| 14:45:08 | mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Timed%20out%20waiting%20for%20Nova%20hypervisor-stats%20count%20%3E%3D%201%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d | |
| 14:45:34 | dansmith | ah, yeah, hmm | |
| 14:45:45 | dansmith | host discovery maybe? | |
| 14:46:06 | mriedem | that'd be my guess | |
| 14:46:31 | mriedem | dtantsur: are you around? | |
| 14:46:43 | dtantsur | mriedem: hi, I am | |
| 14:46:56 | mriedem | dtantsur: we're trying to sort out http://logs.openstack.org/94/436094/35/check/gate-tempest-dsvm-ironic-ipa-wholedisk-bios-agent_ipmitool-tinyipa-ubuntu-xenial-nv/1df37be/logs/devstacklog.txt.gz#_2017-06-12_20_26_29_362 | |
| 14:47:05 | mriedem | that job is timing out waiting for hypervisor stats to show up, | |
| 14:47:11 | dtantsur | I'm on a meeting, but will try to look | |
| 14:47:13 | mriedem | and are wondering if it's not doing proper host discovery | |
| 14:47:15 | mriedem | thanks | |
| 14:47:26 | dtantsur | what's the patch? | |
| 14:47:33 | mriedem | https://review.openstack.org/#/c/436094/35 | |
| 14:47:43 | mriedem | it's a devstack patch that gets us to multiple cells testing | |
| 14:47:50 | mriedem | with 2 levels of nova conductor | |
| 14:47:59 | mriedem | one in the top level controller node, and one in each cell | |
| 14:48:05 | mriedem | dansmith: ^ is that accurate? | |
| 14:48:12 | dtantsur | hmm, I wonder how this all works with ironic | |
| 14:48:25 | dansmith | mriedem: the two levels thing doesn't really matter to him, but yeah | |
| 14:49:36 | dansmith | mriedem: dtantsur I'm poking through lib/ironic now | |
| 14:49:46 | mriedem | i see cell1 created here http://logs.openstack.org/94/436094/35/check/gate-tempest-dsvm-ironic-ipa-wholedisk-bios-agent_ipmitool-tinyipa-ubuntu-xenial-nv/1df37be/logs/devstacklog.txt.gz#_2017-06-12_20_10_32_098 | |
| 14:50:18 | mriedem | this isn't a multinode job, so it should just be a single cell right? | |
| 14:50:18 | dtantsur | I think what we do, or at least what we used to do, is to restart nova-compute for some reason | |
| 14:50:27 | dtantsur | not sure if it's related | |
| 14:50:56 | mriedem | i'm not seeing simple_cell_setup or discover_hosts run anywhere in the devstack log | |
| 14:51:25 | dtantsur | we don't seem to run it indeed.. | |
| 14:51:30 | dtantsur | but then we would break long ago, no? | |
| 14:52:05 | mriedem | dansmith: keep in mind they have an ironic devstack plugin https://github.com/openstack/ironic/tree/master/devstack | |
| 14:52:14 | dansmith | mriedem: right | |
| 14:52:18 | dansmith | mriedem: just pushed a change | |
| 14:52:27 | dansmith | https://review.openstack.org/473868 | |
| 14:52:46 | dtantsur | thanks dansmith, let's try this | |
| 14:53:02 | mriedem | s/.///? in there? | |
| 14:53:10 | dansmith | gah yes | |
| 14:53:10 | dtantsur | to be honest, I have hard time understanding the recent changes around cells, etc.. | |
| 14:53:35 | dansmith | mriedem: fixed | |
| 14:54:37 | dtantsur | dansmith: shouldn't the dependency be the other way around? i.e. the devstack patch to depend on the ironic change? | |
| 14:54:45 | mriedem | yes | |
| 14:55:14 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use context mgr in instance.delete https://review.openstack.org/443764 | |
| 14:55:14 | mriedem | i'm also not sure how any of the ironic jobs have worked w/o discovering the hosts by now | |
| 14:55:15 | mriedem | since ocata | |
| 14:55:24 | dtantsur | this is what I wonder as well | |
| 14:55:32 | dtantsur | we had to do it in tripleo | |
| 14:56:20 | dansmith | dtantsur: not for testing | |
| 14:56:52 | dansmith | dtantsur: I'll swap the order when this shows the job working on this patch | |
| 14:57:00 | dtantsur | ack | |
| 15:05:01 | mriedem | this is what normally runs discover_hosts https://github.com/openstack-dev/devstack/blob/master/stack.sh#L1412 | |
| 15:05:32 | mriedem | maybe the ironic thing happens before that? | |
| 15:05:33 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: Remove only compute ports in deallocate_for_instance https://review.openstack.org/473873 | |
| 15:05:35 | dansmith | right | |
| 15:05:36 | dansmith | right | |
| 15:05:42 | dansmith | that is the late run to pick up stragglers I think, | |
| 15:05:53 | dansmith | and this is earlier | |
| 15:06:07 | dansmith | I don't think the devstack change makes anything like db init happen in any different order, | |
| 15:06:21 | dansmith | so I'm kinda suspecting that we're using the wrong config file somewhere and never installing the maps at the top | |
| 15:06:38 | dansmith | but I can chase that down.. just want to do this as the easy first stab and see where we are | |
| 15:06:49 | dansmith | if this doesn't work, it's definitely a config file thing I think | |
| 15:06:59 | dansmith | definitely I think | |