| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-07-26 | |||
| 14:05:20 | vdrok | good morning folks | |
| 14:05:30 | gibi | mriedem: it seems replacing JsonEqual with DictMatcher passes the sample tests so this direction looks promising | |
| 14:05:48 | gibi | mriedem: I will rework my improvement patch | |
| 14:06:08 | vdrok | mriedem: seeing http://logs.openstack.org/88/473888/12/check/gate-tempest-dsvm-ironic-ipa-wholedisk-agent_ipmitool-tinyipa-multinode-ubuntu-xenial/d56cef7/logs/screen-n-cond-cell1.txt.gz#_Jul_26_11_15_52_052437 again in the multinode job :( I thought the option you've added in the last patchset of devstack change was the workaround for the subnode? | |
| 14:10:02 | dansmith | vdrok: jlvillal said you guys had fixed that | |
| 14:10:45 | dansmith | vdrok: this: https://review.openstack.org/#/c/477313/ | |
| 14:10:55 | dansmith | but I guess that was grenade | |
| 14:11:18 | vdrok | dansmith: we did add the nomulticell for grenade. but for the multinode job I thought the fix was https://review.openstack.org/#/c/477556/4..5/lib/nova | |
| 14:11:21 | dansmith | vdrok: this? http://logs.openstack.org/88/473888/12/check/gate-tempest-dsvm-ironic-ipa-wholedisk-agent_ipmitool-tinyipa-multinode-ubuntu-xenial/d56cef7/logs/screen-n-cond-cell1.txt.gz#_Jul_26_11_15_52_128079 | |
| 14:12:02 | dansmith | vdrok: I don't think so | |
| 14:13:44 | dansmith | vdrok: you won't get any reschedules with the multicell layout, so if you really hit those in regular gate jobs like this then we need to make you non multicell for now I guess | |
| 14:13:47 | dansmith | even in the regular ones | |
| 14:14:43 | jangutter | moshel: regarding my comments on https://review.openstack.org/#/c/398265 it's basically nova sending extra data to os-vif, so that in future os-vif can handle representors uniformly. | |
| 14:14:55 | vdrok | dansmith: hrm, so this means we need to add some stuff to devstack to check that if virt driver = ironic add the nomulticell everywhere? | |
| 14:15:04 | jangutter | moshele: ^^ sorry for misspelling your nick. | |
| 14:15:21 | dansmith | vdrok: don't you start nova-compute from your devstack plugin such that you can just set it there? | |
| 14:16:04 | vdrok | dansmith: we restart it :) but if it was already configured for multicell it fails. at least it failed when I tried this way | |
| 14:16:44 | vdrok | lemme try to find that patch | |
| 14:17:47 | vdrok | dansmith: here https://review.openstack.org/481114. was tested at https://review.openstack.org/481593 | |
| 14:18:37 | moshele | jangutter: Ok, I will update the representor_address | |
| 14:19:37 | vdrok | n-cond-cell1 log is still there, and is empty http://logs.openstack.org/93/481593/2/check/gate-tempest-dsvm-ironic-ipa-wholedisk-bios-pxe_snmp-tinyipa-ubuntu-xenial-nv/5972121/logs/screen-n-cond-cell1.txt.gz | |
| 14:20:09 | dansmith | vdrok: yep | |
| 14:21:05 | dansmith | vdrok: give me a sec | |
| 14:21:15 | vdrok | sure | |
| 14:25:38 | bauzas | mriedem: around ? I'm just reading https://review.openstack.org/#/c/485435/6/novaclient/tests/functional/v2/test_os_services.py@74 and I do wonder if we track somewhere the non-possibility to disable/enable a non-compute service | |
| 14:25:46 | bauzas | that's something I missed | |
| 14:26:00 | sdague | mriedem: can you check out whatever you were verifying yesterday in https://review.openstack.org/#/c/487246/ ( the wait for service patch ), or at least point me to what I'm looking for | |
| 14:26:43 | bauzas | I do remember you modified the REST API to use a PUT /os-services/ instead of an action and you made it cell-aware, but I don't remember we said to not accept enablement/disablement for internal services but computes | |
| 14:28:40 | bauzas | mriedem: nevermind, saw the commit msg in https://review.openstack.org/#/c/464280/ | |
| 14:30:26 | dansmith | vdrok: so you can set this variable early enough to make it work right? https://review.openstack.org/487443 | |
| 14:30:52 | mriedem | bauzas: https://bugs.launchpad.net/python-novaclient/+bug/1700359/comments/2 | |
| 14:30:53 | openstack | Launchpad bug 1700359 in OpenStack Compute (nova) "test_os_service_disable_enable fails with "ERROR (NotFound): Could not find binary nova-scheduler on host"" [Medium,Confirmed] | |
| 14:31:40 | mriedem | bauzas: trying to perform actions on non-compute services literally won't work, | |
| 14:31:52 | mriedem | because we look up the service in the cell via the host mapping, which only exists for compute services | |
| 14:31:55 | bauzas | mriedem: yeah it made sense to me | |
| 14:32:10 | mriedem | i just have the todo to put something in the release notes for it | |
| 14:32:25 | bauzas | mriedem: and because we only care about the enable/disable flag by the scheduler only for finding a target | |
| 14:32:32 | jaypipes | jangutter: where are you getting the vif_plug_ovs package here? https://github.com/Netronome/agilio-ovs-openstack-plugin/blob/d3f53fd3e6eaaa3d4b5f98978b2f3aa504a7a8cd/vif_plug_agilio_ovs/agilio_ovs.py#L29-L31 | |
| 14:32:40 | bauzas | mriedem: given other services and not targets for instances | |
| 14:33:00 | jaypipes | jangutter: I don't see os-vif in the list of requirements.txt Python packages. | |
| 14:33:23 | jangutter | jaypipes: yep, there's a sizeable set of cleanups still on that repo. | |
| 14:33:26 | openstackgerrit | Merged openstack/nova master: Using plain routes for the microversions test https://review.openstack.org/485061 | |
| 14:34:49 | dansmith | jaypipes: we're claiming in the scheduler now right? | |
| 14:35:02 | dansmith | oh no | |
| 14:35:06 | dansmith | nm that one hasn't merged yet | |
| 14:35:32 | dansmith | jaypipes: hopefully after it does, the ironic people won't have reschedules during their test runs :/ | |
| 14:35:33 | jaypipes | dansmith: looking at the -1s on that now. | |
| 14:35:58 | cdent | gibi: on https://review.openstack.org/#/c/487327/ I assume the virtenv’s still build correctly before runnin the actual tests? That error message is produced by wsgi-intercept, which gets upset about proxy variables when using the urllib3 interceptor | |
| 14:36:46 | vdrok | dansmith: yup, seems like it should help, thanks! we'll have to add it to our base builder in project-config | |
| 14:37:34 | dansmith | vdrok: okay, you'll work on that/ | |
| 14:37:42 | vdrok | yup | |
| 14:37:53 | vdrok | will propose a project-config patch now | |
| 14:38:03 | dansmith | vdrok: once we merge the scheduler claiming patch, hopefully you won't get any more reschedules during test runs like this, which will probably be faster anyway | |
| 14:38:22 | dansmith | should make your runs faster I mean | |
| 14:38:39 | dansmith | probably good to do this devstack variable in the meantime anyway | |
| 14:38:55 | vdrok | ++ | |
| 14:41:35 | mriedem | sdague: question in https://review.openstack.org/#/c/486700/2/nova/tests/fixtures.py | |
| 14:44:12 | mriedem | oh dan https://review.openstack.org/#/c/487443/ | |
| 14:44:43 | dansmith | mriedem: I was rushing | |
| 14:45:42 | sdague | mriedem: looking | |
| 14:45:45 | mriedem | why do we have: local nomulticellflag="$1" | |
| 14:45:50 | mriedem | in start_nova_compute | |
| 14:45:55 | mriedem | nothing passes a variable to that function | |
| 14:46:03 | dansmith | mriedem: yes, grenade does | |
| 14:46:07 | dansmith | mriedem: and ironic's grenade does | |
| 14:46:18 | sdague | mriedem: so what is your concern there? | |
| 14:46:19 | mriedem | oh grenade | |
| 14:46:41 | mriedem | sdague: was trying to sort out if that's coming from something within nova, since we do use pkg_resources.parse_version, but nowhere near that code | |
| 14:47:17 | dansmith | mriedem: jlvillal was struggling with that in o.vo recently too | |
| 14:47:18 | sdague | mriedem: stevedore probably | |
| 14:47:25 | dansmith | something changed in setuptools I think | |
| 14:50:08 | sdague | mriedem: actually, I think it's pbr | |
| 14:52:11 | sdague | let me see if I can trigger a deeper stack warn to figure out exactly where | |
| 14:58:50 | sdague | dansmith: for https://review.openstack.org/#/c/487443/2/lib/nova why isn't the answer to disable multicell? | |
| 14:59:40 | dansmith | sdague: not sure what you mean.. default=True=do setup multiple tiering.. set to false will not do that | |
| 15:00:07 | dansmith | maybe my logic needs inverting to make more sense? like disable=True? | |
| 15:00:31 | mriedem | note we aren't actually multi-cell yet | |
| 15:00:35 | mriedem | we're multi-tier | |
| 15:00:57 | dansmith | mriedem: well, you could multi-cell with this, we just don't configure it that way | |
| 15:01:14 | mriedem | dansmith: so i can post a WIP to set NOVA_NUM_CELLS=2 and it should work? | |
| 15:01:36 | dansmith | mriedem: it should configure multiple cells, but not sure it will work :) | |
| 15:01:47 | dansmith | mriedem: you need NUM_CELLS=2 and the CELL_INDEX=2 for the third node | |
| 15:01:53 | mriedem | could tease out some interesting bugs we don't know about yet | |
| 15:02:16 | mriedem | where is CELL_INDEX? | |
| 15:02:16 | dansmith | well, I didn't know we had three node jobs until you told me yesterday | |
| 15:02:49 | dansmith | I rememberd the name wrong, but it's this: https://github.com/openstack-dev/devstack/blob/master/lib/nova#L66-L68 | |
| 15:03:45 | mriedem | couldn't that just be: NOVA_CPU_CELL=${NOVA_CPU_CELL:-$NOVA_NUM_CELLS} ? | |
| 15:03:48 | dansmith | I just want to say that I wish we had taken my original suggestion to call cellsv2 something other than cells | |
| 15:03:49 | mriedem | are they always the same? | |
| 15:03:57 | sdague | ok, I'm trying to figure out a way to explain this in stackrc that isn't going to require poking people in irc. | |
| 15:04:03 | dansmith | no, | |
| 15:04:16 | dansmith | mriedem: you need it to be different on each node you want to be in a different cell | |
| 15:05:02 | mriedem | ah ok | |
| 15:05:04 | sdague | so, the fleetify patch creates a conductor per compute. | |
| 15:05:10 | sdague | plus the api conductor | |
| 15:05:11 | mriedem | conductor per node | |
| 15:05:16 | dansmith | condutor per cell | |
| 15:05:26 | mriedem | well, | |
| 15:05:31 | sdague | right, it's good we're all on the same page :) | |
| 15:05:32 | mriedem | not really | |