| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-07 | |||
| 16:16:13 | mriedem | i'm not sure why we have the || true in there | |
| 16:16:21 | mriedem | probably for the old bug where map_cell0 would return 1 | |
| 16:17:19 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Get instance availability_zone without hitting the api db https://review.openstack.org/439754 | |
| 16:17:20 | openstackgerrit | Dan Smith proposed openstack/nova master: Teach HostAPI about cells https://review.openstack.org/442162 | |
| 16:17:20 | openstackgerrit | Dan Smith proposed openstack/nova master: Enlighten server tags API about cells https://review.openstack.org/440856 | |
| 16:17:21 | openstackgerrit | Dan Smith proposed openstack/nova master: Target cell in super conductor operations https://review.openstack.org/438022 | |
| 16:17:21 | openstackgerrit | Dan Smith proposed openstack/nova master: Make os-availability-zones know about cells https://review.openstack.org/442163 | |
| 16:17:22 | openstackgerrit | Dan Smith proposed openstack/nova master: Make scheduler target cells to get compute node instance info https://review.openstack.org/439891 | |
| 16:17:22 | openstackgerrit | Dan Smith proposed openstack/nova master: Change MQ targeting to honor only what is in the context https://review.openstack.org/438023 | |
| 16:17:23 | openstackgerrit | Dan Smith proposed openstack/nova master: Add cell field to Destination object https://review.openstack.org/438024 | |
| 16:17:23 | openstackgerrit | Dan Smith proposed openstack/nova master: Make consoleauth target the proper cell https://review.openstack.org/441461 | |
| 16:17:24 | openstackgerrit | Dan Smith proposed openstack/nova master: Teach simple_tenant_usage about cells https://review.openstack.org/441388 | |
| 16:17:24 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Make conductor ask scheduler to limit migrates to same cell https://review.openstack.org/438025 | |
| 16:17:25 | openstackgerrit | Dan Smith proposed openstack/nova master: Teach os-aggregates about cells https://review.openstack.org/442200 | |
| 16:30:11 | mriedem | dansmith: the archive_deleted_rows post-test-hook only runs on the nova-next job i guess http://logs.openstack.org/52/440652/2/check/gate-tempest-dsvm-neutron-nova-next-full-ubuntu-xenial-nv/51f2ea4/logs/devstack-gate-post_test_hook.txt.gz | |
| 16:30:15 | mriedem | just fyi | |
| 16:31:52 | dansmith | hmm why is that? | |
| 16:33:06 | mriedem | https://github.com/openstack-infra/project-config/commit/df865a4e9d6830abb553f077ce885d8493210efa | |
| 16:33:31 | mriedem | i'm guessing we just put it there since it's a nova-specific job really, | |
| 16:33:42 | mriedem | and was a fancy snowflake at the time, which is where we put things like that | |
| 16:34:01 | dansmith | ah I see, because it evolved from the placement job/ | |
| 16:34:29 | mriedem | https://review.openstack.org/#/c/376567/ says we could make it fatal | |
| 16:34:32 | mriedem | could/should | |
| 16:34:35 | mriedem | we could think about that now | |
| 16:34:45 | mriedem | so if archive starts breaking then we fail the job | |
| 16:34:51 | dansmith | yeah | |
| 16:37:12 | dansmith | mriedem: why is there no single-create test in this? https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_auto_allocate_network.py | |
| 16:37:36 | mriedem | there is | |
| 16:37:37 | mriedem | https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_auto_allocate_network.py#L150 | |
| 16:37:52 | dansmith | mriedem: that tests no allocate right? | |
| 16:37:57 | mriedem | correct | |
| 16:38:15 | dansmith | but the multi one tests that there _is_ allocation | |
| 16:38:17 | mriedem | but we still create a server | |
| 16:38:30 | mriedem | why would it matter for multi cell? | |
| 16:38:58 | dansmith | I don't know why it's failing, and was looking for a non-multi-create test to see why that works and multi doesn't | |
| 16:39:17 | mriedem | log link me? | |
| 16:39:19 | dansmith | changing it to one server doesn't make it pass, so it must not be related to multi-create, I was just curious | |
| 16:39:47 | dansmith | mriedem: http://logs.openstack.org/94/436094/12/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/051cffa/logs/testr_results.html.gz | |
| 16:42:30 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add test for inventory with invalid ResourceClass https://review.openstack.org/442498 | |
| 16:43:42 | dansmith | I'm not even sure what to look at | |
| 16:46:25 | mriedem | just look into my eyes and don't be afraid | |
| 16:47:09 | mriedem | dansmith: so we're failing on https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_auto_allocate_network.py#L188 | |
| 16:47:19 | mriedem | are we going to the proper cell to get the info_cache? | |
| 16:47:37 | mriedem | that test creates 3 servers and waits for them all to be ACTIVE, | |
| 16:47:55 | mriedem | then iterates over them and lists their addresses to make sure there was a nic assigned | |
| 16:48:09 | dansmith | should be.. you think that's the only test that looks at stuff from info cache? | |
| 16:48:13 | mriedem | so either the nic isn't getting assigned, or we're not pulling the info_cache properly | |
| 16:48:35 | mriedem | there should be other tests that do that for attaching interfaces | |
| 16:48:51 | dansmith | yeah, and those appear to work | |
| 16:49:06 | mriedem | i'm not sure what they assert though | |
| 16:49:11 | mriedem | checking the n-cpu logs now | |
| 16:49:12 | dansmith | mriedem: and if I boot a server manually I see an address | |
| 16:49:21 | dansmith | let me try with auto networks | |
| 16:49:24 | mriedem | dansmith: with devstack? | |
| 16:49:27 | dansmith | yeah | |
| 16:49:30 | mriedem | dansmith: that's not the same, | |
| 16:49:37 | mriedem | because your tenant in devstack has access to the public network already | |
| 16:49:43 | mriedem | so we don't hit the auto-allocate paths | |
| 16:49:45 | mriedem | in n-cou | |
| 16:49:47 | dansmith | right, I'm saying we're pulling those nic details | |
| 16:49:48 | mriedem | *n-cpu | |
| 16:49:50 | dansmith | in the api | |
| 16:50:06 | mriedem | yeah, but this specific test has a tenant which has no access to any existing networks | |
| 16:50:13 | openstackgerrit | Dan Smith proposed openstack/nova master: Make os-availability-zones know about cells https://review.openstack.org/442163 | |
| 16:50:13 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Get instance availability_zone without hitting the api db https://review.openstack.org/439754 | |
| 16:50:14 | openstackgerrit | Dan Smith proposed openstack/nova master: Make scheduler target cells to get compute node instance info https://review.openstack.org/439891 | |
| 16:50:14 | openstackgerrit | Dan Smith proposed openstack/nova master: Change MQ targeting to honor only what is in the context https://review.openstack.org/438023 | |
| 16:50:15 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Make conductor ask scheduler to limit migrates to same cell https://review.openstack.org/438025 | |
| 16:50:15 | openstackgerrit | Dan Smith proposed openstack/nova master: Add cell field to Destination object https://review.openstack.org/438024 | |
| 16:50:16 | openstackgerrit | Dan Smith proposed openstack/nova master: Teach os-aggregates about cells https://review.openstack.org/442200 | |
| 16:50:51 | dansmith | right, I'm just saying I think api is pulling info cache from the right cell db because I see nic details normally | |
| 16:56:04 | mriedem | hmm http://logs.openstack.org/94/436094/12/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/051cffa/logs/screen-n-cpu.txt.gz#_2017-03-07_00_14_21_656 | |
| 16:56:14 | mriedem | No network configured allocate_for_instance | |
| 16:56:19 | mriedem | Instance network_info: |[]| | |
| 16:56:51 | dansmith | is that before it goes to allocate or after? | |
| 16:57:02 | mriedem | during | |
| 16:57:50 | dansmith | once we're on the compute node, there's no cell switching involved, so I'm not sure why multicell has anything to do with this | |
| 16:58:02 | dansmith | is there something we put into info cache from the api side on instance create or something? | |
| 16:58:19 | dansmith | on my test system, I have nothing in cell0 for info caches, so it's not like we're creating them there by accident or something | |
| 16:58:29 | mriedem | i don't see "Automatically allocating a network for project" | |
| 16:59:17 | dansmith | should that be logged by n-cpu? | |
| 16:59:21 | mriedem | yeah | |
| 16:59:45 | mriedem | https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L1495 | |
| 17:00:51 | dansmith | yeah | |
| 17:00:59 | mriedem | i'm also not seeing this https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L1542 | |
| 17:01:04 | mriedem | ^ is called via the api | |
| 17:01:20 | mriedem | oh well i should look for that in the n-api logs | |
| 17:02:44 | mriedem | i don't see that in n-api logs either | |
| 17:02:58 | mriedem | i wonder if somehow the auto_allocate flag isn't getting set on the NetworkRequest object? | |
| 17:03:46 | dansmith | not sure why that would be.. we don't even persist that right? | |
| 17:06:33 | mriedem | we don't persist it | |
| 17:06:41 | mriedem | are we somehow mangling what we pass between n-api and conductor? | |
| 17:07:17 | mriedem | i would doubt it because otherwise we wouldn't have any tests that create a server on a specific nic passing | |
| 17:07:27 | mriedem | which i believe is most of tempest | |
| 17:07:36 | mriedem | they all have tenant-isolated networks | |
| 17:09:12 | dansmith | mriedem: no mangling that I know of, and this all passes on the same stack of patches with normal devstack, AFAICT | |
| 17:11:09 | mriedem | ok. notifications meeting atm and then i'll keep digging. | |
| 17:11:12 | mriedem | we might need a debug logging patch | |
| 17:13:42 | dansmith | okay thanks for helping | |
| 17:14:45 | mriedem | https://www.youtube.com/watch?v=RWc5gsU-eeY | |