| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-23 | |||
| 21:15:38 | mriedem | andreaf: n-cpu barfs that the compute node that represents the local host doesn't exist, and then it creates it | |
| 21:15:42 | mriedem | and stops the barfing | |
| 21:15:42 | mnaser | if that can be approved, i can backport it and then bump it in stable/ocata and stable/pike | |
| 21:16:33 | andreaf | mriedem so I don't know why I get http://logs.openstack.org/24/545724/9/check/tempest-multinode-full/1bbec81/job-output.txt.gz#_2018-02-23_17_22_40_367118 on every single VM created in that job | |
| 21:18:37 | mriedem | is it every one or just the subnode? | |
| 21:18:43 | mriedem | ubuntu-xenial-inap-mtl01-0002692907 is the subnode | |
| 21:19:20 | andreaf | mriedem actually every single VM on the 907 node | |
| 21:19:44 | mriedem | andreaf: so this is the warning you get on first startup | |
| 21:19:45 | mriedem | http://logs.openstack.org/24/545724/9/check/tempest-multinode-full/1bbec81/compute1/logs/screen-n-cpu.txt.gz?level=INFO#_Feb_23_17_22_20_954537 | |
| 21:19:50 | mriedem | No compute node record found for host ubuntu-xenial-inap-mtl01-0002692907. If this is the first time this service is starting on this host, then you can ignore this warning.: ComputeHostNotFound_Remote: Compute host ubuntu-xenial-inap-mtl01-0002692907 could not be found. | |
| 21:20:03 | mriedem | then the compute node record is created | |
| 21:20:04 | mriedem | http://logs.openstack.org/24/545724/9/check/tempest-multinode-full/1bbec81/compute1/logs/screen-n-cpu.txt.gz?level=INFO#_Feb_23_17_22_21_021412 | |
| 21:20:09 | mriedem | Feb 23 17:22:21.021412 ubuntu-xenial-inap-mtl01-0002692907 nova-compute[21947]: INFO nova.compute.resource_tracker [None req-5ceeacfd-f9f6-4b19-9044-6fd13c9adadc None None] Compute node record created for ubuntu-xenial-inap-mtl01-0002692907:ubuntu-xenial-inap-mtl01-0002692907 with uuid: 85868727-6173-4345-bd2e-a92bfaf1de8a | |
| 21:20:29 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: Fix N358 hacking check https://review.openstack.org/547670 | |
| 21:21:06 | mriedem | http://logs.openstack.org/24/545724/9/check/tempest-multinode-full/1bbec81/compute1/logs/devstacklog.txt.gz#_2018-02-23_17_22_18_880 | |
| 21:21:06 | mriedem | on that subnode, wait_for_compute starts here: | |
| 21:21:11 | mriedem | 2018-02-23 17:22:18.880 | + lib/nova:is_nova_ready:1003 : wait_for_compute 60 | |
| 21:21:21 | mriedem | and ends here | |
| 21:21:22 | mriedem | 2018-02-23 17:22:21.607 | + functions:wait_for_compute:463 : return 0 | |
| 21:21:56 | andreaf | so it's successful | |
| 21:21:58 | mriedem | so in that case, it looks like the compute node record was created before devstack started waiting | |
| 21:22:58 | mnaser | mriedem: round 2 https://review.openstack.org/#/c/547431/ | |
| 21:23:07 | andreaf | right... so I'm missing what's going wrong... | |
| 21:23:49 | mnaser | its friday and the servers are lazy | |
| 21:23:51 | mriedem | http://logs.openstack.org/24/545724/9/check/tempest-multinode-full/1bbec81/controller/logs/devstacklog.txt.gz#_2018-02-23_17_13_29_978 | |
| 21:23:51 | mriedem | the host mapping for the n-cpu on the controller node is created here: | |
| 21:23:57 | mriedem | 2018-02-23 17:13:29.978 | Creating host mapping for compute host 'ubuntu-xenial-inap-mtl01-0002692906': adbcb697-55fc-4750-9242-b13aa75cab07 | |
| 21:24:28 | mriedem | discover_hosts doesn't run on the subnode | |
| 21:24:39 | mriedem | and i don't see discover_hosts running after the subnode compute is created to discover it | |
| 21:25:22 | mnaser | oh that's why its different | |
| 21:25:26 | mriedem | andreaf: you'd likely need to compare to an existing multinode job and see where discover_hosts runs, | |
| 21:25:30 | mnaser | grenade uses simple_cell_setup which does discover_hosts | |
| 21:25:37 | mriedem | because discover_hosts *has* to be run every time you startup a new n-cpu | |
| 21:25:45 | andreaf | mriedem yeah that's what I was about to do :) | |
| 21:26:08 | andreaf | should it run on the controller? | |
| 21:26:32 | andreaf | discover_host is a devstack function right? | |
| 21:26:52 | mnaser | andreaf: discover_hosts is a nova-manage cli func | |
| 21:26:59 | mnaser | nova-manage cell_v2 discover_hosts | |
| 21:27:03 | mnaser | it runs on the controller after you add new computes | |
| 21:27:05 | mriedem | andreaf: OH I KNOW THE PROBLEM!!! | |
| 21:27:15 | mriedem | your problem is zuulv3 | |
| 21:27:16 | mriedem | :) | |
| 21:27:22 | andreaf | heh | |
| 21:27:26 | mnaser | i thought that was the solution for everything | |
| 21:27:27 | mriedem | devstack-gate would run discover_hosts on the primary after the subnode was setup | |
| 21:27:42 | mriedem | the devstack multinode guide says this too | |
| 21:27:56 | mriedem | i'll look for the d-g code | |
| 21:28:01 | andreaf | mriedem oh, right | |
| 21:28:07 | andreaf | mriedem I missed that in d-g | |
| 21:28:24 | mriedem | https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L783 | |
| 21:28:28 | mriedem | andreaf: ^ | |
| 21:28:34 | mriedem | d-g would run discover_hosts after the subnode was setup | |
| 21:28:48 | mriedem | so you'll need an ansible task to run discover_hosts on the primary node once the subnode is setup | |
| 21:28:50 | mriedem | that should do the trick | |
| 21:29:24 | andreaf | mriedem heh the one line I did not port to ansible! | |
| 21:29:48 | andreaf | mriedem I guess I kind of assumed cells did not matter for the base job :( but it's cells v2 | |
| 21:29:50 | mriedem | mnaser: need to fix the commit message | |
| 21:29:58 | mriedem | cells v2 is king | |
| 21:30:11 | andreaf | mriedem cool thanks a lot, much appreciated!! | |
| 21:30:11 | mnaser | oops | |
| 21:30:15 | mriedem | yw | |
| 21:31:02 | andreaf | mriedem # NOTE(mriedem): We want to remove this if/when nova supports auto-registration of computes with cells, but that's not happening in Ocata. | |
| 21:31:12 | mnaser | ok fixed the commit msg, andreaf i'd borrow your +2 back please https://review.openstack.org/#/c/547431/ :) | |
| 21:31:18 | andreaf | mriedem I guess it's not happening in pike or queens either :D | |
| 21:31:25 | mnaser | s/Ocata/a while/ | |
| 21:31:59 | andreaf | mnaser +2 | |
| 21:32:08 | mnaser | thank you | |
| 21:33:21 | mriedem | andreaf: right | |
| 21:34:01 | mriedem | andreaf: also we don't run discover_hosts from the subnode because it's not configured to have access to the nova_api db | |
| 21:34:03 | mriedem | on purpose | |
| 21:34:48 | mriedem | we want to make sure that when people like leakypipes try to add code to nova-compute that hits the API it blows up :P | |
| 21:47:16 | leakypipes | mriedem: ha. ha. | |
| 21:52:43 | cfriesen | if we tell nova to boot an instance (specifying that we want interfaces on two networks), then the instance fails to start, so we reschedule to another node, would we expect to re-use the ports that were created for the first compute node? | |
| 21:58:59 | cfriesen | answering my own question, based on a comment in ComputeManager._build_networks_for_instance() the answer seems to be "yes" | |
| 21:59:08 | cfriesen | or at least "maybe" | |
| #openstack-nova - 2018-02-24 | |||
| 02:27:46 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Check for leaked server resource allocations in post_test_hook https://review.openstack.org/538510 | |
| 03:22:01 | Spazmotic | Morning | |
| 03:43:55 | openstackgerrit | Merged openstack/nova master: [Placement] Invalid query parameter could lead to HTTP 500 https://review.openstack.org/539408 | |
| 03:44:09 | openstackgerrit | Merged openstack/nova master: [placement] Add functional tests for traits API https://review.openstack.org/524094 | |
| 04:09:37 | alex_xu | cfriesen: it is yes, but will remove the port for sriov https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1873 | |
| 04:49:06 | openstackgerrit | Merged openstack/nova stable/ocata: Don't update RT in _allocate_network https://review.openstack.org/519943 | |
| 06:49:58 | WangShuaili | When does nova-lifecycle-event trigger | |
| 08:45:09 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova-specs master: Support shared and dedicated VMs in one host https://review.openstack.org/543805 | |
| #openstack-nova - 2018-02-25 | |||
| 02:52:18 | Guest1511 | Nickserv you being hateful over there? | |
| 02:58:01 | Guest1511 | Whew Freenode services are super grumpy | |
| 02:58:39 | Spazmotic | Morning folks. | |
| 11:13:54 | melwitt | mriedem: do you know anything about py3 support in osc-placement? dhellman was asking about it, said it claims support in the classifiers but he didn't see any py3 jobs configured for it | |
| 11:18:52 | mriedem | there isn't a job for it, i added that to the ptg etherpad | |
| 11:19:02 | mriedem | should be easy to add, i can work something up, in doug's honor | |
| 11:19:10 | openstack | Launchpad bug 1751564 in OpenStack Dashboard (Horizon) "unable to attach multiattach volume to server" [Undecided,New] | |
| 11:19:10 | mriedem | ildikov: fyi https://bugs.launchpad.net/horizon/+bug/1751564 | |
| 11:19:28 | mriedem | if you know any horizon people | |
| 11:34:23 | openstackgerrit | Matt Riedemann proposed openstack/osc-placement master: Enable py35 unit and functional test jobs https://review.openstack.org/547809 | |
| 11:34:24 | mriedem | melwitt: ^ | |
| 11:38:04 | melwitt | mriedem: cool, thanks | |
| 12:04:07 | openstackgerrit | Matt Riedemann proposed openstack/osc-placement master: Enable py35 unit test job https://review.openstack.org/547809 | |
| 12:10:14 | openstackgerrit | Matt Riedemann proposed openstack/osc-placement master: Migrate legacy-osc-placement-dsvm-functional job in-tree https://review.openstack.org/547812 | |
| 12:25:49 | openstackgerrit | Matt Riedemann proposed openstack/osc-placement master: Add osc-placement-dsvm-functional-py3 job https://review.openstack.org/547815 | |
| 12:38:12 | Spazmotic | Quiet in here tonight but forgot it's a sunday morning hehe | |
| 12:38:23 | Spazmotic | Night shift things~ | |