| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-13 | |||
| 14:43:50 | EmilienM | it's running "nova-manage cell_v2 simple_cell_setup" as part of the nova database initialization when no compute nodes have been created and it returns 1 | |
| 14:44:05 | openstackgerrit | Vladyslav Drok proposed openstack/nova: Support Ironic interface attach/detach in nova virt https://review.openstack.org/364413 | |
| 14:44:10 | EmilienM | yes there is no trace but I can easily give one to you, let me do something quickly | |
| 14:44:40 | openstackgerrit | Emilien Macchi proposed openstack/nova: Debug https://bugs.launchpad.net/tripleo/+bug/1656276 https://review.openstack.org/419926 | |
| 14:45:25 | EmilienM | bauzas: i'm preparing a CI job with more debug, let me 5 min | |
| 14:45:50 | bauzas | EmilienM: anyway, I think the bug description has a first point to look | |
| 14:46:26 | bauzas | EmilienM: so https://github.com/openstack/puppet-nova/blob/master/manifests/db/sync_cell_v2.pp#L36 means that we try 10 times every 5 secs until the return code is zero, right?N | |
| 14:46:40 | EmilienM | bauzas: "No hosts found to map to cell, exiting" | |
| 14:46:46 | EmilienM | bauzas: exactly | |
| 14:46:59 | EmilienM | bauzas: keep in mind we can change puppet to accept any return code for this Exec | |
| 14:47:08 | EmilienM | bauzas: but I'm afraid accepting 0,1 is not the right thing to do | |
| 14:47:12 | bauzas | EmilienM: okay, so why does the puppet command failed only once ? | |
| 14:47:22 | openstackgerrit | Vladyslav Drok proposed openstack/nova: Generate necessary network metadata for portgroups https://review.openstack.org/388756 | |
| 14:47:22 | EmilienM | because what is the exec fails for a good reason, we won't see it | |
| 14:47:25 | bauzas | EmilienM: it should have failed 10 times ? | |
| 14:47:45 | EmilienM | bauzas: it tried 10 times and show output of last failure | |
| 14:47:52 | EmilienM | but I guess it's 10 times the same failure | |
| 14:48:07 | bauzas | ah ok | |
| 14:48:08 | EmilienM | it returned 1 ten times | |
| 14:48:28 | EmilienM | this "No hosts found to map to cell, exiting" means "no compute has been found, fail"? | |
| 14:49:49 | bauzas | https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L1251 is the reason behind it | |
| 14:50:22 | bauzas | EmilienM: which returns 1 indeed | |
| 14:50:23 | bauzas | https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L1114 | |
| 14:50:42 | EmilienM | indeed | |
| 14:50:47 | amoralej | bauzas, EmilienM, i've been doing some additional tests and if no computes are found when running the command the cell is not added to cell_mappints table | |
| 14:51:03 | amoralej | so, it may be right to fail if no compute nodes are found | |
| 14:51:05 | bauzas | amoralej: that's an expected behaviour | |
| 14:51:22 | bauzas | so, the point is, what is the meaning of that return code equal to 1 | |
| 14:51:25 | EmilienM | so we need to run nova-compute *before* running this command? it might be very tricky | |
| 14:51:28 | openstackgerrit | John Garbutt proposed openstack/nova: WIP: Call preallocate_for_instance before scheduler https://review.openstack.org/413124 | |
| 14:51:32 | EmilienM | in multinode deployments specially | |
| 14:51:51 | amoralej | bauzas, then in any case we need to get compute nodes before running it | |
| 14:52:01 | bauzas | EmilienM: well, I think no computes nodes doesn't mean that the command failed | |
| 14:52:15 | EmilienM | bauzas: https://github.com/openstack/nova/commit/a86d78f5f048e11ddb7e375f2d6cc707108b9478 | |
| 14:52:15 | bauzas | EmilienM: so, there are 2 possibilites | |
| 14:52:27 | EmilienM | alaski: ^ | |
| 14:52:46 | bauzas | either we say it's okay to have no computes when running that command, and then we should soft-fail with return code 0 | |
| 14:53:08 | bauzas | or, we ask users to deploy compute nodes before running that command | |
| 14:53:29 | amoralej | bauzas, in option 1, we'd need to re-run it after adding the compute nodes? | |
| 14:53:37 | bauzas | amoralej: indeed | |
| 14:54:11 | amoralej | then, option 1 converts to option 2 | |
| 14:54:30 | amoralej | even if command exit with rc 0, we need to orchestrate | |
| 14:54:37 | bauzas | you're right | |
| 14:54:40 | mdbooth | lyarwood: That's a separate issue, right? | |
| 14:54:51 | mdbooth | I mean, isn't that broken either way? | |
| 14:54:58 | amoralej | bauzas, i have a doubt about cell0 | |
| 14:55:31 | bauzas | amoralej: EmilienM: I was a bit off the recent cellsv2 discussions, so I don't know if we're planning something like an automatic mapping when adding nodes | |
| 14:55:32 | amoralej | what i find when i'm running the command after adding compute nodes is that a new cell is created in cell_mappings | |
| 14:55:44 | bauzas | amoralej: cell0 is a total different story | |
| 14:55:52 | bauzas | amoralej: cell0 is not a cell :p | |
| 14:55:58 | bauzas | well, it is a cell | |
| 14:56:10 | bauzas | but we could have named it "cemetery cell" | |
| 14:56:28 | amoralej | ok, then i have a new doubt, :) | |
| 14:56:29 | bauzas | it's for failed instances not having a compute node | |
| 14:56:50 | bauzas | ie. when the scheduler returns NoValidHosts, then the instance is mapped to cell0 | |
| 14:56:59 | lyarwood | mdbooth: nope, my change introduces the lookup, previously all the calls made by BlockDevice to libvirt would work with either a path or target dev | |
| 14:57:03 | amoralej | so, for single cell deployments, do i need only cell0 or cell0 and "reall cell" ? | |
| 14:57:05 | bauzas | instead of being mapped to the cell that the compute belongs to | |
| 14:57:17 | mdbooth | lyarwood: Of course. Got it. | |
| 14:57:37 | bauzas | amoralej: for real deployments, you need cell0 *and* a single cell | |
| 14:57:52 | amoralej | ok, understood | |
| 14:58:19 | bauzas | amoralej: that's why we populate cell0 with a db sync | |
| 14:58:35 | bauzas | so that we have everything like if it was a normal cell | |
| 14:58:37 | bauzas | anyway | |
| 14:58:47 | bauzas | EmilienM: amoralej: so, in order to tackle your problem | |
| 14:59:51 | bauzas | https://github.com/openstack/nova/commit/a18f601753f92ff4a2a42be0962a188f583bbfb9 just canary-tested an existing problem | |
| 15:00:13 | bauzas | because previously, it was returning 0 as you were already having a cell0 | |
| 15:00:20 | bauzas | now, we're going past this point | |
| 15:02:02 | amoralej | yes bauzas, i'm going to summarize this findings in the LP | |
| 15:05:19 | mandy | hi | |
| 15:06:32 | mandy | could you anyone help me ?? | |
| 15:08:12 | bauzas | amoralej: so, just verified and FWIW, we create cells (and run the single cell command) *after* starting n-cpu in devstack | |
| 15:08:18 | bauzas | EmilienM: ^ | |
| 15:08:29 | bauzas | that's why the upstream nova jobs aren't trampled | |
| 15:08:35 | figleaf | mandy: just ask your question. | |
| 15:08:41 | openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: [WIP] try py35 functional with a clean slate https://review.openstack.org/419968 | |
| 15:09:01 | amoralej | bauzas, could you take a look to comment #3 in https://bugs.launchpad.net/nova/+bug/1656276 and confirm if it's right? | |
| 15:09:01 | openstack | Launchpad bug 1656276 in tripleo "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Critical,Triaged] | |
| 15:10:39 | mandy | actualy i am successfully create one vm in openstack using devtsack | |
| 15:11:01 | mandy | * devstack but unable to ping that vm ....could you plz help me | |
| 15:11:02 | mandy | ?? | |
| 15:11:34 | EmilienM | bauzas: interesting | |
| 15:11:41 | EmilienM | bauzas: link? | |
| 15:12:09 | mandy | i used private network | |
| 15:12:56 | mandy | is anyone there | |
| 15:12:57 | mandy | ?? | |
| 15:13:26 | mriedem | ceph job is now non-voting, fyi | |
| 15:13:29 | openstackgerrit | sahid proposed openstack/nova: compute: reset instance events https://review.openstack.org/420026 | |
| 15:13:56 | diana_clarke | mandy: Those kinds of support/usage questions don't typically get answered here (see channel topic). | |
| 15:14:08 | mandy | ok | |
| 15:14:11 | mandy | thanks | |
| 15:14:13 | mandy | bye | |
| 15:16:35 | EmilienM | mwhahaha: out of your mind, would it be doable to run this cell_v2 simple_cell_setup *after* starting nova-compute in tripleo? | |
| 15:16:44 | EmilienM | mwhahaha: I'm wondering about the steps | |
| 15:17:02 | bauzas | EmilienM: http://logs.openstack.org/90/409890/1/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/447ab55/logs/devstacklog.txt.gz#_2016-12-12_21_54_28_042 | |
| 15:17:13 | mriedem | mdbooth: how are you feeling about https://review.openstack.org/#/c/312488/ ? | |
| 15:17:16 | bauzas | (sorry, in internal meeting now) | |
| 15:17:26 | mriedem | the vz ci is running test_verify_created_server_ephemeral_disk now | |
| 15:17:36 | openstackgerrit | Vasyl Saienko proposed openstack/nova: Implement interface attach/detach in ironic virt driver https://review.openstack.org/419975 | |
| 15:17:55 | openstackgerrit | Vasyl Saienko proposed openstack/nova: Implement interface attach/detach in ironic virt driver https://review.openstack.org/419975 | |