| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-04-07 | |||
| 12:49:16 | sean-k-mooney | nova-manage --config-file /etc/nova/nova-conductor-1.conf db sync --local_cell | |
| 12:49:25 | sean-k-mooney | i think that woudl be the right path but its something like that | |
| 12:49:58 | sean-k-mooney | ah its /etc/nova/nova_cell1.conf | |
| 12:51:01 | Uggla | nova-manage --config-file /etc/nova/nova_cell1.conf db sync correct ? | |
| 12:51:06 | sean-k-mooney | yes | |
| 12:51:20 | sean-k-mooney | that is what devstack ran | |
| 12:51:24 | Uggla | ERROR: Could not access cell0. | |
| 12:51:52 | sean-k-mooney | odd... | |
| 12:51:56 | sean-k-mooney | let me try that myself | |
| 12:52:42 | bauzas | chateaulav: no worries | |
| 12:53:18 | Uggla | sean-k-mooney, I got this: https://paste.openstack.org/show/bw9C4K0JEaD2eH2kM3TF/ | |
| 12:53:19 | sean-k-mooney | Uggla: you are missing --local_cell | |
| 12:53:40 | sean-k-mooney | that disablise it trying to connect to cell0 | |
| 12:53:46 | sean-k-mooney | when syncign cell 1 | |
| 12:54:06 | sean-k-mooney | stack@controller:/home/vagrant$ nova-manage --config-file /etc/nova/nova_cell1.conf db sync --local_cell | |
| 12:54:10 | sean-k-mooney | that should work | |
| 12:54:11 | Uggla | sean-k-mooney, sounds good ! I'm gonna check | |
| 12:54:38 | bauzas | correct ^ | |
| 12:55:19 | Uggla | it is ok regarding the tables \o/ | |
| 12:56:06 | sean-k-mooney | so how did you configure devstack to use your patch | |
| 12:56:08 | Uggla | but I'm unclear if this is specific onlyfor devstack ? | |
| 12:56:22 | sean-k-mooney | no you have to do this for all installers | |
| 12:57:09 | sean-k-mooney | nova-manage can try and discover all the cells via the api db technially but that assuem either you store the password in the db which we discurage or you use the same password on all db instances | |
| 12:57:11 | bauzas | https://github.com/openstack/devstack/blob/master/lib/nova#L818 | |
| 12:57:46 | bauzas | and https://github.com/openstack/devstack/blob/master/lib/nova#L847 | |
| 12:58:00 | sean-k-mooney | in generall its simpler to have one config file per cell and run the comand once per cell | |
| 12:58:04 | Uggla | for devstack I use my patch by configuring a git remote on my repo and "pulling" it | |
| 12:58:10 | bauzas | context : https://docs.openstack.org/nova/latest/admin/cells.html#configuring-a-new-deployment | |
| 12:58:27 | sean-k-mooney | Uggla: ok so you set NOVA_REPO and NOVA_BRANCH before you stacked | |
| 12:58:48 | sean-k-mooney | and you did not already have /opt/stack/nova right | |
| 12:58:56 | sean-k-mooney | before you ran stack.sh | |
| 12:59:09 | Uggla | not sorry, I had a already installed devstack and replace the code with mine + restart | |
| 12:59:19 | sean-k-mooney | oh ok | |
| 12:59:41 | sean-k-mooney | so doing sudo systemctl restart devstack@n-* | |
| 12:59:47 | Uggla | yep | |
| 12:59:50 | sean-k-mooney | works only for python changes | |
| 12:59:59 | sean-k-mooney | if you chagne the db you have to manually sync again | |
| 13:00:15 | Uggla | ack | |
| 13:00:43 | sean-k-mooney | 99% of the time we are not touching the db so we dont usually call that out | |
| 13:00:52 | Uggla | do we agree that the doc for db migration is this one: https://docs.openstack.org/nova/latest/admin/upgrades.html | |
| 13:01:07 | sean-k-mooney | yes it is | |
| 13:01:34 | sean-k-mooney | """Using the newly installed nova code, run the DB sync. First run nova-manage api_db sync, then nova-manage db sync. nova-manage db sync should be run for all cell databases, including cell0. If necessary, the --config-file argument can be used to point to the correct nova.conf file for the given cell.""" | |
| 13:01:45 | Uggla | any objection if I propose a minor update to make it clearer ? | |
| 13:02:15 | sean-k-mooney | im not sure what is not clear but sure we can review it | |
| 13:02:28 | sean-k-mooney | its always ok to propose updates to docs ectra | |
| 13:02:34 | sean-k-mooney | we will just review it as normal | |
| 13:02:47 | Uggla | sean-k-mooney, that's clearly what I was looking. And I was confuse by the fact that it was not clear for me is all cells will be updated or not. | |
| 13:02:47 | sean-k-mooney | you could add an example i guess | |
| 13:03:52 | Uggla | And more confuse by that: --local_cell Only sync db in the local cell: do not attempt to fan-out to all cells | |
| 13:03:59 | sean-k-mooney | well it says ""nova-manage db sync should be run for all cell databases"" | |
| 13:04:14 | sean-k-mooney | that imperitive directing you to run it for each cell | |
| 13:05:04 | sean-k-mooney | Uggla: feel free to add a note with examples or other changes if you think it would help make it cleaner | |
| 13:05:11 | Uggla | I agree, but I had the doubt if it was automatic or normal. | |
| 13:07:53 | sean-k-mooney | Uggla: https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L193-L195= | |
| 13:08:22 | Uggla | sean-k-mooney, clearly I understood it incorrectly. But I think this is mainly due to: "Only sync db in the local cell: do not attempt to fan-out to all cells" --> so without this option I thought it will configure all cells | |
| 13:08:46 | sean-k-mooney | not quite | |
| 13:09:00 | sean-k-mooney | without it it will do cell0 and the curernt cell | |
| 13:09:21 | sean-k-mooney | each nova.conf will only have one cell db configured | |
| 13:09:46 | sean-k-mooney | we do not have the code required to loop over all the cells in the api db and update them automatically | |
| 13:09:57 | sean-k-mooney | so without --local_cell it does cell0 and the local cell | |
| 13:10:04 | sean-k-mooney | with it it only does the local cell | |
| 13:10:09 | sean-k-mooney | cell1 in this case | |
| 13:10:27 | sean-k-mooney | cell0 is speical its not a real cell | |
| 13:10:42 | sean-k-mooney | it used for instance that could not be schduled to a host | |
| 13:10:48 | Uggla | I'm not clear. What is local cell ? | |
| 13:10:51 | sean-k-mooney | and a few other internal usecases | |
| 13:11:01 | sean-k-mooney | it the one that is reference in the nova.conf | |
| 13:11:26 | sean-k-mooney | in the [database]/connection value | |
| 13:12:18 | sean-k-mooney | so | |
| 13:12:20 | sean-k-mooney | [database] | |
| 13:12:22 | sean-k-mooney | connection = mysql+pymysql://root:password@127.0.0.1/nova_cell1?charset=utf8 | |
| 13:13:09 | sean-k-mooney | by default devstack generates /etc/nova/nova.conf with that pointign at cell0 | |
| 13:13:21 | Uggla | ok back to my previous question, do you have cases where nova.conf does not contain cell0 ? | |
| 13:13:36 | Uggla | or is it only devstack specific ? | |
| 13:13:43 | sean-k-mooney | no you do | |
| 13:13:50 | sean-k-mooney | the conductor config dont point to cell0 | |
| 13:13:57 | sean-k-mooney | they point to there cell db | |
| 13:14:20 | sean-k-mooney | whcih will be cell1 or in osp we call it just nova | |
| 13:17:25 | Uggla | ok it clear now. thx. | |
| 13:22:06 | Uggla | \o/ my code is working. At least the index part... ;) | |
| 13:35:07 | Uggla | and the creation part. Seems I manage to discuss with manila ! | |
| 13:53:52 | spatel | sean-k-mooney i have opened this bug for my issue - https://bugs.launchpad.net/nova/+bug/1968054 | |
| 13:54:25 | sean-k-mooney | that does not seem like a nova issue | |
| 13:54:39 | sean-k-mooney | the error is with conencting to the rabbitmq right | |
| 13:56:41 | bauzas | reminder : nova sessions start in 4 mins | |
| 14:02:29 | bauzas | nova sessions start now | |
| 14:02:37 | bauzas | https://www.openstack.org/ptg/rooms/newton | |
| 14:02:37 | sean-k-mooney | joining | |
| 14:02:50 | sean-k-mooney | spatel: that looks like you are having internal network issue with connectivty between your comptues and the contoler | |
| 14:03:41 | spatel | This is all running behind single switch so its hard to say network issue. all other nodes are happy | |
| 14:04:00 | spatel | as soon as i restart my nova-compute everything came back | |
| 14:04:20 | spatel | look like some thread crashed and never recover until manually restart | |
| 14:04:52 | sean-k-mooney | it could be related to dns | |
| 14:04:53 | spatel | Out of 20 compute 2 nodes encounter with this issue.. | |
| 14:05:06 | sean-k-mooney | https://github.com/openstack/nova/commit/fe1ebe69f358cbed62434da3f1537a94390324bb | |
| 14:06:12 | spatel | I have everything using IP address | |
| 14:06:21 | spatel | i have found similar bug but not sure its related to my issue or not - https://bugs.launchpad.net/oslo.messaging/+bug/1934937 | |
| 14:07:05 | sean-k-mooney | spatel: the error are not coming from nova so i woudl think its unlikely this is a nova bug it proably an oslo one if its not related to your infra | |
| 14:07:35 | spatel | i don't think its related to infra for sure | |
| 14:08:12 | spatel | i may post same bug to oslo | |