| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-12-14 | |||
| 13:04:42 | sean-k-mooney | i assume tempest has the same bug | |
| 13:10:57 | hemanth_n | sean-k-mooney, lyardwood: As discussed on Friday, i started backporting Iabe37dbdc244b019cb68099739b2c1c81b94a30d, and it requires I999fc4aaf14cca02e4ed730afb844a043b59530c for the tests to be successful. So, i am going ahead with backport of I999fc4aaf14cca02e4ed730afb844a043b59530c (after some tests on my local).. hope this is ok | |
| 13:11:20 | openstackgerrit | Mamduh proposed openstack/os-vif stable/ussuri: Refactor code of linux_net to more cleaner and increase performace https://review.opendev.org/c/openstack/os-vif/+/765419 | |
| 13:11:30 | ygk_12345 | hi all | |
| 13:11:53 | ygk_12345 | I am facing this error in nova-api log in the Train nova setup. http://paste.openstack.org/raw/801004/ | |
| 13:11:58 | ygk_12345 | is this a known bug ? | |
| 13:12:32 | sean-k-mooney | hemanth_n: that touches more files then i would like but i think its ok | |
| 13:12:47 | sean-k-mooney | hemanth_n:looking at the change its minimal | |
| 13:13:06 | hemanth_n | sean-k-mooney: yep more backports, ack i will test in local before i submit a patch | |
| 13:14:31 | sean-k-mooney | ygk_12345: that looks like you have not run the required db migration our your backporint some with a db change which is not allowed | |
| 13:14:51 | ygk_12345 | sean-k-mooney what would be the exact command ? | |
| 13:15:13 | openstackgerrit | Mamduh proposed openstack/os-vif stable/train: Refactor code of linux_net to more cleaner and increase performace https://review.opendev.org/c/openstack/os-vif/+/765912 | |
| 13:15:49 | sean-k-mooney | ygk_12345: hum it was added in pike https://github.com/openstack/nova/commit/3674a4268d177230375fa1b581dbdf6f62755cee | |
| 13:16:08 | sean-k-mooney | ygk_12345: "nova-manage db sync" | |
| 13:16:41 | sean-k-mooney | but given how long ago it was added its unlikely you have not run any db migration since pike | |
| 13:16:57 | ygk_12345 | sean-k-mooney is this the one "su -s /bin/sh -c "nova-manage db sync" nova" ? | |
| 13:18:14 | sean-k-mooney | ygk_12345: more or less yes although it proably not the issue | |
| 13:18:23 | ygk_12345 | sean-k-mooney now I got this after the command http://paste.openstack.org/raw/801005/ | |
| 13:18:43 | openstackgerrit | Mamduh proposed openstack/os-vif stable/stein: Refactor code of linux_net to more cleaner and increase performace https://review.opendev.org/c/openstack/os-vif/+/765914 | |
| 13:19:53 | sean-k-mooney | so this is train | |
| 13:20:11 | sean-k-mooney | was the cloud initall deploy as train or upgraded form a previous release | |
| 13:20:11 | ygk_12345 | sean-k-mooney yes | |
| 13:20:21 | ygk_12345 | intially as train only | |
| 13:22:10 | sean-k-mooney | so that failing here https://github.com/openstack/nova/blob/0e7cd9d1a95a30455e3c91916ece590454235e0e/nova/db/sqlalchemy/migrate_repo/versions/316_add_disk_ratio_for_compute_nodes.py | |
| 13:22:59 | ygk_12345 | sean-k-mooney yes exactly | |
| 13:23:32 | sean-k-mooney | so if the migration is failing there which is mitaka then ist possbale that the rest fo the migration have not been run either | |
| 13:23:45 | sean-k-mooney | the service uuid was added in a later migration | |
| 13:24:29 | sean-k-mooney | lyarwood: gibi do you know if we ahve a way to print the crrent db version via nova manage | |
| 13:24:51 | ygk_12345 | sean-k-mooney so what would be the workaround now ? | |
| 13:24:52 | sean-k-mooney | stephenfin: maybe you know ^ | |
| 13:25:16 | sean-k-mooney | ygk_12345: well we need to figure out what version your db actull is | |
| 13:25:35 | sean-k-mooney | it sound like your deployment didnt actully work properly | |
| 13:25:40 | ygk_12345 | sean-k-mooney i am following the train minimal nova installation guide | |
| 13:25:50 | ygk_12345 | sean-k-mooney oh ok | |
| 13:26:35 | ygk_12345 | sean-k-mooney i used this on ubuntu18 add-apt-repository cloud-archive:train | |
| 13:27:22 | sean-k-mooney | the install gudies are not really tested at least not by the project teams | |
| 13:27:25 | lyarwood | sean-k-mooney: yeah isn't there a version subcommand or something? | |
| 13:27:55 | lyarwood | https://docs.openstack.org/nova/latest/cli/nova-manage.html#nova-database - nova-manage db version | |
| 13:28:37 | ygk_12345 | sean-k-mooney its version 315 | |
| 13:28:56 | ygk_12345 | lyarwood sean-k-mooney its db version 315 | |
| 13:28:56 | sean-k-mooney | ygk_12345: ya so that pre mitaka | |
| 13:29:05 | openstackgerrit | Mamduh proposed openstack/os-vif stable/rocky: Refactor code of linux_net to more cleaner and increase performace https://review.opendev.org/c/openstack/os-vif/+/765962 | |
| 13:29:35 | ygk_12345 | sean-k-mooney so what would be the workaroud now ? | |
| 13:29:35 | sean-k-mooney | ygk_12345: so at that point you should not have the disk_allocation_ratio column in that table | |
| 13:30:12 | sean-k-mooney | i think what has happened here is that since you started the services before the db was synced | |
| 13:30:38 | sean-k-mooney | the conductor or one of the other service created teh db column | |
| 13:31:05 | ygk_12345 | sean-k-mooney shall I remove the db and nova pkgs and try again ? | |
| 13:31:21 | sean-k-mooney | you could do that yes if you have not workloads on it | |
| 13:31:38 | ygk_12345 | sean-k-mooney just a test platform. no workloads | |
| 13:31:52 | sean-k-mooney | you dont really need to uninstall the service just stop them | |
| 13:32:00 | sean-k-mooney | then drop the db and recreate it | |
| 13:32:18 | sean-k-mooney | making sure that the db is fully migrated | |
| 13:33:07 | ygk_12345 | sean-k-mooney ok. sounds fine. will do that now | |
| 13:34:31 | sean-k-mooney | you were following https://docs.openstack.org/nova/train/install/controller-install-ubuntu.html#install-and-configure-components | |
| 13:35:29 | ygk_12345 | sean-k-mooney exactly. you spotted it :) | |
| 13:35:30 | sean-k-mooney | that has you restart the services at the end https://docs.openstack.org/nova/train/install/controller-install-ubuntu.html#finalize-installation implying they are running before you populate the db | |
| 13:35:35 | sean-k-mooney | i think that is wrong | |
| 13:35:44 | sean-k-mooney | really it should have you install nova-manage only | |
| 13:35:50 | sean-k-mooney | then populate the db | |
| 13:36:03 | sean-k-mooney | then install the contoler services after | |
| 13:36:19 | sean-k-mooney | i think that is what caused your issue | |
| 13:37:01 | openstackgerrit | Mamduh proposed openstack/os-vif stable/queens: Refactor code of linux_net to more cleaner and increase performace https://review.opendev.org/c/openstack/os-vif/+/765941 | |
| 13:37:08 | ygk_12345 | sean-k-mooney so i have deleted the dbs and created again | |
| 13:38:47 | ygk_12345 | sean-k-mooney i have created the dbs again now. Whats the next step ? | |
| 13:38:50 | sean-k-mooney | ygk_12345: are all the nova services stopoed | |
| 13:38:56 | sean-k-mooney | *stopped | |
| 13:39:18 | ygk_12345 | sean-k-mooney not on the compute node though. will it affect ? | |
| 13:40:27 | sean-k-mooney | possibly | |
| 13:40:32 | sean-k-mooney | although no it should not | |
| 13:40:43 | sean-k-mooney | the compute node only access the db via the conductor | |
| 13:41:19 | ygk_12345 | sean-k-mooney now shall I populate the dbs ? | |
| 13:42:06 | ygk_12345 | sean-k-mooney also the services are shown to be started in the end after the db stuff in that doc. so whats wrong with that ? | |
| 13:42:07 | sean-k-mooney | yep thats the next step | |
| 13:42:52 | sean-k-mooney | starting them at the end is ok | |
| 13:43:09 | sean-k-mooney | but they should not be started before the db is ready | |
| 13:43:47 | sean-k-mooney | steps 3-6 should be done before any of the serviecs are started | |
| 13:44:24 | ygk_12345 | @se | |
| 13:44:37 | ygk_12345 | sean-k-mooney oh ok got it. it went well this time. lets see further | |
| 13:45:16 | sean-k-mooney | before you start the services check the db version | |
| 13:46:03 | ygk_12345 | oh ok | |
| 13:46:07 | sean-k-mooney | i think it shoudl be 402 | |
| 13:46:26 | sean-k-mooney | https://github.com/openstack/nova/tree/stable/train/nova/db/sqlalchemy/migrate_repo/versions based on the last migration | |
| 13:47:21 | sean-k-mooney | if it gets to 402 then your good to go with starting the services | |
| 13:48:30 | ygk_12345 | yes its 402. awesome | |
| 13:49:36 | sean-k-mooney | feel free to submit a patch to update the install guide with this and any other issue you find | |
| 13:50:26 | sean-k-mooney | as i said in this case it should have had you install just nova-manage then set up the db then install the other nova packages for the api conducort and scheduler | |
| 13:51:11 | sean-k-mooney | ygk_12345: when you install the packages the post init script starts the service automatically which can cause issues as you found | |
| 13:51:25 | ygk_12345 | sean-k-mooney ok sure thanks for your time as well | |
| 13:51:39 | sean-k-mooney | no worries | |
| 14:28:29 | gibi | lyarwood: sorry I was away, but I see you found the reasons | |
| 14:59:14 | sean-k-mooney | gibi: i suspect the conductor proably created the column not really sure what actully did it but yes | |
| 14:59:46 | sean-k-mooney | gibi: i have seen that with devstack often enought to know that running the service before the db is ready end is weired issue like that | |
| 15:00:48 | gibi | sean-k-mooney: oh I referred to the aggregate question from lyarwood | |
| 15:00:59 | sean-k-mooney | oh ok | |
| 15:01:04 | sean-k-mooney | sorry | |
| 15:12:39 | openstackgerrit | Lance Bragstad proposed openstack/placement master: Pass context objects to oslo.policy directly https://review.opendev.org/c/openstack/placement/+/766785 | |
| 15:12:40 | openstackgerrit | Lance Bragstad proposed openstack/placement master: Implement secure RBAC for resource providers https://review.opendev.org/c/openstack/placement/+/760240 | |
| 15:41:53 | openstackgerrit | Jan Gutter proposed openstack/os-vif stable/train: DNM: testing lower-constraints https://review.opendev.org/c/openstack/os-vif/+/766975 | |
| 15:45:08 | openstackgerrit | Lee Yarwood proposed openstack/nova master: Add regression test for bug #1908075 https://review.opendev.org/c/openstack/nova/+/766976 | |
| 15:45:08 | openstack | bug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Undecided,New] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood) | |