| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-05-20 | |||
| 10:00:53 | artom | And Neutron? | |
| 10:00:53 | artom | And Neutron? | |
| 10:01:07 | admin0 | its in rocky release .. there is no placement database or api as far as i know | |
| 10:01:07 | admin0 | its in rocky release .. there is no placement database or api as far as i know | |
| 10:02:02 | artom | Wait, when did we move placement out of tree? | |
| 10:02:02 | artom | Wait, when did we move placement out of tree? | |
| 10:02:09 | artom | Before rocky, surely | |
| 10:02:09 | artom | Before rocky, surely | |
| 10:02:18 | admin0 | then maybe i am wrong | |
| 10:02:18 | admin0 | then maybe i am wrong | |
| 10:03:06 | artom | admin0, but the point, the hostname is used in placement resource providers, in neutron port bindings, and in cinder connection_info (sometimes? depending on the volume backend configuration?) I'm less familiar with how that works) | |
| 10:03:06 | artom | admin0, but the point, the hostname is used in placement resource providers, in neutron port bindings, and in cinder connection_info (sometimes? depending on the volume backend configuration?) I'm less familiar with how that works) | |
| 10:03:08 | admin0 | i do not see a database called "placemenet" like i see in newer deployments | |
| 10:03:08 | admin0 | i do not see a database called "placemenet" like i see in newer deployments | |
| 10:05:19 | admin0 | fair point .. so right now, i have around 16 instances that are running in a compute node called h7 (ceph backend) .. which if I want to migrate from h7 -> h9, it says h7- host not found | |
| 10:05:19 | admin0 | fair point .. so right now, i have around 16 instances that are running in a compute node called h7 (ceph backend) .. which if I want to migrate from h7 -> h9, it says h7- host not found | |
| 10:05:35 | admin0 | openstack server show $uuid shows it be on h7 ( without . and without .openstack.local) | |
| 10:05:35 | admin0 | openstack server show $uuid shows it be on h7 ( without . and without .openstack.local) | |
| 10:06:07 | admin0 | if there is a way to know what exactly hostname an instance is supposed to be at, then maybe at least i can migrate by matching the existing hostname to what the instance is expecting | |
| 10:06:07 | admin0 | if there is a way to know what exactly hostname an instance is supposed to be at, then maybe at least i can migrate by matching the existing hostname to what the instance is expecting | |
| 10:06:49 | admin0 | artom, stephenfin this shows the issue and the command I tried: https://gist.github.com/a1git/ce400f1084a184fa5c805ee72554a6e3 | |
| 10:06:49 | admin0 | artom, stephenfin this shows the issue and the command I tried: https://gist.github.com/a1git/ce400f1084a184fa5c805ee72554a6e3 | |
| 10:10:50 | admin0 | because i was 100% sure the hostname was h7 and not h7. before, ( the . got added farily recently ) , i changed the hostname, hostname -f, virsh hostname back to h7 .. but it did not solved the issue | |
| 10:10:50 | admin0 | because i was 100% sure the hostname was h7 and not h7. before, ( the . got added farily recently ) , i changed the hostname, hostname -f, virsh hostname back to h7 .. but it did not solved the issue | |
| 10:11:05 | stephenfin | admin0: so like I said above, fixing this will likely involve manual database fixes | |
| 10:11:05 | stephenfin | admin0: so like I said above, fixing this will likely involve manual database fixes | |
| 10:11:29 | artom | s/likely/definitely/ | |
| 10:11:29 | artom | s/likely/definitely/ | |
| 10:11:31 | stephenfin | if you have a backup (you have a backup, right), you should use that to see what values were there previously | |
| 10:11:31 | stephenfin | if you have a backup (you have a backup, right), you should use that to see what values were there previously | |
| 10:11:34 | admin0 | stephenfin, that is fine .. but which database/tables have this associations | |
| 10:11:34 | admin0 | stephenfin, that is fine .. but which database/tables have this associations | |
| 10:13:17 | stephenfin | admin0: compute_nodes has a host column and hypervisor_hostname column, the migrations table also has source_node and dest_node columns | |
| 10:13:17 | stephenfin | admin0: compute_nodes has a host column and hypervisor_hostname column, the migrations table also has source_node and dest_node columns | |
| 10:14:21 | stephenfin | admin0: tbh, you should look at 'nova/db/sqlalchemy/models.py' for anything hostname-related (i.e. the column name has "host" or "node" in it) and then check your DB to see if entries matching those hostnames appears | |
| 10:14:21 | stephenfin | admin0: tbh, you should look at 'nova/db/sqlalchemy/models.py' for anything hostname-related (i.e. the column name has "host" or "node" in it) and then check your DB to see if entries matching those hostnames appears | |
| 10:14:37 | stephenfin | admin0: and for the love of God, back up your DB before you start messing around with it | |
| 10:14:37 | stephenfin | admin0: and for the love of God, back up your DB before you start messing around with it | |
| 10:14:40 | stephenfin | :) | |
| 10:14:40 | stephenfin | :) | |
| 10:16:15 | admin0 | stephenfin, host says h7 and hypervisor_hostname says h7. | |
| 10:16:15 | admin0 | stephenfin, host says h7 and hypervisor_hostname says h7. | |
| 10:16:18 | artom | I'd also do stuff like check the volume attachments and port bindinds, and the resource provider, see if there are host names in there anywhere. Sorry to not be more specific | |
| 10:16:18 | artom | I'd also do stuff like check the volume attachments and port bindinds, and the resource provider, see if there are host names in there anywhere. Sorry to not be more specific | |
| 10:16:48 | admin0 | so if i put the hostname, hostname-f and virsh hostname to be h7, i think i can safely update hypervisor_hostname to h7 ( removing the .) | |
| 10:16:48 | admin0 | so if i put the hostname, hostname-f and virsh hostname to be h7, i think i can safely update hypervisor_hostname to h7 ( removing the .) | |
| 10:17:11 | stephenfin | admin0: that sounds correct, but you'll need to experiment | |
| 10:17:11 | stephenfin | admin0: that sounds correct, but you'll need to experiment | |
| 10:17:24 | stephenfin | admin0: also, what artom said | |
| 10:17:24 | stephenfin | admin0: also, what artom said | |
| 10:17:29 | admin0 | yeah | |
| 10:17:29 | admin0 | yeah | |
| 10:20:45 | aarents | lyarwood: agree that rebasing thousands of instance isn't worth the effort, and indeed as you said UCA did not backport the "relaxing patch" in focal. I will do downstream, thanks again for the hint | |
| 10:20:45 | aarents | lyarwood: agree that rebasing thousands of instance isn't worth the effort, and indeed as you said UCA did not backport the "relaxing patch" in focal. I will do downstream, thanks again for the hint | |
| 10:22:51 | lyarwood | aarents: np anytime | |
| 10:22:51 | lyarwood | aarents: np anytime | |
| 10:28:02 | admin0 | stephenfin, i think this provides a picture --> https://gist.githubusercontent.com/a1git/0af2d9fbd93787ba34723c9c7bdbba4a/raw/1eb6c6f93ed5495f5ac4c484c97fc53a41ca6f2e/gistfile1.txt | |
| 10:28:03 | admin0 | stephenfin, i think this provides a picture --> https://gist.githubusercontent.com/a1git/0af2d9fbd93787ba34723c9c7bdbba4a/raw/1eb6c6f93ed5495f5ac4c484c97fc53a41ca6f2e/gistfile1.txt | |
| 10:28:13 | admin0 | looking into the original ids at the top . | |
| 10:28:13 | admin0 | looking into the original ids at the top . | |
| 10:28:38 | admin0 | and i think .openstack.local was added by openstack as i did not had any domain in the hostname -f part before | |
| 10:28:38 | admin0 | and i think .openstack.local was added by openstack as i did not had any domain in the hostname -f part before | |
| 10:30:27 | admin0 | so question is, do i set the hypervisor_hostname to just h7 or h7.openstack.local ? | |
| 10:30:27 | admin0 | so question is, do i set the hypervisor_hostname to just h7 or h7.openstack.local ? | |
| 10:31:23 | sean-k-mooney | admin0: it will depend on when the vms were cereated | |
| 10:31:23 | sean-k-mooney | admin0: it will depend on when the vms were cereated | |
| 10:31:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Use jinja2 to generate fake libvirt data https://review.opendev.org/c/openstack/nova/+/790971 | |
| 10:31:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Rename 'ImageBackendFixture' to 'LibvirtImageBackendFixture' https://review.opendev.org/c/openstack/nova/+/792353 | |
| 10:31:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Use jinja2 to generate fake libvirt data https://review.opendev.org/c/openstack/nova/+/790971 | |
| 10:32:05 | sean-k-mooney | if you have had vms created with different values over the lifetime of the host then you wil not be able to avoid having to modify the nova, nueton and cinder dbs to correcct this | |
| 10:32:05 | sean-k-mooney | if you have had vms created with different values over the lifetime of the host then you wil not be able to avoid having to modify the nova, nueton and cinder dbs to correcct this | |
| 10:32:16 | stephenfin | lyarwood: Addressed your comments for last week ^ | |
| 10:32:16 | stephenfin | lyarwood: Addressed your comments for last week ^ | |
| 10:32:18 | sean-k-mooney | admin0: this is why we dont actullly supprot changing the host name | |
| 10:32:19 | sean-k-mooney | admin0: this is why we dont actullly supprot changing the host name | |
| 10:32:57 | sean-k-mooney | admin0: it will also affect the placment resouce provider name too | |
| 10:32:57 | sean-k-mooney | admin0: it will also affect the placment resouce provider name too | |
| 10:33:30 | sean-k-mooney | admin0: i would proably go with h7 but both are valid | |
| 10:33:30 | sean-k-mooney | admin0: i would proably go with h7 but both are valid | |
| 10:34:42 | lyarwood | stephenfin: ack thanks I'll take a look shortly | |
| 10:34:42 | lyarwood | stephenfin: ack thanks I'll take a look shortly | |
| 10:34:53 | sean-k-mooney | stephenfin: artom this has come up enough times at this poitn that i think we need to put a check in init host to block starting the agent if the hypervior_hostname has change | |
| 10:34:54 | sean-k-mooney | stephenfin: artom this has come up enough times at this poitn that i think we need to put a check in init host to block starting the agent if the hypervior_hostname has change | |
| 10:35:02 | sean-k-mooney | what do you think ? | |
| 10:35:02 | sean-k-mooney | what do you think ? | |
| 10:35:38 | artom | sean-k-mooney, ah, interesting... | |
| 10:35:38 | artom | sean-k-mooney, ah, interesting... | |
| 10:36:25 | stephenfin | how would you detect that. I was under the impression that changing the hostname meant the compute node was re-registering itself or something | |
| 10:36:25 | stephenfin | how would you detect that. I was under the impression that changing the hostname meant the compute node was re-registering itself or something | |
| 10:36:45 | sean-k-mooney | not in all cases | |
| 10:36:45 | sean-k-mooney | not in all cases | |
| 10:37:07 | sean-k-mooney | it will but the compute service record wont always change | |
| 10:37:07 | sean-k-mooney | it will but the compute service record wont always change | |
| 10:37:10 | artom | stephenfin, compare the hostname from the OS to the existing DB record? | |
| 10:37:10 | artom | stephenfin, compare the hostname from the OS to the existing DB record? | |
| 10:37:14 | admin0 | in my case, i cannot migrate, spawn new instance, work on security groups, so i think its no harm kind of | |
| 10:37:14 | admin0 | in my case, i cannot migrate, spawn new instance, work on security groups, so i think its no harm kind of | |
| 10:37:38 | sean-k-mooney | artom: there is a chiken and egg problem | |
| 10:37:38 | sean-k-mooney | artom: there is a chiken and egg problem | |
| 10:38:13 | sean-k-mooney | artom: i was thinkign we look up the compute service recored using conf.host then look up the compute node records | |