| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-05-20 | |||
| 08:18:07 | bauzas | and not everyone reads openinfra mailing lists | |
| 08:18:07 | bauzas | and not everyone reads openinfra mailing lists | |
| 08:18:28 | bauzas | actually, I'll reply by this | |
| 08:18:28 | bauzas | actually, I'll reply by this | |
| 08:18:47 | bauzas | * bauzas usually hates to jump on those kind of large and opiniated threads | |
| 08:25:49 | kashyap | Sure; what you suggest should be "obvious" for the Infra folks; and is part of clear communication to the communities. | |
| 08:25:49 | kashyap | Sure; what you suggest should be "obvious" for the Infra folks; and is part of clear communication to the communities. | |
| 08:36:45 | aarents | Morning nova! | |
| 08:36:45 | aarents | Morning nova! | |
| 08:37:04 | aarents | in this change "libvirt: Provide the backing file format when creating qcow2 disks" https://review.opendev.org/c/openstack/nova/+/708745/ , | |
| 08:37:04 | aarents | in this change "libvirt: Provide the backing file format when creating qcow2 disks" https://review.opendev.org/c/openstack/nova/+/708745/ , | |
| 08:37:22 | aarents | we ensure to add the "backing_fmt" field at qcow2 image creation due to newer libvirt constraints.(if not present newer libvirt does not start instance) | |
| 08:37:23 | aarents | we ensure to add the "backing_fmt" field at qcow2 image creation due to newer libvirt constraints.(if not present newer libvirt does not start instance) | |
| 08:37:44 | aarents | there is the Lee following quote: "Nova needs to [...] potentially handle the upgrade case where we may need to rebase existing disks in order to update the metadata." | |
| 08:37:44 | aarents | there is the Lee following quote: "Nova needs to [...] potentially handle the upgrade case where we may need to rebase existing disks in order to update the metadata." | |
| 08:38:01 | aarents | I would like to know if there is an ongoing work with the "upgrade case" ? | |
| 08:38:02 | aarents | I would like to know if there is an ongoing work with the "upgrade case" ? | |
| 08:38:49 | openstackgerrit | Lucas Alvares Gomes proposed openstack/nova master: DO NOT REVIEW: Testing new OVN as default https://review.opendev.org/c/openstack/nova/+/792328 | |
| 08:38:49 | openstackgerrit | Lucas Alvares Gomes proposed openstack/nova master: DO NOT REVIEW: Testing new OVN as default https://review.opendev.org/c/openstack/nova/+/792328 | |
| 08:50:15 | lyarwood | aarents: no as the check was removed in the next minor release of libvirt | |
| 08:50:15 | lyarwood | aarents: no as the check was removed in the next minor release of libvirt | |
| 08:50:49 | lyarwood | aarents: AFAIK UCA is still shipping the original version with the check so if you're running into this you need to raise a bug against UCA and ask them to bump their version of libvirt | |
| 08:50:49 | lyarwood | aarents: AFAIK UCA is still shipping the original version with the check so if you're running into this you need to raise a bug against UCA and ask them to bump their version of libvirt | |
| 08:54:50 | aarents | lyarwood: intersting I will double check, thank you | |
| 08:54:50 | aarents | lyarwood: intersting I will double check, thank you | |
| 08:57:14 | lyarwood | aarents: np, fixing this by rebasing every single instance disk as a result just isn't worth the effort IMHO | |
| 08:57:15 | lyarwood | aarents: np, fixing this by rebasing every single instance disk as a result just isn't worth the effort IMHO | |
| 09:16:25 | artom | Huh, http://status.openstack.org/reviews/ is broken | |
| 09:16:25 | artom | Huh, http://status.openstack.org/reviews/ is broken | |
| 09:56:21 | admin0 | hi all.. how do I rename an existing host to a new one ? | |
| 09:56:21 | admin0 | hi all.. how do I rename an existing host to a new one ? | |
| 09:56:41 | stephenfin | admin0: afaik, you can't | |
| 09:56:41 | stephenfin | admin0: afaik, you can't | |
| 09:56:55 | admin0 | the issue i have is that i started with hostname say h7 .. then at some point, it registered as h7.openstack.local and at some point its registered at h7. ( only a . with no domain at the end) | |
| 09:56:55 | admin0 | the issue i have is that i started with hostname say h7 .. then at some point, it registered as h7.openstack.local and at some point its registered at h7. ( only a . with no domain at the end) | |
| 09:57:10 | admin0 | so now my associations are broken .. i cannot migrate stuff .. i cannot use security groups | |
| 09:57:10 | admin0 | so now my associations are broken .. i cannot migrate stuff .. i cannot use security groups | |
| 09:57:29 | admin0 | because the instances are running, how can i fix this | |
| 09:57:29 | admin0 | because the instances are running, how can i fix this | |
| 09:57:55 | stephenfin | oh, so you had one name and you've inadvertently changed it? | |
| 09:57:55 | stephenfin | oh, so you had one name and you've inadvertently changed it? | |
| 09:58:41 | stephenfin | artom can hopefully sanity check what I'm saying, but I think you'll need to manually correct the ComputeNode entries for the host in the DB and ensure your RabbitMQ queues are correctly named | |
| 09:58:41 | stephenfin | artom can hopefully sanity check what I'm saying, but I think you'll need to manually correct the ComputeNode entries for the host in the DB and ensure your RabbitMQ queues are correctly named | |
| 09:59:16 | admin0 | stephenfin, i did not knowingly changed it .. i think one admin accidently added openstack_domain: "" because he did not wanted to see openstack.local and ran the openstack ansible nova playbooks .. and that caused hosts calling h7 to be re-added as h7. | |
| 09:59:16 | admin0 | stephenfin, i did not knowingly changed it .. i think one admin accidently added openstack_domain: "" because he did not wanted to see openstack.local and ran the openstack ansible nova playbooks .. and that caused hosts calling h7 to be re-added as h7. | |
| 09:59:21 | admin0 | stephenfin, here is what i see now: openstack_domain: "" | |
| 09:59:21 | admin0 | stephenfin, here is what i see now: openstack_domain: "" | |
| 09:59:31 | admin0 | stephenfin, sorry .. https://gist.github.com/a1git/c22ec0c17aaa9dcf95fd7485eb76af2f | |
| 09:59:31 | admin0 | stephenfin, sorry .. https://gist.github.com/a1git/c22ec0c17aaa9dcf95fd7485eb76af2f | |
| 10:00:36 | artom | stephenfin, that sounds about right | |
| 10:00:36 | artom | stephenfin, that sounds about right | |
| 10:00:48 | artom | There may also be placement and/or cinder fallout | |
| 10:00:48 | artom | There may also be placement and/or cinder fallout | |
| 10:00:53 | artom | And Neutron? | |
| 10:00:53 | artom | And Neutron? | |
| 10:01:07 | admin0 | its in rocky release .. there is no placement database or api as far as i know | |
| 10:01:07 | admin0 | its in rocky release .. there is no placement database or api as far as i know | |
| 10:02:02 | artom | Wait, when did we move placement out of tree? | |
| 10:02:02 | artom | Wait, when did we move placement out of tree? | |
| 10:02:09 | artom | Before rocky, surely | |
| 10:02:09 | artom | Before rocky, surely | |
| 10:02:18 | admin0 | then maybe i am wrong | |
| 10:02:18 | admin0 | then maybe i am wrong | |
| 10:03:06 | artom | admin0, but the point, the hostname is used in placement resource providers, in neutron port bindings, and in cinder connection_info (sometimes? depending on the volume backend configuration?) I'm less familiar with how that works) | |
| 10:03:06 | artom | admin0, but the point, the hostname is used in placement resource providers, in neutron port bindings, and in cinder connection_info (sometimes? depending on the volume backend configuration?) I'm less familiar with how that works) | |
| 10:03:08 | admin0 | i do not see a database called "placemenet" like i see in newer deployments | |
| 10:03:08 | admin0 | i do not see a database called "placemenet" like i see in newer deployments | |
| 10:05:19 | admin0 | fair point .. so right now, i have around 16 instances that are running in a compute node called h7 (ceph backend) .. which if I want to migrate from h7 -> h9, it says h7- host not found | |
| 10:05:19 | admin0 | fair point .. so right now, i have around 16 instances that are running in a compute node called h7 (ceph backend) .. which if I want to migrate from h7 -> h9, it says h7- host not found | |
| 10:05:35 | admin0 | openstack server show $uuid shows it be on h7 ( without . and without .openstack.local) | |
| 10:05:35 | admin0 | openstack server show $uuid shows it be on h7 ( without . and without .openstack.local) | |
| 10:06:07 | admin0 | if there is a way to know what exactly hostname an instance is supposed to be at, then maybe at least i can migrate by matching the existing hostname to what the instance is expecting | |
| 10:06:07 | admin0 | if there is a way to know what exactly hostname an instance is supposed to be at, then maybe at least i can migrate by matching the existing hostname to what the instance is expecting | |
| 10:06:49 | admin0 | artom, stephenfin this shows the issue and the command I tried: https://gist.github.com/a1git/ce400f1084a184fa5c805ee72554a6e3 | |
| 10:06:49 | admin0 | artom, stephenfin this shows the issue and the command I tried: https://gist.github.com/a1git/ce400f1084a184fa5c805ee72554a6e3 | |
| 10:10:50 | admin0 | because i was 100% sure the hostname was h7 and not h7. before, ( the . got added farily recently ) , i changed the hostname, hostname -f, virsh hostname back to h7 .. but it did not solved the issue | |
| 10:10:50 | admin0 | because i was 100% sure the hostname was h7 and not h7. before, ( the . got added farily recently ) , i changed the hostname, hostname -f, virsh hostname back to h7 .. but it did not solved the issue | |
| 10:11:05 | stephenfin | admin0: so like I said above, fixing this will likely involve manual database fixes | |
| 10:11:05 | stephenfin | admin0: so like I said above, fixing this will likely involve manual database fixes | |
| 10:11:29 | artom | s/likely/definitely/ | |
| 10:11:29 | artom | s/likely/definitely/ | |
| 10:11:31 | stephenfin | if you have a backup (you have a backup, right), you should use that to see what values were there previously | |
| 10:11:31 | stephenfin | if you have a backup (you have a backup, right), you should use that to see what values were there previously | |
| 10:11:34 | admin0 | stephenfin, that is fine .. but which database/tables have this associations | |
| 10:11:34 | admin0 | stephenfin, that is fine .. but which database/tables have this associations | |
| 10:13:17 | stephenfin | admin0: compute_nodes has a host column and hypervisor_hostname column, the migrations table also has source_node and dest_node columns | |
| 10:13:17 | stephenfin | admin0: compute_nodes has a host column and hypervisor_hostname column, the migrations table also has source_node and dest_node columns | |
| 10:14:21 | stephenfin | admin0: tbh, you should look at 'nova/db/sqlalchemy/models.py' for anything hostname-related (i.e. the column name has "host" or "node" in it) and then check your DB to see if entries matching those hostnames appears | |
| 10:14:21 | stephenfin | admin0: tbh, you should look at 'nova/db/sqlalchemy/models.py' for anything hostname-related (i.e. the column name has "host" or "node" in it) and then check your DB to see if entries matching those hostnames appears | |
| 10:14:37 | stephenfin | admin0: and for the love of God, back up your DB before you start messing around with it | |
| 10:14:37 | stephenfin | admin0: and for the love of God, back up your DB before you start messing around with it | |
| 10:14:40 | stephenfin | :) | |
| 10:14:40 | stephenfin | :) | |
| 10:16:15 | admin0 | stephenfin, host says h7 and hypervisor_hostname says h7. | |
| 10:16:15 | admin0 | stephenfin, host says h7 and hypervisor_hostname says h7. | |
| 10:16:18 | artom | I'd also do stuff like check the volume attachments and port bindinds, and the resource provider, see if there are host names in there anywhere. Sorry to not be more specific | |
| 10:16:18 | artom | I'd also do stuff like check the volume attachments and port bindinds, and the resource provider, see if there are host names in there anywhere. Sorry to not be more specific | |
| 10:16:48 | admin0 | so if i put the hostname, hostname-f and virsh hostname to be h7, i think i can safely update hypervisor_hostname to h7 ( removing the .) | |
| 10:16:48 | admin0 | so if i put the hostname, hostname-f and virsh hostname to be h7, i think i can safely update hypervisor_hostname to h7 ( removing the .) | |
| 10:17:11 | stephenfin | admin0: that sounds correct, but you'll need to experiment | |