| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-09-30 | |||
| 22:31:51 | sean-k-mooney | the only time after its booted that it can not have a host in cell 1 is if its shleved | |
| 22:36:14 | atmark | so it didn't end up in any cell since it doesn't have a host? | |
| 22:36:27 | atmark | host is NULL | |
| 22:36:42 | sean-k-mooney | select * from nova_api.instance_mappings where instance_uuid = "ded221d0-8410-4f89-b322-b9ff0207a0dd"; | |
| 22:36:50 | sean-k-mooney | can you run that but with your instance uuid | |
| 22:37:19 | sean-k-mooney | you should see somethign like this https://paste.openstack.org/show/bAUqQT5Al2Q9GoF9Nln2/ | |
| 22:38:15 | atmark | https://paste.openstack.org/show/b9tx6I2rQ3tUtsGBR98g/ | |
| 22:38:20 | sean-k-mooney | what im wondering is does that exist and is the cell_id set | |
| 22:38:32 | sean-k-mooney | and if so is that cell in your cell_mappings table | |
| 22:38:54 | sean-k-mooney | ok so cell_id is 6 | |
| 22:39:11 | sean-k-mooney | the cell mappings | |
| 22:39:16 | sean-k-mooney | has potitally password inf | |
| 22:39:22 | sean-k-mooney | so dont paste that | |
| 22:39:46 | atmark | 6 exist | |
| 22:39:49 | sean-k-mooney | but does the db connection string for cell id 6 end in nova | |
| 22:39:49 | atmark | https://paste.openstack.org/show/bh9PvsUNrxNuYSRsATga/ | |
| 22:40:08 | atmark | 6 and 9 | |
| 22:40:13 | sean-k-mooney | yep | |
| 22:40:20 | sean-k-mooney | does 6 point to the db where the instance iss | |
| 22:40:25 | sean-k-mooney | i.e. nova in your case | |
| 22:40:48 | sean-k-mooney | it shoudl end in :3306/nova | |
| 22:41:10 | sean-k-mooney | this is how we know which database to look in to delete the instace at the cell level | |
| 22:41:39 | atmark | https://paste.openstack.org/show/blXqFYFedaFGS9U7igPG/ | |
| 22:41:50 | atmark | cell0 | |
| 22:42:06 | atmark | it's pointing to nova_cell0 | |
| 22:42:24 | sean-k-mooney | ya so cell0 is where it should be if it fail | |
| 22:42:43 | sean-k-mooney | can you ceck cell 0 and see if its also in the instance table there | |
| 22:43:48 | atmark | yup it's here | |
| 22:44:05 | sean-k-mooney | ok so its in both nova and nova_cell0 | |
| 22:44:10 | sean-k-mooney | thats really strange | |
| 22:44:14 | atmark | correct | |
| 22:44:40 | sean-k-mooney | so the quickest way to fix this is just to delete that one record | |
| 22:44:50 | sean-k-mooney | but it shoudl only ever end up in one of those two dbs | |
| 22:45:07 | sean-k-mooney | any instnace that failed before it was schduled to a host should end up in nova_cell0 | |
| 22:45:14 | atmark | delete in both db? | |
| 22:45:38 | atmark | probably safe to delete in both right | |
| 22:45:44 | sean-k-mooney | you could try just removing it in nova | |
| 22:45:57 | sean-k-mooney | and then delete again but ya it should be safe to delete in both | |
| 22:46:27 | sean-k-mooney | i dont know of a failrue mode like this unless you change you cell mapping at some point | |
| 22:46:44 | sean-k-mooney | normally cell 0 get id 1 an cell_1 gets id 2 | |
| 22:47:05 | sean-k-mooney | although there is no significance to the speicic id | |
| 22:47:13 | atmark | iirc, we did played with mappings because something failed | |
| 22:47:40 | sean-k-mooney | ack so maybe this change at some point | |
| 22:48:07 | sean-k-mooney | i have never seen this specific issue before | |
| 22:48:23 | atmark | yep. anywany, this is for monday. don't wanna cause something bad | |
| 22:48:40 | atmark | thanks for the help | |
| 22:49:18 | atmark | we have 3 prod envs, only 1 env is exhibiting this issue | |
| 22:49:25 | sean-k-mooney | ack | |
| 22:49:33 | atmark | all same version | |
| 22:50:21 | sean-k-mooney | its my favorite way to install and manage openstack | |
| 22:50:34 | sean-k-mooney | im just sad that the zed release is not avlaible currently | |
| #openstack-nova - 2022-10-01 | |||
| 08:59:46 | opendevreview | Takashi Natsume proposed openstack/nova master: Add a hacking rule for the setDaemon method https://review.opendev.org/c/openstack/nova/+/854653 | |
| #openstack-nova - 2022-10-03 | |||
| 03:13:40 | abdi | OpenStack Nova Team. I filed the following bug against tempest: https://bugs.launchpad.net/tempest/+bug/1989232. However, it has not got a lot of attention from the tempest side. My ongoing investigation suggests nova may be the culprit. Could you review my last posting on the bug and comment? | |
| 08:06:37 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds check if resized to swap zero https://review.opendev.org/c/openstack/nova/+/857339 | |
| 09:32:12 | opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova master: compute: enhance compute evacuate instance to support target state https://review.opendev.org/c/openstack/nova/+/858383 | |
| 09:32:13 | opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova master: api: extend evacuate instance to support target state https://review.opendev.org/c/openstack/nova/+/858384 | |
| 09:45:16 | auniyal | Hi gibi | |
| 09:45:29 | auniyal | I have a doubt in this comment, can you please confirm, the changes | |
| 09:45:29 | auniyal | https://review.opendev.org/c/openstack/nova/+/791135/7 | |
| 09:46:13 | auniyal | last comment in this patch | |
| 09:55:19 | gibi | auniyal: what I ment there is that you have to restore the state of the instance object to point to the source host. And that needs not just the instance.host but also the instnace.node to be set properly. You can copy the instance current node at the start of the post_live_migration_at_destination call in a local variabla and then use that to restore instance.node in the finally block | |
| 10:10:28 | sean-k-mooney | gibi: no | |
| 10:10:48 | sean-k-mooney | gibi: post live migration is after we are allowed to rollback | |
| 10:10:54 | gibi | sorry | |
| 10:10:57 | gibi | I mixed source det | |
| 10:10:58 | gibi | dest | |
| 10:11:03 | sean-k-mooney | oh ok | |
| 10:11:21 | gibi | so it is not enough to set the instance.host but need to set the instance.node | |
| 10:11:24 | gibi | to the dest | |
| 10:11:27 | sean-k-mooney | yes | |
| 10:11:38 | gibi | but then auniyal you have to look up the dest node | |
| 10:12:21 | sean-k-mooney | well your running on the test at this point | |
| 10:12:26 | auniyal | isn't dest node is same as dest host ? or dest | |
| 10:12:31 | sean-k-mooney | so you just need to set it im not sure you need to look it up | |
| 10:12:41 | sean-k-mooney | auniyal: for libvirt yes | |
| 10:12:44 | sean-k-mooney | in general no | |
| 10:12:52 | sean-k-mooney | actully no | |
| 10:13:01 | sean-k-mooney | even for libvirt they can be different | |
| 10:13:19 | sean-k-mooney | although i think that owuld be an error | |
| 10:15:02 | sean-k-mooney | node_name = compute_node.hypervisor_hostname | |
| 10:15:28 | sean-k-mooney | where as instance.host = CONF.host | |
| 10:16:01 | auniyal | queston, what is diff between host and node, till now, I thought the instance.host is where compute service is running, (node is as general we say in any network tree/mapping (machine)) | |
| 10:16:02 | sean-k-mooney | by default those shoudl be the same but if you manually set [DEFAULT]/host | |
| 10:16:09 | sean-k-mooney | then they can be differnt | |
| 10:16:26 | sean-k-mooney | node is the name of the comptue node | |
| 10:16:33 | sean-k-mooney | host is the name of the compute service | |
| 10:16:58 | sean-k-mooney | for non clustered drivers like libvirt they are normally the same | |
| 10:17:52 | sean-k-mooney | for clustered dirvers like ironic where 1 compute service manages many baremetal servers | |
| 10:17:55 | sean-k-mooney | they are differnet | |
| 10:18:31 | sean-k-mooney | you can look up the name like this | |
| 10:18:33 | sean-k-mooney | https://github.com/openstack/nova/blob/1025c9879341d44db33c4cc501435364dd185a9e/nova/compute/manager.py#L9263-L9267 | |
| 10:19:58 | auniyal | ack | |
| 11:48:15 | opendevreview | Andre Aranha proposed openstack/nova master: Replace Centos 8 jobs for Centos 9 https://review.opendev.org/c/openstack/nova/+/858272 | |
| 11:49:47 | opendevreview | Andre Aranha proposed openstack/nova master: Remove the periodic Centos 8 job https://review.opendev.org/c/openstack/nova/+/858272 | |
| 12:02:36 | opendevreview | Andre Aranha proposed openstack/nova stable/yoga: Test setting the nova job to centos-9-stream https://review.opendev.org/c/openstack/nova/+/860087 | |
| 12:39:35 | opendevreview | Maksim Malchuk proposed openstack/nova stable/xena: Fix to implement 'pack' or 'spread' VM's NUMA cells https://review.opendev.org/c/openstack/nova/+/829804 | |
| 14:48:13 | atmark | Hello. Getting this error when trying to temporarily increase allocation_ratio in placement via CLI | |
| 14:48:23 | atmark | JSON does not validate: 'total' is a required property Failed validating 'required' in schema['properties']['inventories']['patternProperties']['^[A-Z0-9_]+$']: | |
| 14:48:53 | atmark | ./openstack resource provider inventory set c80259e6-67fb-47a0-b04f-364b2f2b2969 --resource MEMORY_MB:allocation_ratio=1.0 | |
| 16:14:10 | gibi | atmark: either define the total value in --resource too, or use --amend to instruct the client to only change the allocation_ratio | |