| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-09-20 | |||
| 12:11:21 | kevko_home | tssurya: and what do you think about my above thinkings ? :/ | |
| 12:12:19 | tssurya | looking more into the code.. | |
| 12:14:29 | tssurya | kevko_home: okay so you basically want the user to be warned if placement_database.connection is not set ? | |
| 12:14:40 | tssurya | and nova_api.databaseconnection is also not set ? | |
| 12:15:01 | kevko_home | tssurya: yes, something like that ..let me send you diff ..w8 a minute | |
| 12:16:36 | tssurya | well I guess the Rocky documentation deployment (https://docs.openstack.org/nova/rocky/user/placement.html#deployment) states api_db sync command is used for both syncing and if you set placement_db conn it will be used and if you don't set that, it is expected that you set the api_db conn. | |
| 12:17:10 | tssurya | I understand your concern though, and the need for a check seems ok to me, | |
| 12:19:32 | kevko_home | tssurya: something like this http://paste.openstack.org/show/730414/ | |
| 12:19:41 | kevko_home | tssurya: ok, i will send a review then | |
| 12:22:21 | gibi | bauzas: sorry I was out grabbing food. The fault is that nova does not consider allocations agains the child RPs during the move | |
| 12:22:36 | tssurya | kevko_home: ack | |
| 12:33:32 | Kevin_Zheng | tssurya Hi | |
| 12:33:41 | Kevin_Zheng | does this bothers CERN? | |
| 12:33:43 | Kevin_Zheng | https://launchpad.net/bugs/1793423 | |
| 12:33:43 | openstack | Launchpad bug 1793423 in OpenStack Compute (nova) "Should not read config options while update cells" [Undecided,In progress] - Assigned to Zhenyu Zheng (zhengzhenyu) | |
| 12:33:58 | Kevin_Zheng | or it is made this way | |
| 12:36:27 | tssurya | Kevin_Zheng: we don't use the update_cell command command very often, but what you have reported has always bugged me :) | |
| 12:36:52 | Kevin_Zheng | Ha | |
| 12:36:56 | tssurya | meaning the update_cell command is used for a lot of other stuff than just updating the DB connection and transport_url | |
| 12:37:19 | tssurya | so it is not fair that these two always have to be passed to ensure they don't get over written by the config options | |
| 12:39:52 | tssurya | Kevin_Zheng: although I don't understand your point 2: where you say you can't use nova_ | |
| 12:39:58 | tssurya | nova_cell1.conf | |
| 12:40:25 | tssurya | because you need api_database connection, | |
| 12:40:39 | tssurya | for update_cell ? | |
| 12:40:50 | Kevin_Zheng | sorry disconnected | |
| 12:41:28 | tssurya | np, did you get the messages ^ ? | |
| 12:41:37 | Kevin_Zheng | yeah, I saw them | |
| 12:41:40 | Kevin_Zheng | reading | |
| 12:41:53 | Kevin_Zheng | yeas | |
| 12:41:55 | Kevin_Zheng | yeah | |
| 12:42:04 | Kevin_Zheng | we have to update the records in api_db | |
| 12:42:27 | tssurya | ah okay | |
| 12:42:34 | tssurya | yea sorry, got confused | |
| 12:43:10 | tssurya | but now that you mention "up_calls" we have the same problem for nova-manage db archive_deleted_rows | |
| 12:43:34 | Kevin_Zheng | hmm | |
| 12:52:36 | openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Only update cells' transport_url and database_connection when needed. https://review.openstack.org/603998 | |
| 12:52:54 | openstackgerrit | huanhongda proposed openstack/nova master: Fix instance delete stuck in deleting task_state https://review.openstack.org/598084 | |
| 13:04:16 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/pike: libvirt: Reduce calls to qemu-img during update_available_resource https://review.openstack.org/604039 | |
| 13:16:42 | mnaser | there isn't a way to delete a compute node from the db, is there? | |
| 13:16:53 | bauzas | mnaser: by SQL goos ? | |
| 13:16:57 | mnaser | well, without sql | |
| 13:17:07 | bauzas | ah | |
| 13:17:10 | mnaser | nova service-delete deletes the service but the compute node record is still there | |
| 13:17:23 | mnaser | and scheduler complains constantly about 'No compute service record found for host <foo>' | |
| 13:17:36 | bauzas | that's a good point | |
| 13:17:49 | bauzas | compute_node records are automatically populated by the resource tracker | |
| 13:17:53 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use provider tree in virt FakeDriver https://review.openstack.org/604083 | |
| 13:17:53 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Run ServerMovingTests with nested resources https://review.openstack.org/604084 | |
| 13:18:10 | gibi | bauzas, tetsuro ^^ | |
| 13:18:13 | mnaser | maybe that error should check if there is an existing service, or something | |
| 13:18:21 | bauzas | mnaser: I thought we were checking this in the RT code | |
| 13:18:41 | mnaser | but this is on the nova-scheduler code, i think RT code sits in nova-compute ? | |
| 13:18:49 | bauzas | gibi: all cool, been frankly diverted by working on the reshape patch | |
| 13:18:51 | bauzas | (and by meetings) | |
| 13:19:06 | bauzas | mnaser: yup, run by nova-compute service | |
| 13:19:22 | bauzas | I'm just saying we populate the entries there | |
| 13:19:41 | mnaser | yeah but in this case it's taking a hypervisor out of service forever | |
| 13:19:51 | mnaser | so turn it off, nova service-delete <foo> and that error doesnt go ahway | |
| 13:19:54 | mnaser | s/ahway/away/ | |
| 13:20:12 | lbragstad | johnthetubaguy gmann nice work on https://review.openstack.org/#/c/547850/ | |
| 13:20:13 | bauzas | yup, I see the problem | |
| 13:20:13 | mnaser | https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L716-L720 | |
| 13:20:24 | mnaser | maybe we should clean up there (or delete compute record when a nova-compute binary is deleted) | |
| 13:21:05 | bauzas | mnaser: here is where we create the entry in the DB https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L587 | |
| 13:21:22 | bauzas | mnaser: we had a FK before | |
| 13:21:31 | bauzas | between service and CN | |
| 13:21:41 | mnaser | but i guess with the os-services api addition and making it managable by api | |
| 13:21:43 | mnaser | that might have disappeared | |
| 13:21:53 | mnaser | or some other historic reason you might know more than me :) | |
| 13:22:18 | mnaser | oh you know what | |
| 13:22:32 | mnaser | FK might not do anything anyways, because we just change deleted field | |
| 13:22:42 | mnaser | so for all that sql cares, FK is still satisified | |
| 13:23:06 | bauzas | well, actually that's a good point | |
| 13:23:10 | bauzas | we shadow the services table | |
| 13:23:22 | bauzas | so a CASCADE DELETE isn't like super important | |
| 13:24:31 | bauzas | BBIAB | |
| 13:26:38 | mnaser | filed a bug: https://bugs.launchpad.net/nova/+bug/1793533 -- i can take care of pushing the code to fix it if i can get some guidance on the best approach | |
| 13:26:38 | openstack | Launchpad bug 1793533 in OpenStack Compute (nova) "Deleting a service with nova-compute binary doesn't remove compute node" [Undecided,New] | |
| 13:31:00 | mriedem | mnaser: deleting the compute service should also delete the compute node record in the db | |
| 13:31:08 | mnaser | o | |
| 13:31:10 | mriedem | mnaser: the problem is you need to also have nova-compute for that host stopped | |
| 13:31:16 | mriedem | otherwise the RT will re-create the compute node record | |
| 13:31:33 | mriedem | https://developer.openstack.org/api-ref/compute/#delete-compute-service | |
| 13:32:20 | mnaser | i'm pretty sure it was stopped when i did this (actually, the hypervisor was rebooted with a new hostname so i just deleted the old one) | |
| 13:32:48 | mnaser | let me see the created_at time for the compute node record if there is one | |
| 13:34:14 | mriedem | https://github.com/openstack/nova/blob/d87852ae6a1987b6faa3cb5851f9758b47ef4636/nova/db/sqlalchemy/api.py#L404 | |
| 13:34:32 | mriedem | ^ is the DB API deleting the service and compute node record when you delete the service via DELETE /os-services/{service_id} | |
| 13:34:38 | mnaser | mriedem: created_at for compute node record is '2018-04-13 00:21:53', created_at for service (that was deleted) is '2018-04-13 00:21:53' | |
| 13:35:51 | mnaser | oo | |
| 13:35:58 | mriedem | do the host values align? | |
| 13:35:58 | mnaser | deleted=16 on compute node | |
| 13:36:04 | mnaser | so it was soft deleted | |
| 13:36:50 | mnaser | yes, they do, it was actually soft deleted | |
| 13:37:00 | mnaser | could it be that loop doesn't grab non-soft deleted compute nodes? | |
| 13:37:18 | mnaser | that would be the only logical thing happening | |
| 13:37:47 | mnaser | OR | |
| 13:37:51 | mriedem | by default that query is only going to look up non-soft-deleted cmpute node records that match the query | |
| 13:37:51 | mnaser | it looks like it uses _get_computes_for_cells ? | |
| 13:38:00 | mnaser | so maybe it is looking in nova_api compute node lists? | |
| 13:38:06 | mriedem | which loop are you talking about? | |