| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-05-09 | |||
| 22:58:58 | redkrieg | 2019-05-09 18:57:34.505 12028 ERROR nova.scheduler.client.report [req-d7fa7b90-ada9-4296-8f2f-977b8e041dfc - - - - -] [req-ce0a1ceb-1e0f-486f-afe6-c1dc9ecdbad5] Failed to create resource provider record in placement API for UUID 5f5eecf0-9805-4b0c-892e-cddb70b6a4eb. Got 409: {"errors": [{"status": 409, "request_id": "req-ce0a1ceb-1e0f-486f-afe6-c1dc9ecdbad5", "detail": "There was a conflict when | |
| 22:59:04 | redkrieg | trying to complete your request.\n\n Conflicting resource provider name: nyc-slxc-1 already exists. ", "title": "Conflict"}]}. | |
| 23:04:33 | cdent | redkrieg: quick and dirty: find the resource provider that represent the older compute node (its uuid) and delete that from placement. you can do that with curl calls to placement, the osc-placement client, or looking into the database directly | |
| 23:12:35 | redkrieg | cdent: thanks, I found the entry and deleted it. looks like it registered correctly after. | |
| 23:13:15 | cdent | redkrieg: great. I think there's been some work or at least discussion on making that less of a problem, but quick and dirty seemed like the right way to go in this case | |
| 23:13:48 | redkrieg | yep, it's not a common scenario. I generally avoid reinstalling when I can. | |
| 23:16:37 | mriedem | the proper thing would have been (1) stop the nova-compute process on the host, (2) delete the compute service in the API which (depending on your release) should cleanup the provider in placement automatically | |
| 23:18:07 | mriedem | https://developer.openstack.org/api-ref/compute/?expanded=delete-compute-service-detail#delete-compute-service | |
| 23:19:20 | redkrieg | I actually tried to use the openstack-ansible-ops playbook to remove the host first, but I think it left a placement entry instead of properly cleaning it up. I'm just glad it's working now. thanks for looking! | |
| 23:22:01 | mriedem | hmm, i hope the ansible stuff isn't screwing things up now | |
| 23:22:16 | mriedem | which release are you using? | |
| 23:27:45 | Nick_A | we're on rocky | |
| 23:48:44 | mriedem | am i being blizzarded again? | |
| 23:49:30 | mriedem | anyway, DELETE /os-services/{service_id} should have cleaned that stuff up for you assuming the nova-compute service was not running (it will recreate records on a periodic task) | |
| 23:49:47 | mriedem | so i don't know what that ansible play does, but might be worth investigating, it should stop the service first and then delete it via the api | |
| 23:51:20 | artom | mriedem, you're seriously getting snow? | |
| 23:51:37 | mriedem | artom: no blizzard the company | |
| 23:51:57 | artom | They're a verb now? | |
| 23:52:15 | mriedem | yes | |
| 23:52:26 | mriedem | "it's a blizzard of blizzard operators" | |
| 23:52:29 | mriedem | blizzarded | |
| 23:53:45 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle inactive port binding in _update_port_binding_for_instance https://review.opendev.org/653506 | |
| 23:54:16 | mriedem | fwiw i have no idea who redkrieg and Nick_A work for, i've just noticed the blizzard gang runs in packs | |
| 23:59:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle inactive port binding in _update_port_binding_for_instance https://review.opendev.org/653506 | |
| 23:59:35 | mriedem | tobberydberg: i know your guys are interested in this ^ | |
| 23:59:42 | mriedem | might be a bear to backport but that should have it passing | |
| #openstack-nova - 2019-05-10 | |||
| 00:00:01 | artom | Wait, Blizzard genuinely run OpenStack? | |
| 00:00:05 | mriedem | artom: yes | |
| 00:00:09 | artom | Explains the WoW server queues | |
| 00:00:14 | artom | </joke from 15 years ago> | |
| 00:00:29 | mriedem | https://www.openstack.org/summit/denver-2019/summit-schedule/events/23379/how-blizzard-entertainment-uses-autoscaling-with-overwatch | |
| 00:00:39 | mriedem | eandersson will cut you | |
| 00:01:24 | artom | I probably deserve that. | |
| 00:03:36 | artom | Oh, Designate. | |
| 00:03:47 | artom | My intro to OpenStack was on that, as an intern at eNovance | |
| 00:03:53 | mriedem | it's great that when post live migration fails we just, you know, act like it was good http://logs.openstack.org/64/649464/3/check/nova-live-migration/a0fdcc9/logs/screen-n-cpu.txt.gz?level=TRACE#_May_09_23_00_39_272825 | |
| 00:04:11 | mriedem | https://review.opendev.org/#/c/649464/3/nova/compute/manager.py@6889 | |
| 00:04:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: Test theory behind bug 1822884 https://review.opendev.org/649464 | |
| 00:04:28 | openstack | bug 1822884 in OpenStack Compute (nova) "live migration fails due to port binding duplicate key entry in post_live_migrate" [Medium,In progress] https://launchpad.net/bugs/1822884 - Assigned to sean mooney (sean-k-mooney) | |
| 00:04:57 | artom | *snerk* my goat is still in there: https://github.com/openstack/designate/blob/master/designate/tests/resources/zonefiles/malformed_example.com.zone | |
| 00:05:55 | mriedem | i was supposed to have started making dinner and hour ago so i'm out of here | |
| 00:05:57 | mriedem | o/ | |
| 00:06:01 | mriedem | *an | |
| 00:06:20 | artom | Good man | |
| 01:22:33 | openstackgerrit | Merged openstack/nova stable/queens: Use migration_status during volume migrating and retyping https://review.opendev.org/657579 | |
| 01:49:35 | bhagyashris_ | Artom: Hi | |
| 01:49:54 | bhagyashris_ | artom: Hi | |
| 02:37:20 | openstackgerrit | Merged openstack/nova master: Remove macs kwarg from allocate_for_instance https://review.opendev.org/652749 | |
| 06:50:40 | openstackgerrit | Merged openstack/python-novaclient master: Use SHA256 instead of MD5 in completion cache https://review.opendev.org/658181 | |
| 07:34:52 | tobberydberg | Definitely mriedem - thanks! | |
| 08:58:44 | openstackgerrit | zhufl proposed openstack/nova master: Fix broken url links https://review.opendev.org/658312 | |
| 09:09:02 | openstackgerrit | zhufl proposed openstack/nova master: Fix broken url links https://review.opendev.org/658312 | |
| 09:57:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: reorder conditions in _heal_allocations_for_instance https://review.opendev.org/655458 | |
| 09:57:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: pull out functions from _heal_allocations_for_instance https://review.opendev.org/655457 | |
| 09:57:39 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: pull out put_allocation call from _heal_* https://review.opendev.org/655459 | |
| 09:57:39 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Prepare _heal_allocations_for_instance for nested allocations https://review.opendev.org/637954 | |
| 09:57:40 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: nova-manage: heal port allocations https://review.opendev.org/637955 | |
| 11:17:13 | kashyap | ildikov: Hi, when you're about, do you have a minute to re-read this error message you've added in _set_multiattach_support()? | |
| 11:17:23 | kashyap | LOG.debug('Volume multiattach is not supported based on current ' | |
| 11:17:26 | kashyap | 'versions of QEMU and libvirt. QEMU must be less than ' | |
| 11:17:29 | kashyap | '2.10 or libvirt must be greater than or equal to 3.10.') | |
| 11:17:38 | kashyap | I think the "or" there should be "and", isn't it? | |
| 11:17:43 | kashyap | Otherwise, it doesn't quite resolve. | |
| 11:39:23 | ildikov | kashyap: I think the 'or' was intentional | |
| 11:41:02 | kashyap | ildikov: Hmm, in that case, I've seen someone from HP report a bug with the above error: where they have 2.10.1 QEMU and libvirt as 3.6.0. | |
| 11:41:58 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Server move operations with ports having resource request https://review.opendev.org/652608 | |
| 11:41:59 | ildikov | kashyap: yeah, that should lead to an error | |
| 11:42:20 | ildikov | kashyap: as neither the QEMU version is lower nor the libvirt version is higher than suggested | |
| 11:42:37 | ildikov | kashyap: it's due to how they handle caching or smth | |
| 11:43:00 | ildikov | as we needed to turn that off for volumes being attached to multiple instances | |
| 11:43:31 | kashyap | Yeah, I was reading the related code in the source. Let me go recheck | |
| 11:43:32 | ildikov | and there were some changes in how that's handled with the flags, etc | |
| 11:43:49 | ildikov | in QEMU and libvirt | |
| 11:43:56 | kashyap | (Hmm, but that error message is somewhat confusing.) | |
| 11:44:07 | kashyap | ildikov: Right, I saw the bugzilla linked which describes the mess in libvirt | |
| 11:45:18 | kashyap | ildikov: Oh, bad me -- I mistook 3.6.0 as higher than 3.10.0. Can I be blinder than that... | |
| 11:45:30 | ildikov | well, the error message only says that if the QEMU version is low enough than nothing else matters and the same thing for the libvirt one if it's high enough | |
| 11:45:51 | ildikov | kashyap: happens to everyone :) | |
| 11:49:41 | kashyap | ildikov: Thanks for the clarification. Just someone reading it out loud is just what you need sometimes :-) | |
| 11:50:27 | ildikov | kashyap: np, always happy to help :) | |
| 12:12:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove unused param from _fill_provider_mapping https://review.opendev.org/655107 | |
| 12:12:11 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth https://review.opendev.org/655109 | |
| 12:12:11 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move _fill_provider_mapping to the scheduler_utils https://review.opendev.org/655108 | |
| 12:12:12 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pass network API to the conducor's MigrationTask https://review.opendev.org/655111 | |
| 12:12:12 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: allow getting resource request of every bound ports of an instance https://review.opendev.org/655110 | |
| 12:12:13 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: re-calculate provider mapping during migration https://review.opendev.org/655112 | |
| 12:12:13 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add request_spec to server move RPC calls https://review.opendev.org/655721 | |
| 12:12:14 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Extend NeutronFixture to handle migrations https://review.opendev.org/655114 | |
| 12:12:14 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: update allocation in binding profile during migrate https://review.opendev.org/656422 | |
| 12:12:15 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: func test for migrate server with ports having resource request https://review.opendev.org/655113 | |
| 12:47:05 | artom | Does placement understand AZs? Or I guess a better question would be, if we make the right request to placement (whatever that request may look like), are we guaranteed to get at least some hosts in the correct AZ? | |
| 12:48:59 | amodi | artom: yes, they do, https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#availability-zones-with-placement | |
| 12:49:33 | artom | francoisp, ^^ | |
| 12:49:56 | francoisp | yes thanks amodi, artom | |
| 12:51:12 | artom | mdbooth, by the way, are we going to start including francoisp in our triage call? | |
| 12:51:48 | artom | Doh, wrong channel, answer downstream plz :) | |
| 13:10:49 | openstackgerrit | Merged openstack/nova master: Expose Hyper-V supported image types https://review.opendev.org/655137 | |
| 13:37:01 | openstackgerrit | Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362 | |
| 13:46:53 | openstackgerrit | Merged openstack/nova master: Make libvirt expose supported image types https://review.opendev.org/653454 | |
| 14:50:37 | efried | tssurya: Quick look at https://review.opendev.org/#/c/648662/ if you please, sanity check my comment. | |