| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-28 | |||
| 17:16:29 | mriedem | it's not bothering me | |
| 17:16:35 | lyarwood | mdbooth: meh, can't you read my mind? | |
| 17:16:35 | mriedem | but lyarwood is building on it | |
| 17:17:06 | mdbooth | lyarwood: Not until you push it to gerrit :) | |
| 17:17:11 | mriedem | lyarwood: consequently, what state is https://review.openstack.org/#/c/290793/ in now? | |
| 17:19:08 | lyarwood | mriedem: rotting on the floor, I had an open question around how to actually use the new UUIDs to avoid this iirc | |
| 17:19:41 | jgriffith | omg that change is everywhere :) | |
| 17:19:43 | mriedem | spotz: looking for the errors from here https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L51 | |
| 17:20:03 | mriedem | that will log the warning every 10 tries, so 10 minutes https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L43 | |
| 17:20:15 | mriedem | earlier you said you hit this https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L59 | |
| 17:20:28 | mriedem | spotz: can you share the nova.conf from your compute node? | |
| 17:21:03 | cfriesen | question about aborting live migration...where are users supposed to find the appropriate migration id? I don't see it returned in the response to the live-migrate request or instance-action. Are they supposed to parse the output of migration-list? | |
| 17:21:28 | mriedem | nova migration-list or something | |
| 17:21:54 | spotz | mriedem: Yep, its just the one built through the install guide, nothing prorietary let me get it for you | |
| 17:23:00 | mriedem | sdague: i still owe you that nickel, i totally forgot to pay up in person | |
| 17:23:35 | mriedem | cfriesen: https://docs.openstack.org/cli-reference/nova.html#nova-migration-list | |
| 17:23:47 | mriedem | that api currently filters so that only live migration records are returned | |
| 17:23:48 | sfinucan | tonyb: Just FYI, https://review.openstack.org/#/c/438510/ merged so the next pbr release should be coming shortly (I hope) | |
| 17:24:44 | mriedem | cfriesen: see https://review.openstack.org/#/c/407237/ | |
| 17:25:48 | spotz | mriedem: It's gthe latest comment but it decided it needed a BIG font for some reason | |
| 17:26:08 | mriedem | spotz: ? | |
| 17:26:35 | mriedem | comment where? | |
| 17:26:47 | spotz | mriedem: here new gist https://gist.github.com/spotz/4550ffcd9568d608bf6f483ef5feebab | |
| 17:26:55 | mriedem | efried: https://etherpad.openstack.org/p/pike-nova-priorities-tracking looks grand, thanks | |
| 17:27:05 | efried | mriedem Thanks, still working on it. | |
| 17:28:05 | mriedem | spotz: and a gist for the rc you used to make the curl request to the placement endpoint? | |
| 17:29:32 | openstackgerrit | STEW TY proposed openstack/nova master: Complete verification of servers-action-fixed-ip.inc https://review.openstack.org/338606 | |
| 17:30:29 | spotz | mriedem: https://gist.github.com/spotz/779bd1045c0422b6dd1cc6640dd11787 | |
| 17:31:34 | mriedem | that all looks fine | |
| 17:34:01 | spotz | mriedem: It's gotta be something stupid just not sure what | |
| 17:34:27 | spotz | I do appreciate all the help today mriedem and dansmith! | |
| 17:35:09 | mriedem | yeah i don't really understand it, if the creds are ok for you to make a curl request, then nova-compute should be reporting into the placement api | |
| 17:35:15 | mriedem | and the curl request you made should return something | |
| 17:35:46 | mriedem | and there are no errors or warnings in the nova-compute logs... | |
| 17:36:55 | spotz | mriedem: yeah it's not making much sense and the more I tinker I worry I'm losing track of what I changed. Between Brian and I testing each version of the doc we'll get it just not as quickly as I was hoping | |
| 17:37:12 | mriedem | do you see this debug message in the n-cpu logs? https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L582 | |
| 17:37:57 | mriedem | e.g. http://logs.openstack.org/59/383859/19/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/4377f7d/logs/screen-n-cpu.txt.gz#_2017-02-28_14_35_05_210 | |
| 17:38:39 | spotz | mriedem: Nope, just 2017-02-28 11:37:29.369 2204 INFO nova.scheduler.host_manager [req-b549cd44-392e-4992-a88e-31f7180a3451 - - - - -] Successfully synced instances from host 'compute'. | |
| 17:39:15 | spotz | And one retry filter a while ago | |
| 17:39:20 | cfriesen | mriedem: thanks, I knew about that command. The inconvenience is that there's no way to list the currently active migration for a server, so the user has to list all the migrations, then find the currently active one for the server of interest. | |
| 17:40:08 | cfriesen | mriedem: why doesn't the live migration request return the migration ID? (or the "nova instance-action" command?) | |
| 17:40:24 | antwash | johnthetubaguy, sdague : hey there! I have something for you to look at https://review.openstack.org/#/c/439070/ -- marked as WIP until feedback is received. | |
| 17:40:34 | antwash | ^^ when yo have time | |
| 17:42:12 | spotz | mriedem: Thanks again for the help, need to afk | |
| 17:44:05 | mriedem | cfriesen: i don't think we have a migration id before we return from the live migration request | |
| 17:44:18 | mriedem | cfriesen: would have to look to confirm | |
| 17:44:29 | mriedem | the instance-action is not specific to live migration | |
| 17:44:34 | mriedem | so it wouldn't have a migration id in it | |
| 17:44:54 | mriedem | you'd have to extend instance actions if you wanted to store extra meta per action | |
| 17:47:03 | efried | mriedem Think I'm done; added announcement & link on meeting agenda. | |
| 17:47:17 | mriedem | cfriesen: so we should be pulling migrations per server /servers/{uuid}/migrations | |
| 17:48:20 | mriedem | cfriesen: would have to look at what nova migration-list does, because i thought it would take a server uuid | |
| 17:48:42 | mriedem | ah | |
| 17:48:43 | mriedem | https://docs.openstack.org/cli-reference/nova.html#nova-server-migration-list | |
| 17:48:53 | mriedem | cfriesen: ^ | |
| 17:49:01 | mriedem | that will return the list of live migrations for the given server | |
| 17:49:09 | mriedem | if you have a high enough api to provide that microversion | |
| 17:52:47 | openstackgerrit | Dan Smith proposed openstack/nova master: Target cell in conductor cold migrate https://review.openstack.org/438022 | |
| 17:52:48 | openstackgerrit | Dan Smith proposed openstack/nova master: Add cell field to Destination object https://review.openstack.org/438024 | |
| 17:52:48 | openstackgerrit | Dan Smith proposed openstack/nova master: Change MQ targeting to honor only what is in the context https://review.openstack.org/438023 | |
| 17:52:49 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Make conductor ask scheduler to limit migrates to same cell https://review.openstack.org/438025 | |
| 17:53:54 | dansmith | no whammies no whammies no whammies | |
| 18:00:33 | cfriesen | mriedem: okay, server-migration-list is useful. still requires parsing though. would it maybe make sense to have a "--active" flag on that to return just the in-progress migration for that server? | |
| 18:02:08 | cfriesen | or do we just make client code sort the response by ID and pick the latest to try to get the active one? | |
| 18:03:20 | mriedem | cfriesen: it might be nice to add a status filter query parameter to the /servers/id/migrations REST API | |
| 18:03:27 | mriedem | we could validate that with json schema now | |
| 18:04:37 | mriedem | cfriesen: but it does look like you could filter client side | |
| 18:04:40 | mriedem | we print out the migration status | |
| 18:07:57 | cfriesen | mriedem: hmm...in newton at least it appears that "server-migration-list" doesn't show completed migrations | |
| 18:10:36 | mriedem | cfriesen: that's because it's calling objects.MigrationList.get_in_progress_by_instance | |
| 18:11:01 | mriedem | where in-progress is one of | |
| 18:11:02 | mriedem | 'post-migrating'] | |
| 18:11:02 | mriedem | 'running', | |
| 18:11:02 | mriedem | 'queued', 'preparing', | |
| 18:11:12 | mriedem | why we only return in progress migrations, idk | |
| 18:11:22 | mriedem | i think because that api was added for being able to cancel a live migration | |
| 18:11:28 | mriedem | which you can only do if it's running | |
| 18:11:35 | mriedem | tdurakov might remember more | |
| 18:12:09 | mriedem | cfriesen: so before you ask "why only in progress live migrations?" i think that's the original idea of why, but "can we return all, not just in progress?" - i'd think sure | |
| 18:13:00 | mriedem | i believe the os-migrations API returns all migrations | |
| 18:13:40 | mriedem | imo it's a pretty clunky gd API and is tailor made just for canceling a live migration | |
| 18:13:48 | mriedem | which seemed to be too short sighted | |
| 18:14:34 | cfriesen | mriedem: hmm...the API docs do specify that it's in-progress live migrations, but the CLI docs don't specify that it's only for in-progress migrations. | |
| 18:15:14 | tdurakov | It was done to return adress live-mig abort functionality | |
| 18:17:28 | cfriesen | tdurakov: okay, thanks. I agree it's a bit clunky, but it does at least let you get the info needed to abort. | |
| 18:18:44 | cfriesen | mriedem: tdurakov: is it just me that thinks "nova server-migration-list" is a wierd command for something that should only ever have a single result? | |
| 18:21:09 | mriedem | cfriesen: i believe i wrote the api ref docs because they didn't exist | |
| 18:21:17 | mriedem | and i was sorting this all out about 5 months ago | |
| 18:21:26 | tdurakov | Migration list is awful... | |
| 18:23:37 | mriedem | cfriesen: you could push a patch to update the cli help | |
| 18:23:42 | mriedem | to make it more clear | |
| 19:19:07 | openstackgerrit | Monty Taylor proposed openstack/nova master: Be a little more explicit about checking ok status https://review.openstack.org/439121 | |
| 19:19:07 | openstackgerrit | Monty Taylor proposed openstack/nova master: WIP Clean up and consolidate client creation https://review.openstack.org/438512 | |
| 19:32:00 | openstackgerrit | Sujitha proposed openstack/nova master: Remove mox from nova/tests/unit/db/test_db_api.py https://review.openstack.org/434557 | |
| 19:45:57 | tdurakov | mriedem: I wonder what if we just merge these 2 comands from the client side, I mean ,nova migration-list and nove server-migration-list | |
| 19:47:31 | mriedem | tdurakov: as in like nova migration-list --server <optional server id> which toggles the behavior, i.e. which REST API to call? | |
| 19:48:04 | tdurakov | afair there is no --server param in nova migration list | |
| 19:48:10 | mriedem | there isn't | |
| 19:48:14 | mriedem | but that's how i'd unify the CLIs | |
| 19:48:20 | tdurakov | yeah | |