| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-28 | |||
| 16:58:47 | mdbooth | Sure. Would it be a useful thing? | |
| 16:58:52 | mriedem | idk | |
| 16:59:06 | mdbooth | For eg, in this case we've run gate-tempest-dsvm-lxc-ubuntu-xenial, and it failed, but I don't really care. | |
| 16:59:31 | mriedem | what would have been useful was if the hilton didn't f up charging me for last week so my internal expense reporting tool took over an hour to trick to make up for a separate .05 charge | |
| 16:59:51 | mriedem | recheck hilton | |
| 16:59:55 | mdbooth | lol | |
| 17:00:28 | mriedem | spotz: i get the token via: OS_TOKEN=$(openstack token issue -c id -f value) | |
| 17:00:36 | mdbooth | I'm pretty sure that the scope of expense reporting tools vastly exceeds anything we have to deal with :) | |
| 17:01:57 | spotz | mriedem: Still getting no json object | |
| 17:02:12 | mriedem | spotz: but a 200 or an error? | |
| 17:03:25 | spotz | mriedem: 200, I think it's me I suck at curl | |
| 17:03:54 | mriedem | and the creds you're using for that token are the same as what's in nova.conf on the compute? | |
| 17:04:44 | spotz | mriedem: Yeah I set up an rc and did an openstack token issue | |
| 17:05:04 | spotz | Or I could be doing things ass backwards | |
| 17:05:22 | mriedem | are you still seeing warnings in the nova-compute logs about not being able to connect to placement? | |
| 17:06:42 | spotz | mriedem: Nope | |
| 17:09:58 | mriedem | but there are no resource providers coming back... | |
| 17:10:03 | mriedem | something is janked there then | |
| 17:10:17 | mriedem | because the compute is running a periodic task every minute that will update it's info in the placement api | |
| 17:10:30 | mriedem | if it can't, you should be seeing a warning every 10 minutes or so in the compute logs | |
| 17:11:17 | spotz | nova-compute.log is now lines of 017-02-28 11:05:39.099 2618 INFO nova.compute.resource_tracker [req-8025f2c1-a990-4383-ab04-46e929ffce39 - - - - -] Final resource view: name=compute phys_ram=7983MB used_ram=512MB phys_disk=5GB used_disk=0GB total_vcpus=1 used_vcpus=0 pci_stats=[] | |
| 17:11:33 | mriedem | that's not it | |
| 17:11:34 | cfriesen | with the change to make live migration checks async in Newton, did horizon get updated to properly report any errors in the checks? | |
| 17:11:57 | mriedem | and if so, | |
| 17:11:57 | mriedem | cfriesen: it's a microversion so horizon would have to be opting in | |
| 17:11:58 | mriedem | they have to adjust for the behavior change | |
| 17:12:43 | openstackgerrit | John Griffith proposed openstack/nova master: Add attachment_uuid to BDM table https://review.openstack.org/438744 | |
| 17:13:59 | openstackgerrit | John Griffith proposed openstack/nova master: Add Cinder V3 Detach calls https://review.openstack.org/438750 | |
| 17:14:01 | openstackgerrit | STEW TY proposed openstack/nova master: Complete verification of servers-action-fixed-ip.inc https://review.openstack.org/338606 | |
| 17:14:16 | mriedem | sfinucan: mdbooth: you were asking why lyarwood was adding an attachment_id to the bdm table | |
| 17:14:17 | mriedem | https://review.openstack.org/#/c/373203/ | |
| 17:14:18 | mriedem | is why | |
| 17:14:24 | mriedem | you should read it since we need more eyes on it | |
| 17:15:35 | mriedem | dansmith: what do you think, should we shit or get off this pot? https://review.openstack.org/#/c/242602/ | |
| 17:15:47 | dansmith | mriedem: lol | |
| 17:16:01 | mriedem | we have no immediate need for it right now that i'm aware of | |
| 17:16:05 | mdbooth | mriedem: Coincidentally just reading that | |
| 17:16:15 | mriedem | but it's always something that comes up as "if we only had a uuid on the bdm" | |
| 17:16:16 | dansmith | mriedem: I don't care.. seems like a good thing, and people seem to occasionally need it | |
| 17:16:17 | dansmith | but yeah if no need then whatever | |
| 17:16:20 | mdbooth | lyarwood: You still need a commit message btw :P | |
| 17:16:23 | dansmith | I can abandon if it's bothering you | |
| 17:16:29 | mriedem | it's not bothering me | |
| 17:16:35 | lyarwood | mdbooth: meh, can't you read my mind? | |
| 17:16:35 | mriedem | but lyarwood is building on it | |
| 17:17:06 | mdbooth | lyarwood: Not until you push it to gerrit :) | |
| 17:17:11 | mriedem | lyarwood: consequently, what state is https://review.openstack.org/#/c/290793/ in now? | |
| 17:19:08 | lyarwood | mriedem: rotting on the floor, I had an open question around how to actually use the new UUIDs to avoid this iirc | |
| 17:19:41 | jgriffith | omg that change is everywhere :) | |
| 17:19:43 | mriedem | spotz: looking for the errors from here https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L51 | |
| 17:20:03 | mriedem | that will log the warning every 10 tries, so 10 minutes https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L43 | |
| 17:20:15 | mriedem | earlier you said you hit this https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L59 | |
| 17:20:28 | mriedem | spotz: can you share the nova.conf from your compute node? | |
| 17:21:03 | cfriesen | question about aborting live migration...where are users supposed to find the appropriate migration id? I don't see it returned in the response to the live-migrate request or instance-action. Are they supposed to parse the output of migration-list? | |
| 17:21:28 | mriedem | nova migration-list or something | |
| 17:21:54 | spotz | mriedem: Yep, its just the one built through the install guide, nothing prorietary let me get it for you | |
| 17:23:00 | mriedem | sdague: i still owe you that nickel, i totally forgot to pay up in person | |
| 17:23:35 | mriedem | cfriesen: https://docs.openstack.org/cli-reference/nova.html#nova-migration-list | |
| 17:23:47 | mriedem | that api currently filters so that only live migration records are returned | |
| 17:23:48 | sfinucan | tonyb: Just FYI, https://review.openstack.org/#/c/438510/ merged so the next pbr release should be coming shortly (I hope) | |
| 17:24:44 | mriedem | cfriesen: see https://review.openstack.org/#/c/407237/ | |
| 17:25:48 | spotz | mriedem: It's gthe latest comment but it decided it needed a BIG font for some reason | |
| 17:26:08 | mriedem | spotz: ? | |
| 17:26:35 | mriedem | comment where? | |
| 17:26:47 | spotz | mriedem: here new gist https://gist.github.com/spotz/4550ffcd9568d608bf6f483ef5feebab | |
| 17:26:55 | mriedem | efried: https://etherpad.openstack.org/p/pike-nova-priorities-tracking looks grand, thanks | |
| 17:27:05 | efried | mriedem Thanks, still working on it. | |
| 17:28:05 | mriedem | spotz: and a gist for the rc you used to make the curl request to the placement endpoint? | |
| 17:29:32 | openstackgerrit | STEW TY proposed openstack/nova master: Complete verification of servers-action-fixed-ip.inc https://review.openstack.org/338606 | |
| 17:30:29 | spotz | mriedem: https://gist.github.com/spotz/779bd1045c0422b6dd1cc6640dd11787 | |
| 17:31:34 | mriedem | that all looks fine | |
| 17:34:01 | spotz | mriedem: It's gotta be something stupid just not sure what | |
| 17:34:27 | spotz | I do appreciate all the help today mriedem and dansmith! | |
| 17:35:09 | mriedem | yeah i don't really understand it, if the creds are ok for you to make a curl request, then nova-compute should be reporting into the placement api | |
| 17:35:15 | mriedem | and the curl request you made should return something | |
| 17:35:46 | mriedem | and there are no errors or warnings in the nova-compute logs... | |
| 17:36:55 | spotz | mriedem: yeah it's not making much sense and the more I tinker I worry I'm losing track of what I changed. Between Brian and I testing each version of the doc we'll get it just not as quickly as I was hoping | |
| 17:37:12 | mriedem | do you see this debug message in the n-cpu logs? https://github.com/openstack/nova/blob/45cb83829e5b16dbacb7c71b7a49060c75cad986/nova/scheduler/client/report.py#L582 | |
| 17:37:57 | mriedem | e.g. http://logs.openstack.org/59/383859/19/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/4377f7d/logs/screen-n-cpu.txt.gz#_2017-02-28_14_35_05_210 | |
| 17:38:39 | spotz | mriedem: Nope, just 2017-02-28 11:37:29.369 2204 INFO nova.scheduler.host_manager [req-b549cd44-392e-4992-a88e-31f7180a3451 - - - - -] Successfully synced instances from host 'compute'. | |
| 17:39:15 | spotz | And one retry filter a while ago | |
| 17:39:20 | cfriesen | mriedem: thanks, I knew about that command. The inconvenience is that there's no way to list the currently active migration for a server, so the user has to list all the migrations, then find the currently active one for the server of interest. | |
| 17:40:08 | cfriesen | mriedem: why doesn't the live migration request return the migration ID? (or the "nova instance-action" command?) | |
| 17:40:24 | antwash | johnthetubaguy, sdague : hey there! I have something for you to look at https://review.openstack.org/#/c/439070/ -- marked as WIP until feedback is received. | |
| 17:40:34 | antwash | ^^ when yo have time | |
| 17:42:12 | spotz | mriedem: Thanks again for the help, need to afk | |
| 17:44:05 | mriedem | cfriesen: i don't think we have a migration id before we return from the live migration request | |
| 17:44:18 | mriedem | cfriesen: would have to look to confirm | |
| 17:44:29 | mriedem | the instance-action is not specific to live migration | |
| 17:44:34 | mriedem | so it wouldn't have a migration id in it | |
| 17:44:54 | mriedem | you'd have to extend instance actions if you wanted to store extra meta per action | |
| 17:47:03 | efried | mriedem Think I'm done; added announcement & link on meeting agenda. | |
| 17:47:17 | mriedem | cfriesen: so we should be pulling migrations per server /servers/{uuid}/migrations | |
| 17:48:20 | mriedem | cfriesen: would have to look at what nova migration-list does, because i thought it would take a server uuid | |
| 17:48:42 | mriedem | ah | |
| 17:48:43 | mriedem | https://docs.openstack.org/cli-reference/nova.html#nova-server-migration-list | |
| 17:48:53 | mriedem | cfriesen: ^ | |
| 17:49:01 | mriedem | that will return the list of live migrations for the given server | |
| 17:49:09 | mriedem | if you have a high enough api to provide that microversion | |
| 17:52:47 | openstackgerrit | Dan Smith proposed openstack/nova master: Target cell in conductor cold migrate https://review.openstack.org/438022 | |