| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-09-20 | |||
| 16:17:22 | mnaser | yeah, i mean i can do some checking if you want me to do that before deeleting | |
| 16:18:08 | mriedem | https://review.openstack.org/#/q/If507e23f0b7e5fa417041c3870d77786498f741d | |
| 16:18:23 | mriedem | https://review.openstack.org/#/c/580498/ isn't even released yet on pike | |
| 16:18:47 | dansmith | sheesh | |
| 16:19:06 | mriedem | merged july 9... | |
| 16:19:42 | mriedem | mnaser: so that 108 and 8 orphaned things are pretty safe to cleanup i think | |
| 16:20:08 | mriedem | the 6 incorrect will likely require a bit of investigation, but could just see where is the instance now and compare to what it's allocations are | |
| 16:24:30 | jroll | jaypipes-ooo: way late, but I just found out we also have a john snow in our org (cc stephenfin) | |
| 16:37:33 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Fix upper-constraints link in tox file https://review.openstack.org/604158 | |
| 16:39:07 | larsks | mnaser: that has been useful twice so far :) | |
| 16:39:21 | mnaser | larsks: meanwhile i just spent most of the past few hours doing the same thing | |
| 16:39:36 | mnaser | but in a different approach that does list all servers + all tenants because that would burn down some oof our deployments | |
| 16:47:58 | mriedem | dansmith: you know how we added a microversion to make live migration an async rpc cast from api to conductor? | |
| 16:48:07 | mriedem | i'm kind of wondering if we should do something similar for resize | |
| 16:48:13 | mnaser | mriedem: hacked some more, http://paste.openstack.org/show/730450/ generates http://paste.openstack.org/show/730460/ | |
| 16:48:25 | mnaser | ill test it out a bit more and try to see if i can add it to nova-manage | |
| 16:48:31 | mriedem | with resize we only finally return the api response once conductor casts to prep_resize on compute | |
| 16:48:35 | dansmith | mriedem: is resize not async already? | |
| 16:48:39 | mriedem | nope | |
| 16:48:40 | openstackgerrit | Merged openstack/nova master: Merge extended_volumes extension response into server view builder https://review.openstack.org/596285 | |
| 16:48:40 | dansmith | because it could take a long ass time | |
| 16:48:43 | mriedem | well, | |
| 16:48:47 | openstackgerrit | Merged openstack/nova stable/rocky: Update RequestSpec.flavor on resize_revert https://review.openstack.org/602403 | |
| 16:48:49 | mriedem | it's async once we pick a compute and cast to it | |
| 16:48:57 | dansmith | ah, okay | |
| 16:49:12 | mriedem | the live migration task before that async microversion would do the reschedules and such within conductor | |
| 16:49:16 | mriedem | which could timeout the api response | |
| 16:49:57 | mriedem | for cross-cell resize, i'm considering doing similar to the pre-migration checks *from* conductor and going back to the scheduler well for more hosts if the one selected doesn't pass the pre-check | |
| 16:50:06 | mriedem | *pre-live migration checks | |
| 16:50:34 | mriedem | likely a todo to throw on the pile for my patch at this point | |
| 16:50:39 | dansmith | ack | |
| 16:50:51 | mriedem | i have officially started losing sleep over this | |
| 16:51:16 | mriedem | need to contract out some hot alaski action | |
| 16:54:29 | dansmith | heh | |
| 16:56:59 | mnaser | i dont have time right now to get this audit code into nova-manage so i'll leave it for anyone to hack on later (till iget the time) | |
| 16:57:22 | mriedem | mnaser: can you report a bug and throw it in there so we don't lose track of it? | |
| 16:57:33 | mnaser | mriedem: fair enough | |
| 16:59:25 | mnaser | https://bugs.launchpad.net/nova/+bug/1793569 | |
| 16:59:25 | openstack | Launchpad bug 1793569 in OpenStack Compute (nova) "Add placement audit commands" [Undecided,New] | |
| 17:05:52 | mriedem | thanks | |
| 17:13:40 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion bump for handling-down-cell https://review.openstack.org/591657 | |
| 17:19:36 | lucidguy | if I curl the placment API for an allocation (instance) its | |
| 17:19:36 | lucidguy | responding it exists on the wrong hypervisor ID. Openstack | |
| 17:19:39 | lucidguy | calls show it in the right place. How can I edit this, can't | |
| 17:19:42 | lucidguy | figure out the PUT equivalent syntax. Tempted to delete but | |
| 17:19:45 | lucidguy | that sounds very wrong | |
| 17:19:48 | lucidguy | Ouch, sorry about that | |
| 17:20:56 | lucidguy | if I curl the placment API for an allocation (instance) its responding it exists on the wrong hypervisor ID. Openstack calls show it in the right place. How can I edit this, can't figure out the PUT equivalent syntax. Tempted to delete but that sounds very wrong. | |
| 17:21:03 | lucidguy | That's better | |
| 17:25:04 | dansmith | lucidguy: you should use osc-placement to fix it | |
| 17:25:54 | lucidguy | osc-placement.. hmm, never heard of it. | |
| 17:27:09 | lucidguy | How does it work. | |
| 17:27:43 | dansmith | https://docs.openstack.org/osc-placement/latest/index.html | |
| 17:27:58 | dansmith | lucidguy: also check out https://github.com/larsks/os-placement-tools | |
| 17:28:18 | larsks | dansmith: hah, I just pasted that to him in #openstack :) | |
| 17:28:26 | lucidguy | Sounds like something new and time consuming to learn. | |
| 17:29:48 | lucidguy | I would pay for a quick solution.. :) | |
| 17:30:12 | larsks | lucidguy: I haven't been following the conversation, but that script on github may be a quick solution. | |
| 17:30:30 | dansmith | larsks: good, because that's the right place to discuss :) | |
| 17:30:37 | larsks | Yeah | |
| 17:32:24 | lucidguy | I don't think the orginization would like a random script hitting the production enviroment. | |
| 17:32:38 | lucidguy | I'm sure it's fine, but if things were to go wrong, I would be screwed | |
| 17:32:52 | dansmith | so wat | |
| 17:32:55 | dansmith | *wait | |
| 17:33:04 | dansmith | you wanted a quick solution instead of learning a tool, | |
| 17:33:12 | dansmith | and were just going to curl delete an instance's resource allocation, | |
| 17:33:33 | dansmith | but a "random script" that was written by people that know openstack and have tested it in production is not okay? | |
| 17:35:12 | lucidguy | Again, I'm sure it's safe. But my knowledge of the placment service is little to none. Ideally there was a tool provided that can assist me. | |
| 17:35:14 | cdent | lucidguy: what release are you running? | |
| 17:36:05 | lucidguy | Queens | |
| 17:36:51 | cdent | lucidguy: yeah, in that case osc-placement _is_ the tool provided to assist you | |
| 17:37:24 | dansmith | and check_placement is the automated use of that tool to fix your exact problem without you having to learn everything about the tool | |
| 17:38:00 | cdent | lucidguy: also see https://bugs.launchpad.net/nova/+bug/1793569 | |
| 17:38:00 | openstack | Launchpad bug 1793569 in OpenStack Compute (nova) "Add placement audit commands" [Wishlist,Confirmed] | |
| 17:39:06 | lucidguy | I can see https://paste.ubuntu.com/p/8F3JwHqSjz/ with the appropriate curl GET command, is it as simple as performing a PUT to change that hypervisor ID? | |
| 17:39:29 | bauzas | dansmith: mriedem: others: gentle notice that I'll be on PTO tomorrow | |
| 17:39:51 | bauzas | I'm actively working on libvirt reshape, still work to do on inventories | |
| 17:40:03 | bauzas | then I'll push a WIP for review | |
| 17:40:11 | dansmith | bauzas: ack | |
| 17:40:15 | cdent | lucidguy: which curl GET produced that? | |
| 17:40:23 | dansmith | FWIW, I'm out next week from tuesday->friday | |
| 17:40:30 | bauzas | ok | |
| 17:40:33 | dansmith | bauzas: so just have it all ready on monday, k? :) | |
| 17:40:35 | cdent | me M->W | |
| 17:40:47 | cdent | going to dan's house | |
| 17:40:51 | bauzas | dansmith: I'm really rushing to provide it ASAP :) | |
| 17:41:03 | bauzas | but there are a couple of things I need to consider | |
| 17:41:13 | jiteka | mriedem: thanks for the answer for my question from yesterday about cold-migration support in Mitaka | |
| 17:41:23 | mriedem | jiteka: yw | |
| 17:41:36 | bauzas | since I was a bit off when implementing reshape, I just wonder if the only trigger for reshape is the ReshapeNeeded exception that then sends allocations ? | |
| 17:41:38 | bauzas | dansmith: ^ | |
| 17:41:38 | jiteka | mriedem: Today I did some testing and face the same exception that was I was getting from live-migration | |
| 17:41:40 | lucidguy | cdent: https://paste.ubuntu.com/p/n3GCFR8MSt/ | |
| 17:42:17 | bauzas | dansmith: IIUC, the update_provider_tree() method is just running periodically, but we only pass allocations as a non-null param when we trigger a reshape, right? | |
| 17:42:36 | jiteka | mriedem: http://paste.openstack.org/show/730464/ | |
| 17:42:53 | dansmith | bauzas: I haven't followed the whole thing, but IIRC you only get to raise that on the first call of update so that it's done before compute starts up | |
| 17:43:02 | bauzas | dansmith: yup, that I understood | |
| 17:43:08 | cdent | lucidguy: so the is issue is that you've got the allocations showing up on two different resource providers and it should just be one. If that's the case, edit the json to remove the block identified by the wrong one, and PUT it back to the same URL | |
| 17:43:14 | bauzas | dansmith: and then we call again this method by passing an allocations param | |
| 17:43:31 | bauzas | so we only reshape when allocations is non-None | |