| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-08 | |||
| 18:26:15 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Marker reset option for nova-manage map_instances https://review.openstack.org/539501 | |
| 18:27:58 | sean-k-mooney | Jeffrey4l: hi are you still working on https://bugs.launchpad.net/nova/+bug/1414895 i recently deployed stable pike and have this issue for standard livemigrations not just host evacuate | |
| 18:27:59 | openstack | Launchpad bug 1414895 in OpenStack Compute (nova) "failed compute node didn't delete instance's path directory in init_host" [Low,In progress] - Assigned to Jeffrey Zhang (jeffrey4l) | |
| 18:28:10 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Ensure 'HostState.numa_topology' is always an object https://review.openstack.org/542362 | |
| 18:28:20 | stephenfin | dansmith, bauzas: ^ | |
| 18:28:49 | stephenfin | It's rather low priority, but I would appreciate a sanity check on that at some future date | |
| 18:28:53 | stephenfin | Context here http://paste.openstack.org/show/666746/ | |
| 18:29:00 | sean-k-mooney | stephenfin: am could that break people who use the jason filter also do we care? | |
| 18:29:41 | stephenfin | sean-k-mooney: NUMATopologyFilter, I assume? | |
| 18:30:23 | Jeffrey4l | sean-k-mooney, no i am not working on it. feel free to take it. | |
| 18:30:58 | stephenfin | sean-k-mooney: It should not because we currently have code to handle both objects or strings https://github.com/openstack/nova/blob/master/nova/scheduler/filters/numa_topology_filter.py#L32-L33 | |
| 18:31:01 | sean-k-mooney | stephenfin: no the json filter allow you to match on any key/value pair on the hostsate dictionary | |
| 18:31:19 | stephenfin | ...which is what I want to delete | |
| 18:31:28 | stephenfin | sean-k-mooney: Hmm, I'm not sure | |
| 18:31:58 | sean-k-mooney | stephenfin: i think it might break it but for the second quest i also dont think we should really care | |
| 18:32:26 | sean-k-mooney | stephenfin: this filter to be specific https://github.com/openstack/nova/blob/master/nova/scheduler/filters/json_filter.py | |
| 18:32:53 | stephenfin | sean-k-mooney: I didn't even know that was a thing. I guess we could convert back to a JSON blob specifically for that string | |
| 18:33:00 | stephenfin | Or, more sanely, blacklist that key | |
| 18:33:03 | sean-k-mooney | stephenfin: you use it with scheduler hints | |
| 18:33:41 | stephenfin | I don't even think that would work at the moment. The 'numa_topology' string is one of those o.v.o object-in-a-string kind of strings | |
| 18:34:09 | stephenfin | i.e. with 'nova_object.version' and 'nova_object.changes' keys | |
| 18:34:25 | stephenfin | Probably a good point to note in the review | |
| 18:34:33 | stephenfin | And now, I'm off home o/ | |
| 18:34:35 | sean-k-mooney | Jeffrey4l: ok good to know. i dont know if i will have time to try and fix it but i hit it with kolla ansible deploy stable pike image for docker hub using external ceph | |
| 18:36:19 | openstackgerrit | Merged openstack/nova stable/pike: Stop globally caching host states in scheduler HostManager https://review.openstack.org/539005 | |
| 18:54:14 | openstackgerrit | Dan Smith proposed openstack/nova master: Compute RPC client bump to 5.0 https://review.openstack.org/541035 | |
| 18:54:14 | openstackgerrit | Dan Smith proposed openstack/nova master: Bump compute RPC API to version 5.0 https://review.openstack.org/541005 | |
| 18:59:37 | mriedem | jaypipes: want to send this to it's final resting place? https://review.openstack.org/#/c/541495/ | |
| 19:00:18 | mriedem | we need another core on https://review.openstack.org/#/c/541008/ - the nova-next job is passing again with that regression fix | |
| 19:02:49 | dansmith | mriedem: oh this is the thing from yesterday | |
| 19:03:54 | mriedem | sho is | |
| 19:10:00 | melwitt | dansmith: I updated the instance group get_hosts patch, not sure if that's what you had in mind https://review.openstack.org/#/c/540258 | |
| 19:10:16 | mriedem | pippo likes it | |
| 19:10:21 | mriedem | pippo likes lots of things | |
| 19:10:34 | melwitt | +W, pippo likes it | |
| 19:10:49 | dansmith | melwitt: oh I had looked and didn't think you had, but I must have missed it | |
| 19:11:38 | melwitt | dansmith: it was later in the afternoon, ran into some snags with putting the test together. so I think it was after you were off for the day | |
| 19:11:48 | openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: The documentation build should be changed to use openstackdocstheme instead of oslosphinx. https://review.openstack.org/542378 | |
| 19:11:49 | dansmith | I looked this morning | |
| 19:11:55 | dansmith | I was just .. blind I guess | |
| 19:12:02 | melwitt | oh. blind to the awesomeness I'm sure! | |
| 19:12:36 | dansmith | I'm not sure what this setup_instance_group() method is though | |
| 19:13:15 | mriedem | shoves a bunch of crap in the reqspec i think | |
| 19:13:22 | melwitt | it appeared to me to be the entry point for things that wanted to look at the instance group on the request spec | |
| 19:13:33 | dansmith | yeah, that's just so odd | |
| 19:13:34 | dansmith | and | |
| 19:13:41 | dansmith | not really well named | |
| 19:13:48 | melwitt | so it's called just before select_destinations in conductor, for example. but it's also called for conductor tasks like migrate | |
| 19:14:01 | melwitt | yeah | |
| 19:14:03 | jaypipes | mriedem: done | |
| 19:14:15 | mriedem | nova.scheduler.utils.shove_group_crap_in_reqspec | |
| 19:14:22 | openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | |
| 19:15:21 | dansmith | melwitt: well, I think I would have done it differently so we could scatter/gather and use the cell cache in there | |
| 19:15:36 | dansmith | but this should work I guess if it's in front of all the places where we need it | |
| 19:15:51 | melwitt | dansmith: do you mean like duplicate the get_hosts code there? I considered that but wasn't sure what you would prefer | |
| 19:16:00 | melwitt | I do wish to use scatter-gather ideally | |
| 19:16:26 | dansmith | I mean like just pull the whole group with hosts from the cells and then use the .hosts from those things | |
| 19:16:48 | dansmith | however, | |
| 19:16:49 | dansmith | can we really not pull a group out with the hosts pre-populated? | |
| 19:16:55 | dansmith | I don't see any that let you specify it | |
| 19:16:57 | melwitt | because scatter-gather expects to call a method that takes a context as the first arg and get_hosts uses the object._context | |
| 19:17:11 | melwitt | oh, I see | |
| 19:17:25 | dansmith | right, so just do a query for the group and get those back | |
| 19:17:56 | dansmith | you can always adapt s/g by just using a closure | |
| 19:18:14 | dansmith | oh | |
| 19:18:16 | dansmith | yeah, | |
| 19:18:18 | dansmith | this would be better | |
| 19:18:19 | dansmith | hang on | |
| 19:20:33 | dansmith | https://pastebin.com/Gv26a83b | |
| 19:20:48 | dansmith | that will just do the load with the right context in each cell in parallel | |
| 19:20:57 | dansmith | right? | |
| 19:20:57 | dansmith | then you can stitch the hosts together after | |
| 19:21:56 | melwitt | seems like it ... let me try it out | |
| 19:23:03 | melwitt | I wanted to do something like that, be able to pass something for scatter-gather. so if this works that would be ideal | |
| 19:28:28 | openstackgerrit | Merged openstack/nova master: TrivialFix: Add a blankline https://review.openstack.org/542094 | |
| 19:29:22 | melwitt | I noticed something else weird, another query for get_hosts. trying to figure out what to do with that https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L687 | |
| 19:30:26 | melwitt | it seems like it's redundant. we already got hosts for the group, then we query for an instance group again based on instance uuid, then get the hosts for that one? | |
| 19:32:11 | melwitt | I guess it's that, first get the hosts for the request_spec.instance_group. but if request_spec.instance_uuid is populated, override the old request_spec.instance_group with the instance group associated with the instance_uuid | |
| 19:32:13 | cfriesen | So we recently tripped over an interesting live migration bug....if you have an instance with an rbd-backed root disk and a config drive, libvirt will try to block-migrate the rbd drive and fail. The libvirt folks (dpb) seem to be saying that nova shouldn't set VIR_MIGRATE_NON_SHARED_INC if there aren't any disks to migrate. | |
| 19:32:19 | mriedem | dansmith: just realized we have some compat code doing up-calls in the compute manager, so that's fun https://review.openstack.org/#/c/541005/6 | |
| 19:33:23 | dansmith | mriedem: yep but super old, before multi-cell could be a thing anyway | |
| 19:33:38 | melwitt | cfriesen: that sounds like something artom might know about ^ | |
| 19:33:40 | mriedem | right. comment in https://review.openstack.org/#/c/541035/ btw | |
| 19:34:09 | mriedem | cfriesen: i think mdbooth already patched that | |
| 19:34:50 | dansmith | mriedem: ack, thanks | |
| 19:34:52 | mriedem | https://review.openstack.org/#/q/I9b545ca8aa6dd7b41ddea2d333190c9fbed19bc1 | |
| 19:34:55 | mriedem | cfriesen: ^ | |
| 19:36:27 | cfriesen | mriedem: yeah, I'm pretty sure we're seeing this in pike, which should have that change in it. Will try and bottom it out. | |
| 19:36:45 | mriedem | pike 16.0.4+? | |
| 19:38:08 | cfriesen | I can see the change in our version of the code, I just need to confirm we can reproduce the bug with the current code. | |
| 19:41:16 | dansmith | mriedem: I'm going to fix that self.reservations thing in a follow-on patch because that bubbles up pretty high | |
| 19:41:32 | mriedem | that's fine | |
| 19:54:02 | mriedem | melwitt: given you're going to want to backport this to pike and ocata, https://review.openstack.org/#/c/541442/ - i'm not sure if you want to address nits now or not | |
| 19:54:22 | mriedem | but i'll be around for 2 more hours if you do so i can +W | |
| 19:55:59 | melwitt | mriedem: I'm cool with fixing nits. thanks for the heads up. I'll update it right after I update this instance group thing. adding test coverage | |
| 19:58:00 | dansmith | melwitt: es worky? | |
| 19:58:12 | melwitt | dansmith: yis. thank you | |
| 19:58:16 | dansmith | \o/ | |
| 19:58:24 | melwitt | o/ high five! | |
| 19:59:28 | dansmith | let it be known I had a good idea once | |