| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-05 | |||
| 18:58:03 | efried | head since Denver. But as I said here https://review.openstack.org/#/c/555081/4/specs/rocky/approved/cpu-resources.rst@412 I get that I'm being outvoted. So let's stop trying to convince each other that ours is the one true vision and just get on with making things happen. I'm implementing the granular algorithm right now. If you want to propose an amendment to the spec, let's git r done. | |
| 18:58:03 | efried | Look, I'm (still) not arguing that there are cases where it will be useful to force split. I'm (still) asserting that we're going to need to be able to handle both. One behavior will be the default, and the other will require some extra syntax to make it happen. I'm not convinced we truly need *either* immediately. So I've been advocating for the default being the one that's in the spec, because that's the one that make | |
| 18:59:16 | efried | (But on that note, I don't have a good way to tie separate request groups together unless allocation requests include the anchor provider.) | |
| 19:01:04 | jaypipes | efried: ack | |
| 19:18:56 | eandersson | What is the status of NUMA migration / evacuation etc? | |
| 19:19:40 | mriedem | eandersson: there is a spec for supporting live migration of numa instances https://review.openstack.org/#/c/552722/ | |
| 19:19:43 | mriedem | artom owns that | |
| 19:19:49 | mriedem | evac.... | |
| 19:19:58 | eandersson | Do you know when resize etc was fixed? | |
| 19:20:12 | mriedem | evac should work, not sure when cold migrate was fixed | |
| 19:20:18 | mriedem | cfriesen_ might know | |
| 19:29:10 | melwitt | what sort of testing do we generally do for the metadata API service? mostly only unit tests? | |
| 19:31:22 | melwitt | nvm, I found some func tests too | |
| 19:34:00 | eandersson | mriedem, cfriesen_ thanks - yea was hoping to figure out when (resize, evacuate etc) was actually fixed | |
| 19:34:35 | mriedem | eandersson: i'm not sure what about evac would be broken for instances with numa | |
| 19:34:56 | mriedem | evac goes through the scheduler to find a new host and has to do a claim on that new host, just like normal server create | |
| 19:34:58 | openstack | Launchpad bug 1417667 in OpenStack Compute (nova) "migration/evacuation/rebuild/resize/unshelve of instance with NUMA topology needs to recalculate NUMA topology" [Medium,In progress] - Assigned to sahid (sahid-ferdjaoui) | |
| 19:34:58 | eandersson | Yea - it's possible we were mislead by this bug report https://bugs.launchpad.net/nova/+bug/1417667 | |
| 19:35:50 | mriedem | that does look misleading, | |
| 19:36:07 | mriedem | cfriesen_ reported that for live migration, and then it looks like a blanket statement was made about other operations, including rebuild, which isn't a move operation | |
| 19:37:01 | eandersson | We did do a lot of testing on this, but honestly don't remember what was actually broken (besides live migration) for us | |
| 19:37:05 | mriedem | eandersson: so https://review.openstack.org/#/q/topic:bug/1417667+(status:open+OR+status:merged) | |
| 19:37:14 | mriedem | should handle all of the cold migrate/evac/resize cases i think | |
| 19:38:08 | mriedem | https://review.openstack.org/#/c/226411/ goes back to liberty | |
| 19:38:40 | mriedem | https://review.openstack.org/#/c/218938/ looks like that is the fix for cold migrate / resize | |
| 19:40:37 | eandersson | btw unrelated by is there a reason why filters and weights weren't made into plugins? | |
| 19:40:44 | mriedem | so i think we can probably mark this as fixed for evac and cold migrate, and open a new bug for tracking the live migration issue, which is being resolved via https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/numa-aware-live-migration.html in rocky | |
| 19:41:00 | mriedem | you can plug those in | |
| 19:41:04 | eandersson | oh | |
| 19:41:29 | eandersson | we have been adding them to the nova/schedulers/filters etc :D | |
| 19:41:35 | openstack | Launchpad bug 1417667 in OpenStack Compute (nova) "migration/evacuation/rebuild/resize/unshelve of instance with NUMA topology needs to recalculate NUMA topology" [Medium,In progress] - Assigned to sahid (sahid-ferdjaoui) | |
| 19:41:35 | mriedem | cfriesen_: if you're around and https://bugs.launchpad.net/nova/+bug/1417667/comments/45 is accurate, we should work on closing out that bug to reflect reality since Liberty | |
| 19:41:49 | mriedem | eandersson: we == ? | |
| 19:41:50 | mriedem | suse? | |
| 19:42:02 | eandersson | *I have :D for my deployment | |
| 19:42:23 | melwitt | you mean adding custom out-of-tree ones right? if you have generally useful weighers/filters you can upstream them | |
| 19:42:57 | eandersson | Yea for sure :D | |
| 19:42:59 | melwitt | do we allow custom out of tree ones? I thought maybe not | |
| 19:43:02 | mriedem | eandersson: https://docs.openstack.org/nova/latest/user/filter-scheduler.html#writing-your-own-filter | |
| 19:43:08 | mriedem | melwitt: yes we do | |
| 19:43:15 | melwitt | I can't remember if we restricted it to in-tree only | |
| 19:43:18 | mriedem | that's also why scheduler hints are wild west | |
| 19:43:18 | melwitt | okay, nevermind then | |
| 19:43:40 | mriedem | hell you can even load your own scheduler driver | |
| 19:44:01 | eandersson | I think we may just have backported some of the newer filter fixes | |
| 19:44:16 | mriedem | eandersson: which release are you on? | |
| 19:44:23 | eandersson | mitaka atm | |
| 19:44:33 | openstackgerrit | Tyler Blakeslee proposed openstack/nova stable/queens: Add __repr__ for NovaException https://review.openstack.org/559158 | |
| 19:47:53 | mriedem | ok so i think i was wrong about the plugin, | |
| 19:47:59 | mriedem | i think that might just be the scheduler driver entrypoint | |
| 19:48:14 | mriedem | https://github.com/openstack/nova/blob/master/nova/scheduler/filters/__init__.py#L64 | |
| 19:48:21 | mriedem | loads up the filters from nova/scheduler/filters as you said | |
| 19:48:46 | mriedem | https://docs.openstack.org/nova/latest/user/filter-scheduler.html#writing-your-own-filter doesn't mention that you have to drop them into the code tree | |
| 19:49:09 | eandersson | It would be a nice feature, because someone might write a scheduler to automatically upgrade their computes | |
| 19:49:17 | eandersson | you know sounds like a cool thing | |
| 19:49:27 | melwitt | it used to be free-form pluggable but it locked down more recently, IIRC. and the doc not updated | |
| 19:49:36 | eandersson | I see | |
| 19:49:39 | mriedem | eandersson: typically nova is anti-plugin | |
| 19:49:55 | mriedem | and have slowly evolved over time to cut out plugin points, as they aren't tested and we break them as such | |
| 19:50:07 | mriedem | plus, interop is a concern with too many plug points | |
| 19:50:30 | mriedem | for a private cloud that just wants to customize everything, interop isn't a concern, but it's a concern for public cloud | |
| 19:50:52 | mriedem | well, could be a concern for private cloud if it ever wants to use the same APIs on a public cloud | |
| 19:51:18 | melwitt | yeah, I've seen it bite private cloud when people used other open source tools that talk to openstack | |
| 19:51:29 | melwitt | that expected the APIs to be a certain way | |
| 19:53:09 | eandersson | Yea - makes sense | |
| 19:53:32 | melwitt | they thought "oh I can use this tool to talk to the openstack private cloud" and something custom in the cloud API tripped it up so they couldn't use the tool | |
| 19:54:11 | mriedem | i can't see anything in loadables or filters that used to use extension points | |
| 19:54:23 | mriedem | looks like it's just always been classpath loading since 2012 | |
| 19:54:28 | eandersson | Yea - the first place I always look is setup.cfg | |
| 19:56:52 | melwitt | oh, hm. okay | |
| 19:57:05 | melwitt | I dunno what I was thinking of. maybe I dreamed it | |
| 19:57:19 | mriedem | naw i thought they were loadable via ext point too | |
| 19:57:23 | mriedem | i know the driver is | |
| 19:57:43 | mriedem | and compute rest api extensions used to be loadable via ext point | |
| 19:57:58 | mriedem | i think that was also killed in liberty | |
| 19:58:11 | melwitt | right | |
| 19:58:39 | eandersson | btw more crazy questions... why was a notifaction on instance rename never implemented? | |
| 19:58:58 | mriedem | no one ever asked for/wrote it? | |
| 19:59:09 | eandersson | I see - that simple huh? :D | |
| 19:59:09 | mriedem | there are notifications on instance.update | |
| 19:59:11 | mriedem | which is what a rename is | |
| 19:59:24 | mriedem | https://docs.openstack.org/nova/latest/reference/notifications.html#versioned-notification-samples | |
| 19:59:57 | mriedem | although it's not like create where you have a start/end notification | |
| 20:00:24 | mriedem | oh, well looky here: "old_display_name": null, | |
| 20:00:56 | mriedem | http://git.openstack.org/cgit/openstack/nova/tree/nova/notifications/base.py#n121 | |
| 20:01:00 | mriedem | so yeah, instance.update is what you want | |
| 20:01:15 | eandersson | cool - maybe I can add that to designate | |
| 20:01:29 | eandersson | https://github.com/openstack/designate/blob/master/designate/notification_handler/nova.py#L73 | |
| 20:01:56 | mriedem | hmm, i don't think the display_name is what you're looking for | |
| 20:02:20 | mriedem | "host_name": "some-server", | |
| 20:02:30 | mriedem | host_name is what's used for the DNS entry i think | |
| 20:02:35 | eandersson | Yea - it is | |
| 20:02:50 | melwitt | yeah and host_name is derived from the display_name during create | |
| 20:02:53 | mriedem | and you can't change that via the REST API | |
| 20:03:09 | mriedem | https://developer.openstack.org/api-ref/compute/#update-server | |
| 20:05:08 | melwitt | yeah, display_name -> hostname thing only happens during create, I don't see it for update | |
| 20:05:48 | melwitt | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L524 | |
| 20:06:45 | openstackgerrit | Mathieu Gagné proposed openstack/nova-specs master: Multiple Fixed-IPs support in network information https://review.openstack.org/312626 | |
| 20:07:37 | mriedem | if we updated the hostname that would also likely imply a rebuild needed to get it into the guest | |
| 20:07:41 | melwitt | would this be a problem if anything in the exception message (like string substitutions like display_name) has unicode characters in it? https://review.openstack.org/#/c/555812/5/nova/exception.py@108 | |