| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-05 | |||
| 18:59:16 | efried | (But on that note, I don't have a good way to tie separate request groups together unless allocation requests include the anchor provider.) | |
| 19:01:04 | jaypipes | efried: ack | |
| 19:18:56 | eandersson | What is the status of NUMA migration / evacuation etc? | |
| 19:19:40 | mriedem | eandersson: there is a spec for supporting live migration of numa instances https://review.openstack.org/#/c/552722/ | |
| 19:19:43 | mriedem | artom owns that | |
| 19:19:49 | mriedem | evac.... | |
| 19:19:58 | eandersson | Do you know when resize etc was fixed? | |
| 19:20:12 | mriedem | evac should work, not sure when cold migrate was fixed | |
| 19:20:18 | mriedem | cfriesen_ might know | |
| 19:29:10 | melwitt | what sort of testing do we generally do for the metadata API service? mostly only unit tests? | |
| 19:31:22 | melwitt | nvm, I found some func tests too | |
| 19:34:00 | eandersson | mriedem, cfriesen_ thanks - yea was hoping to figure out when (resize, evacuate etc) was actually fixed | |
| 19:34:35 | mriedem | eandersson: i'm not sure what about evac would be broken for instances with numa | |
| 19:34:56 | mriedem | evac goes through the scheduler to find a new host and has to do a claim on that new host, just like normal server create | |
| 19:34:58 | openstack | Launchpad bug 1417667 in OpenStack Compute (nova) "migration/evacuation/rebuild/resize/unshelve of instance with NUMA topology needs to recalculate NUMA topology" [Medium,In progress] - Assigned to sahid (sahid-ferdjaoui) | |
| 19:34:58 | eandersson | Yea - it's possible we were mislead by this bug report https://bugs.launchpad.net/nova/+bug/1417667 | |
| 19:35:50 | mriedem | that does look misleading, | |
| 19:36:07 | mriedem | cfriesen_ reported that for live migration, and then it looks like a blanket statement was made about other operations, including rebuild, which isn't a move operation | |
| 19:37:01 | eandersson | We did do a lot of testing on this, but honestly don't remember what was actually broken (besides live migration) for us | |
| 19:37:05 | mriedem | eandersson: so https://review.openstack.org/#/q/topic:bug/1417667+(status:open+OR+status:merged) | |
| 19:37:14 | mriedem | should handle all of the cold migrate/evac/resize cases i think | |
| 19:38:08 | mriedem | https://review.openstack.org/#/c/226411/ goes back to liberty | |
| 19:38:40 | mriedem | https://review.openstack.org/#/c/218938/ looks like that is the fix for cold migrate / resize | |
| 19:40:37 | eandersson | btw unrelated by is there a reason why filters and weights weren't made into plugins? | |
| 19:40:44 | mriedem | so i think we can probably mark this as fixed for evac and cold migrate, and open a new bug for tracking the live migration issue, which is being resolved via https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/numa-aware-live-migration.html in rocky | |
| 19:41:00 | mriedem | you can plug those in | |
| 19:41:04 | eandersson | oh | |
| 19:41:29 | eandersson | we have been adding them to the nova/schedulers/filters etc :D | |
| 19:41:35 | openstack | Launchpad bug 1417667 in OpenStack Compute (nova) "migration/evacuation/rebuild/resize/unshelve of instance with NUMA topology needs to recalculate NUMA topology" [Medium,In progress] - Assigned to sahid (sahid-ferdjaoui) | |
| 19:41:35 | mriedem | cfriesen_: if you're around and https://bugs.launchpad.net/nova/+bug/1417667/comments/45 is accurate, we should work on closing out that bug to reflect reality since Liberty | |
| 19:41:49 | mriedem | eandersson: we == ? | |
| 19:41:50 | mriedem | suse? | |
| 19:42:02 | eandersson | *I have :D for my deployment | |
| 19:42:23 | melwitt | you mean adding custom out-of-tree ones right? if you have generally useful weighers/filters you can upstream them | |
| 19:42:57 | eandersson | Yea for sure :D | |
| 19:42:59 | melwitt | do we allow custom out of tree ones? I thought maybe not | |
| 19:43:02 | mriedem | eandersson: https://docs.openstack.org/nova/latest/user/filter-scheduler.html#writing-your-own-filter | |
| 19:43:08 | mriedem | melwitt: yes we do | |
| 19:43:15 | melwitt | I can't remember if we restricted it to in-tree only | |
| 19:43:18 | mriedem | that's also why scheduler hints are wild west | |
| 19:43:18 | melwitt | okay, nevermind then | |
| 19:43:40 | mriedem | hell you can even load your own scheduler driver | |
| 19:44:01 | eandersson | I think we may just have backported some of the newer filter fixes | |
| 19:44:16 | mriedem | eandersson: which release are you on? | |
| 19:44:23 | eandersson | mitaka atm | |
| 19:44:33 | openstackgerrit | Tyler Blakeslee proposed openstack/nova stable/queens: Add __repr__ for NovaException https://review.openstack.org/559158 | |
| 19:47:53 | mriedem | ok so i think i was wrong about the plugin, | |
| 19:47:59 | mriedem | i think that might just be the scheduler driver entrypoint | |
| 19:48:14 | mriedem | https://github.com/openstack/nova/blob/master/nova/scheduler/filters/__init__.py#L64 | |
| 19:48:21 | mriedem | loads up the filters from nova/scheduler/filters as you said | |
| 19:48:46 | mriedem | https://docs.openstack.org/nova/latest/user/filter-scheduler.html#writing-your-own-filter doesn't mention that you have to drop them into the code tree | |
| 19:49:09 | eandersson | It would be a nice feature, because someone might write a scheduler to automatically upgrade their computes | |
| 19:49:17 | eandersson | you know sounds like a cool thing | |
| 19:49:27 | melwitt | it used to be free-form pluggable but it locked down more recently, IIRC. and the doc not updated | |
| 19:49:36 | eandersson | I see | |
| 19:49:39 | mriedem | eandersson: typically nova is anti-plugin | |
| 19:49:55 | mriedem | and have slowly evolved over time to cut out plugin points, as they aren't tested and we break them as such | |
| 19:50:07 | mriedem | plus, interop is a concern with too many plug points | |
| 19:50:30 | mriedem | for a private cloud that just wants to customize everything, interop isn't a concern, but it's a concern for public cloud | |
| 19:50:52 | mriedem | well, could be a concern for private cloud if it ever wants to use the same APIs on a public cloud | |
| 19:51:18 | melwitt | yeah, I've seen it bite private cloud when people used other open source tools that talk to openstack | |
| 19:51:29 | melwitt | that expected the APIs to be a certain way | |
| 19:53:09 | eandersson | Yea - makes sense | |
| 19:53:32 | melwitt | they thought "oh I can use this tool to talk to the openstack private cloud" and something custom in the cloud API tripped it up so they couldn't use the tool | |
| 19:54:11 | mriedem | i can't see anything in loadables or filters that used to use extension points | |
| 19:54:23 | mriedem | looks like it's just always been classpath loading since 2012 | |
| 19:54:28 | eandersson | Yea - the first place I always look is setup.cfg | |
| 19:56:52 | melwitt | oh, hm. okay | |
| 19:57:05 | melwitt | I dunno what I was thinking of. maybe I dreamed it | |
| 19:57:19 | mriedem | naw i thought they were loadable via ext point too | |
| 19:57:23 | mriedem | i know the driver is | |
| 19:57:43 | mriedem | and compute rest api extensions used to be loadable via ext point | |
| 19:57:58 | mriedem | i think that was also killed in liberty | |
| 19:58:11 | melwitt | right | |
| 19:58:39 | eandersson | btw more crazy questions... why was a notifaction on instance rename never implemented? | |
| 19:58:58 | mriedem | no one ever asked for/wrote it? | |
| 19:59:09 | eandersson | I see - that simple huh? :D | |
| 19:59:09 | mriedem | there are notifications on instance.update | |
| 19:59:11 | mriedem | which is what a rename is | |
| 19:59:24 | mriedem | https://docs.openstack.org/nova/latest/reference/notifications.html#versioned-notification-samples | |
| 19:59:57 | mriedem | although it's not like create where you have a start/end notification | |
| 20:00:24 | mriedem | oh, well looky here: "old_display_name": null, | |
| 20:00:56 | mriedem | http://git.openstack.org/cgit/openstack/nova/tree/nova/notifications/base.py#n121 | |
| 20:01:00 | mriedem | so yeah, instance.update is what you want | |
| 20:01:15 | eandersson | cool - maybe I can add that to designate | |
| 20:01:29 | eandersson | https://github.com/openstack/designate/blob/master/designate/notification_handler/nova.py#L73 | |
| 20:01:56 | mriedem | hmm, i don't think the display_name is what you're looking for | |
| 20:02:20 | mriedem | "host_name": "some-server", | |
| 20:02:30 | mriedem | host_name is what's used for the DNS entry i think | |
| 20:02:35 | eandersson | Yea - it is | |
| 20:02:50 | melwitt | yeah and host_name is derived from the display_name during create | |
| 20:02:53 | mriedem | and you can't change that via the REST API | |
| 20:03:09 | mriedem | https://developer.openstack.org/api-ref/compute/#update-server | |
| 20:05:08 | melwitt | yeah, display_name -> hostname thing only happens during create, I don't see it for update | |
| 20:05:48 | melwitt | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L524 | |
| 20:06:45 | openstackgerrit | Mathieu Gagné proposed openstack/nova-specs master: Multiple Fixed-IPs support in network information https://review.openstack.org/312626 | |
| 20:07:37 | mriedem | if we updated the hostname that would also likely imply a rebuild needed to get it into the guest | |
| 20:07:41 | melwitt | would this be a problem if anything in the exception message (like string substitutions like display_name) has unicode characters in it? https://review.openstack.org/#/c/555812/5/nova/exception.py@108 | |
| 20:07:42 | mriedem | or at least a reboot | |
| 20:08:04 | mriedem | melwitt: ah, yes, probably | |