| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-09 | |||
| 19:23:38 | mriedem | hence the name, LiveMigrateData | |
| 19:24:02 | dansmith | yeah, left that detail out | |
| 19:24:24 | sean-k-mooney | mriedem: yes but for cold migrtion we kindo of abuse the migration_context for associting claim with the active migration too right | |
| 19:25:59 | sean-k-mooney | so we dont actully stuff the resocetrack claims into the migration_context as far as i know but i think we must the uuid of the migration_context when claiming the resouce or somthing like that. | |
| 19:27:28 | mriedem | based on my questions above, i'm clearly not the person to ask about the intracacies of how the migration_context is used during resize | |
| 19:27:39 | mriedem | *intricacies even | |
| 19:27:48 | dansmith | sean-k-mooney: migration context has no uuid, it's attached to the instance | |
| 19:28:17 | mriedem | MigrationContext.migration_id could be used to find the migration object if needed | |
| 19:28:22 | mriedem | within the same cell | |
| 19:28:24 | sean-k-mooney | sorry the migrtion record/object reffrence by the migration_context has a uuid which we use | |
| 19:28:24 | dansmith | sean-k-mooney: it has things that we don't need to persist after completion, unlike things we store in the migration record for posterity, like what flavor it was and what flavor it is now | |
| 19:28:46 | dansmith | mriedem: I meant the context doesn't have its own identifier | |
| 19:28:57 | mriedem | ah yeah | |
| 19:29:25 | dansmith | and, unfortunate that we used the id there I guess, as it makes it potentially less helpful for the cross-cell case | |
| 19:29:26 | mriedem | laura is home, the 2:30 vacuuming has started | |
| 19:29:53 | mriedem | for cross-cell everything is scoped to the cell db, so it's not a big deal | |
| 19:30:04 | dansmith | well, | |
| 19:30:07 | mriedem | conductor orchestrates the db switching when needed | |
| 19:30:17 | dansmith | once we want cross-cell live migration it'll probably be relevant | |
| 19:30:47 | mriedem | i think i'll probably be back at ibm working on php / xenapi by the time that happens.. | |
| 19:31:03 | mriedem | you know, the up and coming tech | |
| 19:31:04 | dansmith | it's a thing people want though | |
| 19:31:27 | mriedem | yeah, huawei's public cloud has a beta for cross-cell live migration, | |
| 19:31:28 | dansmith | especially if it were to make it easier to migrate from an older cloud to a newer one by making one a cell of the other until it's emptied | |
| 19:31:30 | mriedem | i'm not sure how they do it but... | |
| 19:31:34 | dansmith | that's been requested since icehouse or so | |
| 19:32:52 | sean-k-mooney | dansmith: we all know livemigration never works :P also if i get sirov live migration working this cycle i have no plan to test cross cell, sriov, nuam aware live migration between different releases during upgrade :) | |
| 19:33:14 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: WIP: Test report_ironic_standard_resource_class_inventory=False https://review.openstack.org/609107 | |
| 19:33:40 | mriedem | there is no cross cell live migration... | |
| 19:34:51 | sean-k-mooney | mriedem: didnt you jsut say huawei's public cloud has beta supprot. i would assume they will ask you to upstream it a some point or is that complete different devision form yours? | |
| 19:35:40 | mriedem | completely different | |
| 19:35:54 | mriedem | their public cloud is still using cascading, which is their proprietary cells v1 | |
| 19:36:17 | mriedem | they are working on migrating off that to cells v2 | |
| 19:36:22 | dansmith | that's more like cross-deployment live migration | |
| 19:36:50 | sean-k-mooney | dansmith: is that still an ask from the edge working group? | |
| 19:37:06 | dansmith | is what? cross-deployment live migration? | |
| 19:37:12 | dansmith | I'm sure it'll be on their list at some point | |
| 19:37:27 | dansmith | I can see the cross-cell thing being fine, but I dunno about cross-deploy | |
| 19:38:17 | sean-k-mooney | ya when i was in the edge room in dublin i spent 15 minute explaining why cross cloud inter hyperviror live migration was never going to be a thing and should not be in the phase 1 basic feature support for edge | |
| 19:39:15 | sean-k-mooney | they litrally wanted to live migrate form libvirt + ceph in one edge site to vmware on another | |
| 19:39:56 | jaypipes | dansmith, mriedem, melwitt: is there a reason we don't pass instance metadata to the RequestSpec? | |
| 19:40:18 | dansmith | jaypipes: why would we need to? | |
| 19:40:34 | jaypipes | dansmith: so that scheduler filters can look at the instance metadata? :) | |
| 19:40:40 | dansmith | they should never do that | |
| 19:40:46 | dansmith | instance metadata is owned by the user not nova | |
| 19:40:52 | jaypipes | dansmith: oh, but Oath disagrees strongly. ;) | |
| 19:40:54 | dansmith | looking at it, especially for placement would violate | |
| 19:40:58 | dansmith | jaypipes: -2 | |
| 19:41:02 | jaypipes | hehe | |
| 19:41:03 | melwitt | but what about... custom filters YALL | |
| 19:41:13 | jaypipes | dansmith: what melwitt said :) | |
| 19:41:17 | dansmith | yeah | |
| 19:41:19 | jaypipes | dansmith: example... | |
| 19:41:23 | sean-k-mooney | jaypipes: jay just stuff the info in a schduler hint and use the json fileter | |
| 19:41:57 | mriedem | the request spec has an instance_uuid on it, you can get the instance from that and pull the metadata off the instance; that won't work for multi-create, but you probably don't care at oath | |
| 19:42:11 | dansmith | yup | |
| 19:42:15 | jaypipes | dansmith: nova boot --property ytag=SOME_CUSTOM_YAHOO_GOOP; nova scheduler has a filter that does an external lookup to our inventory management system of the ytag to grab the availability zone (really, just a power domain) to send the instance to | |
| 19:42:22 | dansmith | or hint yourself to an external artifact, fetch it and go nuts | |
| 19:42:44 | dansmith | jaypipes: yep, I got it, but metadata is off limits | |
| 19:43:17 | dansmith | --hint ytag-fml -> look up fml externally, do a thing | |
| 19:43:19 | jaypipes | mriedem: you can't do that. the instance_uuid is for the mapping, but the metadata doesn't exist yet, so if you try to call Instance.get_by_uuid(instance_uuid) from the filter, that's a dead end. | |
| 19:43:34 | mriedem | jaypipes: get the build request then | |
| 19:43:43 | jaypipes | mriedem: build request doesn't store metadata. | |
| 19:43:44 | melwitt | last I heard, oath does use multi-create. there's auto-scale stuff that boots several instances at once | |
| 19:43:47 | mriedem | jaypipes: sure it does, | |
| 19:43:49 | mriedem | it stores the instance | |
| 19:43:52 | mriedem | which stores the metadata | |
| 19:44:23 | dansmith | right, that's how we know what to create when we've picked a host :) | |
| 19:44:27 | jaypipes | mriedem: https://github.com/openstack/nova/blob/stable/ocata/nova/compute/api.py#L1004-L1007 | |
| 19:44:39 | jaypipes | mriedem: where exactly does the build request store the instance metadata? | |
| 19:44:40 | mriedem | instance=instance | |
| 19:44:50 | dansmith | it's in instance | |
| 19:44:51 | dansmith | heh yeah | |
| 19:45:00 | mriedem | instance.update(base_options) | |
| 19:45:08 | mriedem | build request stores a serialized instance object | |
| 19:45:13 | jaypipes | ah.. | |
| 19:45:19 | jaypipes | I missed that. sorry. | |
| 19:45:30 | jaypipes | ok, so I'll change the filter to pull the build request by instance_uuid. | |
| 19:45:34 | mriedem | https://github.com/openstack/nova/blob/stable/ocata/nova/compute/api.py#L936 | |
| 19:45:35 | jaypipes | thanks y'all | |
| 19:45:45 | mriedem | performance will suck | |
| 19:45:47 | mriedem | but... | |
| 19:45:52 | dansmith | but you're going to hell anyway? | |
| 19:45:53 | dansmith | yah. | |
| 19:45:57 | jaypipes | mriedem: yes, understood. | |
| 19:46:01 | sean-k-mooney | hum in that case the json fileter and compute capablity filters can already read it and do stuff... | |
| 19:46:25 | jaypipes | sean-k-mooney: we already have a custom IronicCapabilitiesFilter. don't get me started :) | |
| 19:46:48 | melwitt | jaypipes: yeah, you might probably run into perf problems during a db lookup in a filter (but I guess you said earlier you're doing an external system lookup in a filter already and that wasn't hurting perf?) | |
| 19:46:58 | sean-k-mooney | jaypipes: are you thinking of pulling this suff out in a pre placement filter or post out of interest | |
| 19:47:27 | jaypipes | this one's actually a network availability filter that looks for num_additional_ipv4 and num_ipv6 custom metadata key/values, calls out to our IPAM system from within the filter itself, and determines if the target system has enough IP addresses available. | |
| 19:47:31 | jaypipes | dansmith: you're welcome ^ | |
| 19:47:35 | melwitt | I remember when I worked at yahoo, we ran into perf issues with an in-tree filter that was doing db lookups and had to patch it out (and upstream fixed it soon after) | |
| 19:48:04 | jaypipes | melwitt: this is even worse. :) it's doing out of band calls to a REST API from within the in-tree filter :) | |
| 19:48:11 | sean-k-mooney | jaypipes: for l3 routeded network we are storing that kindo of info in placement | |
| 19:48:16 | melwitt | yeah. interesting that it's not causing perf issues | |
| 19:48:34 | jaypipes | melwitt: well, it's not like the traffic to the scheduler is huge... | |
| 19:49:00 | jaypipes | melwitt: I mean, it's not like there's thousands of concurrent callers of nova boot for ironic hosts. | |
| 19:49:00 | sean-k-mooney | jaypipes: any way you could jsut write some kind of bridge between neutron and the ipam to model it in placement and not use a scheduler filter | |
| 19:49:06 | melwitt | it used to be, is what I'm saying. but that was back before we had placement filtering the set of compute nodes down | |
| 19:49:14 | jaypipes | sean-k-mooney: baby steps :) | |