| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-13 | |||
| 19:29:08 | artom | Ahh, virtio-sock: http://lists.openstack.org/pipermail/openstack-dev/2017-February/112602.html | |
| 19:29:18 | openstackgerrit | Dan Smith proposed openstack/nova master: Add AggregateList.get_by_metadata() query method https://review.openstack.org/544728 | |
| 19:29:19 | openstackgerrit | Dan Smith proposed openstack/nova master: Add aggregates list to Destination object https://review.openstack.org/544729 | |
| 19:29:20 | openstackgerrit | Dan Smith proposed openstack/nova master: Add request filter functionality to scheduler https://review.openstack.org/544730 | |
| 19:29:20 | openstackgerrit | Dan Smith proposed openstack/nova master: Make get_allocation_candidates() honor aggregate restrictions https://review.openstack.org/547990 | |
| 19:29:21 | openstackgerrit | Dan Smith proposed openstack/nova master: Add require_tenant_aggregate request filter https://review.openstack.org/545002 | |
| 19:29:21 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Honor availability_zone hint via placement https://review.openstack.org/546282 | |
| 19:29:30 | dansmith | edleafe: rebased on top of your patch ^ | |
| 19:30:10 | artom | Wonder how evil it would be support the metadata API with virtio-sock in libvirt only... | |
| 19:34:46 | dansmith | artom: it would be fully contained in the libvirt driver I think.. not sure that'd be a problem | |
| 19:35:16 | artom | dansmith, evil in the sense that we'd be leaving other virt drivers in a "we don't support the metadata API as well as libvirt does" state | |
| 19:35:39 | dansmith | I dunno how addressing works on vsock, and if you'd have to come up with a well-known address or port for it, but other than that it'd just be a proxy to the http interface | |
| 19:36:12 | artom | There'd also be the question of do we let the user decide whether they want a vsock or not | |
| 19:36:30 | dansmith | if it's a thing cloud-init looks for and supports, then it'd try that, like it tries http, and like it looks for a configdrive | |
| 19:36:48 | artom | I should really be thinking about other stuff, but this is more fun :/ | |
| 19:36:48 | dansmith | no, I'd say you just expose it if libvirt can do it, maybe optional via config, | |
| 19:36:51 | dansmith | but don't make it a user thing | |
| 19:37:07 | dansmith | I'm not sure it's a huge problem in need of a solution yeah :) | |
| 19:37:21 | dansmith | worst case, give them one vif on a private network that can get to metadata :) | |
| 19:38:18 | dansmith | ah, ports are strings, so "org.openstack.nova.metadata-api" or something | |
| 19:38:26 | artom | Well, as we use the metadata for more stuff (what did cfriesen want to use it for at the PTG? Something to do with pinning?), the fact that operators don't like deploying it (which I gather is a thing) is becoming a problem | |
| 19:39:50 | cfriesen | Yeah, I use it for CPU scaling since our qemu is too old for proper CPU hotplug...the metadata tells the guest at guest kernel boot time which vCPUs should be "offline". | |
| 19:40:58 | cfriesen | But we also have a terrible terrible feature that lets you specify the desired virtual PCI address for your virtual NICs. This was added for a customer that had legacy apps that had hardcoded PCI addresses for things. | |
| 19:41:15 | artom | cfriesen, hah, we had a customer that wanted to do that | |
| 19:41:26 | artom | I *think* we convinced them that device tagging was the way to go | |
| 19:41:45 | cfriesen | yeah...but until Pike you couldn't attach/detach tagged devices | |
| 19:42:22 | artom | Indeed, and PFs still can't be tagged, it turns out | |
| 19:42:34 | artom | *nudge nudge* https://review.openstack.org/#/c/533805/ ;) | |
| 19:42:34 | cfriesen | hmm, that's kind of lame | |
| 19:43:06 | cfriesen | I'll see if I can get our PCI guy to look at it. | |
| 19:43:43 | artom | cfriesen, thanks :) | |
| 19:58:34 | cdent | sorry, not sorry, I've given lots of people a merge conflict | |
| 20:00:14 | mikal_ | artom: I don't have a particular problem with the vsock thing if it "tastes like" normal network access inside the instance. | |
| 20:00:33 | mikal_ | artom: in fact, it would be nice for such a thing to "sign" the request so we strongly know what instance made it. | |
| 20:00:56 | mikal_ | artom: ala AWS | |
| 20:01:14 | artom | mikal_, hrmm, that's actually a good question - how does the host know what instance a request came from over vsock? | |
| 20:02:18 | mikal_ | artom: its not one vsock per instance? | |
| 20:02:32 | artom | mikal, it is, but... | |
| 20:02:46 | artom | This is where what little knowledge I have of this thing breaks down | |
| 20:02:49 | mikal | So nova knows when it sets the vsock the instance its going to wire it into... | |
| 20:03:18 | artom | But if there's only one service listening on the host side, even if there's one vsock per guest... | |
| 20:04:03 | artom | Presumably they speak TCP over that vsock, so unless every instance has its own "vsock IP" that nova knows about... | |
| 20:04:28 | artom | If berrange was here he'd probably laugh at my ignorance :/ | |
| 20:09:38 | dansmith | vsock works like AF_INET or AF_UNIX | |
| 20:09:42 | dansmith | it's a socket protocol | |
| 20:09:44 | dansmith | so no TCP or IP | |
| 20:09:59 | dansmith | it's 1:1 and uses strings as the addressing on either end | |
| 20:10:35 | dansmith | so it wouldn't "taste like normal networking" in that sense.. cloud-init would have to open an AF_VSOCK socket to some known port name and then try to speak HTTP over it or something, | |
| 20:10:55 | dansmith | which something on the host could proxy to the regular metadata API, or just emulate the metadata server | |
| 20:11:05 | dansmith | mikal: artom ^ | |
| 20:11:52 | artom | dansmith, heh, dunno if that was off the top of your head or you did the research, but thanks regardless :) | |
| 20:12:29 | cfriesen | artom: https://wiki.qemu.org/Features/VirtioVsock | |
| 20:15:04 | dansmith | artom: I had to refresh my memory on the port bit, but yeah, the rest is pretty straightforward | |
| 20:15:14 | dansmith | it's been discussed a couple times at least | |
| 20:15:16 | mikal | dansmith: it doesn't feel hard to get changes into cloud-init, but it would take a long time to propogate | |
| 20:15:34 | dansmith | mikal: yep, and we kinda have to do it first | |
| 20:15:53 | dansmith | mikal: I think it'd just be one of those "use it if it's there" sort of things anyway, so it wouldn't be too bad | |
| 20:15:59 | mikal | I've had a long term dream of getting something a bit like AWS IAMS into nova, but I don't think this is it on reflection | |
| 20:16:12 | mikal | dansmith: yeah, part of the discovery process at boot | |
| 20:16:13 | dansmith | but someone would have to write up both sides and think through a bunch of it | |
| 20:16:30 | mikal | IAMS is more like a transparent http proxy embedded into neutron | |
| 20:16:57 | mikal | So what is the advantage of vsock over the current ways? It would need to be large to be bothered making the change. | |
| 20:17:14 | dansmith | it has no addressing, and requires no networking | |
| 20:17:22 | dansmith | it's an agent transport basically, | |
| 20:17:31 | dansmith | but done in a generic POSIX-compatible way | |
| 20:18:57 | artom | Security is a big advantage, I'd htink | |
| 20:19:18 | artom | You're not sending potentially sensitive metadata over potentially unsecure proxies | |
| 20:19:19 | dansmith | I don't know | |
| 20:19:37 | artom | In a potentially sniffable by other guests way | |
| 20:19:39 | dansmith | it feels more secure but I don't think it's substantially so | |
| 20:19:50 | dansmith | no, I don't think this is less sniffable | |
| 20:20:10 | dansmith | less prone to bugs which might leak sniffability, maybe :) | |
| 20:21:25 | mikal | And instances need networking to be useful, so I guess we're talking about a before-networking-configured case here | |
| 20:21:40 | mikal | The benefits still feel marginal compared to the amount of mucking around to get them | |
| 20:21:51 | dansmith | well, this came up mostly because you need to know some things to plumb yourself for certain types of networking | |
| 20:21:56 | dansmith | chicken and egg like | |
| 20:22:14 | mikal | configdrive! | |
| 20:22:23 | dansmith | yeah, but you can't update that at runtime | |
| 20:22:29 | mikal | (Which cracks me up because configdrive was meant to be long dead by now) | |
| 20:22:30 | dansmith | which is why we landed here :) | |
| 20:22:55 | mikal | So for these network types, you need to have attached the _first_ interface before the config drive is generated? | |
| 20:23:03 | mikal | But the later ones you could use metadata server for? | |
| 20:23:42 | dansmith | if you boot with two SRIOV nics and you need to know which one is public, and you can't get to metadata over them (yet), then you could use configdrive, | |
| 20:23:46 | dansmith | but then if you attach something else later, | |
| 20:24:03 | dansmith | you'd need to know that even though at boot there was no metadata server, now there is and you should use that instead | |
| 20:24:10 | dansmith | it's doable of course | |
| 20:26:23 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add 'member_of' param to GET /allocation_candidates https://review.openstack.org/552098 | |
| 21:07:11 | cfriesen | just noticed something odd...the docs at https://docs.openstack.org/nova/pike/configuration/config.html specify that ram_weight_multiplier/disk_weight_multiplier/etc are under DEFAULT, but that generates a warning log since they're actually under "filter_scheduler" as far as I can tell. | |
| 21:07:42 | tssurya | dansmith : do we still plan to keep this NewtonCellsCheck Test in Rocky ? -> https://github.com/openstack/nova/blob/master/nova/tests/unit/db/test_sqlalchemy_migration.py#L356 ; since it uses a particularly older DBVersion it fails for my new DB version; not sure how to handle it because it is wrong to just update it to the new version I guess. | |
| 21:09:57 | mriedem | cfriesen: https://docs.openstack.org/nova/latest/configuration/config.html says the are under the [cells] section | |
| 21:11:13 | mriedem | oh heh there are two options with the same name | |
| 21:11:13 | mriedem | nice | |
| 21:12:39 | mriedem | cfriesen: it says it's in the [filter_scheduler] group https://docs.openstack.org/nova/latest/configuration/config.html#filter-scheduler | |
| 21:12:54 | mriedem | the 'group' you're looking at is the 'Deprecated Variations¶' | |
| 21:13:05 | mriedem | b/c deprecated_group="DEFAULT", | |
| 21:13:25 | mriedem | we can probably start cleaning up all of those aliases | |
| 21:13:28 | mriedem | they were made in 2016 | |
| 21:16:45 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add check if neutron "binding-extended" extension is available https://review.openstack.org/523548 | |
| 21:16:45 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add code to bind a port against a dest host during live migration https://review.openstack.org/523604 | |
| 21:16:46 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add VIFMigrateData object for live migration https://review.openstack.org/515423 | |