| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-14 | |||
| 19:33:50 | efried | dansmith: I guess so, because boot includes plug, which has to come later. | |
| 19:34:56 | Sundar | How about this model? Accelerators are RCs, pre-published to Nova. Scheduling works with known RPs, traits and RCs, but attach handles are not part of it. Cyborg alone deals with attach handles. When a request comes to n-cpu, one or more attach handles get picked (nova virt either calls a plugin, or calls Cyborg agent, which int turn calls a Cyborg driver/plugin). The pre-create would be done by the plugin | |
| 19:35:47 | dansmith | um | |
| 19:35:51 | dansmith | I'm not sure what to do with that | |
| 19:36:07 | dansmith | Sundar: IMHO, cyborg or os-acc should not be mucking with the instance's allocation in placement | |
| 19:36:25 | Sundar | Yes, it does not | |
| 19:36:40 | dansmith | the discussion above is not about attach handles | |
| 19:36:53 | Sundar | Cyborg publishes RPs etc to placement. Nova/plcaement do all the allocation of candidates | |
| 19:36:58 | openstackgerrit | Brianna Poulos proposed openstack/python-novaclient master: Microversion 2.63 - Add trusted_image_certificates https://review.openstack.org/500396 | |
| 19:37:53 | Sundar | My difficulty is in relating attach handles to accelerators, and making sure all possibilities and use cases are covered :) | |
| 19:38:11 | dansmith | Sundar: I'm confused.. the things you're saying are true and not really negotiable, but it sounds like you're suggesting something else | |
| 19:38:34 | Sundar | Sorry, let me try again | |
| 19:38:44 | Sundar | Consider this flow: | |
| 19:39:52 | Sundar | A request comes to n-api. The n-sch works works with placement to decide allocation candidates and choose a compute node. The request then comes to n-cpu in the compute node. No Cyborg so far (except for populating Placement at the start). is this good so far? | |
| 19:40:26 | efried | cyborg was involved in creating the artifact to begin with. I think we're calling this an "attachment". | |
| 19:40:46 | efried | Sundar: How about this, let me draw up the flow as I think I understand it and then we'll have a starting point for discussion. | |
| 19:41:08 | efried | dansmith: What's this pre-create thing called in the neutron flow? | |
| 19:41:17 | dansmith | create port? | |
| 19:41:17 | Sundar | efried: Cyborg publishes accelerators as RCs. You are saying attach handles (like PCI VFs or Power location codes) should also be published? | |
| 19:42:04 | efried | Sundar: I'm saying the user does a "openstack create accelerator-attachment <details>". Cyborg creates what you're calling an "attachment" and returns its UUID. | |
| 19:42:23 | efried | Sundar: Then the user does "openstack server create <details> --accelerator <that UUID>" | |
| 19:43:04 | Sundar | efired: why is that needed? For a edevice, the #attachments can be discovered automatically., unlike Neutron ports whose # is not fixed or discoverable | |
| 19:43:14 | efried | This gets to n-sch, which asks cyborg for the Acc* object corresponding to the attachment UUID. Cyborg pulls the original profile info out of its database and feeds it back. | |
| 19:43:24 | dansmith | efried: and nova uses that id to ask cyborg what resources it should ask for based on what accelerator the user asked for | |
| 19:43:26 | Sundar | E.g. for a SR-IOV device, | |
| 19:43:30 | dansmith | efried: and include that in the allocation | |
| 19:43:46 | Sundar | the #VFs is known at device discovery time | |
| 19:43:52 | dansmith | it gets that claim, and then can immediately tell cyborg "okay, I just committed $foo to $user on host $bar, cool?" | |
| 19:44:08 | dansmith | and if cyborg says yes, then cyborg reswizzles its inventory if needed, and compute gets started | |
| 19:44:09 | efried | Right, n-sch converts the Acc* object into placement query material to be included in the GET /a-c query. | |
| 19:44:26 | dansmith | when ready, it asks cyborg for "okay how are we going to attach this mofo?", which might be os-acc, and then we're off | |
| 19:46:04 | efried | Okay, I think dansmith and I are on the same page. So dansmith in your view, the plugin needs to have methods for plug(), unplug(), and this prepare_for_plug() thing? | |
| 19:46:24 | dansmith | probablyish | |
| 19:46:42 | dansmith | it could also have the "tell cyborg we chose a host for attachment $foo", which n-cond could use in the early part | |
| 19:47:13 | dansmith | which neutron calls a host binding | |
| 19:47:15 | tlb | any devs around?+ | |
| 19:48:38 | efried | dansmith: check_host_for_attach(), then bind(), then plug() ? | |
| 19:50:08 | dansmith | efried: no, I think just bind(), plug() | |
| 19:50:13 | efried | okay. | |
| 19:50:19 | dansmith | efried: I mean, right? | |
| 19:50:28 | dansmith | we know what host we want to use because we just got granted an allocation for one | |
| 19:50:32 | efried | dansmith: Well, I didn't know this bind step was a thing at all. | |
| 19:50:37 | dansmith | we try to bind(), if that works, then we move on | |
| 19:50:49 | Sundar_ | IRC froze on me. Reconnecting | |
| 19:51:06 | dansmith | bind() tells cyborg that we're consuming a thing, it can adjust its inventory before it says success or failure | |
| 19:51:31 | dansmith | once bind returns true, we know that cyborg is ready to commit that thing to us and we'll next talk to cyborg during the plug() part on the compute | |
| 19:52:08 | tlb | Q: _get_computes_for_cells returning subset of available compute nodes across AZs (wrong subset, as it happens); suggestions in the dbeug log that it's cached somewhere (get_or_set_cached_cell_and_set_connections); any idea how/where? not obvious from the code, or at least, not to me. | |
| 19:52:14 | efried | dansmith: Okay. So the PluginBase should have a bind() method with a default `pass` impl which will probably not be overridden in many cases. The bind() step in cyborg will do stuff that diddles with cyborg's internal db and whatnot, and then call into the plugin's bind() method just in case. | |
| 19:52:18 | tlb | *debug | |
| 19:52:20 | Sundar_ | The difference between Neutron ports and an attach handle is this. A port is a virtual object that needs to be created by CLI. An attach handle (at least in some common cases like PCI VFs) are device properties, which can be discovered. We don't need 'openstack create' for that | |
| 19:52:58 | efried | Sundar_: Discovered... | |
| 19:53:01 | dansmith | efried: anything that has a potentially-varying inventory would need to implement it | |
| 19:53:15 | dansmith | Sundar_: no | |
| 19:53:18 | dansmith | that's not it | |
| 19:53:33 | dansmith | Sundar_: we're discussing an attachment handle as purely a handle to some state on the cyborg side | |
| 19:54:10 | efried | Sundar_: I'm going to tell you what frame rate and display heads and resolution I need from my GPU, and you're going to put that profile into your database and return a UUID to me. | |
| 19:54:18 | dansmith | right | |
| 19:54:24 | dansmith | which may mean you're just going to give me a device, | |
| 19:54:34 | dansmith | or it may mean you'll reconfigure some device to give me a slice of it which will support that | |
| 19:54:37 | dansmith | I don't care | |
| 19:54:37 | efried | This is so that, later on, when nova says "hey, give me the details on this UUID" you can give it back the AccGPU object with those details filled in. | |
| 19:54:46 | dansmith | just give me a uuid | |
| 19:54:51 | dansmith | right | |
| 19:55:02 | efried | It's so we don't have to carry (big random dict) across the nova boot command line and down through the conductor to the scheduler and compute etc. | |
| 19:55:21 | dansmith | which is how it used to work with cinder, and that's why it's been painful every single day since | |
| 19:55:44 | efried | I'm going to work up a seqdiag... | |
| 19:55:47 | Sundar_ | "it may mean you'll reconfigure some device" without knowing which compute node is involved, how do you pick a device to reconfigure? | |
| 19:56:02 | efried | Sundar_: That won't happen until the bind() and/or plug() steps. | |
| 19:56:06 | dansmith | Sundar_: you won't until we bind | |
| 19:56:16 | dansmith | at that point, you get to do that, decide if there's room, etc | |
| 19:56:21 | efried | That's the point - we want to save off configuration details before we know where we're going to land. | |
| 19:57:05 | Sundar_ | "bind() tells cyborg that we're consuming a thing" Is this invoked by n-cpu on os-acc/Cyborg? | |
| 19:57:28 | dansmith | it's likely done by conductor, talking to cyborg, through os-acc | |
| 19:58:28 | dansmith | efried: I have to wander off for a bit, but I think we're on the same page | |
| 19:58:52 | efried | dansmith: ack. I'll make sure you see this diagram :) | |
| 19:59:02 | dansmith | I literally can not friggin wait | |
| 19:59:09 | dansmith | (that's a lie, I can wait) | |
| 19:59:38 | efried | dansmith: If you need something to keep your horses held until then, I still need your comeback on the reshape spec. | |
| 19:59:46 | Sundar_ | efried: You offered to build a flow/sequence diagram. Thanks. Please LMK when it is ready. May be it will make things more concrete. Not sure if we have the same mental model, but let's discuss on an etherpad | |
| 20:00:00 | efried | Sundar_: Roger wilco. | |
| 20:04:29 | melwitt | I started the rocky blueprint status etherpad here, still have some more to go through https://etherpad.openstack.org/p/nova-rocky-blueprint-status | |
| 20:04:49 | mriedem | melwitt: thanks for doing that | |
| 20:04:52 | mriedem | don't you feel better now?! | |
| 20:05:15 | melwitt | sorta | |
| 20:13:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add certificate validation docs https://review.openstack.org/560158 | |
| 20:13:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add trusted certs to feature support matrix docs https://review.openstack.org/574890 | |
| 20:13:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove max_size parameter from fake_libvirt_utils.fetch_*image methods https://review.openstack.org/574911 | |
| 20:13:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix nits from trusted certs notification change https://review.openstack.org/575521 | |
| 20:22:09 | mriedem | huh, for some reason the ironic job is running on our docs-only changes https://review.openstack.org/#/c/574890/ | |
| 20:22:12 | mriedem | ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa SUCCESS in 45m 30s (non-voting) | |
| 20:23:04 | mriedem | http://git.openstack.org/cgit/openstack/ironic/tree/zuul.d/legacy-ironic-jobs.yaml#n4 | |
| 20:23:24 | mriedem | ^doc/.*$ should match and skip | |
| 20:23:27 | melwitt | heh. that's odd | |
| 20:23:38 | melwitt | oh, hold the phone | |
| 20:23:59 | mriedem | because they changed how irrelevant-files works? | |
| 20:24:08 | melwitt | maybe, that's what I was wondering http://lists.openstack.org/pipermail/openstack-dev/2018-June/131304.html | |
| 20:24:23 | melwitt | unless it's just always been like that and no one noticed | |
| 20:25:53 | mriedem | i see why | |
| 20:25:57 | mriedem | http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n10496 | |
| 20:27:35 | mriedem | https://bugs.launchpad.net/nova/+bug/1776981 | |