| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-20 | |||
| 17:42:49 | Sundar | Hmm, ok. I need to think about that. Given the long task list for Cyborg in Stein, may be we can introduce microversions later, as needed | |
| 17:43:20 | mriedem | and it sounds like we don't need something translating JSON responses to OVOs | |
| 17:43:21 | sean-k-mooney | Sundar: what is the status fo teh deployable api endpoint currently | |
| 17:43:34 | Sundar | dansmith: sean-k-mooney: mriedem: What I am gathering is, we need a client for Nova to call Cyborg. That can be just the Cyborg client, as os-acc is not adding much value. Is that correct? | |
| 17:43:35 | mriedem | so what i'm hearing is nova just needs cyborgclient | |
| 17:43:43 | mriedem | Sundar: i think so | |
| 17:44:04 | mriedem | before we make this all so complicated that we never get anything done, we should probably start with that | |
| 17:44:08 | sean-k-mooney | if we can reuse gibi's resouce prvider mapping code for cyborge then yes | |
| 17:45:01 | Sundar | sean-k-mooney: is that a long haul? Can we reasonably expect that to get in by, say, Jan? | |
| 17:46:07 | gibi | Sundar: the non-generic mapping code is up in gerrit, I just got the comment yesterday that it can be done a lot more generic way so I'm working on that right now. I think this week I can publish the generic code | |
| 17:46:24 | gibi | Sundar: https://review.openstack.org/#/c/616239 | |
| 17:46:32 | Sundar | gibi: Excellent, thanks! | |
| 17:46:38 | dansmith | Sundar: as we have said, serializing all of this work is definitely going to blow all your timelines | |
| 17:47:17 | sean-k-mooney | Sundar: has any work progressed on the ci front | |
| 17:47:21 | Sundar | dansmith: sean-k-mooney: Can you please state 'for the record' that you are ok with dispensing os-acc and having Nova call Cyborg directly? | |
| 17:48:03 | Sundar | sean-k-mooney: The concept of Deployables is being reworked to map to RPs. What do you mean by 'CI front'? Zuul checking for Cyborg? | |
| 17:48:11 | sean-k-mooney | Sundar: yes we can call directly with out os-acc via the python clients. | |
| 17:48:15 | dansmith | Sundar: once again, nova will use a client library and not call directly. I do not care what the name of that thing is | |
| 17:48:54 | sean-k-mooney | im agreeing with dan by teh way eventhough we said it differently | |
| 17:49:01 | dansmith | yes | |
| 17:49:01 | Sundar | dansmith: If it is the standard Cyborg client, Nova will have to do the necessary conversions to/from JSON. Hance the question | |
| 17:49:23 | sean-k-mooney | what i meant for the ci front is we talk about the need to do some basic integration testing | |
| 17:49:24 | dansmith | Sundar: when nova uses a client library, it has to deal with the output of that client | |
| 17:49:43 | dansmith | Sundar: no client library returns exactly what we need with no massaging, of course | |
| 17:50:34 | Sundar | dansmith: Great. I will update the spec to skip os-acc. Thanks a lot to you, sean-k-mooney and mriedem. | |
| 17:50:42 | Sundar | I still have the question on colocation | |
| 17:51:13 | Sundar | Re. request groups in device profiles, it is still not clear to me how we would handle co-location without them, i.e., we want 2 accelerators from 2 different RPs in the same device | |
| 17:51:42 | Sundar | sean-k-mooney: You mentioned NUMA ffinity in your rpelies oin the spec. I am looking at co-location within a device | |
| 17:53:21 | sean-k-mooney | Sundar why would you have 2 different RP on the same device instead of 2 inventories in 1 device | |
| 17:53:33 | Sundar | Because they have different traits | |
| 17:53:37 | sean-k-mooney | * 2 inventories in 1 rp | |
| 17:54:01 | sean-k-mooney | ok then have 1 RP for the devce and two nested rp for the two acllerators | |
| 17:54:20 | Sundar | Say an FPGA with 2 regions: one has compression, other has encryption and we want to gang them up together | |
| 17:54:27 | sean-k-mooney | then we can use in_tree=<device rp uuid> for the colocation | |
| 17:55:02 | dansmith | Sundar: those aren't traits, right? | |
| 17:55:11 | Sundar | sean-k-mooney: Would the traits be applied to the parent RP or the children RPs? Ans, more importantly, is that scenario working today? | |
| 17:55:12 | dansmith | Sundar: those are inventories on a single provider, no? | |
| 17:55:36 | Sundar | dansmith: No, they are in 2 different RPs, but on the same device (PCI card for e.g.) | |
| 17:55:47 | sean-k-mooney | Sundar: i was tinking the childe RPs | |
| 17:55:52 | dansmith | Sundar: in that case, what would the inventories be? | |
| 17:56:08 | Sundar | Because they represent different functions, and functions are traits, they would show up as 2 RPs | |
| 17:56:24 | sean-k-mooney | they are different resouce classes also | |
| 17:56:38 | dansmith | sean-k-mooney: right which is why they can be inventories on the same provider | |
| 17:56:41 | Sundar | Each RP would contain resources of the class CUSTOM_ACCELERATOR_FPGA | |
| 17:56:49 | sean-k-mooney | one is COMPRESS_MBs and the other is CRYPTO_MBs | |
| 17:57:03 | dansmith | I thought the whole point of this was to expose functions as consumable things? | |
| 17:57:38 | Sundar | sean-k-mooney: No, we agreed both in rocky and Stein PTGs that the RCs reflect the device type (e.g. GPU, FPGA), not device details or functions | |
| 17:57:48 | dansmith | omg | |
| 17:57:52 | dansmith | that is not what I thought we agreed | |
| 17:57:59 | sean-k-mooney | in anycase i think nested RP can handel this usecase | |
| 17:58:13 | Sundar | So, a requets may look like: resources:CUSTOM_ACCELERATOR_FPGA=1; trait:CUSTOM_FUNCTION_A=required | |
| 17:58:24 | dansmith | because in that case, all cyborg is ever going to expose is a thousand RPs with the same =1 inventories, decorated with super complex traits to describe what is in them at any given point right? | |
| 17:59:02 | Sundar | If the RCs reflect functions, the inventories of RPs will change all the time, as devices get reconfigured | |
| 17:59:13 | dansmith | so will the traits right? | |
| 17:59:58 | sean-k-mooney | the hack here is we cant delete or recreate invtories if there are allocation against them but we can change the traits | |
| 18:00:00 | Sundar | dansmith: Why super-complex traits? I documented a small handful (4 or 5 at the most), plus whatever custom traits that PowerVM guys asked for | |
| 18:00:38 | sean-k-mooney | Sundar: there are more then 5 traits just to dicibe crypto functions | |
| 18:00:46 | dansmith | sean-k-mooney: right, but we can atomically update multiple inventory things at the same time | |
| 18:01:49 | dansmith | I mean, we can atomically update traits too I guess, but.. I totally thought this was going the direction of inventory being functions, and traits being actual, you know, traits about the device like brand, model capabilities, etc | |
| 18:02:16 | Sundar | sean-k-mooney: My point is they all have the same structure: CUSTOM_FUNCTION_foo, CUSTOM_DEVICE_MODEL_bar, etc. | |
| 18:03:24 | sean-k-mooney | ok lets try to get the most basic version of integration working first | |
| 18:03:55 | Sundar | dansmith: It would be simpler if all variations happened in traits, while RCs are more or less static -- a GPU accelerator will never become an FPGA accelerator. | |
| 18:04:45 | sean-k-mooney | Sundar: the expection from a placement point of view it that both traits and resouce classes would be largly stattic but could change over time | |
| 18:04:52 | Sundar | dansmith: This is documented in the specs -- both before and after the Stein PTG. But I am *not trying to guilt-trip you :) | |
| 18:05:55 | openstackgerrit | Elod Illes proposed openstack/nova master: Transform scheduler.select_destinations notification https://review.openstack.org/508506 | |
| 18:06:11 | sean-k-mooney | is there a way today to use cyborge to deploy somthing without special hardware that we could use a piplot to test the workflow | |
| 18:06:11 | dansmith | Sundar: " a GPU will never be an FPGA" is not an argument that means anything to me in this context | |
| 18:06:16 | Sundar | sean-k-mooney: Makes sense. A device model trait is not expected to change much -- except perhaps on firmware updates. A function trait will change only when orchestration programs it (not if the VM programs it, which is the Device as a Service use case) | |
| 18:07:14 | dansmith | Sundar: I thought the discussion had previously gone that a user says "I want two TLS offload devices, and they need to be able to support crypto $foo" | |
| 18:07:45 | dansmith | Sundar: but what you're saying is that they will need to say "I need two FPGAs and they need to have traits TLS_OFFLOAD and TLS_MECH_FOO" | |
| 18:07:58 | dansmith | which means cyborg isn't providing us much in the way of abstraction | |
| 18:08:16 | dansmith | anyway, I'm about out of energy for discussing this at this point, so I'll leave it to the others that are more invested | |
| 18:09:41 | Sundar | dansmith: Are you ok if sean-k-mooney and I continue the discussion? And are you ok with whatever conclusion we reach? :) | |
| 18:10:59 | Sundar | sean-k-mooney: You have been closely following my specs (and thanks for that). Are you in alignment with this representation? | |
| 18:11:36 | dansmith | Sundar: you can of course discuss anything you want, and no I'm not signing off on something I haven't read | |
| 18:12:14 | sean-k-mooney | Sundar: i am honest gettin quite tired also can we pick this up later in the week | |
| 18:12:27 | Sundar | dansmith: Ok, tried my luck there. Can we talk tomorrow same time? | |
| 18:12:48 | sean-k-mooney | perhaps we should do it on the cyborge channel not to flood nova | |
| 18:13:05 | dansmith | Sundar: honestly, I'm not sure we're making progress here | |
| 18:13:35 | dansmith | Sundar: and no, I can't be involved in every discussion, I'm just saying I reserve the right to be unhappy with the next round of the spec | |
| 18:13:35 | Sundar | sean-k-mooney: Sure. Same time tomorrow? | |
| 18:13:44 | dansmith | Sundar: you need more than just sean-k-mooney in agreement on this | |
| 18:13:49 | sean-k-mooney | i would honestly love to jsut protoype something end to end that works and see what it looked like | |
| 18:14:01 | dansmith | and getting everyone into a single irc channel at the same time is just not going to happen repetitively | |
| 18:14:08 | dansmith | sean-k-mooney: ++ | |
| 18:14:35 | Sundar | dansmith: Sorry to hear that. I thought we made progress by agreeing to skip os-acc. | |
| 18:15:08 | dansmith | sean-k-mooney: I'm getting spec fatigue I think.. a series of patches on both sides that actually does something we can evaluate might be a better stepping stone | |
| 18:15:18 | Sundar | sean-k-mooney: I am with you, but Cyborg folks are reluctant to move till Nova spec converges, so I am facing a catch-22 | |
| 18:15:33 | dansmith | sean-k-mooney: it'll be trivial to look at that and evaluate how things are being done | |
| 18:15:34 | sean-k-mooney | so lets create a feature branch | |
| 18:15:38 | dansmith | Sundar: no, that's not a legit argument | |
| 18:15:53 | dansmith | Sundar: you can put up patches against nova and cyborg and test them together without merging anything | |
| 18:15:59 | dansmith | we do it all the time for big complex things like this | |
| 18:16:31 | dansmith | if there isn't already, cyborg should have a fake driver that can just pretend to offer up devices and program them, | |
| 18:17:06 | dansmith | and that should be enough to do some interaction testing between the two services, even if nothing actually gets attached at the final step to the vm | |
| 18:17:07 | dansmith | sean-k-mooney: agree with that ^ ? | |
| 18:17:18 | Sundar | A feature branch upstream? | |
| 18:17:36 | sean-k-mooney | yes | |
| 18:17:38 | spatel | sean-k-mooney: do you have experience with rabbitmq ? | |
| 18:17:48 | sean-k-mooney | spatel: not much sorry | |