| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-20 | |||
| 17:47:21 | Sundar | dansmith: sean-k-mooney: Can you please state 'for the record' that you are ok with dispensing os-acc and having Nova call Cyborg directly? | |
| 17:48:03 | Sundar | sean-k-mooney: The concept of Deployables is being reworked to map to RPs. What do you mean by 'CI front'? Zuul checking for Cyborg? | |
| 17:48:11 | sean-k-mooney | Sundar: yes we can call directly with out os-acc via the python clients. | |
| 17:48:15 | dansmith | Sundar: once again, nova will use a client library and not call directly. I do not care what the name of that thing is | |
| 17:48:54 | sean-k-mooney | im agreeing with dan by teh way eventhough we said it differently | |
| 17:49:01 | dansmith | yes | |
| 17:49:01 | Sundar | dansmith: If it is the standard Cyborg client, Nova will have to do the necessary conversions to/from JSON. Hance the question | |
| 17:49:23 | sean-k-mooney | what i meant for the ci front is we talk about the need to do some basic integration testing | |
| 17:49:24 | dansmith | Sundar: when nova uses a client library, it has to deal with the output of that client | |
| 17:49:43 | dansmith | Sundar: no client library returns exactly what we need with no massaging, of course | |
| 17:50:34 | Sundar | dansmith: Great. I will update the spec to skip os-acc. Thanks a lot to you, sean-k-mooney and mriedem. | |
| 17:50:42 | Sundar | I still have the question on colocation | |
| 17:51:13 | Sundar | Re. request groups in device profiles, it is still not clear to me how we would handle co-location without them, i.e., we want 2 accelerators from 2 different RPs in the same device | |
| 17:51:42 | Sundar | sean-k-mooney: You mentioned NUMA ffinity in your rpelies oin the spec. I am looking at co-location within a device | |
| 17:53:21 | sean-k-mooney | Sundar why would you have 2 different RP on the same device instead of 2 inventories in 1 device | |
| 17:53:33 | Sundar | Because they have different traits | |
| 17:53:37 | sean-k-mooney | * 2 inventories in 1 rp | |
| 17:54:01 | sean-k-mooney | ok then have 1 RP for the devce and two nested rp for the two acllerators | |
| 17:54:20 | Sundar | Say an FPGA with 2 regions: one has compression, other has encryption and we want to gang them up together | |
| 17:54:27 | sean-k-mooney | then we can use in_tree=<device rp uuid> for the colocation | |
| 17:55:02 | dansmith | Sundar: those aren't traits, right? | |
| 17:55:11 | Sundar | sean-k-mooney: Would the traits be applied to the parent RP or the children RPs? Ans, more importantly, is that scenario working today? | |
| 17:55:12 | dansmith | Sundar: those are inventories on a single provider, no? | |
| 17:55:36 | Sundar | dansmith: No, they are in 2 different RPs, but on the same device (PCI card for e.g.) | |
| 17:55:47 | sean-k-mooney | Sundar: i was tinking the childe RPs | |
| 17:55:52 | dansmith | Sundar: in that case, what would the inventories be? | |
| 17:56:08 | Sundar | Because they represent different functions, and functions are traits, they would show up as 2 RPs | |
| 17:56:24 | sean-k-mooney | they are different resouce classes also | |
| 17:56:38 | dansmith | sean-k-mooney: right which is why they can be inventories on the same provider | |
| 17:56:41 | Sundar | Each RP would contain resources of the class CUSTOM_ACCELERATOR_FPGA | |
| 17:56:49 | sean-k-mooney | one is COMPRESS_MBs and the other is CRYPTO_MBs | |
| 17:57:03 | dansmith | I thought the whole point of this was to expose functions as consumable things? | |
| 17:57:38 | Sundar | sean-k-mooney: No, we agreed both in rocky and Stein PTGs that the RCs reflect the device type (e.g. GPU, FPGA), not device details or functions | |
| 17:57:48 | dansmith | omg | |
| 17:57:52 | dansmith | that is not what I thought we agreed | |
| 17:57:59 | sean-k-mooney | in anycase i think nested RP can handel this usecase | |
| 17:58:13 | Sundar | So, a requets may look like: resources:CUSTOM_ACCELERATOR_FPGA=1; trait:CUSTOM_FUNCTION_A=required | |
| 17:58:24 | dansmith | because in that case, all cyborg is ever going to expose is a thousand RPs with the same =1 inventories, decorated with super complex traits to describe what is in them at any given point right? | |
| 17:59:02 | Sundar | If the RCs reflect functions, the inventories of RPs will change all the time, as devices get reconfigured | |
| 17:59:13 | dansmith | so will the traits right? | |
| 17:59:58 | sean-k-mooney | the hack here is we cant delete or recreate invtories if there are allocation against them but we can change the traits | |
| 18:00:00 | Sundar | dansmith: Why super-complex traits? I documented a small handful (4 or 5 at the most), plus whatever custom traits that PowerVM guys asked for | |
| 18:00:38 | sean-k-mooney | Sundar: there are more then 5 traits just to dicibe crypto functions | |
| 18:00:46 | dansmith | sean-k-mooney: right, but we can atomically update multiple inventory things at the same time | |
| 18:01:49 | dansmith | I mean, we can atomically update traits too I guess, but.. I totally thought this was going the direction of inventory being functions, and traits being actual, you know, traits about the device like brand, model capabilities, etc | |
| 18:02:16 | Sundar | sean-k-mooney: My point is they all have the same structure: CUSTOM_FUNCTION_foo, CUSTOM_DEVICE_MODEL_bar, etc. | |
| 18:03:24 | sean-k-mooney | ok lets try to get the most basic version of integration working first | |
| 18:03:55 | Sundar | dansmith: It would be simpler if all variations happened in traits, while RCs are more or less static -- a GPU accelerator will never become an FPGA accelerator. | |
| 18:04:45 | sean-k-mooney | Sundar: the expection from a placement point of view it that both traits and resouce classes would be largly stattic but could change over time | |
| 18:04:52 | Sundar | dansmith: This is documented in the specs -- both before and after the Stein PTG. But I am *not trying to guilt-trip you :) | |
| 18:05:55 | openstackgerrit | Elod Illes proposed openstack/nova master: Transform scheduler.select_destinations notification https://review.openstack.org/508506 | |
| 18:06:11 | sean-k-mooney | is there a way today to use cyborge to deploy somthing without special hardware that we could use a piplot to test the workflow | |
| 18:06:11 | dansmith | Sundar: " a GPU will never be an FPGA" is not an argument that means anything to me in this context | |
| 18:06:16 | Sundar | sean-k-mooney: Makes sense. A device model trait is not expected to change much -- except perhaps on firmware updates. A function trait will change only when orchestration programs it (not if the VM programs it, which is the Device as a Service use case) | |
| 18:07:14 | dansmith | Sundar: I thought the discussion had previously gone that a user says "I want two TLS offload devices, and they need to be able to support crypto $foo" | |
| 18:07:45 | dansmith | Sundar: but what you're saying is that they will need to say "I need two FPGAs and they need to have traits TLS_OFFLOAD and TLS_MECH_FOO" | |
| 18:07:58 | dansmith | which means cyborg isn't providing us much in the way of abstraction | |
| 18:08:16 | dansmith | anyway, I'm about out of energy for discussing this at this point, so I'll leave it to the others that are more invested | |
| 18:09:41 | Sundar | dansmith: Are you ok if sean-k-mooney and I continue the discussion? And are you ok with whatever conclusion we reach? :) | |
| 18:10:59 | Sundar | sean-k-mooney: You have been closely following my specs (and thanks for that). Are you in alignment with this representation? | |
| 18:11:36 | dansmith | Sundar: you can of course discuss anything you want, and no I'm not signing off on something I haven't read | |
| 18:12:14 | sean-k-mooney | Sundar: i am honest gettin quite tired also can we pick this up later in the week | |
| 18:12:27 | Sundar | dansmith: Ok, tried my luck there. Can we talk tomorrow same time? | |
| 18:12:48 | sean-k-mooney | perhaps we should do it on the cyborge channel not to flood nova | |
| 18:13:05 | dansmith | Sundar: honestly, I'm not sure we're making progress here | |
| 18:13:35 | dansmith | Sundar: and no, I can't be involved in every discussion, I'm just saying I reserve the right to be unhappy with the next round of the spec | |
| 18:13:35 | Sundar | sean-k-mooney: Sure. Same time tomorrow? | |
| 18:13:44 | dansmith | Sundar: you need more than just sean-k-mooney in agreement on this | |
| 18:13:49 | sean-k-mooney | i would honestly love to jsut protoype something end to end that works and see what it looked like | |
| 18:14:01 | dansmith | and getting everyone into a single irc channel at the same time is just not going to happen repetitively | |
| 18:14:08 | dansmith | sean-k-mooney: ++ | |
| 18:14:35 | Sundar | dansmith: Sorry to hear that. I thought we made progress by agreeing to skip os-acc. | |
| 18:15:08 | dansmith | sean-k-mooney: I'm getting spec fatigue I think.. a series of patches on both sides that actually does something we can evaluate might be a better stepping stone | |
| 18:15:18 | Sundar | sean-k-mooney: I am with you, but Cyborg folks are reluctant to move till Nova spec converges, so I am facing a catch-22 | |
| 18:15:33 | dansmith | sean-k-mooney: it'll be trivial to look at that and evaluate how things are being done | |
| 18:15:34 | sean-k-mooney | so lets create a feature branch | |
| 18:15:38 | dansmith | Sundar: no, that's not a legit argument | |
| 18:15:53 | dansmith | Sundar: you can put up patches against nova and cyborg and test them together without merging anything | |
| 18:15:59 | dansmith | we do it all the time for big complex things like this | |
| 18:16:31 | dansmith | if there isn't already, cyborg should have a fake driver that can just pretend to offer up devices and program them, | |
| 18:17:06 | dansmith | and that should be enough to do some interaction testing between the two services, even if nothing actually gets attached at the final step to the vm | |
| 18:17:07 | dansmith | sean-k-mooney: agree with that ^ ? | |
| 18:17:18 | Sundar | A feature branch upstream? | |
| 18:17:36 | sean-k-mooney | yes | |
| 18:17:38 | spatel | sean-k-mooney: do you have experience with rabbitmq ? | |
| 18:17:48 | sean-k-mooney | spatel: not much sorry | |
| 18:18:03 | spatel | no worry!! | |
| 18:18:12 | sean-k-mooney | dansmith: i would love to take that approch | |
| 18:20:08 | Sundar | dansmith: sean-k-mooney: OK, thanks for your time. | |
| 18:22:19 | openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: Re-propose numa-aware-live-migration spec https://review.openstack.org/599587 | |
| 18:23:24 | jaypipes | holy crap, I missed a bunch... :( sorry, reading back up... | |
| 18:25:12 | sean-k-mooney | mriedem: i found a relitvly simple and reliable way to repoduce https://bugs.launchpad.net/nova/+bug/1751923 by the way | |
| 18:25:12 | openstack | Launchpad bug 1751923 in OpenStack Compute (nova) "_heal_instance_info_cache periodic task bases on port list from nova db, not from neutron server" [Medium,In progress] - Assigned to Maciej Jozefczyk (maciej.jozefczyk) | |
| 18:50:57 | mriedem | sean-k-mooney: how is that? take down the neutron agent and reboot the vm or something? | |
| 18:52:53 | sean-k-mooney | mriedem: i added a scipt to the bug | |
| 18:53:16 | sean-k-mooney | you can cause it or a similar effect via the api | |
| 18:54:27 | sean-k-mooney | basically if you send the api request to detach a port to neutron and reboot the vm you end up with it broken | |
| 18:54:56 | sean-k-mooney | and you cant use openstack server add port or remove port to fix it | |
| 18:55:12 | sean-k-mooney | mriedem: https://bugs.launchpad.net/nova/+bug/1751923/comments/10 | |
| 18:55:12 | openstack | Launchpad bug 1751923 in OpenStack Compute (nova) "_heal_instance_info_cache periodic task bases on port list from nova db, not from neutron server" [Medium,In progress] - Assigned to Maciej Jozefczyk (maciej.jozefczyk) | |