| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-20 | |||
| 15:48:13 | prometheanfire | the update is being held back atm by reqs cross gating | |
| 15:49:48 | prometheanfire | question is, are old versions of nova going to work with the oslo.service change (18.0.2 and the like), it sounds like not, which means packagers should be made aware | |
| 15:49:56 | sean-k-mooney | prometheanfire: https://review.openstack.org/#/c/619019/1 and https://review.openstack.org/#/c/619022/1 will fix the nova compatiblity | |
| 15:50:47 | sean-k-mooney | prometheanfire: old versions of nova would work but the unites would not which may break packager build systems | |
| 15:52:39 | prometheanfire | unites / unit tests? | |
| 15:55:30 | openstackgerrit | Merged openstack/nova master: Add description of custom resource classes https://review.openstack.org/616721 | |
| 15:55:38 | openstackgerrit | Merged openstack/nova master: Add CellsV2 FAQ about API design decisions https://review.openstack.org/617898 | |
| 16:09:23 | Sundar | jaypipes, dansmith, sean-k-mooney, cdent: Thanks for discussing the Nova-Cyborg spec in IRC y'day. I caught up with that. Will remove the Cyborg API signatures. and | |
| 16:10:19 | Sundar | I still have some questions on what jaypipes expects. The os-acc is not going to handle devices by itself. It neds access to Cyborg db and drivers, which means the majority of work will happen in Cyborg. | |
| 16:13:10 | Sundar | sean-k-mooney: Re. request groups in device profiles, it is still not clear to me how we would handle co-location without them, i.e., we want 2 accelerators from 2 different RPs in the same device. | |
| 16:17:08 | mriedem | os-acc is going to have direct db access to cyborg? | |
| 16:23:20 | Sundar | mdriedem: No. os-acc needs to call Cyborg REST APIs, and those calls do the bulk of the work. | |
| 16:23:50 | jaypipes | Sundar: currently on a call with sean-k-mooney and jangutter about os-vif. give me a little while to respond. | |
| 16:24:15 | mriedem | Sundar: ok, if os-acc were like os-brick and os-vif, i would expect it to deal with the physical devices on the host | |
| 16:24:24 | mriedem | and something like python-cyborgclient would be used for dealing with the cyborg rest API | |
| 16:24:27 | mriedem | or the openstacksdk | |
| 16:24:47 | mriedem | at least that's the model nova has for dealing with volumes and ports | |
| 16:24:54 | Sundar | jaypipes, Sure, NP | |
| 16:26:40 | Sundar | mriedem: I understand. os-acc is not an exact clone of os-vif or os-brick. For example, to bind an ARQ, a device may need to be configured or re-programmed. That requires a Cyborg driver which knows the details of that device. | |
| 16:27:04 | Sundar | mriedem: That is more like what a Neutron mechanism driver does. | |
| 16:27:42 | mriedem | hmm, | |
| 16:27:58 | mriedem | os-brick and os-vif definitely have plugins/drivers that do things based on the 'type' of device | |
| 16:28:07 | mriedem | but i'm just sitting in the peanut gallery here so ignore me | |
| 16:33:47 | Sundar | If we were to have separate drivers for os-acc and Cyborg, it would be cumbersome -- for example, tasks needed for device discovery/initialization (handled by Cyborg drivers) and tasks required for ARQ binding (initiated via os-acc) will have many commonalities. For instance, both may need ways to reset the device (or some part of it). | |
| 16:34:28 | Sundar | Apart from having two different driver installs/configures etc. | |
| 16:38:39 | Sundar | The os-vif plugins, from what I have seen, are handling Linux bridges, OVS, etc., not hardware per se. | |
| 16:41:11 | mriedem | i believe cinder (the service) uses os-brick | |
| 16:41:19 | mriedem | to avoid doing the same things in both places | |
| 16:41:24 | mriedem | jungleboyj: ^? | |
| 16:43:34 | dansmith | mriedem: Sundar I think it's entirely legit to think that not all device programming can be contained within os-acc | |
| 16:44:01 | dansmith | it's a lot more complicated of a thing than configuring an initiator or a bridge | |
| 16:44:21 | Sundar | dansmith: ^ +1 | |
| 16:44:26 | jungleboyj | mriedem: You understanding is correct and we have different drivers in there depending on the type of device. | |
| 16:45:44 | mriedem | ok, again, peanut gallery | |
| 16:45:57 | jungleboyj | Both Cinder and Nova use os-brick so that we aren't duplicating code. | |
| 16:45:59 | sean-k-mooney | o/ | |
| 16:46:20 | jungleboyj | That does the work locally on the compute node and then anything that required work from the volume driver is done through the Cinder-API. | |
| 16:47:40 | dansmith | if we use brick as the analogy, | |
| 16:47:53 | dansmith | it would be like putting all the stuff that talks to the backend volume providers into os-brick | |
| 16:49:02 | sean-k-mooney | dansmith: the programin of the device id not really done by cyborg either howver | |
| 16:49:15 | sean-k-mooney | it will be delegating the fpga progroming in the intel case to opae | |
| 16:49:35 | Sundar | sean-k-mooney: Not quite true. | |
| 16:49:35 | dansmith | sean-k-mooney: sure, just like cinder-volume doesn't actually sort the bits on the disk, but asks whatever api it has for the backend to do it | |
| 16:50:27 | Sundar | Cyborg will indeed call a Cyborg driver, which can call into device/vendor-specific drivers, such as OPAE or i915 (for GPUs), etc. | |
| 16:53:45 | sean-k-mooney | so im conused are we all happy with the statemet nova only interaction point with cyborg should be via os-acc and that os-acc should only interact with cyborg via its rest api | |
| 16:55:20 | dansmith | I guess I'm not sure what is confusing.. we interact with cinder via the cinderclient/brick | |
| 16:55:32 | dansmith | I think we're hoping that os-acc can serve both purposes, right? | |
| 16:55:42 | mriedem | that's what i'm hearing be described | |
| 16:55:51 | mriedem | os-acc is both rest api client and low-level host device thing | |
| 16:56:05 | sean-k-mooney | yes and that os-acc can hold the definitions of any data structre that nova and cyboge have to share | |
| 16:56:26 | sean-k-mooney | mriedem: yes to form no to later | |
| 16:56:37 | mriedem | what? | |
| 16:56:40 | dansmith | huh? | |
| 16:56:43 | dansmith | jinx | |
| 16:57:02 | sean-k-mooney | os-acc would only be a rest client | |
| 16:57:04 | mriedem | i got jinxed by a couple of 7 year old girls the other night, couldn't talk for 10 minutes, it was....not hard | |
| 16:57:31 | sean-k-mooney | the low level programing is handeled by the cyboge drivers and the tools they invoke | |
| 16:58:17 | Sundar | sean-k-mooney: True. But I think what is being said is that Nova views os-acc as performing the low-level tasks, i.e, it calls os-acc and leaves the details to it | |
| 16:58:23 | jaypipes | I really don't see why os-acc can't start off being a plug-the-device-into-the-VM library. | |
| 16:58:25 | dansmith | sean-k-mooney: I don't know why you're so intent on declaring that os-acc is only one thing or another | |
| 16:58:40 | dansmith | jaypipes: that's not a separate action | |
| 16:58:48 | jaypipes | dansmith: what do you mean? | |
| 16:58:57 | dansmith | jaypipes: plugging involves writing pci attachment into the virt xml for boot, right? | |
| 16:59:19 | dansmith | jaypipes: if there's any massaging of the device needed, like writing to sys to discover a new pci endpoint or something, then that seems like it could/should be in os-acc | |
| 16:59:23 | sean-k-mooney | jaypipes: would you be happy saying os-acc could start as beeing a program the device for the vm lib | |
| 16:59:46 | mriedem | in other news, we broke postgresql http://logs.openstack.org/periodic/git.openstack.org/openstack/neutron/master/neutron-tempest-postgres-full/a52bcf9/logs/screen-n-api.txt.gz?level=ERROR | |
| 17:00:28 | jaypipes | dansmith: yes, I agree with you. I just don't believe os-acc should be a REST API client to cyborg. | |
| 17:00:48 | dansmith | jaypipes: so we need a cyborgclient? | |
| 17:00:59 | dansmith | jaypipes: I'm not sure what os-acc would be doing if it's not talking to cyborg | |
| 17:02:08 | sean-k-mooney | dansmith: i would like to clearly scope what os-acc is so that i can understand what componets prefrom what actions | |
| 17:03:40 | sean-k-mooney | Sundar: the reason i care about if os-acc is just interacting with the rest api is because if it not. e.g it use the RPC bus or cyborge db directly then it has deployment impact | |
| 17:03:54 | sean-k-mooney | e.g. the credetials an connection info | |
| 17:04:09 | dansmith | sean-k-mooney: nobody is arguing for that are they? | |
| 17:04:12 | sean-k-mooney | if it interacts with the devices directly it has packaging impacts | |
| 17:04:17 | Sundar | sean-k-mooney: I agree. os-acc will indeed talk to Cyborg API, not the agents or drivers directly. | |
| 17:04:20 | slaweq | mriedem: bug reported: https://bugs.launchpad.net/nova/+bug/1804271 | |
| 17:04:20 | openstack | Launchpad bug 1804271 in OpenStack Compute (nova) "nova-api is broken in postgresql jobs" [Undecided,New] | |
| 17:04:26 | mriedem | slaweq: thanks again | |
| 17:04:30 | dansmith | sean-k-mooney: crossing over to the db/mq between projects would be a major issue I think | |
| 17:04:39 | jaypipes | (sorry, folks, I'm in another meeting...) | |
| 17:05:04 | slaweq | yw :) | |
| 17:05:08 | sean-k-mooney | dansmith: in past version of the spec it was allowed to | |
| 17:05:21 | Sundar | Taking a step back: pretty much everything that needs to be done either requires Cyborg db access or device poking via Cyborg drivers. | |
| 17:06:05 | dansmith | Sundar: you understand you can't poke the cyborg db from os-acc or nova though right? | |
| 17:06:40 | Sundar | dansmith: Yea, thats why I am saying os-acc needs to call Cyborg API | |
| 17:07:03 | dansmith | yeah | |
| 17:07:19 | dansmith | I definitely never saw where in the spec it said that os-acc would talk directly to the cyborg db | |
| 17:07:20 | sean-k-mooney | Sundar: and simlarly do you want os-acc to be able interact with the device directly? i assume no | |
| 17:07:52 | Sundar | Yup, no direct access | |
| 17:08:14 | mriedem | i think the confusion was because of this question / statement earlier, which prompted me to ask about direct db access: | |
| 17:08:15 | mriedem | (10:10:21 AM) Sundar: I still have some questions on what jaypipes expects. The os-acc is not going to handle devices by itself. It neds access to Cyborg db and drivers, which means the majority of work will happen in Cyborg. | |
| 17:08:42 | Sundar | sean-k-mooney: Even in past specs, os-acc wouldn;t talk to CYborg drivers -- it was talking to Cyborg agent on the same compute node -- and that was all prior to the Stein PTG | |
| 17:08:43 | dansmith | right, which is saying "it can't, because .. access to db" | |
| 17:08:44 | dansmith | right? | |
| 17:09:01 | mriedem | yes i realize it meant, "access to the db and drivers via the cyborg rest api" | |
| 17:09:16 | sean-k-mooney | Sundar: that was going to be my next question | |
| 17:09:27 | dansmith | well, I think it means "it has to ask cyborg via api to do that, because only cyborg has access to the drivers and db" but.. same difference | |
| 17:09:41 | sean-k-mooney | yes at the ptg we said os-acc would not talk to the agent directly either | |
| 17:09:58 | sean-k-mooney | so does os-acc talk to anything other then the cyborge api | |
| 17:10:54 | Sundar | If you look at the os-acc API notes in the Nova spec, I have even identified which Cyborg API whill be called in each scenario | |