Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-11-20
15:55:30 openstackgerrit Merged openstack/nova master: Add description of custom resource classes https://review.openstack.org/616721
15:55:38 openstackgerrit Merged openstack/nova master: Add CellsV2 FAQ about API design decisions https://review.openstack.org/617898
16:09:23 Sundar jaypipes, dansmith, sean-k-mooney, cdent: Thanks for discussing the Nova-Cyborg spec in IRC y'day. I caught up with that. Will remove the Cyborg API signatures. and
16:10:19 Sundar I still have some questions on what jaypipes expects. The os-acc is not going to handle devices by itself. It neds access to Cyborg db and drivers, which means the majority of work will happen in Cyborg.
16:13:10 Sundar sean-k-mooney: Re. request groups in device profiles, it is still not clear to me how we would handle co-location without them, i.e., we want 2 accelerators from 2 different RPs in the same device.
16:17:08 mriedem os-acc is going to have direct db access to cyborg?
16:23:20 Sundar mdriedem: No. os-acc needs to call Cyborg REST APIs, and those calls do the bulk of the work.
16:23:50 jaypipes Sundar: currently on a call with sean-k-mooney and jangutter about os-vif. give me a little while to respond.
16:24:15 mriedem Sundar: ok, if os-acc were like os-brick and os-vif, i would expect it to deal with the physical devices on the host
16:24:24 mriedem and something like python-cyborgclient would be used for dealing with the cyborg rest API
16:24:27 mriedem or the openstacksdk
16:24:47 mriedem at least that's the model nova has for dealing with volumes and ports
16:24:54 Sundar jaypipes, Sure, NP
16:26:40 Sundar mriedem: I understand. os-acc is not an exact clone of os-vif or os-brick. For example, to bind an ARQ, a device may need to be configured or re-programmed. That requires a Cyborg driver which knows the details of that device.
16:27:04 Sundar mriedem: That is more like what a Neutron mechanism driver does.
16:27:42 mriedem hmm,
16:27:58 mriedem os-brick and os-vif definitely have plugins/drivers that do things based on the 'type' of device
16:28:07 mriedem but i'm just sitting in the peanut gallery here so ignore me
16:33:47 Sundar If we were to have separate drivers for os-acc and Cyborg, it would be cumbersome -- for example, tasks needed for device discovery/initialization (handled by Cyborg drivers) and tasks required for ARQ binding (initiated via os-acc) will have many commonalities. For instance, both may need ways to reset the device (or some part of it).
16:34:28 Sundar Apart from having two different driver installs/configures etc.
16:38:39 Sundar The os-vif plugins, from what I have seen, are handling Linux bridges, OVS, etc., not hardware per se.
16:41:11 mriedem i believe cinder (the service) uses os-brick
16:41:19 mriedem to avoid doing the same things in both places
16:41:24 mriedem jungleboyj: ^?
16:43:34 dansmith mriedem: Sundar I think it's entirely legit to think that not all device programming can be contained within os-acc
16:44:01 dansmith it's a lot more complicated of a thing than configuring an initiator or a bridge
16:44:21 Sundar dansmith: ^ +1
16:44:26 jungleboyj mriedem: You understanding is correct and we have different drivers in there depending on the type of device.
16:45:44 mriedem ok, again, peanut gallery
16:45:57 jungleboyj Both Cinder and Nova use os-brick so that we aren't duplicating code.
16:45:59 sean-k-mooney o/
16:46:20 jungleboyj That does the work locally on the compute node and then anything that required work from the volume driver is done through the Cinder-API.
16:47:40 dansmith if we use brick as the analogy,
16:47:53 dansmith it would be like putting all the stuff that talks to the backend volume providers into os-brick
16:49:02 sean-k-mooney dansmith: the programin of the device id not really done by cyborg either howver
16:49:15 sean-k-mooney it will be delegating the fpga progroming in the intel case to opae
16:49:35 Sundar sean-k-mooney: Not quite true.
16:49:35 dansmith sean-k-mooney: sure, just like cinder-volume doesn't actually sort the bits on the disk, but asks whatever api it has for the backend to do it
16:50:27 Sundar Cyborg will indeed call a Cyborg driver, which can call into device/vendor-specific drivers, such as OPAE or i915 (for GPUs), etc.
16:53:45 sean-k-mooney so im conused are we all happy with the statemet nova only interaction point with cyborg should be via os-acc and that os-acc should only interact with cyborg via its rest api
16:55:20 dansmith I guess I'm not sure what is confusing.. we interact with cinder via the cinderclient/brick
16:55:32 dansmith I think we're hoping that os-acc can serve both purposes, right?
16:55:42 mriedem that's what i'm hearing be described
16:55:51 mriedem os-acc is both rest api client and low-level host device thing
16:56:05 sean-k-mooney yes and that os-acc can hold the definitions of any data structre that nova and cyboge have to share
16:56:26 sean-k-mooney mriedem: yes to form no to later
16:56:37 mriedem what?
16:56:40 dansmith huh?
16:56:43 dansmith jinx
16:57:02 sean-k-mooney os-acc would only be a rest client
16:57:04 mriedem i got jinxed by a couple of 7 year old girls the other night, couldn't talk for 10 minutes, it was....not hard
16:57:31 sean-k-mooney the low level programing is handeled by the cyboge drivers and the tools they invoke
16:58:17 Sundar sean-k-mooney: True. But I think what is being said is that Nova views os-acc as performing the low-level tasks, i.e, it calls os-acc and leaves the details to it
16:58:23 jaypipes I really don't see why os-acc can't start off being a plug-the-device-into-the-VM library.
16:58:25 dansmith sean-k-mooney: I don't know why you're so intent on declaring that os-acc is only one thing or another
16:58:40 dansmith jaypipes: that's not a separate action
16:58:48 jaypipes dansmith: what do you mean?
16:58:57 dansmith jaypipes: plugging involves writing pci attachment into the virt xml for boot, right?
16:59:19 dansmith jaypipes: if there's any massaging of the device needed, like writing to sys to discover a new pci endpoint or something, then that seems like it could/should be in os-acc
16:59:23 sean-k-mooney jaypipes: would you be happy saying os-acc could start as beeing a program the device for the vm lib
16:59:46 mriedem in other news, we broke postgresql http://logs.openstack.org/periodic/git.openstack.org/openstack/neutron/master/neutron-tempest-postgres-full/a52bcf9/logs/screen-n-api.txt.gz?level=ERROR
17:00:28 jaypipes dansmith: yes, I agree with you. I just don't believe os-acc should be a REST API client to cyborg.
17:00:48 dansmith jaypipes: so we need a cyborgclient?
17:00:59 dansmith jaypipes: I'm not sure what os-acc would be doing if it's not talking to cyborg
17:02:08 sean-k-mooney dansmith: i would like to clearly scope what os-acc is so that i can understand what componets prefrom what actions
17:03:40 sean-k-mooney Sundar: the reason i care about if os-acc is just interacting with the rest api is because if it not. e.g it use the RPC bus or cyborge db directly then it has deployment impact
17:03:54 sean-k-mooney e.g. the credetials an connection info
17:04:09 dansmith sean-k-mooney: nobody is arguing for that are they?
17:04:12 sean-k-mooney if it interacts with the devices directly it has packaging impacts
17:04:17 Sundar sean-k-mooney: I agree. os-acc will indeed talk to Cyborg API, not the agents or drivers directly.
17:04:20 slaweq mriedem: bug reported: https://bugs.launchpad.net/nova/+bug/1804271
17:04:20 openstack Launchpad bug 1804271 in OpenStack Compute (nova) "nova-api is broken in postgresql jobs" [Undecided,New]
17:04:26 mriedem slaweq: thanks again
17:04:30 dansmith sean-k-mooney: crossing over to the db/mq between projects would be a major issue I think
17:04:39 jaypipes (sorry, folks, I'm in another meeting...)
17:05:04 slaweq yw :)
17:05:08 sean-k-mooney dansmith: in past version of the spec it was allowed to
17:05:21 Sundar Taking a step back: pretty much everything that needs to be done either requires Cyborg db access or device poking via Cyborg drivers.
17:06:05 dansmith Sundar: you understand you can't poke the cyborg db from os-acc or nova though right?
17:06:40 Sundar dansmith: Yea, thats why I am saying os-acc needs to call Cyborg API
17:07:03 dansmith yeah
17:07:19 dansmith I definitely never saw where in the spec it said that os-acc would talk directly to the cyborg db
17:07:20 sean-k-mooney Sundar: and simlarly do you want os-acc to be able interact with the device directly? i assume no
17:07:52 Sundar Yup, no direct access
17:08:14 mriedem i think the confusion was because of this question / statement earlier, which prompted me to ask about direct db access:
17:08:15 mriedem (10:10:21 AM) Sundar: I still have some questions on what jaypipes expects. The os-acc is not going to handle devices by itself. It neds access to Cyborg db and drivers, which means the majority of work will happen in Cyborg.
17:08:42 Sundar sean-k-mooney: Even in past specs, os-acc wouldn;t talk to CYborg drivers -- it was talking to Cyborg agent on the same compute node -- and that was all prior to the Stein PTG
17:08:43 dansmith right, which is saying "it can't, because .. access to db"
17:08:44 dansmith right?
17:09:01 mriedem yes i realize it meant, "access to the db and drivers via the cyborg rest api"
17:09:16 sean-k-mooney Sundar: that was going to be my next question
17:09:27 dansmith well, I think it means "it has to ask cyborg via api to do that, because only cyborg has access to the drivers and db" but.. same difference
17:09:41 sean-k-mooney yes at the ptg we said os-acc would not talk to the agent directly either
17:09:58 sean-k-mooney so does os-acc talk to anything other then the cyborge api
17:10:54 Sundar If you look at the os-acc API notes in the Nova spec, I have even identified which Cyborg API whill be called in each scenario
17:11:13 Sundar sean-k-mooney: No
17:11:55 sean-k-mooney ok so the statement i made earilar that os-acc will only talk to the rest-api and is not the lovel device lib was corect
17:12:13 mriedem in that case, why not just use openstacksdk?
17:12:27 mriedem i realize this is bike shedding a bit,
17:12:40 mriedem but os-acc makes me think of os-vif and os-brick which definitely do not call REST APIs in cinder or neutron,

Earlier   Later