Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-11-20
15:26:59 BjoernT Hello, Is someone here aware of the implementation of ComputeManager._run_image_cache_manag as we run in to performance issues on a NFS mounted /var/lib/nova/instances directory and now had to increase rpc response timeout?
15:32:12 sean-k-mooney that ^ sound like an mdbooth kind of question but he does not seam to be about currently
15:38:57 openstackgerrit Hervé Beraud proposed openstack/nova stable/rocky: Use SleepFixture instead of mocking _ThreadingEvent.wait https://review.openstack.org/619022
15:42:25 openstackgerrit Jack Ding proposed openstack/nova master: Add I/O Semaphore to limit concurrent disk ops https://review.openstack.org/609180
15:47:30 prometheanfire efried_pto: sean-k-mooney I'd say we are fine for now, nova may want to add a exclusion to it's reqs.txt, or not
15:48:13 prometheanfire the update is being held back atm by reqs cross gating
15:49:48 prometheanfire question is, are old versions of nova going to work with the oslo.service change (18.0.2 and the like), it sounds like not, which means packagers should be made aware
15:49:56 sean-k-mooney prometheanfire: https://review.openstack.org/#/c/619019/1 and https://review.openstack.org/#/c/619022/1 will fix the nova compatiblity
15:50:47 sean-k-mooney prometheanfire: old versions of nova would work but the unites would not which may break packager build systems
15:52:39 prometheanfire unites / unit tests?
15:55:30 openstackgerrit Merged openstack/nova master: Add description of custom resource classes https://review.openstack.org/616721
15:55:38 openstackgerrit Merged openstack/nova master: Add CellsV2 FAQ about API design decisions https://review.openstack.org/617898
16:09:23 Sundar jaypipes, dansmith, sean-k-mooney, cdent: Thanks for discussing the Nova-Cyborg spec in IRC y'day. I caught up with that. Will remove the Cyborg API signatures. and
16:10:19 Sundar I still have some questions on what jaypipes expects. The os-acc is not going to handle devices by itself. It neds access to Cyborg db and drivers, which means the majority of work will happen in Cyborg.
16:13:10 Sundar sean-k-mooney: Re. request groups in device profiles, it is still not clear to me how we would handle co-location without them, i.e., we want 2 accelerators from 2 different RPs in the same device.
16:17:08 mriedem os-acc is going to have direct db access to cyborg?
16:23:20 Sundar mdriedem: No. os-acc needs to call Cyborg REST APIs, and those calls do the bulk of the work.
16:23:50 jaypipes Sundar: currently on a call with sean-k-mooney and jangutter about os-vif. give me a little while to respond.
16:24:15 mriedem Sundar: ok, if os-acc were like os-brick and os-vif, i would expect it to deal with the physical devices on the host
16:24:24 mriedem and something like python-cyborgclient would be used for dealing with the cyborg rest API
16:24:27 mriedem or the openstacksdk
16:24:47 mriedem at least that's the model nova has for dealing with volumes and ports
16:24:54 Sundar jaypipes, Sure, NP
16:26:40 Sundar mriedem: I understand. os-acc is not an exact clone of os-vif or os-brick. For example, to bind an ARQ, a device may need to be configured or re-programmed. That requires a Cyborg driver which knows the details of that device.
16:27:04 Sundar mriedem: That is more like what a Neutron mechanism driver does.
16:27:42 mriedem hmm,
16:27:58 mriedem os-brick and os-vif definitely have plugins/drivers that do things based on the 'type' of device
16:28:07 mriedem but i'm just sitting in the peanut gallery here so ignore me
16:33:47 Sundar If we were to have separate drivers for os-acc and Cyborg, it would be cumbersome -- for example, tasks needed for device discovery/initialization (handled by Cyborg drivers) and tasks required for ARQ binding (initiated via os-acc) will have many commonalities. For instance, both may need ways to reset the device (or some part of it).
16:34:28 Sundar Apart from having two different driver installs/configures etc.
16:38:39 Sundar The os-vif plugins, from what I have seen, are handling Linux bridges, OVS, etc., not hardware per se.
16:41:11 mriedem i believe cinder (the service) uses os-brick
16:41:19 mriedem to avoid doing the same things in both places
16:41:24 mriedem jungleboyj: ^?
16:43:34 dansmith mriedem: Sundar I think it's entirely legit to think that not all device programming can be contained within os-acc
16:44:01 dansmith it's a lot more complicated of a thing than configuring an initiator or a bridge
16:44:21 Sundar dansmith: ^ +1
16:44:26 jungleboyj mriedem: You understanding is correct and we have different drivers in there depending on the type of device.
16:45:44 mriedem ok, again, peanut gallery
16:45:57 jungleboyj Both Cinder and Nova use os-brick so that we aren't duplicating code.
16:45:59 sean-k-mooney o/
16:46:20 jungleboyj That does the work locally on the compute node and then anything that required work from the volume driver is done through the Cinder-API.
16:47:40 dansmith if we use brick as the analogy,
16:47:53 dansmith it would be like putting all the stuff that talks to the backend volume providers into os-brick
16:49:02 sean-k-mooney dansmith: the programin of the device id not really done by cyborg either howver
16:49:15 sean-k-mooney it will be delegating the fpga progroming in the intel case to opae
16:49:35 dansmith sean-k-mooney: sure, just like cinder-volume doesn't actually sort the bits on the disk, but asks whatever api it has for the backend to do it
16:49:35 Sundar sean-k-mooney: Not quite true.
16:50:27 Sundar Cyborg will indeed call a Cyborg driver, which can call into device/vendor-specific drivers, such as OPAE or i915 (for GPUs), etc.
16:53:45 sean-k-mooney so im conused are we all happy with the statemet nova only interaction point with cyborg should be via os-acc and that os-acc should only interact with cyborg via its rest api
16:55:20 dansmith I guess I'm not sure what is confusing.. we interact with cinder via the cinderclient/brick
16:55:32 dansmith I think we're hoping that os-acc can serve both purposes, right?
16:55:42 mriedem that's what i'm hearing be described
16:55:51 mriedem os-acc is both rest api client and low-level host device thing
16:56:05 sean-k-mooney yes and that os-acc can hold the definitions of any data structre that nova and cyboge have to share
16:56:26 sean-k-mooney mriedem: yes to form no to later
16:56:37 mriedem what?
16:56:40 dansmith huh?
16:56:43 dansmith jinx
16:57:02 sean-k-mooney os-acc would only be a rest client
16:57:04 mriedem i got jinxed by a couple of 7 year old girls the other night, couldn't talk for 10 minutes, it was....not hard
16:57:31 sean-k-mooney the low level programing is handeled by the cyboge drivers and the tools they invoke
16:58:17 Sundar sean-k-mooney: True. But I think what is being said is that Nova views os-acc as performing the low-level tasks, i.e, it calls os-acc and leaves the details to it
16:58:23 jaypipes I really don't see why os-acc can't start off being a plug-the-device-into-the-VM library.
16:58:25 dansmith sean-k-mooney: I don't know why you're so intent on declaring that os-acc is only one thing or another
16:58:40 dansmith jaypipes: that's not a separate action
16:58:48 jaypipes dansmith: what do you mean?
16:58:57 dansmith jaypipes: plugging involves writing pci attachment into the virt xml for boot, right?
16:59:19 dansmith jaypipes: if there's any massaging of the device needed, like writing to sys to discover a new pci endpoint or something, then that seems like it could/should be in os-acc
16:59:23 sean-k-mooney jaypipes: would you be happy saying os-acc could start as beeing a program the device for the vm lib
16:59:46 mriedem in other news, we broke postgresql http://logs.openstack.org/periodic/git.openstack.org/openstack/neutron/master/neutron-tempest-postgres-full/a52bcf9/logs/screen-n-api.txt.gz?level=ERROR
17:00:28 jaypipes dansmith: yes, I agree with you. I just don't believe os-acc should be a REST API client to cyborg.
17:00:48 dansmith jaypipes: so we need a cyborgclient?
17:00:59 dansmith jaypipes: I'm not sure what os-acc would be doing if it's not talking to cyborg
17:02:08 sean-k-mooney dansmith: i would like to clearly scope what os-acc is so that i can understand what componets prefrom what actions
17:03:40 sean-k-mooney Sundar: the reason i care about if os-acc is just interacting with the rest api is because if it not. e.g it use the RPC bus or cyborge db directly then it has deployment impact
17:03:54 sean-k-mooney e.g. the credetials an connection info
17:04:09 dansmith sean-k-mooney: nobody is arguing for that are they?
17:04:12 sean-k-mooney if it interacts with the devices directly it has packaging impacts
17:04:17 Sundar sean-k-mooney: I agree. os-acc will indeed talk to Cyborg API, not the agents or drivers directly.
17:04:20 openstack Launchpad bug 1804271 in OpenStack Compute (nova) "nova-api is broken in postgresql jobs" [Undecided,New]
17:04:20 slaweq mriedem: bug reported: https://bugs.launchpad.net/nova/+bug/1804271
17:04:26 mriedem slaweq: thanks again
17:04:30 dansmith sean-k-mooney: crossing over to the db/mq between projects would be a major issue I think
17:04:39 jaypipes (sorry, folks, I'm in another meeting...)
17:05:04 slaweq yw :)
17:05:08 sean-k-mooney dansmith: in past version of the spec it was allowed to
17:05:21 Sundar Taking a step back: pretty much everything that needs to be done either requires Cyborg db access or device poking via Cyborg drivers.
17:06:05 dansmith Sundar: you understand you can't poke the cyborg db from os-acc or nova though right?
17:06:40 Sundar dansmith: Yea, thats why I am saying os-acc needs to call Cyborg API
17:07:03 dansmith yeah
17:07:19 dansmith I definitely never saw where in the spec it said that os-acc would talk directly to the cyborg db
17:07:20 sean-k-mooney Sundar: and simlarly do you want os-acc to be able interact with the device directly? i assume no
17:07:52 Sundar Yup, no direct access
17:08:14 mriedem i think the confusion was because of this question / statement earlier, which prompted me to ask about direct db access:
17:08:15 mriedem (10:10:21 AM) Sundar: I still have some questions on what jaypipes expects. The os-acc is not going to handle devices by itself. It neds access to Cyborg db and drivers, which means the majority of work will happen in Cyborg.
17:08:42 Sundar sean-k-mooney: Even in past specs, os-acc wouldn;t talk to CYborg drivers -- it was talking to Cyborg agent on the same compute node -- and that was all prior to the Stein PTG
17:08:43 dansmith right, which is saying "it can't, because .. access to db"
17:08:44 dansmith right?
17:09:01 mriedem yes i realize it meant, "access to the db and drivers via the cyborg rest api"

Earlier   Later