Earlier  
Later
New Search
Posted Nick Remark
#openstack-cyborg - 2018-08-09
21:27:11 sean-k-mooney efried: when i was refering to running code on the hypervieros i was refing to the nova-compute agent and how it typically deployed on the hypervior with libvirt
21:27:47 sean-k-mooney im aware for ironic and orther hyperviros that the compute agent is not always colocated with the hypervior
21:28:42 sean-k-mooney although i also thend to reseve the them hypervior for the softare layer and not the phyical server that the vm/compute context executs on
21:29:33 efried By definition, the compute service runs on a system that knows how to talk to the hypervisor. Whether that's by exec'ing commands or communicating over a REST API. And either way, it's the responsibility of platform-specific code to do whatever that is.
21:30:49 Sundar_ efried: What is your take w.r.t Power/CAPI/IBM technologies on passing just the PCI address (or mediated device UUID, or some such) from os-acc to the virt driver?
21:31:15 sean-k-mooney efried: yes and by said definition the nova compute agent should only interfact with the phyical system through the api exposed by the hypervior that driver is talking to though there are some expetion to that in the libvirt case
21:32:37 sean-k-mooney Sundar_: in the CAPI case if this was libvirt kvm. the CAPI protocal would be impemnted between qemu and the device not libvirt and would require passing the memory regoins to qemu.
21:33:18 Sundar_ How would such device-specific data like memory regions be communicated to qemu?
21:33:26 sean-k-mooney Sundar_: we map a dma region in a simlar fashion for vhost user. in that case qemu passes the virt to phyical page traslation table to dpdk via the unix socket
21:33:42 efried Sundar_: Passing just the <identifier> to the virt driver for what operation? Plug? That works as long as there's some other place at which any necessary configuration has been done. Which again would need to be platform-specific code.
21:34:39 sean-k-mooney efried: vhost-user is useing a shared memory interface via dma to hugepages without ever passing that memory mapping to nova
21:35:11 sean-k-mooney the mappings are negociated between the qemu frontend and the dpdk backend
21:35:39 Sundar_ My model is that Cyborg discovers devices and their properties. There needs to be some way to communicate their properties and requirements to the hypervisor. We could extend the VAN with specific fields for each such property/requirement, or ...
21:35:58 Sundar_ we could use whatever the corresponding hypervisor already expects, as long as it is standard
21:38:56 sean-k-mooney how many different protocls do you expect to have to supprot
21:40:14 sean-k-mooney we will have mmio, ivshmem and capi but seam to take a simlar approch to do mmio via pci bars
21:41:15 sean-k-mooney we will have vfio-mediated devices
21:41:48 sean-k-mooney we will have pci passthough of different kinds.
21:41:53 Sundar_ That is because PCI MMIO is standardized. Whereas newer things like OpenCAPI and device-coherent SVM are all relatively new and likely to grow. The short answer is, I don't know how many such protocols there will be, nor do I think that anybody can tell you for sure today
21:43:23 sean-k-mooney Sundar_: so you are porposing creating a libary that needs to support not just multiple protocols, but multiple hyperviors, and multiple versions of those hyperviors instad of create an abstration layer container the common datataype
21:43:30 Sundar_ Going back to the Nova's philosophy, I think Nova wants to be agnostic of devices. Even things like NUMA affinity of devices may need to be modularized to minimize Nova involvement
21:46:06 Sundar_ 'support multiple hypervisors' -- yes, by using plugins/drivers. The existing Nova virt drivers are not (or should not be) device-aware. The os-acc plugins are expected to fill that need
21:47:44 sean-k-mooney https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/os-vif-library.html#proposed-change if your spec does not already contain a detailed breakdown of the split of responiblities as was done for os-vif can i ask that you add one
21:48:26 sean-k-mooney Sundar_: i think the only interface between opentack and the hypervior should be via the nova virt driver
21:49:05 sean-k-mooney at least wehn it comre to the virtualisation fo the guest
21:52:34 Sundar_ Yes, ultimately the Nova virt driver handles all hypervisor-related actions for launching an instance. The os-acc (or its plugins) are helpers -- specific areas are delegated by Nova virt to os-acc.
21:52:54 Sundar_ The os-acc should not directly modift the instance XML, for instanc
21:53:33 Sundar_ The split of ownership can be added to the spec. Sure.
21:53:43 Sundar_ Does that address your concern?
21:54:11 sean-k-mooney Sundar_: if the nova-virt driver calls oss acc and is returned an ovo with then the virt driver uses that to generate the xml or the equvalent for another driver then yes.
21:54:57 sean-k-mooney but only there is a clear seperation between preparing the device. and generating the hyperviors specific configration input
22:02:18 Sundar_ I am wondering how the separation would be so clear-cut for device/cpu/memory interaction, such as conveying device properties for NUMA etc. while keeping Nova device-agnostic. Anyways, I need to join a call now. We can touch base later.
22:03:27 Sundar_ Thanks, efried and sean-k-mooney
#openstack-cyborg - 2018-08-10
04:12:47 openstackgerrit Merged openstack/cyborg master: Fix deployable get all sort issue and unit test failure. https://review.openstack.org/590075
05:17:47 openstackgerrit YumengBao proposed openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590647
05:26:52 openstackgerrit YumengBao proposed openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590647
08:02:17 openstackgerrit ShaoHe Feng proposed openstack/cyborg master: support sub provider https://review.openstack.org/585146
08:54:10 openstackgerrit YumengBao proposed openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590706
08:55:13 openstackgerrit YumengBao proposed openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590706
12:07:53 openstackgerrit zhurong proposed openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590706
14:43:11 openstackgerrit Merged openstack/cyborg master: support sub provider https://review.openstack.org/585146
16:06:10 annabelleB zhipengh[m]: I’m reviewing your cycle-highlights now; does the “quota support” you mention on line 30 correspond to this review? https://review.openstack.org/#/c/564968/
#openstack-cyborg - 2018-08-11
04:28:15 openstackgerrit YumengBao proposed openstack/cyborg master: Update html_theme so cyborg doc page displays in standard page theme https://review.openstack.org/591120
04:56:03 openstackgerrit YumengBao proposed openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590706
#openstack-cyborg - 2018-08-13
08:27:55 openstackgerrit Merged openstack/cyborg master: Update html_theme so cyborg doc page displays in standard page theme https://review.openstack.org/591120
10:26:14 openstackgerrit Merged openstack/os-acc master: Update reno for stable/rocky https://review.openstack.org/584362
13:37:10 openstackgerrit Merged openstack/cyborg master: Follow the new PTI for document build https://review.openstack.org/590706
14:01:10 shaohe_feng #startmeeting openstack-cyborg-driver
14:01:11 openstack Meeting started Mon Aug 13 14:01:10 2018 UTC and is due to finish in 60 minutes. The chair is shaohe_feng. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:13 shaohe_feng #topic Roll Call
14:01:13 openstack Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:15 openstack The meeting name has been set to 'openstack_cyborg_driver'
14:01:40 shaohe_feng #info shaohe_feng
14:02:16 edleafe #info ed leafe
14:05:13 Sundar #info Sundar
14:05:54 shaohe_feng Sundar, edleafe morning
14:06:10 edleafe good UGT morning to you!
14:06:36 shaohe_feng :)
14:06:43 Sundar Good day, Shaohe
14:07:50 shaohe_feng edleafe, Sundar now only we three on line.
14:08:28 Li_Liu #info Li_Liu
14:08:40 Li_Liu do we still have the meeting today?
14:08:41 Sundar shaohe: I had sent an email yesterday to openstack-dev. Shall we talk about that?
14:09:01 shaohe_feng Sundar: OK, go ahead.
14:09:16 shaohe_feng Li_Liu: morning.
14:09:46 shaohe_feng good evening, xinran
14:09:48 Li_Liu shaohe_feng: good evening :P
14:09:58 xinran Hi all
14:10:33 shaohe_feng Sundar: You can talk about your email
14:10:40 Sundar It was apparently decided in some meeting that, to record the discovered devices, Cyborg agent will call Cyborg REST API. Also, to allocate and deallocate accelerators. If so, that will make the public and ha many disadvantages.
14:11:45 Sundar Among other things, it means it is not internal to Cyborg any more. Any user can call it. So, we should authenticate. Even if we open it only to operators, it is still error-prone. We can just keep it Cyborg-internal, right?
14:12:43 xinran When Cyborg agent will call restful API?
14:13:38 shaohe_feng Cyborg agent should not call Cyborg REST API.
14:13:53 Li_Liu Agent should stay with rpc
14:13:59 Sundar xinran: Rest API is meant for things that can be accessed by external users, operators or other services.
14:14:44 xinran Sundar: can you give us an example when agent call restful api
14:15:31 Sundar shaohe, Li_Liu: Agreed. I amreferring to #link https://etherpad.openstack.org/p/cyborg-rocky-development (Line 44)
14:15:38 wangzhh Hi all. Test connection.
14:16:03 xinran IMHO agent should not call restful api
14:16:32 Li_Liu Sundar, that is exposed by cyborg-api, not meat to be used by Agent
14:17:46 Sundar Li_Liu: Was that agreed only for GET, not for PUT/POST?
14:18:19 shaohe_feng wangzhh: good evening.
14:19:10 wangzhh shaohe_feng: good evening. :)
14:19:15 Li_Liu Sundar, PUT/POST are open to admin I think, in case they want to tune some of the deployables
14:20:18 Sundar Li_Liu: the issue there is that we will have to maintain backwards compat for REST APIs. That is going to constrain our development and/or cause upgrade issues.
14:21:13 Li_Liu wangzhh, do you have comments on this?
14:22:00 wangzhh Sundar, what do you mean about "we will have to maintain backwards compat for REST APIs. That is going to constrain our development and/or cause upgrade issues."
14:23:11 wangzhh Can u give me an example which current APIs can not handle?
14:23:32 Sundar wangzhh: In #link https://etherpad.openstack.org/p/cyborg-rocky-development (Line 44), if we allow PUT/POST in addition to get, we may have such considerations
14:25:15 wangzhh We have PATCH now. If we want to update deployable. we can use it.
14:25:46 shaohe_feng sure we should allow PUT/POST, that depends on
14:26:09 Sundar wangzhh: Can you elaborate?
14:26:35 wangzhh shaohe_feng: Do we need create deployable via API?
14:27:52 shaohe_feng That depends on.
14:28:07 Li_Liu wangzhh, maybe we should allow that for now
14:28:23 shaohe_feng at present, agent creates deployable.
14:28:30 Li_Liu to let admin able to add stuff in manaully
14:29:02 Sundar Li_Liu, shaohe: can you explain why that is necessary? And how we will ensure backwards compat and avoid upgrade issues?
14:29:04 wangzhh Sundar: https://github.com/openstack/cyborg/blob/719f3dee01b6f0b0f4f2ce9b45ffd4978eeac287/cyborg/api/controllers/v1/deployables.py#L217
14:30:03 shaohe_feng Sundar: we should allow users to update some attribution of a deployable

Earlier   Later