| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2017-11-15 | |||
| 15:31:38 | LiLiu_ | liliu1@huawei.com | |
| 15:33:04 | zhipeng | did some googling | |
| 15:33:05 | LiLiu_ | I wanna know more about the scope/size of the system you guys are targeting | |
| 15:33:28 | masuberu | it will depends on the performance of this tests | |
| 15:33:29 | zhipeng | seems like the pairHMM will be the part that need acceleration from FPGA or GPU ? | |
| 15:34:09 | masuberu | for instance we have capacity to sequence 1200 genomes per month so it all depends how much hardware we need for that | |
| 15:34:31 | zhipeng | #link https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/wp/wp-accelerating-genomics-opencl-fpgas.pdf | |
| 15:34:47 | shaohe_feng__ | zhipeng: actually, every accelerator in Intel's FPGA is a sriov device. | |
| 15:35:02 | masuberu | zhipeng, yep I got that document yesterday from Albert | |
| 15:35:22 | zhipeng | shaohe_feng__ how about the altera ones ? | |
| 15:35:26 | zhipeng | the same ? | |
| 15:35:37 | shaohe_feng__ | zhipeng: altera means intel :) | |
| 15:35:54 | zhipeng | haha | |
| 15:36:16 | shaohe_feng__ | zhipeng: there maybe 3 method to support FPGA. | |
| 15:36:16 | masuberu | nallatech 385A is what I am getting | |
| 15:36:30 | masuberu | getting for testing | |
| 15:36:54 | shaohe_feng__ | zhipeng: 1. pre-programming FPGA on the host. | |
| 15:38:02 | shaohe_feng__ | zhipeng: 2. the orchestration program FPGA | |
| 15:38:15 | shaohe_feng__ | zhipeng: 3. VM program FPGA. | |
| 15:38:36 | shaohe_feng__ | zhipeng: I have got a FPGA server, and I'm trying to do some test on it | |
| 15:38:54 | masuberu | shaohe_feng__, the person coming tomorrow is called AlbertZQ Wang, not sure if you know him... | |
| 15:39:25 | shaohe_feng__ | zhipeng: AlbertZQ Wang from intel? | |
| 15:39:36 | shaohe_feng__ | ^ masuberu: | |
| 15:39:52 | masuberu | yes | |
| 15:40:02 | shaohe_feng__ | which site? China or US | |
| 15:40:17 | masuberu | I guess China but not 100% sure | |
| 15:40:22 | masuberu | maybe either of them | |
| 15:41:48 | masuberu | any questions? | |
| 15:41:53 | shaohe_feng__ | masuberu: haha, we can connect each other convenience in intel's internal connection tool. | |
| 15:42:23 | masuberu | yeah they showed me that tool last time they came to my office | |
| 15:42:38 | masuberu | any plan? | |
| 15:42:59 | zhipeng | i think this is definitely a very interesting use case for us | |
| 15:43:22 | zhipeng | it would be great if you could come back with more details from the Intel guys tomorrow | |
| 15:43:34 | zhipeng | and we could hammer out details for future work | |
| 15:43:49 | zhipeng | i think getting Nova connect with FPGA is easy | |
| 15:44:15 | zhipeng | but how to sched and manage your FPGAs is what we will be working on | |
| 15:44:21 | masuberu | zhipeng, you mean because of SRIOV? | |
| 15:44:49 | masuberu | it can be nova or k8s | |
| 15:44:55 | masuberu | or another method | |
| 15:45:05 | zhipeng | yes because the current support for Nova is good enough | |
| 15:45:08 | zhipeng | to set it up | |
| 15:45:11 | masuberu | ok | |
| 15:45:18 | masuberu | what about overhead? | |
| 15:45:36 | zhipeng | what type of overhead you are thinking about ? | |
| 15:45:38 | masuberu | is it going to have much overhead due to virtualization? | |
| 15:45:50 | zhipeng | i think that depends | |
| 15:46:05 | zhipeng | how you setup the workload between VM and FPGA | |
| 15:46:37 | masuberu | ok, is there anything I can read about that? because I don't know much to be honest | |
| 15:46:46 | zhipeng | for example if you gonna have FPGA do all the GATK heavy lifting and VM just for running a data pipe application | |
| 15:46:53 | masuberu | and I want to get the most of this project | |
| 15:47:04 | zhipeng | i don't think there will be much overhead | |
| 15:47:05 | masuberu | I don't know for how long I can have the FPGAs | |
| 15:47:54 | masuberu | lets say we decide to run on the numa nodes, then I can even enable cpu afinity | |
| 15:48:10 | masuberu | to make sure process use local memory | |
| 15:48:16 | masuberu | but that is a different story | |
| 15:48:31 | zhipeng | yes | |
| 15:48:48 | masuberu | I am more worry about the lack of documentation about openstack+FPGAs | |
| 15:49:16 | masuberu | also we have another project coming to run GPUs for machine learning stuff | |
| 15:49:16 | zhipeng | you can just go through the nova pci support docs | |
| 15:49:39 | masuberu | but that will come in 3-4 months time | |
| 15:49:46 | masuberu | ok | |
| 15:49:50 | zhipeng | cool :) | |
| 15:50:00 | masuberu | another thing I normally deploy openstack using kolla... | |
| 15:50:10 | masuberu | kolla-ansible, not sure if that will be an issue | |
| 15:50:14 | zhipeng | crushil | |
| 15:50:29 | zhipeng | any thoughts on making cyborg kolla-able ? :P | |
| 15:51:01 | crushil | zhipeng, My coworker who presented for us is kolla-kubernetes core | |
| 15:51:12 | masuberu | nice | |
| 15:51:13 | crushil | I'm working with him to get Cyborg kollafied | |
| 15:51:24 | masuberu | that's cool | |
| 15:51:24 | zhipeng | masuberu there you go :) | |
| 15:51:32 | zhipeng | we have everyone you need in cyborg :P | |
| 15:51:36 | crushil | It's already on my plate for a future sprint | |
| 15:51:52 | masuberu | I am quite flexible we can do whatever we want with this nodes as they are running nothing | |
| 15:52:19 | zhipeng | masuberu just give a howler any time you have an issue on this channel | |
| 15:52:45 | zhipeng | core members are always logged in , so as long as any of us are awake, we will respond :) | |
| 15:52:47 | crushil | Btw you should play with this tool https://review.openstack.org/#/c/487972/. I will be integrating Cyborg into this tool | |
| 15:53:13 | zhipeng | you could also just send email to the openstack-dev mailinglist, or openstack-ops | |
| 15:55:07 | masuberu | I think I know rwellum | |
| 15:55:33 | masuberu | ok | |
| 15:56:02 | masuberu | by the way GATK4 is just a jar file | |
| 15:56:10 | masuberu | very simple to install and run | |
| 15:57:04 | masuberu | is it possible to share an FPGU with multiple instances? | |
| 15:57:25 | masuberu | or do I need to assign 1 FPGU card per vm? | |
| 15:57:31 | LiLiu_ | if the FPGA has multiple PFs/VFs, then yes | |
| 15:58:08 | LiLiu_ | It really depends on how the IP is developed | |
| 15:58:45 | masuberu | ok, and the FPGUs has cores? | |
| 15:59:29 | LiLiu_ | nope | |
| 15:59:44 | masuberu | I mean can I break the resources of a FPGU card and assign it to multiple vms? | |
| 15:59:50 | LiLiu_ | FPGA can be virtualized and shared by multiple VMs | |
| 16:00:01 | LiLiu_ | you are right | |
| 16:00:40 | masuberu | for example with memory and cpus I can "define" flavors, can I do same with FPGUs? | |
| 16:01:08 | LiLiu_ | yup | |
| 16:01:31 | LiLiu_ | That's one of the most common ways of managing FPGA in Openstack | |
| 16:01:46 | masuberu | like for instance 0.3Teraflops for one instance and 0.7Teraflops to another | |
| 16:01:49 | masuberu | ok | |
| 16:02:08 | LiLiu_ | That's totally doable | |
| 16:02:21 | masuberu | do you have experience with openhpc? | |
| 16:02:40 | LiLiu_ | I don | |
| 16:02:47 | masuberu | we run rocks cluster but the development is quite slow, I am wondering whether openhpc would be better option | |