| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2017-11-15 | |||
| 15:09:54 | shaohe_feng_ | zhipeng: got it. | |
| 15:10:01 | zhipeng | could everyone log their names ? | |
| 15:10:04 | zhipeng | #info Howard | |
| 15:10:23 | zhuli | Can’t get my pc connected, have to use cellphone | |
| 15:10:32 | mpaolino | #info Michele | |
| 15:10:41 | zhipeng | zhuli we will just irc tonight | |
| 15:10:41 | Guest69871 | #info Li | |
| 15:10:42 | zhuli | #info zhuli | |
| 15:10:51 | zhuli | Got it | |
| 15:11:01 | Guest69871 | Do we have another meeting tomorrow? | |
| 15:11:20 | crushil | I'm here | |
| 15:11:35 | zhipeng | okey seems like full house almost :) | |
| 15:11:36 | crushil | #info Rushil | |
| 15:11:58 | zhipeng | #topic GATK4 FPGA requirement discussion | |
| 15:12:23 | zhipeng | the first order of business today is that we have masuberu | |
| 15:12:49 | masuberu | o/ | |
| 15:12:54 | zhipeng | system engineer from one of the largest genome testing reserach testing centers in the world | |
| 15:13:05 | zhuli | Welcome masuberu | |
| 15:13:19 | masuberu | thank you all for having me here | |
| 15:13:33 | zhipeng | to talk with us about their upcoming trail with Intel FPGA card to run GTAK4 application for genome analysis | |
| 15:13:42 | zhipeng | the floor is yours masuberu :) | |
| 15:13:50 | masuberu | than you | |
| 15:14:05 | masuberu | I work for a medical research intitute | |
| 15:14:16 | masuberu | and as sequence and analyse genomes | |
| 15:14:22 | shaohe_feng__ | #info shaohe | |
| 15:14:29 | masuberu | right now we are in a project with intel | |
| 15:15:04 | masuberu | they are helping us to build an environment to run GATK4 which a set of tools of best practice genome analysis | |
| 15:15:25 | masuberu | tomorrow a guy from intel is coming to my office to give me FPGA cards | |
| 15:15:50 | masuberu | because GATK4 can either work on HPC cluster, Spark cluster and GDC | |
| 15:16:16 | masuberu | so my idea is to test GATK4 on FPGAs through openstack and compare performance with apache spark | |
| 15:16:36 | masuberu | obviously FPGA should be much faster but we want to see how much faster | |
| 15:17:25 | masuberu | and I was looking for documentation about how to integrate FPGAs on openstack and I found your project | |
| 15:17:52 | zhipeng | shaohe_feng_ is from Intel :) | |
| 15:17:53 | masuberu | so wanted to say hi and see if this would be a possibility of collaboration | |
| 15:18:02 | masuberu | ok | |
| 15:18:10 | masuberu | well intel is quite big | |
| 15:18:12 | masuberu | I guess | |
| 15:18:16 | shaohe_feng__ | masuberu: hi | |
| 15:18:22 | zhipeng | haha yes | |
| 15:18:33 | Guest69871 | Hi Masuberu, This is Li from Huawei, our team has a lot of experience using FPGAs in Openstack | |
| 15:18:36 | masuberu | the person coming tomorrow is from either China or Singapore I think | |
| 15:18:58 | zhipeng | masuberu could you describe a possible setup ? | |
| 15:19:20 | zhipeng | For example if you are gonna have a VM connected to the FPGA | |
| 15:19:38 | zhipeng | or just feeding data directly to GTAK4 on that FPGA ? | |
| 15:20:04 | masuberu | what ever is easier, I was thinking of giving a go and see if could be possible to integrate FPGAs with nova and see if the vms could see it | |
| 15:20:13 | shaohe_feng__ | masuberu: I'm also working on FPGA. And some of us has started to support FPGA on k8s | |
| 15:20:14 | masuberu | otherwise I am happy of using different methods | |
| 15:20:27 | masuberu | ok | |
| 15:20:36 | masuberu | I am totally open to suggestions | |
| 15:20:45 | masuberu | the setup is | |
| 15:21:13 | masuberu | 4 compute nodes with AMD cpus (56 cores) 512 RAM and ~4.5TB spinning disks 10k | |
| 15:21:17 | Guest69871 | Do you have the developed IPs to use for the FPGA? | |
| 15:21:26 | zhipeng | #info masuberu from Garvan Institute of Medical Research | |
| 15:21:35 | masuberu | and 3 nodes with intel cpus (numa 2 sockets 14 cpus each) 512 GB RAM and 20 TB nvmes | |
| 15:21:56 | zhipeng | #link https://www.youtube.com/watch?v=HSAJpAxORAw | |
| 15:21:57 | masuberu | all connected through 2x25GB bond network | |
| 15:22:04 | masuberu | network is mellanox ethernet | |
| 15:22:11 | masuberu | mellanos switch 100GB | |
| 15:22:44 | masuberu | it is not big setup but I think it is a good starting point | |
| 15:23:39 | LiLiu_ | #info Li | |
| 15:23:41 | masuberu | I don't know how many FPGAs I will get, I will know tomorrow | |
| 15:23:53 | zhipeng | shaohe_feng_ any tips on nova integrates with FPGA ? | |
| 15:24:11 | masuberu | so I am not sure if that explains most of the questions? please feel free to ask | |
| 15:24:32 | zhipeng | masuberu will GTAK4 run as an IP for the FPGA ? | |
| 15:25:06 | masuberu | tomorrow I have a meeting with Albert from intel and he will brief me about how it works | |
| 15:25:16 | LiLiu_ | Question: is there multiple IPs to be used in the GATK4 | |
| 15:25:42 | masuberu | LiLiu_, what do you mean by multiple IPs? | |
| 15:26:06 | LiLiu_ | I mean different bitstreams | |
| 15:26:50 | masuberu | so GATK4 is a bunch of tools bioinformatitians use to create pipelines | |
| 15:27:39 | masuberu | you can run a tool at a time or multiple ones at the same time, depends on what you are doing | |
| 15:27:54 | zhipeng | #link https://github.com/broadinstitute/gatk | |
| 15:28:33 | LiLiu_ | OK, I assume you will have multiple types of IPs in your system. Do you chain the FPGAs together or each FPGA only talk to the tied CPU ? | |
| 15:29:00 | masuberu | ok, I will ask that question to the intel guy tomorrow | |
| 15:29:09 | masuberu | I honestly never used a FPGA before | |
| 15:29:45 | zhipeng | shaohe_feng__ any tips on nova integrates with FPGA ? | |
| 15:31:07 | LiLiu_ | I see, my team works a lot on FPGA heterogeneous system in Openstack, we may talk more offline if you are interested, masuberu. | |
| 15:31:28 | masuberu | sure | |
| 15:31:38 | LiLiu_ | liliu1@huawei.com | |
| 15:33:04 | zhipeng | did some googling | |
| 15:33:05 | LiLiu_ | I wanna know more about the scope/size of the system you guys are targeting | |
| 15:33:28 | masuberu | it will depends on the performance of this tests | |
| 15:33:29 | zhipeng | seems like the pairHMM will be the part that need acceleration from FPGA or GPU ? | |
| 15:34:09 | masuberu | for instance we have capacity to sequence 1200 genomes per month so it all depends how much hardware we need for that | |
| 15:34:31 | zhipeng | #link https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/wp/wp-accelerating-genomics-opencl-fpgas.pdf | |
| 15:34:47 | shaohe_feng__ | zhipeng: actually, every accelerator in Intel's FPGA is a sriov device. | |
| 15:35:02 | masuberu | zhipeng, yep I got that document yesterday from Albert | |
| 15:35:22 | zhipeng | shaohe_feng__ how about the altera ones ? | |
| 15:35:26 | zhipeng | the same ? | |
| 15:35:37 | shaohe_feng__ | zhipeng: altera means intel :) | |
| 15:35:54 | zhipeng | haha | |
| 15:36:16 | shaohe_feng__ | zhipeng: there maybe 3 method to support FPGA. | |
| 15:36:16 | masuberu | nallatech 385A is what I am getting | |
| 15:36:30 | masuberu | getting for testing | |
| 15:36:54 | shaohe_feng__ | zhipeng: 1. pre-programming FPGA on the host. | |
| 15:38:02 | shaohe_feng__ | zhipeng: 2. the orchestration program FPGA | |
| 15:38:15 | shaohe_feng__ | zhipeng: 3. VM program FPGA. | |
| 15:38:36 | shaohe_feng__ | zhipeng: I have got a FPGA server, and I'm trying to do some test on it | |
| 15:38:54 | masuberu | shaohe_feng__, the person coming tomorrow is called AlbertZQ Wang, not sure if you know him... | |
| 15:39:25 | shaohe_feng__ | zhipeng: AlbertZQ Wang from intel? | |
| 15:39:36 | shaohe_feng__ | ^ masuberu: | |