| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2017-11-08 | |||
| 15:49:44 | zhipeng | accelerators just got so many types and combinations | |
| 15:50:09 | jkilpatr | so we let operators aggregate things when their workload could run on either and schedule agains that? | |
| 15:50:32 | jkilpatr | against* | |
| 15:51:06 | zhipeng | not necessarily aggreagating, but just select a (super/sub) set of resource that satisfy the fuzzy request | |
| 15:51:24 | zhipeng | if it deems to be a super set, then it is aggregation | |
| 15:51:43 | zhipeng | if it just needs a subset, say just some of the FPGA card funcationalities | |
| 15:51:53 | zhipeng | we could also do that mitigation | |
| 15:52:07 | jkilpatr | still not sure I understand the usecase | |
| 15:52:26 | zhipeng | I think I will just writeup a spec and we could review it :P | |
| 15:52:40 | zhipeng | I need to elaborate more | |
| 15:52:40 | jkilpatr | sure, a better outline of the potential usecase would help a ton | |
| 15:52:44 | zhipeng | yep | |
| 15:52:50 | zhipeng | okey | |
| 15:53:29 | zhipeng | #agreed have weekly sprint video conf using team meeting timeslot starting Nov 15th | |
| 15:53:47 | zhipeng | #topic AoB | |
| 15:54:02 | zhipeng | I think we have covered the most important ones | |
| 15:54:07 | zhipeng | any other business ? | |
| 15:58:54 | zhipeng | okey let's conclude the meeting then, thanks everyone :) | |
| 15:58:59 | zhipeng | #endmeeting | |
| 15:59:00 | openstack | Meeting ended Wed Nov 8 15:58:59 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 15:59:01 | openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_cyborg/2017/openstack_cyborg.2017-11-08-15.01.html | |
| 15:59:02 | openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_cyborg/2017/openstack_cyborg.2017-11-08-15.01.txt | |
| 15:59:03 | openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_cyborg/2017/openstack_cyborg.2017-11-08-15.01.log.html | |
| #openstack-cyborg - 2017-11-13 | |||
| 13:27:24 | jkilpatr | zhipengh[m], so the agent can't interact with placement well IMO. Each agent is on each compute and knows only about it's own local resources. If we want the agent to go and tell Nova 'hey we have N of these devices' it needs to know global state | |
| 13:27:27 | jkilpatr | which is the job of the conductor. | |
| 13:27:52 | jkilpatr | Otherwise we assume that no two computes will have the same type of accelerator, which is fatally flawed IMO | |
| 13:41:21 | zhipengh[m] | Does it have to know the global state ? | |
| 13:42:05 | zhipengh[m] | Currently for Nova, it is the Nova compute you n each node interacting with Placement, as far as I understand | |
| #openstack-cyborg - 2017-11-15 | |||
| 08:03:17 | masber | good afternoon, I just found you project, is cybord ready to be installed? | |
| 08:04:05 | masber | I would like to be able to use FPGAs on openstack, can I get some help within this channel? | |
| 08:21:59 | zhipengh[m] | Hi masber , thx for the interest ! | |
| 08:22:24 | zhipengh[m] | Could u introduce a little bit about yourself and your requirement ? | |
| 08:22:34 | masber | oh yes sorry | |
| 08:22:48 | masber | I work for a medical research intitute | |
| 08:23:09 | masber | and we are playing with openstack/ceph in order to see if it can fit out use cases | |
| 08:23:20 | masber | now we are doing a project with intel | |
| 08:23:35 | masber | and we will borrow some FPGA cards from them | |
| 08:24:16 | zhipengh[m] | That sounds interesting :) | |
| 08:24:47 | masber | I would like to see if we can make those cards to work on openstack, I mean been able to access the FPGAs from the instance | |
| 08:24:55 | masber | I did a talk in the openstack summit in sydney | |
| 08:25:13 | masber | I talk about what we do etc | |
| 08:25:49 | masber | I am not an openstack expert just, installed it and played with it a little bit | |
| 08:26:23 | masber | but now with these FPGAs coming I feel I should start getting my hands dirty | |
| 08:26:40 | zhipengh[m] | Cool ! Can you provide the title of your Sydney topic ? So that I could search on youtube :) | |
| 08:26:46 | zhipengh[m] | As for cyborg | |
| 08:27:07 | masber | so, if I am not mistaken we are receiving some nallatech 385a | |
| 08:27:16 | zhipengh[m] | Our current code base cannot function yet, and we are undergoing active development | |
| 08:27:40 | masber | the name of the talk was "building a genome future with openstack" | |
| 08:28:04 | masber | https://www.youtube.com/watch?v=HSAJpAxORAw | |
| 08:28:10 | zhipengh[m] | I will definitely check it out :) | |
| 08:28:52 | masber | it is quite basic talk | |
| 08:28:55 | zhipengh[m] | So at the moment you could run up the three main cyborg processes, but no function yet | |
| 08:29:02 | masber | I see | |
| 08:29:12 | zhipengh[m] | We are now try to define data models | |
| 08:29:24 | zhipengh[m] | And would be great to have input from you | |
| 08:29:34 | zhipengh[m] | For example on FPGA | |
| 08:29:58 | masber | ok, I am not an user with lots of expertise on these kind of things, but happy to give it a go | |
| 08:30:05 | zhipengh[m] | What is your main use case for that and what would you like to see FPGA resource represented | |
| 08:30:25 | zhipengh[m] | That'd be great ! | |
| 08:30:31 | masber | this is to run a tool called GATK4 | |
| 08:30:53 | masber | it is a best practice set of tools to run genome analysis | |
| 08:31:05 | masber | so this tool can work on HPC or in a SPARK cluster | |
| 08:31:25 | masber | our goal is to see how much efficiency we can get by running these tools on FPGAs | |
| 08:31:40 | masber | compared to running on Apache Spark | |
| 08:32:25 | masber | the thing is I am receiving the cards tomorrow from the intel guy who is coming from overseas (not sure from singapose, china or US) | |
| 08:32:32 | masber | *singapore | |
| 08:32:48 | masber | and I am not sure for how long I can keep the cards either | |
| 08:33:05 | zhipengh[m] | that is very interesting | |
| 08:34:06 | masber | another problem is that currently I am deploying openstack using kolla... | |
| 08:34:28 | zhipengh[m] | Ah~ | |
| 08:36:32 | zhipengh[m] | I think we should first look into GATK4 , see what are the resource requirements for running it on the FPGA card | |
| 08:37:01 | zhipengh[m] | Those requirements could compose what we call accelerator capability in cyborg | |
| 08:37:10 | masber | I think I can get that information from intel I think | |
| 08:37:40 | masber | I see | |
| 08:37:56 | zhipengh[m] | And then I think our team could come up the very first data model outline for FPGA | |
| 08:38:04 | masber | I think intel got GATK4 running on FPGAs | |
| 08:38:05 | zhipengh[m] | With a real use case :) | |
| 08:38:27 | masber | yes my use case is quite real because we are planing to deploy this pipeline in production | |
| 08:38:38 | masber | we are actually running GATK3 in production | |
| 08:38:41 | zhipengh[m] | BTW we will have video conference about 6:20 hour later | |
| 08:38:51 | zhipengh[m] | Would it be possible for you to join ? | |
| 08:39:03 | masber | in 6.20 hours | |
| 08:39:06 | masber | let me check | |
| 08:39:11 | zhipengh[m] | Haha right | |
| 08:39:21 | masber | 2:00 am my time | |
| 08:39:28 | masber | yes I can join | |
| 08:39:56 | zhipengh[m] | Wow that is pretty late | |
| 08:40:20 | zhipengh[m] | Anyways I will send out the details later on openstack-dev mailing list | |
| 08:40:32 | masber | yep, well as I said Im not a quite competent openstack user so most of the time I have to stay till late so I can get help from the community | |
| 08:40:47 | masber | many nights working until 4am | |
| 08:41:02 | zhipengh[m] | Man that is hardcore | |
| 08:41:02 | masber | but it is ok, I am happy to try and see if I can help you | |
| 08:41:18 | masber | my boss gives me free days | |
| 08:41:20 | masber | it is ok | |
| 08:41:45 | zhipengh[m] | :) | |
| 08:42:11 | masber | plus I'm a pragmatic person which means I learn by experience so for me it is quite good to get in contact with people in get things done | |
| 08:42:15 | masber | anyway | |
| 08:42:51 | masber | yeah I will join the call so if there any doubt I can talk with the intel guy tomorrow | |
| 08:43:03 | masber | he is giving me the cards and then we will have a meeting | |
| 08:43:21 | masber | so maybe I can ask questions in your behalf | |