| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2018-08-30 | |||
| 20:08:39 | efried | I don't really have a stake in getting lots of code into these plugins. | |
| 20:08:48 | efried | I just want to make sure we don't end up with linux-isms in common code paths. | |
| 20:08:54 | efried | linux/libvirt | |
| 20:09:16 | Sundar | OK. Could you look at the flow diagram I shared this morning? | |
| 20:09:26 | efried | If I didn't have to write a separate driver/plugin at all, I would be pretty happy. But I'm not sure that is going to happen. | |
| 20:10:08 | Sundar | Not to spoil your afternoon, but I think you would need PowerVM drivers for GPUs, FPGAs, or whatever you support :) | |
| 20:11:23 | Sundar | Are you looking at Power+KVM hypervisor too? | |
| 20:12:10 | Sundar | brb in 5 min | |
| 20:12:19 | efried | I don't know anything about pkvm | |
| 20:14:00 | Sundar | back. ok | |
| 20:15:43 | Sundar | Back to os-acc plugins. The possible role for os-acc plugins is to handle device-compute interactions, such as NUMA affinity for interrupt vectors. But it may be too much to bring them in this spec. Shall we put os-acc plugins as a placeholder until we get to that? | |
| 20:18:26 | efried | "NUMA affinity for interrupt vectors" <== Greek to me. | |
| 20:18:51 | Sundar | OK, NUMA affinity for devices in general? | |
| 20:19:11 | efried | If you don't have an actual use case for os-acc plugins, then... | |
| 20:19:35 | efried | NUMA affinity is something we want to be able to handle via resource provider structure. | |
| 20:19:48 | efried | we can't yet | |
| 20:19:56 | efried | but it's something we should bring up (again) in Denver. | |
| 20:20:49 | Sundar | Sure. Shall we have a f2f chat at Denver? It'll be good to sync up and drive the specs to a close shortly thereafter | |
| 20:21:14 | efried | definitely | |
| 20:21:30 | efried | If I were you, I would ask for a nova/cyborg cross-project session. | |
| 20:21:52 | efried | Based on what I know of people's schedules, I would say Tuesday would be a good day for it. | |
| 20:22:03 | Sundar | I have entered a slot in Nova etherpad. May be I should ping melwitt? | |
| 20:23:02 | efried | It might be better to schedule something during cyborg's time (M/T) where you can get the "placement cores" to visit the cyborg room and spend a couple of hours hashing stuff out. | |
| 20:23:33 | efried | And keep it on the nova schedule in case we a) don't get everything sorted; and/or b) need a wider audience. | |
| 20:23:48 | Sundar | OK | |
| 20:24:04 | efried | Send something to the dev ML, like what Blazar did, to organize the Tuesday thing. | |
| 20:24:05 | efried | oh | |
| 20:24:14 | efried | Sorry, I forgot, Blazar sent that directly to people. | |
| 20:24:18 | efried | not to the ML. | |
| 20:24:34 | Sundar | Meanwhile, please LMK if you are ok with the flow diagram. Did you get it? | |
| 20:26:34 | efried | I got it. | |
| 20:27:47 | Sundar | Great. I'll grab lunch and get back in 15 min. We can shake it out after that. | |
| 20:29:57 | efried | give me 20 | |
| 21:07:14 | Sundar | efried: Ready any time | |
| 21:08:00 | efried | Sundar: Nova meeting in progress. Not sure how long it will last, but not beyond top of the hour. Will you be around? | |
| 21:09:31 | Sundar | Yes | |
| 21:21:08 | efried | Sundar: ō/ | |
| 21:23:57 | Sundar | That was quick. | |
| 21:24:32 | efried | now where did I put that diagram... | |
| 21:24:56 | efried | got it | |
| 21:25:14 | Sundar | The main thing to note in the flow is that there isn't much that os-acc is doing. | |
| 21:25:59 | Sundar | Most of it is Nova virt or Cyborg. I am still pondering what os-acc can do that is not device-specific (and so in Cyborg) or hypervisor-specific (and so in Nova virt) | |
| 21:26:59 | efried | Sundar: as a touchpoint/router and future extension point for those pieces of the requests, it may be useful. | |
| 21:27:05 | efried | but I'm really not sure. | |
| 21:27:30 | Sundar | Yes, I agree. That's what I was thinking | |
| 21:28:21 | efried | not sure it makes much sense for the calls from os-acc to cyborg API/conductor to be asynchronous if it's just going to poll for them to complete right away... | |
| 21:29:37 | Sundar | The calls may take milliseconds to possibly seconds, depending on whether Glance bitstreams need to fetched, one or more FPGAs need to be programmed, etc | |
| 21:30:20 | efried | so? | |
| 21:30:27 | Sundar | That is why it is async | |
| 21:30:31 | efried | The caller is blocking on their completion anyway | |
| 21:30:39 | efried | So why does it matter if they take "a long time"? | |
| 21:30:48 | Sundar | Ah, the n-cpu could do other things while this is blocked, right? | |
| 21:30:53 | efried | could it? | |
| 21:31:06 | efried | I guess. | |
| 21:31:18 | Sundar | The allocation of other resources -- networking, storage -- could go in parallel | |
| 21:31:21 | efried | Just sounds like it would make things pretty complicated. | |
| 21:31:36 | efried | If n-cpu wanted to parallelize, it could send that request off in its own thread. | |
| 21:31:46 | efried | But having it be async *forces* n-cpu to deal with that async-ness. | |
| 21:31:49 | efried | Anyway, this is really a nit. | |
| 21:31:59 | efried | not a substantive thing. | |
| 21:32:03 | Sundar | The n-cpu to os-acc could be async too. | |
| 21:32:15 | Sundar | So, os-acc deals with the async weirdness. | |
| 21:32:59 | Sundar | Hmm, well, os-acc to Cyborg can be sync if os-acc itself is called in an async way | |
| 21:33:13 | Sundar | Just wondering if a REST API call cna really block for seconds | |
| 21:33:48 | efried | absolutely. | |
| 21:34:10 | efried | I'm sure there's a connection timeout at the HTTP level. But seconds shouldn't be a problem. | |
| 21:34:16 | Sundar | Are there precedents in OpenStack where a REST API blocks for seconds? | |
| 21:35:57 | efried | I have no idea. | |
| 21:36:10 | efried | That might be something edleafe and/or cdent would know. | |
| 21:37:22 | Sundar | OK. Another note: The proposal calls for persisting VAN objects in Cyborg db, but Nova db maintains the association between the instance and its VAN UUIDs . That s because, on a VM suspend for example, Nova would need to call os-acc to detach VANs but not deallocate them | |
| 21:37:55 | Sundar | Never mind, the detach would be in Nova virt. | |
| 21:38:03 | Sundar | But you get the point | |
| 21:38:29 | Sundar | On a termination, Nova virt would detach each VAN and then call os-acc to release the resources | |
| 21:38:47 | efried | right, some kind of handle (i.e. the VAN UUID) would need to be associated with the instance in the nova db. | |
| 21:38:54 | Sundar | Yes | |
| 21:39:14 | efried | As long as we can query cyborg with the UUID to get the rest of the VAN info, the UUID should be the only thing we need to store, I would think. | |
| 21:39:21 | Sundar | Yes | |
| 21:42:33 | Sundar | Good. Since we are in agreement, I'll write this up in the spec. | |
| 21:43:55 | efried | Sundar: Note that I'm only one person, and my stake in the details we've just discussed is not an especially strong one. | |
| 21:45:20 | Sundar | We need some path to get this converged. I am incorporating sean-mooney's comments and responding to them. But, if somebody else were to come along in a month, we can't keep waiting. | |
| 21:45:40 | Sundar | I hope to get this closed during the PTG or at most a week after | |
| 21:46:25 | Sundar | I requested melwitt for a PTG session. Could I ask for your help in ensuring that all Nova feedback is given by that time? | |
| 22:13:03 | efried | hah | |
| 22:13:14 | efried | You're asking me to move mountains. | |
| 22:14:25 | efried | There's no way to force all the stakeholders to review a thing. And there's no way to prevent taking weeks or months to revise something with the collaboration of many people only to have someone who should have been involved from the start come along and throw a huge monkey wrench in the works. | |
| 22:16:04 | efried | I think your best bet is to be ready to present an overview of the architecture at the PTG, counting on at least some key players in the audience to have *not* reviewed the specs, and talk out some of the issues there. | |
| 22:16:08 | efried | Sundar: ^ | |
| #openstack-cyborg - 2018-09-02 | |||
| 03:02:52 | openstackgerrit | Sundar Nadathur proposed openstack/cyborg master: Cyborg/Nova/Glance interaction in compute node, including os-acc. https://review.openstack.org/577438 | |
| #openstack-cyborg - 2018-09-04 | |||
| 03:21:14 | openstackgerrit | zhulingjie proposed openstack/cyborg-specs master: fix bug link in readme https://review.openstack.org/599484 | |
| 03:21:15 | openstackgerrit | zhulingjie proposed openstack/python-cyborgclient master: fix bug link in readme https://review.openstack.org/599486 | |
| 03:21:17 | openstackgerrit | zhulingjie proposed openstack/cyborg master: fix bug link in readme https://review.openstack.org/599489 | |
| 06:01:17 | openstackgerrit | wangzhh proposed openstack/cyborg master: Add gpu driver https://review.openstack.org/574075 | |
| #openstack-cyborg - 2018-09-05 | |||
| 00:09:05 | openstackgerrit | Sundar Nadathur proposed openstack/cyborg master: Cyborg/Nova/Glance interaction in compute node, including os-acc. https://review.openstack.org/577438 | |
| 01:32:44 | openstackgerrit | Sundar Nadathur proposed openstack/cyborg master: Cyborg/Nova/Glance interaction in compute node, including os-acc. https://review.openstack.org/577438 | |
| 08:00:03 | openstackgerrit | Xinran WANG proposed openstack/cyborg master: Improve Intel FPGA driver https://review.openstack.org/599956 | |
| 08:06:48 | openstackgerrit | Xinran WANG proposed openstack/cyborg master: Improve Intel FPGA driver https://review.openstack.org/599960 | |
| 14:03:13 | zhipeng | #startmeeting openstack-cyborg | |
| 14:03:14 | openstack | Meeting started Wed Sep 5 14:03:13 2018 UTC and is due to finish in 60 minutes. The chair is zhipeng. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 14:03:15 | openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 14:03:18 | openstack | The meeting name has been set to 'openstack_cyborg' | |