| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-23 | |||
| 18:35:08 | sean-k-mooney | ya they are | |
| 18:35:13 | sean-k-mooney | 81:00.0 VGA compatible controller: NVIDIA Corporation GK104GL [Quadro K5000] (rev a1) | |
| 18:35:15 | sean-k-mooney | 81:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1) | |
| 18:35:44 | sean-k-mooney | so if we we made ^ work it would work for you too right | |
| 18:38:08 | deke997 | yes | |
| 18:38:27 | deke997 | That looks pretty much identical to my use case | |
| 18:42:09 | sean-k-mooney | ok what i would basicaly suggest is as follows. extent the alias with a child tag that can reference other alisas. | |
| 18:42:48 | sean-k-mooney | when you add an alias to a flaovr which has a child tag we woudl claim the deivce and all its childern | |
| 18:43:19 | sean-k-mooney | and expose them as a multi funcitn device | |
| 18:44:16 | sean-k-mooney | the pci pasthogh through filter would have to be enhanced to ensure it only passes host where the parent and child alias can be allocated form the same slot | |
| 18:44:51 | sean-k-mooney | the pci manger would also need to updated so that when we do the pci claim it claims the pair form the same slot | |
| 18:44:54 | deke997 | wouldn't that always be the case? | |
| 18:45:09 | deke997 | I don't see why the passthrough filter needs to be updated | |
| 18:45:21 | sean-k-mooney | well if you put two of those card in the same server it would break | |
| 18:46:07 | sean-k-mooney | as we coudl get 1 fucntion of type a form one card and one fucntion of type b from another | |
| 18:46:21 | sean-k-mooney | so the filter and assignment code would have to be hardened to prevent that | |
| 18:48:27 | deke997 | The assignment code, yes, but the filter, I don't think so | |
| 18:48:59 | deke997 | because before assignment, it would always be true that both the parent and child could be allocated together | |
| 18:49:31 | sean-k-mooney | its true if you alwasy need to request both | |
| 18:49:38 | deke997 | right | |
| 18:49:38 | sean-k-mooney | which is true in your case | |
| 18:49:42 | sean-k-mooney | not in the gpu case | |
| 18:49:47 | deke997 | I see | |
| 18:50:12 | deke997 | in that case, then yea the filter too | |
| 18:50:16 | sean-k-mooney | anyway the assignment code and filter share the same code | |
| 18:50:41 | sean-k-mooney | the filter basicaly passes if it would be able to assign | |
| 18:51:44 | sean-k-mooney | deke997: https://github.com/openstack/nova/blob/master/nova/scheduler/filters/pci_passthrough_filter.py#L51 | |
| 18:52:01 | sean-k-mooney | see we just loop over the request and ask can it suppor the request | |
| 18:52:20 | sean-k-mooney | well the loopin is don internally in the suport_request funtion | |
| 18:53:06 | sean-k-mooney | we just make a copy of the avliable devices and try to assigntem on hte copy https://github.com/openstack/nova/blob/6db486e9fd4f6b8dd02371b043e66808cdd1e0cd/nova/pci/stats.py#L374-L375 | |
| 18:54:40 | sean-k-mooney | so if you update _apply_request to handel the assignment constriatt the filter gets updated for free | |
| 18:55:13 | deke997 | got it | |
| 18:56:10 | deke997 | are we proposing to let libvirt handle initial target device config and then have nova take over, or are we proposing to have nova handle target device config? | |
| 18:56:30 | deke997 | These are the only two options so far, yea? | |
| 18:56:52 | sean-k-mooney | the only two i am aware of. if we were to do this updatream i would prefer to do it properly | |
| 18:57:15 | sean-k-mooney | and have nova do it. if you need to do it downstream quick well that is up to you | |
| 18:57:42 | sean-k-mooney | deke997: what is the eta you had in mind for this | |
| 18:57:52 | deke997 | 2 days ago haha | |
| 18:58:05 | deke997 | I won't be sleeping much till I get this working | |
| 18:58:19 | sean-k-mooney | ya fair warning to land this upstream its proably going to be next cycle | |
| 18:58:35 | deke997 | yea so I think we'll do both | |
| 18:58:44 | deke997 | downstream quick, and then work on upstream | |
| 18:58:51 | deke997 | when is the next cycle? | |
| 18:58:51 | sean-k-mooney | i could be ussuri if it was really pusshed for but this code is complex and a pain to test | |
| 18:59:16 | sean-k-mooney | it starts in 3 monts os like september/october | |
| 18:59:30 | sean-k-mooney | there is still about 2-3 weeks to propose specs for this cycle | |
| 18:59:57 | sean-k-mooney | you would need to figure out what need to chagne and write up a propals and get it review by then | |
| 19:00:17 | sean-k-mooney | then there is able 2 months left to land the feature | |
| 19:00:30 | sean-k-mooney | but beign realisting if you need this quick | |
| 19:00:52 | sean-k-mooney | then likely createing a poc and upstreaming it next cycle is what your going to have to do | |
| 19:01:32 | sean-k-mooney | for the poc you could do the hack where you fix up the adresses | |
| 19:01:32 | deke997 | One thing to check on: | |
| 19:01:52 | deke997 | I think cyborg is doing some stuff with multi function too | |
| 19:01:59 | deke997 | I saw it while doing research | |
| 19:02:04 | sean-k-mooney | ya so cyborg would be another option | |
| 19:02:15 | sean-k-mooney | if the integration is done this cycle | |
| 19:02:34 | sean-k-mooney | then you could write a device dirver for you custom multifunion device | |
| 19:02:50 | sean-k-mooney | but the xml change would still need to be done | |
| 19:03:03 | sean-k-mooney | you would not have to touch the filer or pci manager in that case | |
| 19:03:14 | sean-k-mooney | but you would need to do all the cyborg work | |
| 19:04:51 | sean-k-mooney | deke997: cyborg support in nova is planned for this cycle but we said that last cycle too | |
| 19:05:02 | sean-k-mooney | deke997: its much closer to being read this time however | |
| 19:06:35 | deke997 | Good to know | |
| 19:07:48 | deke997 | I need to look into how they're implementing multi function a bit more | |
| 19:08:13 | deke997 | but even so, I can't wait till the next cycle to get a beta working here | |
| 19:09:05 | sean-k-mooney | at present without the libvirt support for multifuntion devices in nova cybporg cant support what you need | |
| 19:09:06 | deke997 | I think the options in order of increasing time, complexity, and correctness would probably be | |
| 19:09:17 | deke997 | 1. hack | |
| 19:09:22 | deke997 | 2. nova | |
| 19:09:24 | deke997 | 3. cyborg | |
| 19:09:58 | sean-k-mooney | 0 heiring an intern to manually update every vm as it spawns | |
| 19:10:07 | deke997 | hahahah | |
| 19:10:13 | deke997 | but I can't even do that | |
| 19:10:32 | sean-k-mooney | actully there is one other hack you cloud do | |
| 19:10:32 | deke997 | the xml changes don't go into effect till you reboot | |
| 19:10:46 | deke997 | and when you reboot, it gets overwritten | |
| 19:11:07 | deke997 | is there a way to make the xml changes before boot? or to make the changes go live while the instance is running? | |
| 19:11:32 | sean-k-mooney | for reasons in the past i did have need to do horible things | |
| 19:11:36 | sean-k-mooney | https://opendev.org/x/networking-ovs-dpdk/src/branch/master/devstack/libs/ovs-dpdk#L79-L130 | |
| 19:12:16 | sean-k-mooney | that is an example of a script that replace the qemu executable and rewrites the qemu commandline on the fly | |
| 19:13:39 | sean-k-mooney | what that sciprt is acatlly doing is making sure nic offload that did not work with ancnet version of qemu and ovs-dpdk are disabeld and it makes ure the guest memory has the mmap shared flag set | |
| 19:14:25 | sean-k-mooney | you could install a qemu wrapper script as a hack to match on parts of the qemu command line and alter it | |
| 19:14:46 | sean-k-mooney | it would have 0 support form anyone but it is a -1 on your scale | |
| 19:17:04 | deke997 | haha i'll have to look into that too i guess | |
| 19:17:38 | sean-k-mooney | adding flags to existing optins is nasty but actully kind of simple | |
| 19:17:51 | sean-k-mooney | doing what you need to do would be harder | |
| 19:18:18 | sean-k-mooney | but at the end of the day the commandline is fairly deterministic so it would be doable | |
| 19:18:36 | sean-k-mooney | definelty not my first choise but we dont always get to do our first choice | |
| 19:19:57 | deke997 | I'll see what I can figure out tonight and get back to you soon with the progress | |
| 19:20:07 | deke997 | lmk if you have any other ideas | |
| #openstack-nova - 2020-01-24 | |||
| 10:05:01 | lyarwood | ?pyyamk | |
| 10:05:08 | lyarwood | whops sorry | |
| 10:54:07 | stephenfin | efried: Looking through that now. A few of them look like races ("The specified subnet does not have enough free addresses to satisfy the request.") but yet more might be issues ("Unknown error occurred."). Will keep rooting | |
| 14:23:43 | gibi | dansmith: I've replied to your comments in https://review.opendev.org/#/c/702261/ | |
| 14:24:28 | artom | gibi, according to our internal PTO calendar, he's off in the woods somewhere | |
| 14:25:25 | gibi | artom: thanks. | |
| 14:28:32 | gibi | lyarwood: coud you re-review https://review.opendev.org/#/c/702181/ as you were +2 before but I fixed comments from elod | |
| 14:38:01 | bauzas | why for gosh's sake we aren't using the same driver method for finishing a resize and finishing a resize revert ? graaaah | |
| 14:45:58 | lyarwood | gibi: done, thanks! | |
| 14:46:13 | lyarwood | random Friday question, do we not have an ImageService fixture? | |