Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-05-15
14:59:02 dansmith see the package names you have to get installed for each type of device?
15:00:02 jaypipes dansmith: yeah.
15:00:25 dansmith those being distro packages either bundle or download proprietary binaries for that layer
15:00:26 jaypipes dansmith: back to hardware-defined software it seems. :(
15:00:59 dansmith pretty sure the nvidia ones at least are very closely-guarded and require an account with endowments in order to download them
15:02:16 dansmith also pretty sure most of these have pretty specific kernel versions that are required,
15:02:30 dansmith as they also utilize the kernel-side driver, fglrx, nvidia, etc
15:02:48 jaypipes dansmith: my fear is that we will get locked in to vendor-specific identifiers and behaviour in the user-facing API and when these vendors inevitably change directions, we'll be forced to change our APIs.
15:03:31 dansmith jaypipes: I don't think we are though, because the traits are just enums.. if nothing is exposing m60 right now, then.. nothing boots that requires it
15:04:53 dansmith and going back to your point of trying to shame me,
15:05:04 jaypipes dansmith: what does m60 give you though? I can understand a potential need for traits that represent all these vendor drivers for OpenCL and CUDA, but the model name doesn't give any more information than the resolution, display heads and API library versions, right?
15:05:18 dansmith I don't want vendor lock-in of course, as you know, but I don't think enumerating the things that could be out there is doing that
15:05:24 dansmith jaypipes: of course it does
15:05:45 dansmith jaypipes: max resolution and display heads means nothing to me, I need cores/shaders/memory for GPGPU
15:05:55 dansmith jaypipes: and of course, lineage so I know if my drivers are appropriate
15:06:14 jaypipes but the drivers are for a version of CUDA or OpenCL, though, right?
15:06:17 dansmith jaypipes: if I'm an amd instance and I have a specific version of the drivers, I may be able to support a foobar99 but not a foobar128
15:06:25 jaypipes you don't have drivers for M60, drivers for M61, etc, right?
15:06:26 dansmith I'm not sure what the appropriate level of abstraction is exactly,
15:06:44 dansmith maybe m60 is too fine-grained, but I think probably not because there are cards with 1, 2, 4 m60 chips, etc
15:08:14 openstackgerrit Vladyslav Drok proposed openstack/nova-specs master: Allow having placement inventories with reserved value equal to total https://review.openstack.org/568613
15:12:23 openstackgerrit Merged openstack/nova master: trivial: Explain how the marker works for instance-cell mapping https://review.openstack.org/567597
15:26:46 jaypipes dansmith: ok, so I'll just point out one last word on this VGPU stuff... AWS' elastic GPUs don't mention any vendor at all (https://aws.amazon.com/ec2/elastic-gpus/). They only mention "supports OpenGL 4.3" and up to 8G of GPU memory. Now, I know it's just one data point, but it does seem they are trying not to get into the game of leaking out vendor information ...
15:28:39 dansmith jaypipes: check out this in their FAQ: Q. Do I need a driver for Elastic GPUs?
15:29:00 dansmith jaypipes: looks like they have network-virtualized the GPUs so they can be attached anywhere, and they consume network bandwidth
15:29:20 dansmith they use OpenGL as their interface,
15:29:21 dansmith which means you're pretty far removed from the thing as a compute device, AFAIK
15:29:35 dansmith they also seem to focus on "graphics acceleration", which would be in line with being OpenGL-only
15:29:46 openstackgerrit Vladyslav Drok proposed openstack/nova-specs master: Allow having placement inventories with reserved value equal to total https://review.openstack.org/568613
15:30:18 dansmith jaypipes: ah, they say use P3 for GPGPU
15:30:39 dansmith jaypipes: https://aws.amazon.com/ec2/instance-types/p3/
15:30:42 dansmith jaypipes: tesla v100 only
15:30:46 dansmith jaypipes: so, yeah, easy :)
15:32:12 dansmith and note they're not saying "you'll get some nvidia somethingorother" they say .. "it'll be a damned v100, alwas"
15:32:13 jaypipes dansmith: I see them say tesla v100. those are passthrough direct GPU devices, though, not vGPU "types", right?
15:32:35 jaypipes "Alternatively, you can also use the NVIDIA AMI with GPU driver and CUDA toolkit pre-installed." ... interesting.
15:32:52 dansmith jaypipes: yeah I dunno if it's a vGPU or them just giving you a whole card
15:33:04 jaypipes dansmith: it's the whole card. or 8 of them :)
15:33:32 dansmith doesn't have anything to do with our discussion though right? they're very specific about what you get, and thus what driver you need
15:35:08 jaypipes dansmith: yes and no? :) the "elastic GPUs" are very much non-vendor-specific. The P3 instances are clearly very much "here is this specific machine configuration with 8 NVIDIA GPUs", etc.
15:35:24 jaypipes dansmith: but I see your point.
15:35:38 jaypipes dansmith: we need to support both cloudy and non-cloudy things.
15:35:52 dansmith jaypipes: but they're not for GPGPU, they're for graphics acceleration
15:36:01 jaypipes dansmith: I've just been trying my best to keep the vendor-specific stuff out of the way.
15:36:05 dansmith and they virtualize the OpenGL interface, so you're far removed
15:36:09 jaypipes dansmith: yes, I understand.
15:38:02 openstackgerrit Vladyslav Drok proposed openstack/nova-specs master: Allow having placement inventories with reserved value equal to total https://review.openstack.org/568613
15:38:24 dansmith and, they achieve the vendor neutrality of the elastic gpu thing by creating their own vendor so to speak, and provide signed drivers for supported operating systems
15:38:28 dansmith if they didn't have that,
15:38:45 dansmith they'd have to have specific details about which drivers you needed to have to support all their stuff, or provide flavors to let you choose which ones,
15:38:56 dansmith because drivers are required one way or the other
15:39:20 dansmith so unless you're going to write a driver and backend system to do that, I don't see how even the elastic gpu case alters the discussion
15:40:22 jaypipes dansmith: sounds like we need traits that represent the vendor software drivers, then...
15:41:53 dansmith or we expect that people have to do that on their own by using custom traits
15:42:11 dansmith which is what I said in that review, but then that generates a much less interoperable situation for everyone
15:42:33 jaypipes agreed.'
15:42:48 jaypipes dansmith: ok, thx for talking through this with me. was very useful.
15:43:01 jaypipes and for the record, I did not try to shame you. sorry if anything came across that way.
15:43:16 dansmith we also can punt the problem for a while, saying people need to use custom traits for now and see how it goes
15:45:45 jaypipes ack
15:46:21 openstackgerrit Merged openstack/nova master: placement: Fix HTTP error generation https://review.openstack.org/568567
16:01:39 openstackgerrit Chris Dent proposed openstack/nova master: [placement] default to accept of application/json when */* https://review.openstack.org/568630
16:08:27 mriedem stephenfin: have to talked with claudiu about whether or not the hyperv driver can support https://review.openstack.org/#/c/541290/ also?
16:09:07 stephenfin mriedem: I have not, but HyperV's support for all things NUMA is minimal at best
16:09:23 stephenfin Far as I can tell, they let you define guest topologies but not anything to do with the host topology
16:10:23 stephenfin So they don't do things like "restrict my instance's vCPU processes to CPUs from the same NUMA nodes as the attached PCI device". Wouldn't make sense (yet!) to do that for networks either
16:20:02 mriedem stephenfin: ok, as dansmith mentioned in the spec, it would be good to call out what is lacking in hyper-v to restrict them from being able to do the new magic
16:20:20 mriedem stephenfin: if you haven't seen yet, lots of comments on that spec, mostly concerned about the data modeling changes being proposed
16:20:26 dansmith yeah, sounds like it's cool, just a sentence saying "they don't do host topo" would be good I think
16:21:06 mriedem i'd also like to know what's going on with bauzas' numa topology modeling in placement spec
16:21:19 mriedem since this could technically depend on that and then we wouldn't need to build in more claims logic in the RT for this stuff
16:21:34 bauzas mriedem: I need to update my spec
16:21:44 mriedem like, if numa support is not even close for rocky, then i just need to know that
16:22:12 mriedem bauzas: you should probably read https://review.openstack.org/#/c/541290/ if you haven't already
16:22:54 bauzas mriedem: I'm a bit on and off given the previous week and the next one, but okay
16:23:16 bauzas maybe we could discuss that in Vancouver ?
16:24:55 cfriesen anyone know offhand where the code is that determines how much disk to consume if the flavor specifies a size of zero?
16:30:01 openstackgerrit Eric Fried proposed openstack/nova master: Use placement.inventory.inuse in report client https://review.openstack.org/568639
16:30:43 mriedem bauzas: i don't think we have any related forum sessions for something like this
16:30:52 mriedem unless you just mean between sessions
16:30:57 bauzas between, yeah
16:30:59 bauzas anyway
16:31:09 bauzas I'll look at the spec tomorrow if I can
16:31:16 bauzas I still have 3 free days
16:55:22 openstackgerrit Matt Riedemann proposed openstack/nova stable/queens: Don't reschedule on RequestedVRamTooHigh errors https://review.openstack.org/568642
17:10:53 openstackgerrit Merged openstack/nova stable/queens: Added ability to configure default architecture for ImagePropertiesFilter https://review.openstack.org/568575
17:10:59 openstackgerrit Merged openstack/nova master: Update auth_url in install docs https://review.openstack.org/568002
17:13:06 mriedem dansmith: that queens backport is ready to launch now https://review.openstack.org/#/c/568363/
17:13:27 dansmith right on
17:16:53 openstackgerrit Merged openstack/nova master: Add INVENTORY_INUSE to DELETE /rp/{u}/inventories https://review.openstack.org/568578
17:32:26 melwitt gibi, efried: there's not much of a plan. I'm going to put together a short slide deck similar to this cinder one from the boston summit (in case we need it) https://www.slideshare.net/JayBryant2/openstack-cinder-onboarding-education-boston-summit-2017
17:33:31 melwitt I was thinking I'll also prepare a set of logs from a gate run showing how to trace a request through the various nova services, in case anyone is interested in that
17:34:21 melwitt if you have any other topic ideas, it would help to have more choices available because what we usually do is poll the room and ask what topic they most like to hear about
17:35:58 efried melwitt: When sdague did that (walk through logs) in Boston I found it super useful. So ++ to that idea.
17:36:48 efried um, Boston or Austin, can't remember.
17:37:21 melwitt I think you're right that it was boston
17:41:44 openstackgerrit Eric Fried proposed openstack/nova master: Use placement.inventory.inuse in report client https://review.openstack.org/568639
18:20:24 openstackgerrit Eric Fried proposed openstack/nova master: Use placement.inventory.inuse in report client https://review.openstack.org/568639
18:20:46 openstackgerrit Eric Fried proposed openstack/nova master: Use placement.inventory.inuse in report client https://review.openstack.org/568639
18:28:28 openstackgerrit melanie witt proposed openstack/nova-specs master: Propose counting quota usage from placement and API database https://review.openstack.org/509042

Earlier   Later