Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-12-04
13:09:07 efried BJJ is one of the harder ones, generally speaking. Most times it takes around a decade to get there (though my buddy did it in a little over 7 years, which is fairly quick - he's got talent).
13:11:38 sean-k-mooney2 jaypipes: o/ its faily early still in your neck of the woods right?
13:11:52 jaypipes sean-k-mooney2: quarter after eight
13:12:53 sean-k-mooney2 jaypipes: ya i taught you were -5 from me.
13:14:34 sean-k-mooney2 so general question. in pike there was no way to request a spcecific resouce class to be allocated as part of a vm boot request correct e.g. in placement
13:18:55 efried sean-k-mooney2 I believe that is correct, except for the Ironic case.
13:19:08 efried ...which isn't really a resource class.
13:19:22 sean-k-mooney2 right
13:20:29 jaypipes sean-k-mooney2: no, you could/can specify an override in the flavor's extra specs "resources" key.
13:20:31 sean-k-mooney2 ok thats what i taught. am for neted resouce provided sriov is going to be the first example resouce correct
13:20:46 sean-k-mooney2 jaypipes: oh really?
13:20:53 jaypipes sean-k-mooney2: no. Ironic has used resources in the flavor extra specs already
13:21:22 jaypipes sean-k-mooney2: and vCPU, memory and disk can also be overridden from the flavor.vcpus/memory_mb/disk_gb values
13:22:29 efried sean-k-mooney2 I think the first real world example may be VGPUs in the xen driver.
13:23:16 sean-k-mooney2 jaypipes: and if i was to do that it wont break any of the internal accounting? e.g. if flavor says 4 vcpus and i override the resouce request to 2 or 8 bad thing wont happen? or this is a gun and i should be careful where i point it
13:23:54 jaypipes sean-k-mooney2: accounting will be done with the override.
13:25:14 sean-k-mooney2 ok cool so if i were to do this always claim >= what is used by normal flavor properties.
13:26:19 efried sean-k-mooney2 For the Standard Three (VCPU, MEMORY_MB, and DISK_GB) in pike, it's in fact the exact same code path by the time it gets to actually claiming anything. The override/conversion gets done really early.
13:27:11 sean-k-mooney2 efried: yes but i would assume the xml generation would use the VCPU not the override in resouces
13:27:34 efried sean-k-mooney2 Maybe I'm not understanding what you mean by "override". Where are you overriding?
13:27:59 jaypipes efried: the extra specs resources key/value pair are overrides from what is in, say, flavor.vcpus
13:28:16 efried Right, sean-k-mooney2 in ^ this case, you're safe.
13:28:33 efried For a sec I was scared you were, like, doing something in the virt driver that was different from what came down in the flavor.
13:30:41 sean-k-mooney2 efried: well i was just trying to make sure you couldnt have flavor.vpus=4 and resouces.vcpus=2 and end up spawning a vm with 4 cpus but only claim 2
13:31:10 jaypipes sean-k-mooney2: no, you can't do that
13:31:29 efried sean-k-mooney2 oh, it's an interesting point. I imagine it's up to the virt driver what they key off of to decide how much resource to allocate. Let me check whether we actually overwrite the flavor.vcpus value for the copy we send down to virt...
13:31:37 jaypipes sean-k-mooney2: but you can have flavor.vcpus = 2 and resources:VCPU=2 and the instance will get 2.
13:31:53 jaypipes doh, I meant resources:VCPU=4 and get 4.
13:33:03 sean-k-mooney2 jaypipes: ok well since the flavor is admin only i guess that is ok. if this was available via image metadata that could be bad
13:33:24 jaypipes sean-k-mooney2: ack
13:37:20 sean-k-mooney2 ok well that good to know it will likely be helpful at some point.
13:38:18 sean-k-mooney2 have ye heard of vhost-vfio
13:41:17 sean-k-mooney2 it is a form of vhost offload like vhost-user which leaverages the kenel vfio-pci driver to provide sriov like performance that can also support livemigration/ovs hardware offload.
13:43:02 jaypipes sean-k-mooney2: no, haven't heard of it until just now. sounds like yet another avenue for hard-to-debug issues in live migration :)
13:43:59 sean-k-mooney2 jaypipes: hehe proably an im likely going to be one of the first to enjoy that experience
13:44:18 jaypipes sean-k-mooney2: you always are :)
13:44:33 jaypipes strace and gdb ftw
13:45:45 sean-k-mooney2 this is the semi internal poc that im being pull upstream work to work on. it leaverages mdevs instead of vfs to represent the virtual interfaces which is fun...
13:46:29 jaypipes sean-k-mooney2: what advantage does Nvidia's mdev framework have over the SR-IOV VF interface?
13:47:10 sean-k-mooney2 jaypipes: live migrtation and the ability to create them at runtime are 2 of the advantages
13:47:34 jaypipes sean-k-mooney2: why can't VFs be created at runtime?
13:48:18 sean-k-mooney2 jaypipes: the linux kernel does not have an interface to do so without removeing and reallocating all vfs on the pf
13:48:51 sean-k-mooney2 our nics and technically dpdk can
13:49:14 jaypipes sean-k-mooney2: interesting. why didn't nvidia enhance the existing SR-IOV drivers/VF interface instead of inventing a whole new interface?
13:49:34 openstackgerrit Jay Pipes proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807
13:49:34 openstackgerrit Jay Pipes proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215
13:49:35 openstackgerrit Jay Pipes proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693
13:50:34 sean-k-mooney2 jaypipes: well sriov is a standard by the pcisig and when you create a vf you need to create a new pci endpoint on the host pcie bus for that vf which is no trivial in hardware at runtime
13:50:53 jaypipes alex_xu: great catch on https://review.openstack.org/#/c/384807. I've updated and added a new func test to catch CannotDeleteParentResourceProvider
13:51:24 jaypipes sean-k-mooney2: and what's different about mdev in that respect?
13:52:37 sean-k-mooney2 they are attached to a new bus /sys/bus/mdev and not the pci bus. you create them by echoing a uuid to a file under a partent device then acress them via the uuid after tehy are created
13:53:16 jaypipes sean-k-mooney2: ack. but why couldn't the PCI spec be changed to allow a similar functionality? I'm just curious
13:54:27 sean-k-mooney2 jaypipes: im guessing backwards compatiblity but im not sure. one of the other advanatages of vfio-mdevs ist that it does not need hardware support. it can all be done in the driver
13:54:47 jaypipes sean-k-mooney2: ack
13:55:40 sean-k-mooney2 so if you had a nic without sriov with say 4 tx/rx pairs and you impmented mdev support you could create 4 mdevs one for each queue pair and use it like 4 nics. sriov would require hardware support to do the same
13:57:18 jaypipes sean-k-mooney2: or just use PowerVM, which solves all problems for everyone.
13:57:37 efried Correction: Since Newton.
13:57:49 jaypipes heh
13:58:11 sean-k-mooney2 jaypipes: :) certily solve my problem of exposing intel ad vaule in hardware
13:59:09 openstackgerrit Lajos Katona proposed openstack/nova master: Extend ServerMovingTests with custom resources https://review.openstack.org/497399
13:59:33 sean-k-mooney2 so any way i may have to backport whatever i come upwith to pike hence my earlir question though im trying to avoid that give howmuch is changeing in this space
14:00:01 edleafe Scheduler subteam meeting starting now in #openstack-meeting-alt
14:00:23 sean-k-mooney2 oh im late for a meeting too...
14:01:13 efried sean-k-mooney2 It would be a tough backport, because yeah, queens is going to have huge deltas in this area.
14:02:01 alex_xu jaypipes: there is another comment at https://review.openstack.org/#/c/377215/66
14:03:37 sean-k-mooney2 efried: yep and doing it in a way that will be upgradable to queens will be very hard which is why im trying to push that out of scope but first i need to understand if its technically possible or not
14:04:50 efried sean-k-mooney2 It might be doable for custom resource classes. What did you have in mind?
14:06:59 sean-k-mooney2 custom resouce class create by neutron ovs agent extention, + os-vif plugin to create mdevs during plug + new vnic type to request a vhost-vfio port like we do with sriov
14:07:51 sean-k-mooney2 + the smallest patch to nova possibel to cover xml generation changes in libvirt driver and doing the resouce claim in placement.
14:13:18 gibi sean-k-mooney2: be aware that in pike custom resource classes only works if it is reported by the nova-compute in the RP that represents the compute node. All the other option needs either shared RP or nested RP
14:13:50 jaypipes alex_xu: responded.
14:16:22 alex_xu jaypipes: yea, the uuid and name aren't in opposition, but sorry, I didn't get what your mean about the 'member_of'
14:20:52 openstackgerrit Silvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts https://review.openstack.org/522245
14:23:18 openstackgerrit Silvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts https://review.openstack.org/522245
14:27:13 jaypipes alex_xu: that was responding to bauzas' comment about using member_of instead of tree. member_of is for aggregates, not for provider trees.
14:28:19 alex_xu jaypipes: ah, got it
14:31:00 bauzas jaypipes: oh my bad, you're right
14:31:57 bauzas jaypipes: tbc, I'm not opiniated by which key we should use, just the point that we need to make sure it works with a call having all of them :)
14:32:06 openstackgerrit Matthew Booth proposed openstack/nova master: Make BlockDeviceMapping object support uuid https://review.openstack.org/242603
14:32:06 openstackgerrit Matthew Booth proposed openstack/nova master: Add uuid column to BlockDeviceMapping https://review.openstack.org/242602
14:32:14 jaypipes bauzas: ack
14:32:31 bauzas "tree" is maybe a bit difficult to understand, but I'm not an English folk :)
14:45:22 sean-k-mooney2 gibi: ah yes that is a good point. i would prefer to use nested resouce providers as logically the mdev resouce are a nested resouce of a parent device. in my case a pcie device but technically mdevs could be childern of device on buses other the pci bus.
14:46:39 gibi sean-k-mooney2: I agree that the best would be to have nest RPs for it
14:46:52 efried jaypipes bauzas alex_xu Oh, I wasn't following the conversation in here; I left a (verbose, as usual) comment.
14:47:05 jaypipes efried: on what?
14:48:15 jaypipes efried: in any case, I'm currently adding func tests that add member_of and uuid filters to a tree filter.
15:01:03 alex_xu efried: I don't the tree and resources will work as your expected. The 'GET /resource_providers' just a simple filtering, it is different with the filtering of 'GET /allocation_candidates'
15:02:40 efried jaypipes alex_xu Sorry, I was talking about https://review.openstack.org/#/c/377215/
15:02:44 efried I'm coming at it from the perspective of how I think the REST API should work, not looking at the code.
15:03:14 efried The func tests jaypipes is suggesting should bear out what I said in my comment; and if not, it's a bug that we can fix, right?
15:04:43 openstackgerrit Takashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in delete https://review.openstack.org/525231
15:07:13 jianghuaw_ bauzas, the vGPU relative calls have been there since the minimal version of XS supported by nova. So there is no need to check XenServer's version.
15:07:38 openstackgerrit Jay Pipes proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807
15:07:38 stephenfin dansmith: Question on this patch for you https://review.openstack.org/#/c/514309/
15:07:38 openstackgerrit Jay Pipes proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215
15:07:39 openstackgerrit Jay Pipes proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693
15:07:52 jaypipes bauzas, alex_xu, efried: k, func tests added for tree filter + uuid/member_of
15:08:19 dansmith stephenfin: where?
15:08:29 bauzas jianghuaw_: gtk

Earlier   Later