| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-23 | |||
| 14:17:11 | jaypipes | alex_xu: if the virt driver isn't changed to assign one of the dedicated host CPUs, yep. But from placement (and resource tracking) perspective, we don't care about that. All we care about is that some amount of dedicated (or shared) CPU resources are being deducted from the appropriate inventory of that class of resource (either VCPU or PCPU) | |
| 14:18:14 | sean-k-mooney | alex_xu: jays spec is basicaly discribing how we will keep a tally count of PCPU and VCPU in placement | |
| 14:19:01 | sean-k-mooney | the asiginment of vms to host dedicated or shared cpu sets will be handeled by the virt driver not placment using the exisitng numa toplogy blob in the nova db as we do today | |
| 14:19:26 | sean-k-mooney | placement will just make sure we have enough cpus to fulltile the request without tracking which ones are free | |
| 14:19:44 | sean-k-mooney | thats the virt driver/ resouce trakers jobs | |
| 14:19:45 | jaypipes | sean-k-mooney: and yes, you're right that my spec proposes deprecating the cpu_policy extra spec. | |
| 14:21:15 | sean-k-mooney | i think i left a comment about may using it to translate the flavor VCPU filed into resouces:VCPU=X or resources:PCPU=x to ease transition but long term it would nolonger be needed | |
| 14:21:30 | alex_xu | ah....I probably I see...give me more seconds... | |
| 14:24:41 | mriedem | is tpatil intel? | |
| 14:24:51 | mriedem | oh NTT | |
| 14:25:29 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Move live_migration.pre.start to the start of the method https://review.openstack.org/612714 | |
| 14:25:30 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Ensure attachment cleanup on failure in driver.pre_live_migration https://review.openstack.org/612715 | |
| 14:25:39 | artom | mriedem, ^^ it has begun *dun dun dun* | |
| 14:26:06 | pvc_ | hi anyone | |
| 14:26:14 | pvc_ | how can i remove a pci devices? | |
| 14:26:29 | pvc_ | nova_libvirt searching for it but it is not existing | |
| 14:28:54 | openstackgerrit | Matthew Booth proposed openstack/nova master: Fix test bug when host doesn't have /etc/machine-id https://review.openstack.org/612717 | |
| 14:32:07 | openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | |
| 14:32:07 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733 | |
| 14:32:37 | alex_xu | sean-k-mooney: jaypipes with that spec, the request with resources:PCPU=1 and without any HW:NUMA_.. stuff, that vcpu is also floating on all the pcpus? | |
| 14:33:15 | alex_xu | since that spec is only about the counting pcpu and vcpu... | |
| 14:33:55 | openstackgerrit | Matthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate https://review.openstack.org/578846 | |
| 14:35:06 | mriedem | pvc_: just fyi, today is a spec review sprint in nova so most people are busy with that. you could try asking your questions in the #openstack or #openstack-operators channels. for pci passthrough questions i'd normally direct you to sahid or cfriesen or moshele but none of them are online right now. | |
| 14:35:35 | mriedem | i'd also think that excluding the pci devices you don't want to expose from https://docs.openstack.org/nova/latest/configuration/config.html#pci.passthrough_whitelist would work, but i don't know a lot about that code | |
| 14:35:55 | mriedem | pvc_: you could also post a question to the openstack-dev mailing list | |
| 14:36:08 | mriedem | if this is a common problem and we don't have documentation for it, then we should have a docs bug | |
| 14:36:14 | pvc_ | thank you so much | |
| 14:36:17 | pvc_ | i will do that | |
| 14:36:20 | mriedem | yw | |
| 14:38:33 | jaypipes | alex_xu: yes | |
| 14:39:01 | jaypipes | alex_xu: since the hw:numa_xxx tags are currently the only way to trigger any pinning behaviour. | |
| 14:39:31 | sean-k-mooney | well not quite you can use hw:cpu_policy | |
| 14:39:33 | alex_xu | jaypipes: ok, i see now, then in the future, we want that case work correctly, right? | |
| 14:39:55 | jaypipes | alex_xu: and since my spec doesn't propose any changes to that, then the existing behaviour if an instance does not have the hw:numa_xxx specs means its vCPU threads float over whatever host processors are in CONF.cpu_shared_set. | |
| 14:40:10 | alex_xu | sean-k-mooney: yea, without hw_cpu_polciy also | |
| 14:40:20 | sean-k-mooney | alex_xu: you should assume that if you have resouce:PCPU=X the virt drive will pin those cores but how it does that is not really realted to jays spec | |
| 14:40:39 | jaypipes | sean-k-mooney: you should NOT assume that. | |
| 14:41:05 | sean-k-mooney | jaypipes: why that was the prerequeit for deprecating hw:cpu_policy | |
| 14:41:09 | jaypipes | sean-k-mooney: the only thing that guarantees assignment to a particular host CPU is the presence of hw:numa_xxx specs | |
| 14:41:45 | sean-k-mooney | the hw:numa_xxx specs today do not do that | |
| 14:41:55 | jaypipes | sean-k-mooney: that is a change that the virt driver will need to make, yes. but that change isn't part of my spec... | |
| 14:43:42 | alex_xu | so...probably we need to doc at somewhere for the user, resources:PCPU doesn't means you get a dedicated cpu for your guest... | |
| 14:44:17 | sean-k-mooney | alex_xu: if we deprecated hw:cpu_policy we dont need to because it will. if we dont then yes | |
| 14:47:20 | sean-k-mooney | i guess we should document it in either case but the point being that if the only way to remove hw:cpu_policy is to either make resources:PCPU mean the virt driver will pin you or add a trait for pinned cpus but that seams dumb | |
| 14:48:07 | sean-k-mooney | from a placement point of view it does not care care if you are pinned or not | |
| 14:48:40 | sean-k-mooney | its jsut a resouce class the fact that we are giving it special semantic is a nova thing not placement | |
| 14:49:46 | alex_xu | sean-k-mooney: I see now | |
| 15:14:20 | melwitt | ||
| 15:24:12 | alex_xu | sean-k-mooney: jaypipes thanks for helping me understand correctly, I see now. I leave my to 0 now, still not sure we let resources:PCPU works as that, that confuses for the end user. maybe we should set cpu policy to dedicated in numa implement when only have resources:PCPU, but we deprecate and remove the cpu_policy extra spec. so not sure | |
| 15:24:13 | mriedem | Kevin_Zheng: the detach/attach root volume on stopped instance spec might have applications in the rescue a volume-backed instance spec https://review.openstack.org/#/c/532410/ | |
| 15:24:19 | mriedem | just FYI | |
| 15:24:40 | mriedem | alex_xu: o/ | |
| 15:25:29 | alex_xu | mriedem: enjoy~ | |
| 15:27:29 | mriedem | oh you know i will :) | |
| 15:28:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Migrate "reboot an instance" user guide docs https://review.openstack.org/612730 | |
| 15:41:51 | pvc_ | hi guys can i ask? So if ever i want to use vgpu i can just use one vgpu on enabled_vgpu_types = nvidia-35, then how about its performance if i launch 5 instance on that flavor? | |
| 15:44:01 | mriedem | bauzas: ^ | |
| 15:44:27 | melwitt | dansmith: could you take a look at this bug fix for a wrong pike compute rpc api alias causing problems with rolling upgrade pike => queens? https://review.openstack.org/612231 needs a +W | |
| 15:49:48 | pvc_ | is there a way that i can use all the vgpus of my gpu considering this on docs As of the Queens release, Nova only supports a single type. If more than one vGPU type is specified (as a comma-separated list), only the first one will be used. | |
| 15:53:51 | dansmith | melwitt: yep done, looks legit | |
| 15:54:27 | melwitt | dansmith: ty | |
| 15:56:30 | melwitt | pvc_: I can't comment on the performance but the limitation on vGPU type means having multiple enabled_vgpu_types on one compute host. at present, you can't have more than one type on the same compute host | |
| 15:57:32 | melwitt | we are working on adding support for multiple types this cycle | |
| 15:58:12 | pvc_ | thank you melwitt, i can launch many instance on flavor with gpu resources but im worried on the performance since we will install an gpu application on it. | |
| 16:02:12 | sean-k-mooney | pvc_: be aware that supporting vGPU types on the same host will not enable multipel gpus to be consumed by a singel vm | |
| 16:03:54 | pvc_ | i login to instance then query the nvidia, may i know if this is the driver that is need to be show? 0:05.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:15f8] (rev a1) | |
| 16:05:31 | openstackgerrit | Jan Gutter proposed openstack/os-vif master: Extend port profiles with datapath offload type https://review.openstack.org/572081 | |
| 16:36:07 | melwitt | dansmith: I've been meaning to ask you if you could review our cycle priorities doc where I've written down themes https://review.openstack.org/609807 | |
| 16:37:04 | pvc_ | melwitt but it is possible to launch many instance on that flavor even if it just only one right? | |
| 16:38:18 | melwitt | pvc_: yes, it is. it's just that all the instances on the same compute host will have to use the same gpu type, for example nvidia-35 | |
| 16:38:35 | dansmith | mriedem: especially given it's spec day, could you circle back to this at some point? https://review.openstack.org/#/c/609709/3 | |
| 16:39:17 | mriedem | do i have to? | |
| 16:39:50 | pvc_ | so that is an issue on performance, so if ever i have 24 gpus on my compute node i can only use 1 right? | |
| 16:40:20 | dansmith | melwitt: ack | |
| 16:40:53 | dansmith | mriedem: no, but you were the last to -1 it, I asked a question which you never answered, and jroll has updated it | |
| 16:41:50 | mriedem | ok i hadn't seen, nor was looking | |
| 16:42:28 | mriedem | i didn't pay attention to any of this at the PTG, so if there are more details from the ptg on alternatives and such, those should be in the spec, including caveats about what happens if the conductor group for a node is updated (as i had several questions about that) | |
| 16:42:50 | mriedem | it sounds like, well that might work automagically, or it might not, shrug | |
| 16:44:06 | mriedem | and i guess we don't want anything in the hypervisors API for this because it would be too ironic specific... | |
| 16:44:26 | melwitt | pvc_: hm, looks like you're right, it says only one vGPU per instance. I don't know why that is limited https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#configure-a-flavor-controller | |
| 16:44:52 | melwitt | bauzas ^ | |
| 16:45:10 | sean-k-mooney | melwitt: its limited because libvirt cannot support multiple vgpus on a singel instance currently | |
| 16:45:21 | melwitt | sean-k-mooney: thanks | |
| 16:45:51 | pvc_ | yes but the problem is we launch instance it will use the same vgpu enabled on the nova.conf, is there no way to use the another vgpu? | |
| 16:46:06 | sean-k-mooney | pvc_: that is not how that works | |
| 16:46:21 | pvc_ | it's just for sharing sean-k-mooney? | |
| 16:46:25 | sean-k-mooney | what you are enabling in the nova.conf is the type of vgpu | |
| 16:46:55 | dansmith | mriedem: there are probably sequencing recommendations for when you change the mappings, but I think those are docs and don't need to be in the spec in great detail, IMHO. I don't think the behaviors are really any different than just spinning up a new or stopping a hashring-balanced compute today | |
| 16:46:56 | pvc_ | so if i use nvida-211 | |
| 16:47:05 | sean-k-mooney | your are not enabvling a specific mdev instnce jsut the type of mdev that will be allocated when a vm requests a vgpu | |
| 16:47:37 | pvc_ | so in terms of performance it is okay sean-k-mooney? | |
| 16:47:46 | dansmith | mriedem: if there's more that needs to be here, let's tell them what that is and let them move on | |
| 16:48:27 | sean-k-mooney | pvc_: mdevs are not shared between instances. so the perfromance of the vgpus will be determined by the mdev type you select | |
| 16:48:44 | pvc_ | so if ever i have 3 mdev types ( nvidia-1, nvidia-2, nvidia-3), i use the nvidia-1 on nova.conf. What will happen to the 2 mdev types? | |
| 16:49:02 | sean-k-mooney | higher perfroamce mdev types consume more physical resouce and allow less vgpus to be createed so its a tradeoff | |
| 16:49:32 | sean-k-mooney | if you set nvidia-1 the nova will only create mdevs of the nvida-1 type | |
| 16:50:22 | sean-k-mooney | there are stict rules about if and how mdev types can be mixed | |
| 16:50:41 | sean-k-mooney | they are vendor specific so initally we chose not to allow mixing them | |
| 16:51:02 | pvc_ | is that okay to be use for many instances? | |
| 16:51:49 | sean-k-mooney | yes we will limit you to the number of mdevs that the enabled type reporst it can allocate | |