| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-23 | |||
| 14:40:39 | jaypipes | sean-k-mooney: you should NOT assume that. | |
| 14:41:05 | sean-k-mooney | jaypipes: why that was the prerequeit for deprecating hw:cpu_policy | |
| 14:41:09 | jaypipes | sean-k-mooney: the only thing that guarantees assignment to a particular host CPU is the presence of hw:numa_xxx specs | |
| 14:41:45 | sean-k-mooney | the hw:numa_xxx specs today do not do that | |
| 14:41:55 | jaypipes | sean-k-mooney: that is a change that the virt driver will need to make, yes. but that change isn't part of my spec... | |
| 14:43:42 | alex_xu | so...probably we need to doc at somewhere for the user, resources:PCPU doesn't means you get a dedicated cpu for your guest... | |
| 14:44:17 | sean-k-mooney | alex_xu: if we deprecated hw:cpu_policy we dont need to because it will. if we dont then yes | |
| 14:47:20 | sean-k-mooney | i guess we should document it in either case but the point being that if the only way to remove hw:cpu_policy is to either make resources:PCPU mean the virt driver will pin you or add a trait for pinned cpus but that seams dumb | |
| 14:48:07 | sean-k-mooney | from a placement point of view it does not care care if you are pinned or not | |
| 14:48:40 | sean-k-mooney | its jsut a resouce class the fact that we are giving it special semantic is a nova thing not placement | |
| 14:49:46 | alex_xu | sean-k-mooney: I see now | |
| 15:14:20 | melwitt | ||
| 15:24:12 | alex_xu | sean-k-mooney: jaypipes thanks for helping me understand correctly, I see now. I leave my to 0 now, still not sure we let resources:PCPU works as that, that confuses for the end user. maybe we should set cpu policy to dedicated in numa implement when only have resources:PCPU, but we deprecate and remove the cpu_policy extra spec. so not sure | |
| 15:24:13 | mriedem | Kevin_Zheng: the detach/attach root volume on stopped instance spec might have applications in the rescue a volume-backed instance spec https://review.openstack.org/#/c/532410/ | |
| 15:24:19 | mriedem | just FYI | |
| 15:24:40 | mriedem | alex_xu: o/ | |
| 15:25:29 | alex_xu | mriedem: enjoy~ | |
| 15:27:29 | mriedem | oh you know i will :) | |
| 15:28:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Migrate "reboot an instance" user guide docs https://review.openstack.org/612730 | |
| 15:41:51 | pvc_ | hi guys can i ask? So if ever i want to use vgpu i can just use one vgpu on enabled_vgpu_types = nvidia-35, then how about its performance if i launch 5 instance on that flavor? | |
| 15:44:01 | mriedem | bauzas: ^ | |
| 15:44:27 | melwitt | dansmith: could you take a look at this bug fix for a wrong pike compute rpc api alias causing problems with rolling upgrade pike => queens? https://review.openstack.org/612231 needs a +W | |
| 15:49:48 | pvc_ | is there a way that i can use all the vgpus of my gpu considering this on docs As of the Queens release, Nova only supports a single type. If more than one vGPU type is specified (as a comma-separated list), only the first one will be used. | |
| 15:53:51 | dansmith | melwitt: yep done, looks legit | |
| 15:54:27 | melwitt | dansmith: ty | |
| 15:56:30 | melwitt | pvc_: I can't comment on the performance but the limitation on vGPU type means having multiple enabled_vgpu_types on one compute host. at present, you can't have more than one type on the same compute host | |
| 15:57:32 | melwitt | we are working on adding support for multiple types this cycle | |
| 15:58:12 | pvc_ | thank you melwitt, i can launch many instance on flavor with gpu resources but im worried on the performance since we will install an gpu application on it. | |
| 16:02:12 | sean-k-mooney | pvc_: be aware that supporting vGPU types on the same host will not enable multipel gpus to be consumed by a singel vm | |
| 16:03:54 | pvc_ | i login to instance then query the nvidia, may i know if this is the driver that is need to be show? 0:05.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:15f8] (rev a1) | |
| 16:05:31 | openstackgerrit | Jan Gutter proposed openstack/os-vif master: Extend port profiles with datapath offload type https://review.openstack.org/572081 | |
| 16:36:07 | melwitt | dansmith: I've been meaning to ask you if you could review our cycle priorities doc where I've written down themes https://review.openstack.org/609807 | |
| 16:37:04 | pvc_ | melwitt but it is possible to launch many instance on that flavor even if it just only one right? | |
| 16:38:18 | melwitt | pvc_: yes, it is. it's just that all the instances on the same compute host will have to use the same gpu type, for example nvidia-35 | |
| 16:38:35 | dansmith | mriedem: especially given it's spec day, could you circle back to this at some point? https://review.openstack.org/#/c/609709/3 | |
| 16:39:17 | mriedem | do i have to? | |
| 16:39:50 | pvc_ | so that is an issue on performance, so if ever i have 24 gpus on my compute node i can only use 1 right? | |
| 16:40:20 | dansmith | melwitt: ack | |
| 16:40:53 | dansmith | mriedem: no, but you were the last to -1 it, I asked a question which you never answered, and jroll has updated it | |
| 16:41:50 | mriedem | ok i hadn't seen, nor was looking | |
| 16:42:28 | mriedem | i didn't pay attention to any of this at the PTG, so if there are more details from the ptg on alternatives and such, those should be in the spec, including caveats about what happens if the conductor group for a node is updated (as i had several questions about that) | |
| 16:42:50 | mriedem | it sounds like, well that might work automagically, or it might not, shrug | |
| 16:44:06 | mriedem | and i guess we don't want anything in the hypervisors API for this because it would be too ironic specific... | |
| 16:44:26 | melwitt | pvc_: hm, looks like you're right, it says only one vGPU per instance. I don't know why that is limited https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#configure-a-flavor-controller | |
| 16:44:52 | melwitt | bauzas ^ | |
| 16:45:10 | sean-k-mooney | melwitt: its limited because libvirt cannot support multiple vgpus on a singel instance currently | |
| 16:45:21 | melwitt | sean-k-mooney: thanks | |
| 16:45:51 | pvc_ | yes but the problem is we launch instance it will use the same vgpu enabled on the nova.conf, is there no way to use the another vgpu? | |
| 16:46:06 | sean-k-mooney | pvc_: that is not how that works | |
| 16:46:21 | pvc_ | it's just for sharing sean-k-mooney? | |
| 16:46:25 | sean-k-mooney | what you are enabling in the nova.conf is the type of vgpu | |
| 16:46:55 | dansmith | mriedem: there are probably sequencing recommendations for when you change the mappings, but I think those are docs and don't need to be in the spec in great detail, IMHO. I don't think the behaviors are really any different than just spinning up a new or stopping a hashring-balanced compute today | |
| 16:46:56 | pvc_ | so if i use nvida-211 | |
| 16:47:05 | sean-k-mooney | your are not enabvling a specific mdev instnce jsut the type of mdev that will be allocated when a vm requests a vgpu | |
| 16:47:37 | pvc_ | so in terms of performance it is okay sean-k-mooney? | |
| 16:47:46 | dansmith | mriedem: if there's more that needs to be here, let's tell them what that is and let them move on | |
| 16:48:27 | sean-k-mooney | pvc_: mdevs are not shared between instances. so the perfromance of the vgpus will be determined by the mdev type you select | |
| 16:48:44 | pvc_ | so if ever i have 3 mdev types ( nvidia-1, nvidia-2, nvidia-3), i use the nvidia-1 on nova.conf. What will happen to the 2 mdev types? | |
| 16:49:02 | sean-k-mooney | higher perfroamce mdev types consume more physical resouce and allow less vgpus to be createed so its a tradeoff | |
| 16:49:32 | sean-k-mooney | if you set nvidia-1 the nova will only create mdevs of the nvida-1 type | |
| 16:50:22 | sean-k-mooney | there are stict rules about if and how mdev types can be mixed | |
| 16:50:41 | sean-k-mooney | they are vendor specific so initally we chose not to allow mixing them | |
| 16:51:02 | pvc_ | is that okay to be use for many instances? | |
| 16:51:49 | sean-k-mooney | yes we will limit you to the number of mdevs that the enabled type reporst it can allocate | |
| 16:52:03 | sean-k-mooney | there no over subsiptoin or sharing of vgpus | |
| 16:52:50 | pvc_ | so if ever i have 24 mdev types i can launch 24 instance, correct me if im wrong. | |
| 16:53:20 | sean-k-mooney | pvc_: no | |
| 16:54:33 | sean-k-mooney | an mdev type is like a nove flavor. its corresponed to a set of phyical resouces form the gpu. in the nvida case its a set of max reolutions, cuda cores, and vRAM | |
| 16:54:52 | sean-k-mooney | this is all detailed in the nvidia documentation | |
| 16:55:09 | pvc_ | thank you sean-k-mooney | |
| 16:55:28 | pvc_ | so if i have a chance to launch an instance is it okay and nova will tell me if ever i cannot allocate omre | |
| 16:55:55 | sean-k-mooney | yes you will get a no valid host error if all vpus have been consumed | |
| 16:56:43 | sean-k-mooney | nova will read the number of allication that can be made form the mdev-type specified by the nova.conf and report that to placement | |
| 16:57:09 | pvc_ | i understand now, it is okay if you only specifiy one enabled_vgpu_types on nova.conf. But by default it will consume the mdev types. Thank you :) | |
| 16:57:21 | pvc_ | But by default it will consume the all mdev types* | |
| 16:57:23 | pvc_ | am i right? | |
| 16:58:10 | sean-k-mooney | pvc_: no if you dont specyu an enabled vgpu type the no vgpus will be consumable | |
| 16:58:31 | pvc_ | i already specify one enabled ( example nvidia-160) | |
| 17:00:34 | pvc_ | thank you sean-k-mooney. :) | |
| 17:01:49 | sean-k-mooney | pvc_: you should look at https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#vgpu-types-tesla-p100 to determin how your tesla P100 can be subdevied | |
| 17:03:57 | pvc_ | noted on this. thanks for your help. :) | |
| 17:07:35 | openstackgerrit | Merged openstack/nova-specs master: Document Stein review priorities https://review.openstack.org/609807 | |
| 17:17:04 | openstack | Launchpad bug 1787910 in OpenStack Compute (nova) rocky "OVB overcloud deploy fails on nova placement errors" [High,Fix committed] - Assigned to Matt Riedemann (mriedem) | |
| 17:17:04 | melwitt | dtantsur: I've been meaning to ask you a question about this ironic bug we fixed around RC time https://bugs.launchpad.net/nova/+bug/1787910 in comment #2 you mentioned the regression broke the ironic-inspector CI upstream. do you happen to know why the ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa job we run in nova did not break the same way? | |
| 17:17:17 | dtantsur | melwitt: looking | |
| 17:18:17 | melwitt | thanks. I don't know the differences between the ironic-inspector job and our job | |
| 17:18:35 | dtantsur | melwitt: okay, so tripleo got broken because it still used disk/ram filters with ironic | |
| 17:18:38 | melwitt | trying to learn if there's a test coverage gap we can close to catch more issues | |
| 17:18:54 | dtantsur | as to inspector, I don't remember why exactly I mentioned that. but there is at least one difference | |
| 17:19:17 | dtantsur | ironic CI does not set vcpus/memory_mb on nodes, which leads to them not exposed to nova for some time | |
| 17:19:29 | dtantsur | ironic-inspector, as part of its functioning, discovers these properties and sets them | |
| 17:19:45 | sean-k-mooney | dtantsur: i was under the impression we ended up keeping the disk/ram filter for that reason in rocky and it was going to be fixed in ironic this cycle | |
| 17:20:24 | dtantsur | sean-k-mooney: I don't think it was for this reason, but I may be missing something | |
| 17:20:40 | dtantsur | for ironic these properties (and filters) are optional since.. mmm.. pike? maybe queens | |
| 17:21:23 | melwitt | dtantsur: thanks for the pointer | |
| 17:21:40 | dtantsur | np | |
| 17:21:46 | sean-k-mooney | dtantsur: we defintly had a bug in the RC period related to them during rocky release. mriedem do you remeber what the bug related to ironc and the disk/ram filter was? | |
| 17:22:25 | dtantsur | sean-k-mooney: the link melwitt posted above? yeah, essentially disk/ram filter stopped working for ironic. since nobody cares about this case any more, we just ended up fixing tripleo to not enable these filters. | |
| 17:22:33 | dtantsur | no further investigation was done IIUC | |
| 17:23:17 | sean-k-mooney | dtantsur: maybe not sure | |