| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-14 | |||
| 20:48:24 | efried | jroll: minor updates, fyi ^ | |
| 20:48:36 | jroll | noted | |
| 20:51:20 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Revert "openstack server create" to "nova boot" in nova docs https://review.opendev.org/693239 | |
| 20:52:11 | mriedem | amodi: done | |
| 20:52:27 | amodi | thanks! | |
| 20:52:36 | mriedem | efried: i don't even remember approving those. i think it was someone that hacked my account. | |
| 20:53:23 | efried | mriedem: This ain't discover card. You ARE responsible for charges made on your account. | |
| 20:58:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Reset vm_state to original value if rebuild claim fails https://review.opendev.org/692185 | |
| 21:27:24 | tonyb | mriedem, sean-k-mooney: Not if I do my job right. It'll be everyone's problem | |
| 21:28:30 | sean-k-mooney | power i could live with if they gave me a machine. im not sure i would have room for a mainframe in my hallway | |
| 21:29:09 | sean-k-mooney | so z is not a think i want to deal with | |
| 21:29:16 | openstackgerrit | Eric Fried proposed openstack/nova master: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934 | |
| 21:29:16 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Add emulated TPM support to Nova https://review.opendev.org/631363 | |
| 21:30:14 | tonyb | sean-k-mooney: Well I'd hope you have access to some at work, at home you're on your own but https://www.raptorcs.com/TALOSII/ is about the same price I paid for my x86_64 dev box | |
| 21:30:55 | sean-k-mooney | ya i have seen those before | |
| 21:36:07 | sean-k-mooney | tonyb: im sure there is a less painful way to get acesss to hardware at work but i generally do all my deve either my my personal server i bought or on my laptop in a vm where i fake out numa toploiges with libvirt | |
| 21:36:36 | sean-k-mooney | technically i could also just fake a power system that way too i guess i just would be slow without kvm | |
| 21:37:22 | tonyb | sean-k-mooney: Yes it works but is slow | |
| 21:38:02 | sean-k-mooney | ya its just because kvm only supprot acclerate of your native instruction set | |
| 21:39:05 | tonyb | yeah TCG is nice but slow | |
| 21:39:38 | sean-k-mooney | have you played with the multi threaded tcg backend e.g. mttcg | |
| 21:39:55 | sean-k-mooney | thats actully not bad in the little i have used it | |
| 21:40:03 | tonyb | No, I didn't know that existed | |
| 21:40:17 | sean-k-mooney | you need really new qemu | |
| 21:40:35 | sean-k-mooney | it was added for acclerating other architure mainly arm | |
| 21:40:35 | tonyb | I have F30, is that new enough? | |
| 21:41:04 | sean-k-mooney | i can check. it getts enabeld automaticaly if you cpu model is supproted | |
| 21:41:19 | tonyb | I'll go googling ;p | |
| 21:41:50 | sean-k-mooney | oh looks like you can force it too now https://wiki.qemu.org/Features/tcg-multithread | |
| 21:43:00 | sean-k-mooney | looks like aarch64 was in 2.9 | |
| 21:43:11 | sean-k-mooney | not sure if power has been added | |
| 21:48:42 | sean-k-mooney | look like mips,arm,aarch64,x86_64 and aplha have support in the latest version but no power supprot at elast form the release notes | |
| 21:48:57 | tonyb | Pfft | |
| 21:57:51 | openstackgerrit | Merged openstack/nova master: Join migration_context and flavor in Migration.instance https://review.opendev.org/689846 | |
| 21:57:57 | openstackgerrit | Merged openstack/nova master: Improve metadata server performance with large security groups https://review.opendev.org/656084 | |
| 22:04:34 | sean-k-mooney | ... | |
| 22:04:51 | sean-k-mooney | i was just about to push and then call it a day and now i have a merg conflcit | |
| 22:06:48 | openstackgerrit | sean mooney proposed openstack/nova master: support pci numa affinity policies in flavor and image https://review.opendev.org/674072 | |
| 22:06:54 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Join migration_context and flavor in Migration.instance https://review.opendev.org/694408 | |
| 22:07:33 | sean-k-mooney | ill fix that tomorrow but it now has the func tests and artoms nits adressed | |
| 22:07:39 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Improve metadata server performance with large security groups https://review.opendev.org/694409 | |
| 22:18:47 | sean-k-mooney | actully it looks trivial ill fix it now | |
| 22:19:37 | openstackgerrit | sean mooney proposed openstack/nova master: support pci numa affinity policies in flavor and image https://review.opendev.org/674072 | |
| 23:30:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Store instance action event exc_val fault details https://review.opendev.org/694428 | |
| #openstack-nova - 2019-11-15 | |||
| 00:43:13 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Expose instance action event details out of the API https://review.opendev.org/694430 | |
| 02:20:29 | openstackgerrit | Merged openstack/nova master: Move compute_node_to_inventory_dict to test-only code https://review.opendev.org/693438 | |
| 02:20:35 | openstackgerrit | Merged openstack/nova master: Remove get_minimum_version mocks from test_resource_tracker https://review.opendev.org/693439 | |
| 02:20:43 | openstackgerrit | Merged openstack/nova stable/queens: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/694357 | |
| 02:59:56 | openstackgerrit | Merged openstack/nova master: Stop using NoAuthMiddleware in tests https://review.opendev.org/687416 | |
| 03:10:17 | openstackgerrit | Merged openstack/nova master: Remove TODO from ComputeTaskManager._live_migrate https://review.opendev.org/693696 | |
| 04:31:02 | openstackgerrit | Archit Modi proposed openstack/nova stable/pike: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/694450 | |
| 05:41:05 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Add resources metadata of instance https://review.opendev.org/663563 | |
| 06:45:33 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Add resources metadata of instance https://review.opendev.org/663563 | |
| 07:18:39 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Handle new is_volume_backend join column query https://review.opendev.org/694462 | |
| 07:18:39 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Ignore root_gb if instance is booted from volume https://review.opendev.org/612626 | |
| 07:18:40 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Instance object changes for the new 'is_volume_backed' expected_attr https://review.opendev.org/694463 | |
| 08:08:32 | bauzas | good morning Nova | |
| 08:39:17 | awalende | Hello, I have trouble passing through an nvidia t4 card to my guests on Rocky. nova-compute log reports the final resource as: pci_stats=[PciDevicePool(count=1,numa_node=1,product_id='1eb8',tags={dev_type='type-PF'},vendor_id='10de')] | |
| 08:40:57 | awalende | However, when I set the alias as alias={"name":"T4","vendor_id":"10de","product_id":"1eb8"} on the controller and alias={"name":"T4","vendor_id":"10de", "product_id":"1eb8"} on compute node does not work | |
| 08:41:30 | awalende | Also I wonder, why it is declared as a "PF". I clearly inserted an PCIe card | |
| 08:42:16 | stephenfin | awalende: How have you configured the PCI whitelist in nova.conf? | |
| 08:42:40 | awalende | passthrough_whitelist={"vendor_id":"10de"} | |
| 08:45:16 | stephenfin | awalende: So this is interesting. The Tesla T4 supports SR-IOV https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-t4/t4-tensor-core-product-brief.pdf | |
| 08:46:21 | stephenfin | and because we're detecting that, we're flagging it as a PF https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6562-L6565 | |
| 08:46:44 | awalende | So there is probably a difference passing them through as other regular gpus? | |
| 08:47:45 | awalende | Can we pass them even at all? | |
| 08:47:54 | stephenfin | Possibly. I could be wrong, but I do recall that type-PF devices were disabled by default because you typically wouldn't want to use them when you talk about NICs | |
| 08:47:59 | stephenfin | but it can be enabled | |
| 08:48:08 | stephenfin | do you want to pass through the PF or the VF? | |
| 08:48:23 | awalende | the whole hardware | |
| 08:48:53 | awalende | I guess probably the PF....so the usecase is, we want to pass the whole card to the guest. so that they have to install nvidia drivers by themselfs on the guest machine | |
| 08:49:09 | awalende | like we did with our older Tesla V100 cards | |
| 08:50:06 | stephenfin | Seems like you could get better resource utilization if you passed through the VFs though (since apparently that would give you up to 16 devices) | |
| 08:50:11 | stephenfin | That doesn't matter for now though | |
| 08:51:03 | stephenfin | If you want to use type-PF devices, I _think_ you need to either (a) set the 'device_type' flag in the 'alias' config option or (b) make sure the VFs are not whitelisted | |
| 08:51:46 | awalende | ye we already tried to set 'device_type' but it didn't make a difference :( | |
| 08:52:06 | stephenfin | have you tried (b)? | |
| 08:52:16 | stephenfin | to do that, you'll need to also set the 'product_id' in the 'passthrough_whitelist' | |
| 08:52:44 | stephenfin | to whatever the product_id of the PF is, assuming the PF and VF have different product_id's (they should) | |
| 08:52:54 | awalende | my colleague means he "maybe" tried it as well. We try it again and let you know | |
| 08:53:23 | stephenfin | If they don't, you're going to need to use the 'address' field, I'm afraid | |
| 08:53:31 | stephenfin | okay, do please | |
| 08:53:59 | stephenfin | sean-k-mooney might be able to help too but they're definitely fast asleep atm :) | |
| 08:54:13 | awalende | How do you make sure that "VFs are not whitelisted"? | |
| 08:55:08 | bauzas | the T4 ? | |
| 08:55:19 | bauzas | awalende: which grid driver are you using ? | |
| 08:55:25 | bauzas | GRID8 ? | |
| 08:55:53 | awalende | no driver at all, we plugged the T4 into our hypervisor nodes and want to pass them through to guests | |
| 08:56:05 | stephenfin | awalende: If SR-IOV is enabled on the device, you will have additional PCI devices appearing at e.g. 'lspci' | |
| 08:56:19 | awalende | one moment | |
| 08:57:06 | stephenfin | 'lspci | grep -i nvidia' should flag them, I'd imagine | |
| 08:57:07 | bauzas | oh right, then that's normal | |
| 08:57:26 | bauzas | Nvidia plans to deliver virtual GPUs as VFs later | |
| 08:57:36 | bauzas | I guess they already started to expose the GPU as a PF | |
| 08:57:44 | awalende | stephenfin, it only shows me one card as af:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) | |
| 08:57:45 | stephenfin | I'm not sure how you'd turn them on/off. Probably by echoing stuff to sysfs like you do for SR-IOV NICs, I guess | |
| 08:57:48 | bauzas | but I thought the Tesla models wouldn't get it | |
| 08:57:55 | bauzas | just the Volta ones | |
| 08:58:20 | bauzas | I'm not a nVidia salesperson, (un?)fornutately :p | |
| 08:58:23 | stephenfin | okay, it's probably disabled so | |