| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-21 | |||
| 13:04:21 | jaypipes | sean-k-mooney: yeah, the func tests should *not* take more than around 20 minutes max. | |
| 13:04:33 | efried | bauzas: You around? | |
| 13:05:13 | gibi | jaypipes, sean-k-mooney: I think we had to increase the functional timeout recently due to slowness of the instances running the tests in CI | |
| 13:05:23 | sean-k-mooney | jaypipes: its stange i can see the func tests running fine until here http://logs.openstack.org/72/553072/3/check/nova-tox-functional-py35/6ccdce3/job-output.txt.gz#_2018-03-20_20_02_28_641084 and then it just stops outputing until it hits the time out | |
| 13:05:48 | gibi | jaypipes, sean-k-mooney: https://review.openstack.org/#/c/537933/ | |
| 13:06:06 | sean-k-mooney | gibi: perhaps but it looks like the func tests are haning after like 5 mins | |
| 13:06:17 | gibi | sean-k-mooney: OK then that is a different issue | |
| 13:06:32 | gibi | sean-k-mooney: anyhow the current func test timeout is 60 minutes | |
| 13:06:44 | sean-k-mooney | gibi: ya that should be more then enough | |
| 13:08:24 | jaypipes | indeed. | |
| 13:12:52 | openstackgerrit | Jon Schlueter proposed openstack/os-vif stable/queens: Fix VF-rep lookup routine to use parent PF number https://review.openstack.org/554917 | |
| 13:21:22 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: [WIP] Cleanup RP and HM records while deleting a compute service. https://review.openstack.org/554920 | |
| 13:23:39 | kashyap | sahid: Hey, thanks for the review here: https://review.openstack.org/#/c/534384/ | |
| 13:24:01 | kashyap | Looking now... | |
| 13:37:06 | openstackgerrit | Merged openstack/nova stable/pike: Add regression test for BFV+IsolatedHostsFilter failure https://review.openstack.org/543602 | |
| 13:40:35 | efried | bauzas: I wanted to get some feedback on vGPU inventorying/reporting through the Nova API, when you've got a minute. | |
| 13:47:31 | bauzas | efried: sure ? | |
| 13:48:40 | efried | bauzas: What I'm wondering is: 1) Does the host's inventory of (v)GPUs show up in some nova API?; 2) Once you've created an instance with a vGPU, does that vGPU show up anywhere on the instance when you query it through the nova API? | |
| 13:48:50 | efried | ...and then assuming 'yes' to either/both, more details... | |
| 13:49:03 | bauzas | efried: 1) no | |
| 13:49:15 | bauzas | efried: 2) it's just a new PCI device | |
| 13:50:13 | efried | bauzas: Okay, cool. | |
| 13:51:58 | efried | edmondsw: ^ | |
| 13:52:57 | edmondsw | bauzas any plans to address that gap? | |
| 13:54:07 | efried | jaypipes, edleafe, cdent: Is there a Nova API (as opposed to a placement API) where I can see an instance's allocations? In particular, for an arbitrary resource class I sent through via flavor extra_specs. | |
| 13:54:36 | bauzas | edmondsw: you mean, how to know the vGPU inventory by the API ? | |
| 13:54:44 | bauzas | well, good question | |
| 13:54:49 | bauzas | I haven't planned it yet | |
| 13:55:05 | bauzas | using the Placement API, that said, you can ask for it | |
| 13:55:12 | cdent | efried: I don't know, but you might dig around in the instance object to see which objects it links to and what method it and the have | |
| 13:56:22 | jaypipes | efried: os-simple-tenant-usage | |
| 13:56:27 | jaypipes | efried: is about the closest. | |
| 13:56:35 | jaypipes | efried: doesn't look at placement though. | |
| 13:56:50 | jaypipes | efried: and doesn't look at flavor extra specs :( | |
| 13:57:02 | bauzas | yeah, for the moment, Placement API rather | |
| 13:57:04 | efried | so resources other than proc, mem, disk, and PCI won't appear | |
| 13:57:12 | cdent | efried: did you mean HTTP API? | |
| 13:57:16 | bauzas | efried: for vGPUs yeah | |
| 13:57:36 | efried | cdent: Um, I think so? | |
| 13:57:54 | efried | As opposed to, like, driver methods or whatever. | |
| 13:58:11 | efried | Not sure whether it's HTTP or RPC - but yeah, remote officially-supported API. | |
| 13:58:14 | efried | I guess it's HTTP. | |
| 13:58:41 | mriedem | vgpu is stored as a vgpu resource / allocation in placement isn't it? | |
| 14:03:44 | bauzas | mriedem: that's correct, a specific RC | |
| 14:04:06 | bauzas | so, specific inventories and allocations | |
| 14:05:18 | bauzas | I have a devstack running somewhere | |
| 14:05:21 | bauzas | I can show it | |
| 14:05:44 | bauzas | mriedem: replied on https://bugs.launchpad.net/nova/+bug/1752463 | |
| 14:05:46 | openstack | Launchpad bug 1752463 in OpenStack Compute (nova) "Attaching virtual GPU devices to guests in nova" [Medium,Incomplete] | |
| 14:05:55 | bauzas | mriedem: thoughts on providing docs related to a specific device ? | |
| 14:06:02 | mriedem | yeah so vgpu will show up in placement, and you can use osc-placement to get the CLI | |
| 14:06:20 | mriedem | simple tenant usage is going to be woefully out of date wrt the new fangled placement resources | |
| 14:06:33 | bauzas | mriedem: IMHO, those kinds of driver-specific docs should be done downstream (like with RH OSP) | |
| 14:06:55 | edmondsw | use case 1... need to query inventory data (type, whether / to what VM they are allocated, etc.) of GPUs and vGPUs on a given host | |
| 14:07:04 | bauzas | mriedem: but if you'd like to get some nvidia specific details in https://docs.openstack.org/nova/queens/admin/virtual-gpu.html , lemme know | |
| 14:07:05 | mriedem | bauzas: a small 'driver notes' section or something for known issues seems ok in the openstack docs | |
| 14:07:14 | bauzas | mriedem: okay,n | |
| 14:07:18 | bauzas | mriedem: I can add a note then | |
| 14:07:19 | edmondsw | use case 2... need to query inventory data of GPUs and vGPUs allocated to a given VM | |
| 14:07:22 | mriedem | doesn't answer my question if https://review.openstack.org/#/c/459753/ handles this | |
| 14:07:24 | mriedem | for nvidia | |
| 14:07:31 | edmondsw | I think we can get some of that from placement, but not all? | |
| 14:07:32 | bauzas | mriedem: no, it's not helping | |
| 14:07:46 | mriedem | edmondsw: i think you can get all of that from placement | |
| 14:07:48 | bauzas | mriedem: hiding is just for PCI passthrough | |
| 14:08:02 | mriedem | edmondsw: and with what's already available in the CLI https://docs.openstack.org/osc-placement/latest/index.html | |
| 14:08:21 | bauzas | mriedem: the problem with OVH is that they use some product lines that are not accepted by the nvidia driver :p | |
| 14:08:35 | bauzas | mriedem: so they hide the fact they're virtualizging | |
| 14:08:58 | bauzas | mriedem: but in order to make virtual GPUs, you need a specific product line anyway | |
| 14:10:47 | edmondsw | mriedem can you query the placement API by server instance? | |
| 14:10:52 | bauzas | shit, pilgrimstack isn't here | |
| 14:11:28 | edmondsw | if not, then placement doesn't give you use case 2 | |
| 14:11:30 | bauzas | can't remember the product that OVH used for their production boxes, but it's not a Tesla line | |
| 14:11:45 | bauzas | hence the need for hiding the virt driver | |
| 14:13:26 | bauzas | edmondsw: https://developer.openstack.org/api-ref/placement/#list-allocations | |
| 14:13:44 | bauzas | edmondsw: the consumer_uuid is the instance UUID | |
| 14:14:00 | mriedem | edmondsw: yes | |
| 14:14:01 | edmondsw | bauzas ah tx | |
| 14:14:06 | bauzas | edmondsw: it'll show up the existing allocations for each RP | |
| 14:14:18 | mriedem | edmondsw: in the CLI, the instance == the consumer | |
| 14:14:33 | mriedem | edmondsw: https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-allocation-show | |
| 14:14:37 | mriedem | that's the one you want i think | |
| 14:14:43 | mriedem | shows allocations for a given consumer (instance) | |
| 14:15:03 | mriedem | https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-inventory-show shows inventory for a given provider (compute node) | |
| 14:15:11 | edmondsw | thanks | |
| 14:15:38 | mriedem | if you love that stuff, https://review.openstack.org/#/q/status:open+project:openstack/osc-placement+branch:master+topic:bp/placement-osc-plugin-rocky | |
| 14:15:42 | mriedem | reviews welcome | |
| 14:20:12 | bauzas | jaypipes: cfriesen: stephenfin: efried: thanks for your reviews on https://review.openstack.org/#/c/552924/3 | |
| 14:20:25 | bauzas | I need to look at all of them | |
| 14:22:25 | Kevin_Zheng | Hi, could anyone kindly provide some suggestions on funcional tests in https://review.openstack.org/#/c/553288/ | |
| 14:24:39 | bauzas | jaypipes: efried: stephenfin: could we maybe do a hangout based on https://review.openstack.org/#/c/552924/3/specs/rocky/approved/numa-topology-with-rps.rst ? | |
| 14:25:13 | jaypipes | bauzas: maybe tomorrow morning? | |
| 14:25:23 | jaypipes | bauzas: currently working on a number of things that might affect that | |
| 14:25:25 | Kevin_Zheng | I'm adding 'request_id' field to the nofitications, while testing, I have to compare the 'request_id' field in the payload with the reference, as req_id is in the response header, so it seems I have to modify the return value in common methods in nova.tests.functional.api.client, don't know if this is a suitable approach or not | |
| 14:25:44 | bauzas | jaypipes: I'm not in a rush | |
| 14:25:56 | efried | bauzas: I'm game. Just let me know. | |
| 14:26:06 | bauzas | jaypipes: I just felt that sharding our resources between NUMA nodes is at risk | |
| 14:26:20 | bauzas | jaypipes: hence the use of specific RCs | |
| 14:26:48 | openstackgerrit | Dan Smith proposed openstack/nova stable/queens: Add --by-service to discover_hosts https://review.openstack.org/554600 | |
| 14:30:54 | stephenfin | bauzas, jaypipes: Tomorrow afternoon (GMT) would be OK with me, yes | |