| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-12 | |||
| 11:46:31 | openstackgerrit | Zhenyu Zheng proposed openstack/nova-specs master: Amend allow abort live migrations in queued status spec https://review.openstack.org/560872 | |
| 11:58:09 | bhagyashris | alex_xu: Hi, I just want to know about the spec https://docs.openstack.org/oslo.config/latest/reference/mutable.html which basically give the provision to make the conf parameter as mutable | |
| 11:59:51 | Shilpa | stephenfin: checked logs http://logs.openstack.org/72/550172/2/check/tempest-full/f6945b6/job-output.txt.gz, and observed that 1 test case is failed, and that is tempest.api.compute.servers.test_novnc.NoVNCConsoleTestJSON.test_novnc[id-c640fdff-8ab4-45a4-a5d8-7e6146cbd0dc] | |
| 11:59:52 | alex_xu | bhagyashris: maybe gcb is good person, but looks like he isn't in this channel | |
| 11:59:53 | bhagyashris | alex_xu: there is one patch submitted in nova https://review.openstack.org/#/c/319203/3 so what are the criteria to make the conf parameter as mutable in any project | |
| 12:00:48 | alex_xu | bhagyashris: ok | |
| 12:01:53 | bhagyashris | alex_xu: as per the operators point of view every parameter should be make it as mutable but why only these two parameters are made it as mutable | |
| 12:02:01 | openstackgerrit | Matthew Booth proposed openstack/nova master: Give volume DriverBlockDevice classes a common prefix https://review.openstack.org/526346 | |
| 12:02:02 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add DriverLocalImageBlockDevice https://review.openstack.org/526347 | |
| 12:02:03 | openstackgerrit | Matthew Booth proposed openstack/nova master: Expose driver_block_device fields consistently https://review.openstack.org/528362 | |
| 12:02:04 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add local_root to block_device_info https://review.openstack.org/529029 | |
| 12:02:05 | openstackgerrit | Matthew Booth proposed openstack/nova master: Pass DriverBlockDevice to driver.attach_volume https://review.openstack.org/528363 | |
| 12:02:06 | openstackgerrit | Matthew Booth proposed openstack/nova master: Fix libvirt volume tests passing invalid disk_info https://review.openstack.org/529328 | |
| 12:03:06 | bhagyashris | alex_xu: in this patch https://review.openstack.org/#/c/319203/3 | |
| 12:05:28 | alex_xu | bhagyashris: I guess that BP only target to something people just need at that point | |
| 12:07:28 | openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Deduplicate notification samples Rocky - 7 https://review.openstack.org/560748 | |
| 12:07:40 | alex_xu | bhagyashris: I'm not sure all the config option works, but I also didn't see any reject for other options other than that BP | |
| 12:10:59 | bhagyashris | alex_xu: Actually I have few queries in nova only those two parameters are marked as mutable = True in (https://review.openstack.org/#/c/319203/3) so is that means other all the conf parameters are non-mutable. | |
| 12:11:40 | openstackgerrit | pippo proposed openstack/nova master: Update eventlet https://review.openstack.org/560876 | |
| 12:32:41 | efried | Good UGT morning bhagyashris | |
| 12:33:33 | efried | bhagyashris: Are you going to take over https://review.openstack.org/560444 and https://review.openstack.org/560459 ? | |
| 12:34:55 | bhagyashris | efried: Good morning :) | |
| 12:42:22 | bhagyashris | efried: ok i will take. | |
| 12:43:34 | efried | bhagyashris: Great. Can you put your name down on the etherpad https://etherpad.openstack.org/p/rocky-nova-priorities-tracking see L115 | |
| 12:43:54 | efried | jaypipes: FYI ^ | |
| 12:47:39 | bauzas | stephenfin: quick question, if I'm providing a extra spec saying --property hw:numa_nodes=N, does it mean that the instance will use 2 physical NUMA nodes, or just that the instance XML will have 2 nodes ? | |
| 12:48:13 | bauzas | if the latter, are we supporting to not have sharded the NUMA nodes ? | |
| 12:48:37 | efried | sean-k-mooney, cfriesen: ^ | |
| 12:48:40 | bauzas | stephenfin: context is https://review.openstack.org/#/c/552924/7/specs/rocky/approved/numa-topology-with-rps.rst@185 | |
| 12:52:02 | bauzas | efried: as you can see, I wonder if we would still need a specific sharding param | |
| 12:52:47 | bauzas | efried: because when we discussed yesterday, I was only thinking of vGPUs affinited to CPUs, not about vCPUs being anti-affinited by NUMA nodes | |
| 12:53:05 | bauzas | so the latter would need something missing now | |
| 12:53:21 | bauzas | and then, in that case, that would be in my spec : | |
| 12:57:33 | efried | bauzas: Yup; and I totally can't answer that question (whether we need explicit anti-affinity/sharding) | |
| 12:57:47 | bauzas | efried: when looking at the doc, see " FLAVOR-NODES: (integer) The number of host NUMA nodes to restrict execution of instance vCPU threads to. If not specified, the vCPU threads can run on any number of the host NUMA nodes available." | |
| 12:57:47 | efried | But your spec definitely must. | |
| 12:57:51 | bauzas | https://docs.openstack.org/nova/latest/user/flavors.html#extra-specs-numa-topology | |
| 12:58:17 | efried | bauzas: Oh, that's interesting. I read that as a *maximum*. | |
| 12:58:41 | efried | In which case, we're covered. | |
| 12:58:50 | efried | sort of | |
| 12:58:59 | efried | We need to split into N numbered request groups | |
| 12:59:24 | efried | The only issue is that we can't specify that in a flexible way - it would have to be specific numbers of procs in each group. | |
| 12:59:36 | efried | and the only reasonable way to do that is to divide them evenly. | |
| 13:00:03 | efried | That still means we get *at most* N separate NUMA nodes, but because granular doesn't guarantee separation, we could get anywhere from 1..N | |
| 13:00:13 | bauzas | that said, in https://docs.openstack.org/nova/latest/admin/cpu-topologies.html | |
| 13:00:38 | bauzas | it says " Inadequate per-node resources will result in scheduling failures. Resources that are specific to a node include not only CPUs and memory, but also PCI and SR-IOV resources. It is not possible to use multiple resources from different nodes without requesting a multi-node layout. As such, it may be necessary to ensure PCI or SR-IOV resources are associated with the same NUMA node or force a multi-node layout." | |
| 13:01:02 | efried | That's fair. | |
| 13:01:22 | bauzas | what I understand from the above is that if I'm asking for 2 nodes but then only have 1 node, then NoValidHost | |
| 13:01:33 | bauzas | that's honeslty confusing | |
| 13:01:35 | efried | no, that's not how I read it. | |
| 13:01:52 | bauzas | also given https://bugs.launchpad.net/nova/+bug/1466780 and " There is no correlation required between the NUMA topology exposed in the instance and how the instance is actually pinned on the host. This is by design. See this invalid bug for more information." | |
| 13:01:52 | openstack | Launchpad bug 1466780 in OpenStack Compute (nova) "nova libvirt pinning not reflected in VirtCPUTopology" [Undecided,Invalid] - Assigned to Stephen Finucane (stephenfinucane) | |
| 13:02:06 | bauzas | but I take that only for CPU pinning | |
| 13:02:15 | efried | I read it to mean, "if you're restricting to one NUMA node, but no single node has all the resources available, NoValidHost" Which is totally legit. | |
| 13:02:35 | bauzas | let's wait for the others, then | |
| 13:02:37 | kholkina | the spec for user-data update need your review https://review.openstack.org/#/c/547964/ | |
| 13:02:43 | efried | But it brings us to a different issue, which is whether your spec claims to handle device affinity as well as proc/mem | |
| 13:03:05 | bauzas | efried: that said, if we need to shard, what kind of query param are you thinking ? | |
| 13:03:33 | openstackgerrit | sahid proposed openstack/nova master: libvirt: move version to string in utils https://review.openstack.org/560455 | |
| 13:03:34 | openstackgerrit | sahid proposed openstack/nova master: libvirt: refactor get_base_config to accept host arg https://review.openstack.org/560456 | |
| 13:03:35 | openstackgerrit | sahid proposed openstack/nova master: libvirt: add support for virtio-net rx/tx queue sizes https://review.openstack.org/484997 | |
| 13:03:36 | bauzas | that's the problem with numbered request groups | |
| 13:03:41 | efried | bauzas: For explicitly separating request groups to separate resource providers? I was thinking something like ?separate_providers=resources1,resources2,... | |
| 13:03:55 | efried | ...if placement is where we want to handle it. | |
| 13:04:26 | efried | bauzas: But the other option is to handle it in the NUMATopologyFilter. Placement would give us back all the candidates, which would include the ones that are sharded and the ones that are combined. And the filter would just pick the ones that are appropriately sharded. | |
| 13:04:35 | bauzas | so, ?separate_providers=resources1,resources2&resources1:VCPU=1&resources2:VCPU=1 ? | |
| 13:04:44 | efried | bauzas: yes | |
| 13:05:14 | efried | Implementing that will be a bear, but yes. | |
| 13:05:20 | bauzas | if placement folks are accepting that, then I'd prefer to do that by Placement instead of the filter | |
| 13:05:34 | bauzas | because the less we have in the filter, the better it will be | |
| 13:05:42 | efried | I don't see it making Rocky, tbh | |
| 13:05:46 | bauzas | longer term of course | |
| 13:05:50 | bauzas | efried: yeah, I know | |
| 13:06:00 | bauzas | efried: what I'd love is some consensus on that spec for Rocky | |
| 13:06:03 | efried | But yes, eventually we recognize we're going to need that functionality in placement. | |
| 13:06:20 | efried | bauzas: which spec? | |
| 13:06:21 | bauzas | but then, earlyj working on Rocky-3 if I'm lucky so we can land Stein-1 | |
| 13:06:29 | bauzas | efried: the NUMA one | |
| 13:06:53 | efried | bauzas: Sure, agreed; but we need to crisp up what exactly you're trying to address, and how. | |
| 13:07:00 | efried | That's not yet clear IMO | |
| 13:07:19 | bauzas | efried: for Rocky, I'm only planning to implement nested RPs for vGPUs and fix the vGPU caveats | |
| 13:07:25 | bauzas | plus that spec | |
| 13:07:34 | bauzas | but implementing that spec for Rocky-3 or later | |
| 13:07:49 | bauzas | unless someone picks the ball | |
| 13:07:54 | efried | Okay, gotcha. | |
| 13:08:28 | efried | So you're not thinking to land the code related to the NUMA spec until "later". But you want the spec baked by Rocky-3 | |
| 13:10:03 | stephenfin | bauzas: The latter | |
| 13:10:32 | efried | johnthetubaguy_: You around? I approved https://review.openstack.org/#/c/553605/ on the basis that your -1 has now been addressed, but there's still time to pull it out if you disagree. | |
| 13:10:39 | stephenfin | bauzas: typically those guest nodes would be scheduled to different NUMA nodes, but if that's not possible they'll be squeezed onto the same one | |
| 13:11:00 | bauzas | efried: that's correct | |
| 13:11:00 | stephenfin | bauzas: At least that's the case to the best of my recollection | |
| 13:11:09 | efried | ahjeez. "anti-affinity preferred" is a use case we haven't even thought about yet. | |
| 13:11:14 | bauzas | efried: trying to get a consensus for Rocky-1 or Rocky-2 | |
| 13:11:25 | bauzas | efried: while I'm implementing other things | |
| 13:11:35 | efried | bauzas: roger that | |
| 13:11:42 | bauzas | efried: but once I'm done with the other things, going back to NUMA for implementing | |
| 13:12:04 | bauzas | honestly, if we don't need new Placement version, I think it can be done very quickly | |
| 13:12:07 | bauzas | efried: ^ | |
| 13:12:21 | efried | yes | |
| 13:12:25 | bauzas | efried: it's just about translating specs into Placement queries | |