Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-04-27
16:25:17 finucannot If we do that, the only resources the cores will be sharing is cache and a couple of buses
16:25:43 sean-k-mooney finucannot: when you say cache you mean l3 cache
16:26:05 finucannot and unless you're planning on dedicating every core on the NUMA node to the guest, you're still at risk of other guests on that node screwing with said cache
16:26:12 finucannot sean-k-mooney: yes on both counts
16:26:59 sean-k-mooney l1 cache is per HT l2 cache is shared between HT on same physical core and l3 cache is shared across socket(with cluster on die ist across socket but 1 regoin per numa node)
16:27:30 sean-k-mooney * /across socket/within socket/
16:28:03 finucannot Yup, that was my understanding of things too
16:28:46 finucannot lstopo is a nifty tool to that end
16:28:56 finucannot Aaaaaanyway, it's Friday evening and thisaguy is going to watch Thanos get his ass kicked (hopefully)
16:30:31 openstack bugzilla.redhat.com bug 1468217 in kernel-rt "KVM: paravirt raw_spinlock priority bump for housekeeping vcpus" [Unspecified,Closed: wontfix] - Assigned to mtosatti
16:30:31 sahid that is one example: https://bugzilla.redhat.com/show_bug.cgi?id=1468217, it's really easy to increase latency on realtime application. my thinking is we should not fake guest numa topology, our current behavior is good and we should keep it.
16:31:19 sahid but probably suer have workaround by using some other isolation so, i'm not going to argue more of that :)
16:35:06 sean-k-mooney sahid: our current behavior is ok but suboptimal in several cases. the main one being multi numa node guest do not create multi numa pcie root trees meaning we can effectly force cross numa traffic event though we have cores of the guess on the same numa node pci device.
16:35:41 sahid sean-k-mooney: what about to have an option?
16:38:18 sahid i'm just scary about to break some specific use cases
16:38:29 sahid if we try to be simple, why user would ask for 2 nodes?
16:38:40 sahid they can just ask for 1
16:38:51 sean-k-mooney sahid: the user does not the admin does
16:39:13 sahid yeah... it's the user who is going to select the flavor, no?
16:39:18 sean-k-mooney one reason was to allow guest with hugepages to have more cores then fit on a single socket
16:40:28 sahid sean-k-mooney: you do not reply for what they could ask for 2
16:40:38 sahid s/wht/why
16:41:10 sean-k-mooney that was an example of why they would ask for 2 numa nodes
16:41:37 sean-k-mooney if you ask for one numa nodes with 16 cpus and your only have 10 cpus per numa node that vm cannont boot
16:42:22 sean-k-mooney so when operators want to allow vms with more cores tehn will fit in one host numa node the set hw:numa_nodes=2+
16:44:52 sahid sean-k-mooney: no no... guest can have 16 vCPUs, 1 numa node - the vCPUs are going to be pinned on different host numa node
16:44:59 sean-k-mooney sahid: finucannot mriedem if you want to know the full simantics of the numa stuff this is the oldes spec i can still find that covers it http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/juno/implemented/virt-driver-numa-placement.rst there were 2 before that that did not merge in icehose but
16:45:19 sahid they is no problem with that
16:45:21 sean-k-mooney sahid: no that is not allowed if hw:numa_nodes=1 is set
16:45:44 sean-k-mooney sahid: that is only allowed if hw:numa_nodes is not set at all
16:46:07 sean-k-mooney sahid: in whcich case the bevavior is virt driver defied
16:46:39 openstackgerrit Vladyslav Drok proposed openstack/nova master: Placement: allow to set reserved value equal to total for inventory https://review.openstack.org/564838
16:46:44 sahid yes, if you defined 1 it's not allowed, and that looks good, no?
16:46:59 sahid so basically you want to change that, right?
16:47:06 sean-k-mooney sahid: no
16:47:37 openstackgerrit Matt Riedemann proposed openstack/python-novaclient master: Make sure microversion < 2.62 does not show host(Id) for instance actions https://review.openstack.org/564782
16:47:56 sean-k-mooney sahid: what we said is that the virt dirver today is allowed to have 2 virtual numa nodes mapped to the same host numa node
16:48:56 sean-k-mooney sahid: a virtual numa node is never allowed to span a phyical one.
16:51:12 sahid sean-k-mooney: so it means that for an user asking for 1 or 2 virtual numa node mean the same behavior?
16:52:04 sahid but he will be happy to see 2 NUMA nodes when doing "lscpu", right? :)
16:52:25 sean-k-mooney sahid: asking for hw:numa_node=x just means you will see x numa nodes within the guest and you are guarenteed that a numa node from the guest view point will not span host numa nodes,
16:53:07 sahid not today
16:53:32 sean-k-mooney sahid: today that is what the api of the hw:numa_nodes extra spec says
16:53:46 sahid the guest vCPUs are really running on different X different numa nodes
16:54:31 sahid well the virt driver as well
16:54:34 sean-k-mooney sahid: yes but that is a libvirt dirive implentation detail. hyperv support numa tology too and at least initally it did not have the same implmentation
16:56:04 sean-k-mooney sahid: this is the most importat section of the spec in this regard http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/juno/implemented/virt-driver-numa-placement.rst#n94
16:56:59 sahid this section is just saying that, you can ask for a specific host numa node
16:57:11 sahid can't
16:58:14 sean-k-mooney that part of it but that is not the only intent of that section.
17:00:16 sean-k-mooney there is also this wiki on the topic https://wiki.openstack.org/wiki/VirtDriverGuestCPUMemoryPlacement
17:01:18 sahid ok sean-k-mooney i will try to discuss with some people iternally to ensure that is not going to break anything
17:01:53 sean-k-mooney do you have a specifc change you are proposing ot make to numa topology
17:01:56 sahid but my point is to do not let user think it's guest run 2 numa nodes where it's not
17:02:31 sean-k-mooney sahid: the guest has never had any guarteee that 2 virtual numa node ment 2 host numa nodes.
17:02:31 sahid sean-k-mooney: i'm not sure i understand?
17:02:57 sahid sean-k-mooney: please it's the current behavior and how the spec has been defined and implemented
17:03:57 sean-k-mooney sahid: i helped write that spec and that how it was impemented in the libvirt driver. anyway we are not currently proposing changing it but future virtdirver are not bound to do the same thing as libvirt
17:04:46 sahid yes sean-k-mooney me as-well so there is nothing new for me
17:04:59 sahid it's late and i want to enjoy my long weekend :)
17:05:26 sean-k-mooney sure enjoy :) i still have to resping the mtu patch
17:05:47 sahid sean-k-mooney: cool thanks for that
17:20:51 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Implement granular policy rules for placement https://review.openstack.org/524425
17:20:52 openstackgerrit Matt Riedemann proposed openstack/nova master: Deduplicate config/policy reference docs from main index https://review.openstack.org/564846
17:41:59 openstackgerrit Merged openstack/python-novaclient master: Microversion 2.62 - Add host/hostId to instance action event https://review.openstack.org/564667
18:33:28 openstackgerrit Brianna Poulos proposed openstack/nova master: Add notification support for trusted_certs https://review.openstack.org/563269
18:58:13 jgwentworth has anyone else been noticing request-id changing across nova service calls lately? like, I'll see a request start off with one request-id in nova-api and then by the time it gets to nova-conductor, it's a new one
19:00:20 fried_rice jgwentworth: Don't quote me on this, but I think there's a *global* request ID that's supposed to stay the same. The original request ID was per service, so it did change across those boundaries, but that's why they implemented the global one.
19:00:48 fried_rice jgwentworth: Are you seeing the global request ID changing across service boundaries?
19:01:04 fried_rice jgwentworth: In any case, I think cdent and dhellmann were involved in that work - they would know more.
19:01:27 jgwentworth no, I mean the request-id staying within nova is changing. it used to stay consistent per request
19:02:05 jgwentworth like you used to be able to trace the same request id from nova-api -> nova-conductor -> nova-scheduler -> nova-compute
19:02:30 jgwentworth it's true that request-ids didn't use to span across nova to glance, for example, and that's what the global request-id work was about
19:14:56 openstackgerrit Dan Smith proposed openstack/nova master: Make service all-cells min version helper use scatter-gather https://review.openstack.org/539590
19:14:57 openstackgerrit Dan Smith proposed openstack/nova master: Make host_manager use scatter-gather and ignore down cells https://review.openstack.org/539617
20:06:40 openstackgerrit Brianna Poulos proposed openstack/nova master: Implement certificate_utils https://review.openstack.org/479949
20:06:41 openstackgerrit Brianna Poulos proposed openstack/nova master: Plumb trusted_certs through libvirt driver image paths https://review.openstack.org/561262
20:06:42 openstackgerrit Brianna Poulos proposed openstack/nova master: Add trusted_image_certificates to REST API https://review.openstack.org/486204
20:06:43 openstackgerrit Brianna Poulos proposed openstack/nova master: Add notification support for trusted_certs https://review.openstack.org/563269
20:11:26 jgwentworth bpoulos: hi, just wanted to confirm, is someone from your team going to propose a spec amendment to openstack/nova-specs as described on L80 here? https://etherpad.openstack.org/p/nova-runways-rocky
20:12:06 jgwentworth once we get that reviewed and merged, we can continue to review the remaining patches in the implementation
20:12:25 bpoulos jgwentworth: yes, we have a spec amendment internally, but it has to receive internal review approval before we can post the update. I hope to post the update soon!
20:12:45 jgwentworth bpoulos: perfect, thanks! just wanted to make sure we were on the same page
20:13:05 jgwentworth we'll be on the lookout for the update
20:13:12 bpoulos jgwentworth: no problem, thank you for checking
20:22:50 openstackgerrit Jay Pipes proposed openstack/nova master: add root parent NULL online migration https://review.openstack.org/564876
21:30:53 openstackgerrit sean mooney proposed openstack/nova master: add mtu to libvirt xml for ethernet and bridge types https://review.openstack.org/553072
21:42:22 openstackgerrit Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247
22:08:55 openstackgerrit Hongbin Lu proposed openstack/nova master: Use revision_number on adding/removing security group https://review.openstack.org/535510
22:24:32 openstackgerrit Eric Fried proposed openstack/nova master: Base test module/class for functional placement db https://review.openstack.org/564590
22:24:33 openstackgerrit Eric Fried proposed openstack/nova master: Use test_base symbols directly https://review.openstack.org/564592
22:24:34 openstackgerrit Eric Fried proposed openstack/nova master: Use helpers in test_resource_provider (func) https://review.openstack.org/564638
23:06:25 fried_rice ō/
#openstack-nova - 2018-04-28
00:05:39 openstackgerrit Arvind Nadendla proposed openstack/nova master: update scheduler to use image-traits https://review.openstack.org/560596
02:43:53 openstackgerrit Lars Kellogg-Stedman proposed openstack/nova master: log stale allocations as WARNING instead of DEBUG https://review.openstack.org/564964
02:44:46 openstackgerrit Lars Kellogg-Stedman proposed openstack/nova master: log stale allocations as WARNING instead of DEBUG https://review.openstack.org/564964
06:37:25 openstackgerrit jichenjc proposed openstack/nova master: [doc] Add soft_deleted flag https://review.openstack.org/562897
07:20:50 openstackgerrit jichenjc proposed openstack/nova master: Avoid raise InstanceNotRunning exception https://review.openstack.org/541152
07:22:08 openstackgerrit jichenjc proposed openstack/nova master: remove ec2 object definitions https://review.openstack.org/557150
07:22:13 openstackgerrit jichenjc proposed openstack/nova master: remove ec2 db functions https://review.openstack.org/557572

Earlier   Later