| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-11-27 | |||
| 12:50:11 | sean-k-mooney | nkorabli: that depends on your definition of assigin to a numa cell | |
| 12:50:51 | sean-k-mooney | nkorabli: you can request a vm to have a single virtual numa cell in which case openstack will create it and pin it to a single numa cell | |
| 12:51:30 | sean-k-mooney | nkorabli: you cannot however say that i want to boot this vm on numa cell 0 of host X. that would break the cloud abstration | |
| 12:57:01 | openstackgerrit | Claudiu Belu proposed openstack/nova master: POC: tests: autospecs all the mock.patch usages https://review.openstack.org/470775 | |
| 12:59:05 | nkorabli | sean-k-moooney: I played a bit with that - I had single compute node with 2 numa cells(nodes). I launched a lot of VMs with a single CPU. Then checked how those VMs got distributed across those numa cells. Apparently, all of them got pinned to the first numa node and none to second. | |
| 12:59:45 | sean-k-mooney | are you using openstack kilo | |
| 13:00:02 | sean-k-mooney | or using a pci passthrough device. | |
| 13:00:03 | nkorabli | I used Liberty for that | |
| 13:00:10 | sean-k-mooney | thats your issue | |
| 13:00:26 | nkorabli | I also noticed this in the code: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4099-L4104 | |
| 13:00:42 | sean-k-mooney | liberty had a bug where the numa virtual numa node was used to select the phyical numa node | |
| 13:00:57 | sean-k-mooney | that was expcitly never intended to be the behavior | |
| 13:03:11 | nkorabli | So, this behavior shouldn't be the same in older releases? I supposed that according to the link above - this issue hasn't been fixed yet..' | |
| 13:04:44 | sean-k-mooney | no you are miss understanding what i said. https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4099-L4104 simply states that if we do not request a numa topology for a vm we leave the kernel spread the instaces. | |
| 13:05:51 | sean-k-mooney | by default vms created in openstack are not assumed to have memory or cpus that come from a single numa node | |
| 13:08:41 | sean-k-mooney | nkorabli: are several indirect ways that a numa topology can be requested on your behalf without you seting the hw:numa_nodes extra spec. | |
| 13:09:47 | sean-k-mooney | nkorabli: for example if you request hugepages a request for a singel numa node will implicitly be created. unless your request a numa topology via hw:numa_nodes | |
| 13:13:12 | nkorabli | If I specified `hw:numa_nodes=1` then it will use just one cell, right? (like `cellid=0`). I can't specify `hw:numa_nodes=2` for a single CPU instance either. So, I can't explicitly make use of both numa nodes when working with single CPU VMs? | |
| 13:15:56 | sean-k-mooney | nkorabli: if you specify hw:numa_nodes=1 for a single core vm flavor that vm can be placed on either of numa nodes on the host system but it will be affinitesed to which ever node it first starts on | |
| 13:16:49 | sean-k-mooney | nkorabli: if you dont spceify hw:numa_nodes at all it will implcitly float across all numa nodes and cores at the whim of the host kernel scheduler | |
| 13:18:28 | sean-k-mooney | nkorabli: to be more precise it will float across the cores specified via the vcpu_pin_set option in the nova.conf | |
| 13:19:01 | sean-k-mooney | nkorabli: if vcpu_pin_set is not specified it defaults to all cores | |
| 13:21:45 | nkorabli | alright, that's great! I was expecting behavior like this, just couldn't reproduce in Liberty. When I specified hw:numa_nodes=1, it started throwing errors like this: "No valid host was found. There are not enough hosts available." So, I thought it fills up the first node and never sees the second. | |
| 13:23:37 | nkorabli | This line also made me think this way: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L1976 | |
| 13:24:15 | nkorabli | If I understand you right, this is an issue that appears in Liberty only? | |
| 13:25:32 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L1976 is referring to the virtual numa topology of the guest. not the host numa topology | |
| 13:26:32 | sean-k-mooney | the first release of openstack with numa support did a 1:1 mapping between the virtual cellid to the host cellid howerver that was a mistake that was correct in later releases | |
| 13:28:48 | sean-k-mooney | nkorabli: there is also a rather little known option hw:numa_mempolicy which defaults to strict which allows you to specify if the guest memory must come from the same numa node as the vCPUS | |
| 13:29:11 | sean-k-mooney | i cant find the upstream docs but its memtioned here in the redhat docs https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/6/html/Administration_Guide/section-flavors.html | |
| 13:30:12 | efried | edleafe Sched meeting is in half an hour, right? | |
| 13:36:04 | cdent | efried: appears so | |
| 13:36:25 | efried | cdent Takes me a couple weeks to get the time shift right. DST is stupid. | |
| 13:36:33 | cdent | yes and yes | |
| 13:37:01 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform instance.resize_revert notification https://review.openstack.org/396811 | |
| 13:39:48 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform instance.resize_revert notification https://review.openstack.org/396811 | |
| 13:43:54 | nkorabli | sean-k-mooney: thanks a lot! Let me test that out! | |
| 13:52:15 | openstackgerrit | Claudiu Belu proposed openstack/nova master: POC: tests: autospecs all the mock.patch usages https://review.openstack.org/470775 | |
| 13:54:09 | edleafe | efried: yes, scheduler subteam meeting in 6 minutes in #openstack-meeting-alt | |
| 13:54:16 | efried | thx | |
| 13:56:22 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform instance.exists notification https://review.openstack.org/403660 | |
| 13:56:23 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add sample test for instance audit https://review.openstack.org/480955 | |
| 14:18:18 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform instance.exists notification https://review.openstack.org/403660 | |
| 14:18:19 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add sample test for instance audit https://review.openstack.org/480955 | |
| 14:41:09 | alex_xu | efried: looks like https://review.openstack.org/#/c/498737/2/nova/tests/functional/db/test_resource_provider.py@2981 same with https://review.openstack.org/#/c/518633/15/nova/tests/functional/db/test_allocation_candidates.py@1719 | |
| 14:42:12 | alex_xu | efried: I didn't found one similar to this yet https://review.openstack.org/#/c/498737/2/nova/tests/functional/db/test_resource_provider.py@3207 | |
| 14:46:03 | mriedem | gibi: are you looking for changes on this patch? https://review.openstack.org/#/c/516782/ | |
| 14:46:09 | mriedem | i see you didn't -1 it | |
| 14:46:54 | mriedem | alex_xu: regarding https://review.openstack.org/#/c/522305/ i'm inclined to just fix the bug since it's an admin-only API by default, so not really an interop concern, and it's a 1-line fix which we can then backport | |
| 14:47:09 | mriedem | i'll need to consider updating the microversion dev ref docs on admin APIs | |
| 14:47:57 | alex_xu | mriedem: ok, got it | |
| 14:50:04 | openstack | bug 1730730 in OpenStack Compute (nova) "AllocationCandidates.get_by_filters returns garbage with only sharing providers" [Undecided,Triaged] https://launchpad.net/bugs/1730730 | |
| 14:50:04 | gibi | mriedem: It would be nice to close the bug 1730730 report as well with the patch but jaypipes seems has different oppinion | |
| 14:52:32 | gibi | jaypipes: is it OK for you if I quickly resping https://review.openstack.org/#/c/516782/ and add the closes-bug tag to the commit message? | |
| 14:52:50 | jaypipes | gibi: go for it. | |
| 14:53:06 | gibi | jaypipes: OK | |
| 14:54:32 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: finish refactor AllocCandidates._get_by_filters() https://review.openstack.org/516782 | |
| 14:54:39 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM support matrix update https://review.openstack.org/523140 | |
| 14:55:38 | gibi | mriedem, alex_xu, jaypipes: here is the quick update, i'm +2 now on https://review.openstack.org/516782 | |
| 14:56:22 | alex_xu | gibi: done | |
| 15:00:02 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: [placement] Add x-openstack-request-id in API ref https://review.openstack.org/523007 | |
| 15:04:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix invalid minRam error message https://review.openstack.org/523142 | |
| 15:13:08 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: required traits for no sharing providers https://review.openstack.org/517027 | |
| 15:14:13 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test alloc_cands with indirectly sharing RPs https://review.openstack.org/519601 | |
| 15:14:51 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add ProviderSummary.resource_class_names @property https://review.openstack.org/517646 | |
| 15:24:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Refined fix for validating image on rebuild https://review.openstack.org/521186 | |
| 15:24:57 | openstackgerrit | Dan Smith proposed openstack/nova master: Fix doubling allocations on rebuild https://review.openstack.org/521662 | |
| 15:24:57 | openstackgerrit | Dan Smith proposed openstack/nova master: Add regression test for rebuild with new image doubling allocations https://review.openstack.org/521153 | |
| 15:27:08 | efried | dansmith Is https://review.openstack.org/#/c/507638/ related to the "Migration fixups" priority at https://etherpad.openstack.org/p/nova-ptg-queens-placement L49? | |
| 15:27:25 | efried | dansmith Perhaps a better question: what all is related to the "migration fixups" priority at ^ ? | |
| 15:27:46 | dansmith | that's the last one | |
| 15:27:51 | dansmith | all the others merged already | |
| 15:28:07 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Implement query param schema for delete assisted vol https://review.openstack.org/513668 | |
| 15:28:12 | openstackgerrit | Lajos Katona proposed openstack/nova master: Extend ServerMovingTests with custom resources https://review.openstack.org/497399 | |
| 15:28:57 | efried | dansmith Cool. Looks like it needs a rebase? | |
| 15:29:02 | dansmith | hmm, yeah I guess so | |
| 15:30:13 | efried | dansmith (cdent) Is POST /allocations for multiple consumers (https://review.openstack.org/#/c/500073/) not related to the same effort? | |
| 15:30:23 | dansmith | it is | |
| 15:30:30 | efried | okay, thanks. | |
| 15:42:58 | openstackgerrit | Dan Smith proposed openstack/nova master: Make live migration hold resources with a migration allocation https://review.openstack.org/507638 | |
| 15:45:02 | openstackgerrit | jiangpf proposed openstack/nova master: Encode libvirt domain XML in UTF-8 https://review.openstack.org/522161 | |
| 15:45:12 | efried | jaypipes edleafe dansmith Draft: http://paste.openstack.org/show/627496/ -- As owners of the Three Priorities, would you please vet this for proper inclusion of linkitude? | |
| 15:49:42 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove unnecessary self.flags and ConfPatcher https://review.openstack.org/522993 | |
| 15:53:35 | edleafe | efried: except for the double spaces after periods, it looks good | |
| 15:53:58 | efried | edleafe Thanks. Appreciate you taking a look. I'll work on the spacing thing. | |
| 15:54:21 | edleafe | It probably looks fine on a typewriter | |
| 15:55:35 | efried | edleafe That's how I proofread all my work. | |
| 16:24:01 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Versioned notifications for service create and delete https://review.openstack.org/519588 | |
| 16:36:25 | jaypipes | efried: sure, looks fine to me, thank you! | |
| 16:36:32 | efried | jaypipes Thanks. | |
| 16:43:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Address nits from service create/destroy notification review https://review.openstack.org/523162 | |
| 16:44:15 | cdent | nice placement functional test to get in: https://review.openstack.org/#/c/493865/ | |
| 17:00:27 | openstackgerrit | Merged openstack/nova master: Implement query param schema for flavor index https://review.openstack.org/515413 | |
| 17:00:34 | openstackgerrit | Merged openstack/nova master: Implement query param schema for fping index https://review.openstack.org/513890 | |
| 17:13:52 | dansmith | mriedem: this is about to pass check: https://review.openstack.org/#/c/521186/ | |
| 17:13:57 | dansmith | we gonna get that in soon? | |
| 17:15:26 | mriedem | haven't looked at the latest, | |
| 17:15:36 | mriedem | was going through your live migration swap allocs change quick | |
| 17:30:18 | mriedem | dansmith: ok looks like the only change in https://review.openstack.org/#/c/521186/ is the NUMATopologyFilter (which is looking at image properties so we should run it on rebuild) and the commit message title | |
| 17:32:17 | mriedem | artom: are you good with ^ now? | |
| 17:32:25 | mriedem | i think the only thing remaining is a release note | |