| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-03 | |||
| 10:30:23 | bauzas | but AFAICT, we fully pin | |
| 10:30:47 | sudipto | we could choose to pin it to a NUMA node or choose to just pin a cpuset (older nova behaviour) | |
| 10:31:18 | bauzas | whatever | |
| 10:31:21 | sudipto | My problem is with the representation of the inventory data for cpusets itself with the current inventory field called 'total' | |
| 10:31:51 | bauzas | and ? | |
| 10:31:55 | johnthetubaguy | so the current inventroy doesn't work right, hence the need for nested, and the new resources of NUMA_CORE, etc | |
| 10:32:07 | bauzas | if you consider a single pCPU being a RP | |
| 10:32:25 | sudipto | so we start considering every single pCPU as a RP and then do what bauzas suggested. | |
| 10:32:27 | bauzas | with an VCPU inventory attached to it | |
| 10:32:28 | johnthetubaguy | well, no, I think a NUMA_CORE is a colleciton of pCPU, and you consume one of them | |
| 10:32:47 | johnthetubaguy | but I may have this upside down | |
| 10:32:51 | bauzas | johnthetubaguy: you mean a NUMA cell | |
| 10:33:04 | johnthetubaguy | well, no | |
| 10:33:16 | bauzas | I'm confused then | |
| 10:33:21 | johnthetubaguy | a NUMA cell, reports as a provider of NUMA_CORE | |
| 10:33:27 | johnthetubaguy | contains lots of pCPUs you can claim | |
| 10:33:28 | johnthetubaguy | I think | |
| 10:33:31 | bauzas | righrt | |
| 10:33:42 | bauzas | that's just another indirection | |
| 10:33:56 | johnthetubaguy | the pCPUs are just the NUMA_CORE resources | |
| 10:33:57 | johnthetubaguy | I think | |
| 10:33:58 | bauzas | I just feel for CPU pinning that we need to have an inventory of 1 | |
| 10:34:12 | sudipto | ok | |
| 10:34:17 | bauzas | so you "take" that pCPU | |
| 10:34:21 | johnthetubaguy | you don't care about the particular pCPU though | |
| 10:34:31 | johnthetubaguy | you only care about the NUMA node its on relative to the others you have | |
| 10:34:32 | bauzas | that's a scheduling decision | |
| 10:34:40 | bauzas | the fact is | |
| 10:34:43 | johnthetubaguy | doesn't seem to need the ironic 1:1 treatment | |
| 10:34:52 | bauzas | the current scheduler doesn't choose the ping | |
| 10:34:53 | bauzas | pin | |
| 10:35:00 | bauzas | it leaves the compute picking one pin | |
| 10:35:13 | johnthetubaguy | sure, the new one can pick the NUMA node, doesn't have to pick the pCPU | |
| 10:35:21 | bauzas | it just verifies that the the cell has at least one pin left | |
| 10:35:23 | bauzas | but | |
| 10:35:34 | bauzas | we have a shit ton of corner cases in the current model | |
| 10:35:43 | bauzas | and libvirt is pretty laxist on that | |
| 10:36:12 | bauzas | I just discovered that libvirt is totally okay having two VMs asking the same full excluse access to a pCPU | |
| 10:36:22 | bauzas | exclusive* | |
| 10:36:24 | johnthetubaguy | yep, the key thing is what do you care about, you don't care its pin 7, you care its in the same NUMA_NODE as your other resources | |
| 10:36:39 | bauzas | yeah, but there are situations where we fail | |
| 10:36:57 | bauzas | I'm not a NUMA specialist but I swear having seen problems with that | |
| 10:37:06 | johnthetubaguy | we do today, because we don't lock on the NUMA cell, only on the compute node | |
| 10:37:11 | bauzas | mostly because we don't claim that in the scheduler | |
| 10:37:12 | johnthetubaguy | at least thats my understanding of the issue | |
| 10:37:24 | bauzas | right | |
| 10:37:31 | johnthetubaguy | hence the change above | |
| 10:37:37 | lyarwood | johnthetubaguy / bauzas ; apologies if it's too late to ask but would https://review.openstack.org/#/c/420146/ be something we can squeeze in ahead of the RC1 tag? ./me lost track of it this week with everything else flying around. | |
| 10:37:55 | bauzas | anyway, I think sorting that problem is not trival and could be discussed at PTH | |
| 10:37:57 | bauzas | PTG even | |
| 10:38:36 | johnthetubaguy | the buckets are NUMA_NODEs rather than the computes | |
| 10:38:46 | bauzas | lyarwood: I need to pick my daughter | |
| 10:38:58 | johnthetubaguy | lyarwood: lets take a look, there is always RC2 | |
| 10:39:00 | bauzas | oh I thought RC1 sailed already | |
| 10:39:09 | bauzas | but it's not AFAICT | |
| 10:39:21 | bauzas | lemme verify the releases repo | |
| 10:39:27 | lyarwood | I thought it was pushed to Monday, maybe I'm mistaken, but it can wait | |
| 10:39:27 | johnthetubaguy | yeah, I don't see the tag right now | |
| 10:39:50 | bauzas | yeah https://review.openstack.org/#/c/428531/ | |
| 10:40:02 | johnthetubaguy | oh, this is based on the we don't do releases on fridays because it breaks weekends | |
| 10:40:24 | johnthetubaguy | although this one shouldn't impact others | |
| 10:41:14 | bauzas | all the Depends-On landed | |
| 10:41:29 | bauzas | so nothing prevents https://review.openstack.org/#/c/428531/ to be merged except the -W of course | |
| 10:41:58 | johnthetubaguy | lyarwood: I kinda like this use of os.path.ismount, seems sensible, shame out findmnt | |
| 10:42:07 | bauzas | johnthetubaguy: the only problem with tagging late is that we delay the requirements unfreeze | |
| 10:42:32 | bauzas | honestly, I don't think tagging RC1 requires to wait over monday | |
| 10:42:40 | lyarwood | johnthetubaguy: yeah the regression in findmnt is being addressed but IMHO we just don't need to use it | |
| 10:42:40 | bauzas | but I leave mriedem decide | |
| 10:42:53 | bauzas | oh that reminds me, let me drop a courtesy highlight | |
| 10:43:05 | johnthetubaguy | bauzas: we have tagged on friday before, using the timezone of thursday on some pacific island | |
| 10:43:11 | lyarwood | lol | |
| 10:43:14 | bauzas | \o/ | |
| 10:43:17 | bauzas | problem solved. | |
| 10:43:22 | johnthetubaguy | indeed :) | |
| 10:43:26 | bauzas | anyway, I need to bail out, school time | |
| 10:43:55 | bauzas | I could pretend the ISS is still on Thursday given they cross the world every 1.5h | |
| 10:44:13 | johnthetubaguy | that might give us a few more hours, sweet | |
| 10:44:32 | bauzas | just ask Thomas Pesquet to pull the +W on the patch | |
| 10:46:32 | johnthetubaguy | lyarwood: are we sure our source is always the same format at what is listed in /proc/mounts | |
| 10:46:44 | johnthetubaguy | lyarwood: just wondering about nfs vs nfs3, etc | |
| 10:48:50 | lyarwood | johnthetubaguy: hmm I'm pretty sure they are the same but let me double check | |
| 10:49:21 | johnthetubaguy | I guess if it works for findmnt it kinda has to be, but just wanted to double check | |
| 10:55:23 | cdent | anything special I should poke at, or let the etherpad be my guide? | |
| 10:55:47 | johnthetubaguy | I think the etherpad is still king | |
| 10:56:04 | johnthetubaguy | at this point, seems like its looking for things to backport to RC2 | |
| 11:02:45 | cdent | i'll get the fixes for https://review.openstack.org/#/c/428375/ | |
| 11:06:02 | tdurakov | johnthetubaguy: got question on that one and xen-api(https://review.openstack.org/#/c/424428/) live-migration process - what is the common process upgrading xen server in openstack( 6.5 > 7.0) | |
| 11:06:43 | tdurakov | any recommendations on xenapi multinode deployment? | |
| 11:07:57 | johnthetubaguy | tdurakov: the way we used to do it was live migrate everything off the box, re-image box. I am sure you could do in place upgrade, but all the VMs will be turned off for the duration | |
| 11:08:39 | johnthetubaguy | tdurakov: also, never attempt to use the pools feature with Nova, just have several standalone boxes | |
| 11:08:55 | johnthetubaguy | tdurakov: but the expert these days is BobBall, he can help you through the details | |
| 11:09:25 | tdurakov | BobBall: are you around? | |
| 11:09:47 | tdurakov | johnthetubaguy: thanks for details | |
| 11:10:09 | johnthetubaguy | tdurakov: no worries | |
| 11:10:38 | tdurakov | johnthetubaguy: no ci for xen multinode, so it's kind of tough thing to reveiw | |
| 11:11:07 | johnthetubaguy | tdurakov: yeah, not sure if they have an internal one | |
| 11:12:42 | openstackgerrit | Bob Ball proposed openstack/nova master: Fix live migrate with XenServer https://review.openstack.org/424428 | |
| 11:13:10 | BobBall | hi tdurakov :) I am indeed around. | |
| 11:13:30 | tdurakov | BobBall: hi, want to make some testing of ^ | |
| 11:13:52 | BobBall | And no, no multinode CI. Current testing is done manually. We recently got some more hardware installed in our CI env though with the purpose of enabling multinode for live migrate testing. Hopefully that'll be added in the next few weeks. | |