| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-29 | |||
| 18:45:00 | artom | sean-k-mooney, yeah, they wanted to test NUMA LM as well | |
| 18:45:08 | openstackgerrit | Merged openstack/nova master: Allow strict_proxies for sdk Connection https://review.opendev.org/676926 | |
| 18:45:13 | sean-k-mooney | donnyd its the generation before intel added cluser on die | |
| 18:45:14 | artom | But... it might just be faster to get FN running in CI :) | |
| 18:45:26 | sean-k-mooney | but as i said 2 numa nodes is all that is needed | |
| 18:46:06 | sean-k-mooney | intel is also ment to be fixing the nfv ci at some point | |
| 18:46:14 | sean-k-mooney | maybe | |
| 18:46:19 | donnyd | so will mine work? | |
| 18:46:23 | sean-k-mooney | yep | |
| 18:46:46 | artom | sean-k-mooney, let's make a bet on who gets there first, FN in CI, Intel, or stx :D | |
| 18:47:03 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Rename the nova-grenade-live-migration job to nova-grenade-multinode https://review.opendev.org/678841 | |
| 18:47:03 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Move live_migration test hooks under gate/ https://review.opendev.org/679251 | |
| 18:47:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cold migrate and resize to nova-grenade-multinode https://review.opendev.org/679210 | |
| 18:47:37 | mriedem | artom: you can ask them to use bluejeans during their meeting | |
| 18:48:04 | artom | It's OK, I'm not dogmatic about my video conferencing solutions :D | |
| 18:48:36 | sean-k-mooney | i think me testing stuff manully will be fater tehn all 3. i should start commint with my sean-mooney-ci gerrit account to confuse people | |
| 18:49:09 | donnyd | I can work on it later tonight (after my real job) or tomorrow morning (before my real job) | |
| 18:49:20 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Introduce live_migration_claim() https://review.opendev.org/635669 | |
| 18:49:21 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: Use Claims to update numa-related XML on the source https://review.opendev.org/635229 | |
| 18:49:21 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects for NUMA live migration https://review.opendev.org/634827 | |
| 18:49:22 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | |
| 18:49:22 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: NUMA live migration support https://review.opendev.org/634606 | |
| 18:49:23 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: DNM: Run LM integration tests with NUMA flavor https://review.opendev.org/678887 | |
| 18:49:23 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595 | |
| 18:49:34 | donnyd | any good docs to point me to for setup? | |
| 18:50:23 | sean-k-mooney | donnyd: i can send you the command or document them in an etherpad if you like | |
| 18:50:46 | sean-k-mooney | over all there is not too much that you need too do | |
| 18:50:54 | sean-k-mooney | we jus tneed to know the name of the flavor | |
| 18:50:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Rename the nova-grenade-live-migration job to nova-grenade-multinode https://review.opendev.org/678841 | |
| 18:50:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cold migrate and resize to nova-grenade-multinode https://review.opendev.org/679210 | |
| 18:51:00 | sean-k-mooney | and you need to add a few extra specs | |
| 18:51:06 | mriedem | artom: ha just did ours | |
| 18:51:34 | sean-k-mooney | and then 2 patches 1 for the project-config repo to add the new lable to nodepool and 1 to nova to add the perodic job | |
| 18:51:35 | mriedem | unbelievably, the teacher didn't comment on my new fresh haircut | |
| 18:51:42 | artom | mriedem, D: | |
| 18:51:56 | mriedem | "oh maya nice to meet....WHO IS THIS HANDSOME DEVIL?!" | |
| 18:51:58 | artom | The nerve of some people. | |
| 18:52:21 | artom | Seriously. The female version of https://www.youtube.com/watch?v=4pXfHLUlZf4. | |
| 18:52:23 | artom | (NFSW) | |
| 18:52:48 | sean-k-mooney | see now there is plasible deniablity | |
| 18:54:18 | donnyd | sure that would work great sean-k-mooney | |
| 18:54:33 | donnyd | I can name things however you like | |
| 18:55:06 | sean-k-mooney | we just need to know the name to put it in the project-config patch so it does not really matter | |
| 19:03:20 | donnyd | have to be something like ubuntu-bionic-fn-numa | |
| 19:03:28 | donnyd | centos-7-fn-numa | |
| 19:03:39 | sean-k-mooney | yep | |
| 19:03:45 | donnyd | something like that.. .whatever you come up with, I will just put it in FN | |
| 19:03:49 | sean-k-mooney | although the ubunutu one is proably enough | |
| 19:04:18 | donnyd | why not just use the label that is already there and i can fixup the flavor | |
| 19:04:26 | sean-k-mooney | ubuntu-bionic-fn-numa is fine the the lable and then we need a flavor name that can be numa or whatever | |
| 19:04:27 | donnyd | ubuntu-bionic-expanded | |
| 19:04:33 | donnyd | ok | |
| 19:05:01 | donnyd | i don't think anyone has ever scheduled a job against it | |
| 19:05:02 | sean-k-mooney | donnyd: we could but other jobs are not expecting muliptle numa nodes | |
| 19:05:21 | sean-k-mooney | well you also have the standard lables right | |
| 19:05:26 | donnyd | other jobs don't run on it right now | |
| 19:05:33 | sean-k-mooney | on FN | |
| 19:05:43 | donnyd | i created a label that had 16G of memory | |
| 19:05:45 | donnyd | yes | |
| 19:05:51 | sean-k-mooney | oh ok | |
| 19:05:57 | donnyd | look in nl02 | |
| 19:06:19 | donnyd | i already have a custom label that is completely unused | |
| 19:07:28 | sean-k-mooney | these ones https://github.com/openstack/project-config/blob/master/nodepool/nl02.openstack.org.yaml#L330 | |
| 19:07:41 | donnyd | yep | |
| 19:07:49 | sean-k-mooney | if you added hw:numa_nodes=2 to those it woudl work too | |
| 19:08:06 | sean-k-mooney | that would be very close to how the intel ci used to run | |
| 19:08:12 | donnyd | what do you mean? | |
| 19:08:20 | donnyd | you mean in project-config | |
| 19:08:28 | sean-k-mooney | no in your nova | |
| 19:08:32 | donnyd | oh | |
| 19:08:45 | donnyd | ok, so I just have to add that meta-data to the flavor | |
| 19:09:11 | sean-k-mooney | openstack flavor set "hw:numa_nodes=2" "8cpu-16GBram-80GBdisk" i think | |
| 19:09:39 | sean-k-mooney | not quite | |
| 19:09:53 | donnyd | maybe --property | |
| 19:10:02 | sean-k-mooney | openstack flavor set --property hw:numa_nodes=2 "8cpu-16GBram-80GBdisk" | |
| 19:10:09 | sean-k-mooney | yep | |
| 19:11:08 | donnyd | ok | |
| 19:11:10 | donnyd | done | |
| 19:11:28 | donnyd | but don't I have to setup something on the hypervisors or OS for NUMA to work? | |
| 19:11:39 | sean-k-mooney | nope | |
| 19:11:50 | donnyd | Ok, well its there | |
| 19:11:55 | sean-k-mooney | you have kvm enabeld correct | |
| 19:12:01 | sean-k-mooney | and netsted virt? | |
| 19:12:10 | donnyd | no... .def not nested | |
| 19:12:15 | sean-k-mooney | kvm is all that is required on the host | |
| 19:12:27 | donnyd | but i can enable nested-virt | |
| 19:12:49 | donnyd | not sure how that would tinker with other jobs | |
| 19:12:54 | sean-k-mooney | ok nested virt woudl be need for use to use kvm in the guest which we would need to test pinning and numa currently | |
| 19:13:08 | sean-k-mooney | the standard jobs hardcode qemu | |
| 19:13:12 | sean-k-mooney | so they wont use it | |
| 19:13:18 | sean-k-mooney | so they should be fine | |
| 19:13:21 | donnyd | ok | |
| 19:13:32 | sean-k-mooney | what os are you running by they way | |
| 19:13:36 | sean-k-mooney | on the host | |
| 19:13:50 | donnyd | It will take a bit to drain each hypervisor so I can unload and reload kvm with nested virt | |
| 19:13:55 | donnyd | ubuntu 18.04 | |
| 19:14:37 | sean-k-mooney | nested virt used to be broken on ubuntu 18.04 but i think they have fixed in in more recent kernels | |
| 19:14:43 | sean-k-mooney | there is no rush | |
| 19:15:28 | sean-k-mooney | i can start getting the job working and then when ever you get a chance to enable nested virt we can do some more testing. | |
| 19:15:33 | donnyd | i stand corrected - nested virt is already enabled too | |