| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-29 | |||
| 19:13:32 | sean-k-mooney | what os are you running by they way | |
| 19:13:36 | sean-k-mooney | on the host | |
| 19:13:50 | donnyd | It will take a bit to drain each hypervisor so I can unload and reload kvm with nested virt | |
| 19:13:55 | donnyd | ubuntu 18.04 | |
| 19:14:37 | sean-k-mooney | nested virt used to be broken on ubuntu 18.04 but i think they have fixed in in more recent kernels | |
| 19:14:43 | sean-k-mooney | there is no rush | |
| 19:15:28 | sean-k-mooney | i can start getting the job working and then when ever you get a chance to enable nested virt we can do some more testing. | |
| 19:15:33 | donnyd | i stand corrected - nested virt is already enabled too | |
| 19:15:44 | sean-k-mooney | cool | |
| 19:16:15 | donnyd | kernel is 4.15.0-58-generic | |
| 19:16:50 | sean-k-mooney | in that case if i pivot https://review.opendev.org/#/c/652197/ to use nested virt i can add a muti numa version. that targets FN and reuns as a perodic | |
| 19:17:58 | sean-k-mooney | donnyd: am i need to go do a few tings before i finish up for the day | |
| 19:18:05 | donnyd | ok | |
| 19:18:23 | sean-k-mooney | but ill touch base on monday or tueseday next week when i get a chance to look that this again | |
| 19:19:14 | donnyd | Ok, lmk how else i can help | |
| 19:19:28 | donnyd | In my BIOS i have node-interleaving turned off | |
| 19:19:36 | donnyd | is that required for NUMA? | |
| 19:19:40 | sean-k-mooney | no | |
| 19:19:45 | sean-k-mooney | infact if you trun it on | |
| 19:19:49 | sean-k-mooney | it disable numa | |
| 19:20:11 | donnyd | are there any kernel params i can check to see if it will work | |
| 19:20:58 | sean-k-mooney | ls /sys/bus/node/devices/ | |
| 19:21:07 | sean-k-mooney | is ther more then one node | |
| 19:21:34 | sean-k-mooney | if so it should work. you can always try booting a vm with that flavor too | |
| 19:21:41 | sean-k-mooney | if it works then its all good | |
| 19:23:02 | sean-k-mooney | on a numa host it will look like this | |
| 19:23:04 | sean-k-mooney | stack@cloud-3:~/devstack$ ls /sys/bus/node/devices/ | |
| 19:23:06 | sean-k-mooney | node0 node1 | |
| 19:23:21 | sean-k-mooney | or in your case you should have node 0-3 | |
| 19:31:18 | openstackgerrit | Merged openstack/nova master: rt: soften warning case in _remove_deleted_instances_allocations https://review.opendev.org/673873 | |
| 19:33:11 | donnyd | sean-k-mooney: I will check it out and report back | |
| 19:35:22 | donnyd | ubuntu@compute-1:~$ sudo ls /sys/bus/node/devices/ | |
| 19:35:22 | donnyd | node0 node1 node2 node3 | |
| 19:37:39 | donnyd | woot woot | |
| 19:38:05 | sean-k-mooney | yep that all looks fine | |
| 19:39:08 | donnyd | another plus is that flavor also already has a bit more for memory, so that may be helpful on the nested virt side | |
| 19:39:32 | sean-k-mooney | it will be helpful for hugepage testing in the vm | |
| 19:40:05 | sean-k-mooney | because it means its less of a blancing act between the memroy needed for openstack vs the nested vms | |
| 19:40:25 | donnyd | i don't have hugepages enabled | |
| 19:40:33 | sean-k-mooney | you dont need them on the host | |
| 19:40:41 | sean-k-mooney | if you look at https://review.opendev.org/#/c/652197/20/playbooks/nfv/nfv.yaml | |
| 19:41:00 | sean-k-mooney | you see in my curent nfv job i ame allocating only a limited amount | |
| 19:41:01 | donnyd | do i look for the same parameters inside the vm? | |
| 19:41:24 | sean-k-mooney | how do you mean | |
| 19:41:39 | sean-k-mooney | do you mean the cpu flags | |
| 19:41:52 | donnyd | looking for numa nodes inside the instance launched with that flavor | |
| 19:42:00 | donnyd | the instance created fine | |
| 19:42:09 | donnyd | but how do i tell if its right | |
| 19:42:15 | sean-k-mooney | oh ya if you run the sam command in the vm you shoudl see 2 | |
| 19:42:24 | prometheanfire | how's the lxml issue coming? | |
| 19:42:41 | sean-k-mooney | i need to rework it to use the helper function | |
| 19:42:44 | donnyd | there is no /sys/bus/node dir | |
| 19:42:45 | sean-k-mooney | but its fixed | |
| 19:43:05 | sean-k-mooney | really? | |
| 19:43:09 | donnyd | no | |
| 19:43:24 | sean-k-mooney | what kernel/os is the vm using | |
| 19:43:31 | donnyd | i am going to try again with a different image and maybe not use one i am developing on | |
| 19:43:45 | sean-k-mooney | prometheanfire: https://review.opendev.org/#/c/678964/ | |
| 19:44:14 | sean-k-mooney | ill respin it sortly to use self.assertXmlEqual | |
| 19:44:25 | sean-k-mooney | but once thats done it should be good to go. | |
| 19:44:33 | sean-k-mooney | prometheanfire: can you unabandon your change by the ay | |
| 19:44:39 | sean-k-mooney | *way | |
| 19:44:49 | sean-k-mooney | prometheanfire: https://review.opendev.org/#/c/673848/ | |
| 19:45:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Test heal port allocations in nova-next https://review.opendev.org/669879 | |
| 19:45:55 | prometheanfire | the lxml change? | |
| 19:45:58 | donnyd | yea, i tried with a different image and its good to go | |
| 19:46:10 | prometheanfire | done | |
| 19:46:21 | sean-k-mooney | donnyd: cool can you provide the output of lcpu from the guest too | |
| 19:46:25 | prometheanfire | should I update it to 4.4.1? | |
| 19:46:35 | sean-k-mooney | i just want to check the flags | |
| 19:46:50 | sean-k-mooney | prometheanfire: i used the latest version on pypi for my testing | |
| 19:46:59 | sean-k-mooney | so you can updated it to whatever that is | |
| 19:47:11 | prometheanfire | wfm | |
| 19:47:11 | donnyd | u mean lscpu | |
| 19:47:19 | sean-k-mooney | yes | |
| 19:47:33 | donnyd | https://www.irccloud.com/pastebin/dvV8SBKk/ | |
| 19:48:30 | sean-k-mooney | yep so that supports hugepages and vmx is there so nested virt will work in the vm | |
| 19:48:46 | sean-k-mooney | and it have avx/sse4 so dpdk would work too | |
| 19:49:13 | sean-k-mooney | donnyd: also you can see it has 2 numa nodes | |
| 19:49:26 | donnyd | pretty slick | |
| 19:49:29 | donnyd | that was easy | |
| 19:49:45 | sean-k-mooney | donnyd: this is totally optionaly but can i ask you to tweek somthing else that annoys me | |
| 19:49:47 | donnyd | anything else you need to see | |
| 19:49:50 | donnyd | sure | |
| 19:50:26 | sean-k-mooney | can you add hw:cpu_sockets=2 hw:cpu_cores=2 hw:cpu_threads=2 | |
| 19:50:47 | donnyd | sure thing | |
| 19:50:57 | sean-k-mooney | that will emulatate a 2 socekt system with 2 core cpus with hyperthreading | |
| 19:51:12 | sean-k-mooney | currently the vm thing it has 8 sockets | |
| 19:52:08 | donnyd | more sockets are more better | |
| 19:52:31 | sean-k-mooney | not when windows and vmware licening used to be per socket | |
| 19:52:39 | sean-k-mooney | although now its per core | |
| 19:53:24 | donnyd | sean-k-mooney: more sockets is web scale | |
| 19:53:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add FUP unit test for port heal allocations https://review.opendev.org/672142 | |
| 19:54:47 | sean-k-mooney | the e7 line goes up to 8 i belive an then you get int the craze hyper scaleble skus | |
| 19:54:49 | donnyd | https://www.irccloud.com/pastebin/QLruofuP/ | |
| 19:55:12 | sean-k-mooney | thanks | |
| 19:55:15 | sean-k-mooney | much sainer | |
| 19:55:23 | donnyd | NP | |
| 19:55:51 | donnyd | at some point we probably want to update the labels to reflect what they actually do - but its fine for now | |
| 19:55:52 | sean-k-mooney | ok ill try this configurtion and see if i hit any road blocks. proably next week however. | |