| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-09 | |||
| 16:20:30 | sean-k-mooney | mriedem: if vexxhost would be will to add one like the ones FN have added then yes we could use both vexxhost and FN for this | |
| 16:20:32 | artom | The changes since then are basically comments: https://review.opendev.org/#/c/635229/60..61/nova/tests/unit/virt/libvirt/test_driver.py | |
| 16:20:48 | artom | mriedem, yep, I even added debug logging to the _update_numa_xml method | |
| 16:20:54 | mriedem | artom: yeah i saw and commented on PS61 | |
| 16:22:02 | sean-k-mooney | mnaser: not to put you on the spot but is creating a custom flavor for the upstream ci to consume possible/something you/vexxhost would be willing to consider? | |
| 16:22:15 | artom | So for example input: https://zuul.opendev.org/t/openstack/build/dd0f5dad770d40a2afb3c506327d1b3e/log/compute/logs/screen-n-cpu.txt.gz#2081 | |
| 16:22:24 | sean-k-mooney | mnaser: i can provide the detail later if so. | |
| 16:22:34 | artom | And then output: https://zuul.opendev.org/t/openstack/build/dd0f5dad770d40a2afb3c506327d1b3e/log/compute/logs/screen-n-cpu.txt.gz#2195 | |
| 16:23:17 | artom | We see the vcpupin go from 0-3 and 4-6 to 1-3 and 4-7 | |
| 16:23:49 | artom | And then for the allocations_cpu_tuning, I have https://review.opendev.org/#/c/680806/ | |
| 16:24:00 | openstackgerrit | Chris Dent proposed openstack/nova master: Don't duplicate PlacementFixture in libvirt func tests https://review.opendev.org/681059 | |
| 16:24:04 | cdent | mriedem, efried ^ | |
| 16:24:20 | artom | Again, input: https://zuul.opendev.org/t/openstack/build/f79144ae28114cea96e708d0bb5ebfb1/log/logs/screen-n-cpu.txt.gz#2565 | |
| 16:24:30 | artom | And output: https://zuul.opendev.org/t/openstack/build/f79144ae28114cea96e708d0bb5ebfb1/log/logs/screen-n-cpu.txt.gz#2666 | |
| 16:24:33 | donnyd | sean-k-mooney: artom correct FN is at zero for the moment | |
| 16:24:50 | artom | <vcpu placement="static" cpuset="0">1</vcpu> in both cases | |
| 16:25:02 | artom | Because I couldn't figure out how to set vcpu_pin_set per-node | |
| 16:25:08 | artom | But at least it runs through the code | |
| 16:25:09 | donnyd | The electrical work is all done.... Now just waiting on propane and FN will be back in business | |
| 16:25:28 | sean-k-mooney | artom: i have set vcpu pin set per node in one of them | |
| 16:25:59 | sean-k-mooney | artom: https://review.opendev.org/#/c/679805/ | |
| 16:26:18 | sean-k-mooney | artom: and that also passed on saturday | |
| 16:26:27 | artom | sean-k-mooney, what's with a NUMA flavor though, right? | |
| 16:26:33 | mriedem | artom: ok can you link those into the patch for reference later? | |
| 16:26:34 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/43f69768e18a4e1594810e606f101b6a | |
| 16:26:41 | sean-k-mooney | artom: oh yes | |
| 16:26:50 | artom | My DNM thing hits the other, non-NUMA case, to show that allocation_cpu_pins works | |
| 16:27:06 | sean-k-mooney | well we could do it without a numa flavor later or i can stack and check it by hand now | |
| 16:27:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 16:27:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: fakelibvirt: Make 'Connection.getHostname' unique https://review.opendev.org/681060 | |
| 16:27:35 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Mock 'libvirt_utils.file_open' properly https://review.opendev.org/681061 | |
| 16:29:08 | stephenfin | artom: https://review.opendev.org/#/c/681060/ | |
| 16:30:58 | artom | stephenfin, way to steal credit brah ;) | |
| 16:31:15 | stephenfin | desperate times /o\ | |
| 16:31:21 | mriedem | co-authored-by | |
| 16:31:25 | artom | stephenfin, you proll'y want to explain in more details what those lambda scoping issues were | |
| 16:31:35 | stephenfin | I should have set the authorship now that I'm thinking about it | |
| 16:31:49 | cdent | mriedem: I did a quick survey to look for other duplicate placement fixtures and did not find | |
| 16:32:00 | bauzas | melwitt: gibi: I'm +W with the first 3 changes | |
| 16:32:00 | artom | mriedem, to be fair, I didn't write *that* solution | |
| 16:32:05 | artom | I just found the problem | |
| 16:32:06 | cdent | that's me done for today I reckon | |
| 16:32:10 | mriedem | cool | |
| 16:32:12 | mriedem | thanks | |
| 16:32:12 | bauzas | melwitt: gibi: see you tomorrow | |
| 16:32:34 | artom | stephenfin, because they're really not obviousy | |
| 16:32:36 | artom | *obvious | |
| 16:33:41 | sean-k-mooney | i just clicked on it and no ideay lambda were related to this other then th efact i saw lambdas in the code so ya | |
| 16:44:15 | artom | sean-k-mooney, btw, Matt's expecting a vote from you on that NUMA LM patch - just making sure everyone's on the same page :) | |
| 16:44:37 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Microversion 2.79: Add delete_on_termination to volume-attach API https://review.opendev.org/673485 | |
| 16:47:32 | mriedem | ^ i'm +2 on that novaclient change for 2.79 now | |
| 16:47:37 | mriedem | need to get that merged and released by thursday, | |
| 16:47:41 | mriedem | the osc change is also dependent o nit | |
| 16:48:52 | sean-k-mooney | artom: ok i keep getting distracted by jumping between your stuff stephns patach and email. | |
| 16:49:26 | sean-k-mooney | ill kick off a local devstack run with your latest patches then finish my review of stephns code and then review your code | |
| 16:49:35 | artom | sean-k-mooney, I'll stop harassing you - just making sure expectations were clear | |
| 16:49:51 | sean-k-mooney | sorry i ment to finish reviewing stephens code already | |
| 16:50:35 | sean-k-mooney | no worries i just need to focus. i started late today so ill be around for another few hours today | |
| 16:54:04 | mnaser | sean-k-mooney, artom: i know we talked about this but i dont know what that implies on my side | |
| 16:54:46 | mnaser | i can do the custom flavor side if someone wants to volunteer to update the nodepool/infra side of things, i'm a little backed up on things right now | |
| 16:55:39 | artom | mnaser, thanks, but we may have jumped the gun on that | |
| 16:55:54 | mnaser | np :) | |
| 16:55:56 | sean-k-mooney | mnaser: i can do the nodepool/project-config changes | |
| 16:56:09 | sean-k-mooney | mnaser: ideally in U it would be nice to have two providers | |
| 16:56:10 | mnaser | someone will have to let me know how to do/setup the flavor | |
| 16:56:14 | artom | sean-k-mooney, is that... necessary, at this point? | |
| 16:56:27 | sean-k-mooney | artom: nessisary no prefered yes | |
| 16:56:27 | artom | I know FN is... intermittent, but we got the runs we needed, didn't we? | |
| 16:56:59 | artom | sean-k-mooney, agree on that. I guess you're thinking long term? | |
| 16:57:06 | sean-k-mooney | artom: i want to get a perminet job running in U even if its only nightly | |
| 16:57:10 | sean-k-mooney | artom: yes | |
| 16:57:11 | artom | I was concentrating on the short term of getting the series merged | |
| 16:57:25 | sean-k-mooney | we dont need a second provider to merge it correct | |
| 16:57:35 | artom | No | |
| 16:58:04 | sean-k-mooney | mnaser: its relitivly simple change i was going to wait until after feature freeze when you were less busy to bring it up | |
| 16:58:19 | artom | So, mnaser, while I would definitely love to work with you and Sean (or, let's he honest, let Sean do all the work ;) on getting multi-NUMA flavors from vexxhost, I think we can wait until after FF | |
| 16:58:35 | mnaser | no worries, does it require a patch or is it just a flavor with some extra_specs? | |
| 16:58:46 | artom | mnaser, the latter | |
| 16:58:51 | mnaser | ok so thats easy | |
| 16:58:56 | sean-k-mooney | mnaser: but bassicaly jsut need to add hw:numa_nodes=2 an preferably also add hw:cpu_socket=2 and hw:cpu_threads=2 | |
| 16:59:14 | donnyd | mnaser: yea I put in the flavor data and it just worked on FN | |
| 16:59:27 | mnaser | cool, it should be ok for me to do that | |
| 16:59:54 | donnyd | https://www.irccloud.com/pastebin/KXLeMoFH/ | |
| 17:00:02 | donnyd | mnaser: snippet from FN | |
| 17:00:15 | sean-k-mooney | mnaser: donnyd create a multi-numa lable in nodepool for FN so ideally we woudl use the same one for vexhost to allow it to run on either. again thats a long term thing | |
| 17:00:37 | sean-k-mooney | if we had that it would allow us to propaly test all the numa/pinning/hugepage things in the first party ci | |
| 17:00:53 | sean-k-mooney | with multiple providers to make it redundatnt | |
| 17:01:22 | artom | If we have multiple provider we could even make it more than just experimental (eventually) | |
| 17:02:08 | sean-k-mooney | artom: yep, which is why longter i was hoping to reach out to vexhost and maybe packet to be addtional provier of this lable/flavor | |
| 17:02:30 | sean-k-mooney | but that is a U thing not a now thing | |
| 17:05:08 | artom | yep | |
| 17:13:16 | donnyd | sean-k-mooney: I am going to put the compute service back in operation for FN | |
| 17:13:58 | donnyd | Time is a little short and these resources are kinda needed...I don't want to hold up progress on the *hope that the propane guy actaully even makes it here today | |
| 17:14:20 | donnyd | electrical work is all done.. | |
| 17:15:43 | sean-k-mooney | we had good test result form satusrday and im locall redeploying the latest code | |
| 17:19:57 | donnyd | sean-k-mooney: just waiting on https://review.opendev.org/#/c/681075/ | |
| 17:20:22 | donnyd | FN is up.. .just waiting on this to be able to run jobs | |
| 17:20:47 | sean-k-mooney | ack. thanks | |
| 17:48:10 | mriedem | sean-k-mooney: when you're dug out from the current set of tasks, i found this last week and it's a question for you (or adrianc): https://review.opendev.org/#/c/634606/75..80/nova/compute/manager.py@a6504 | |
| 17:51:37 | sean-k-mooney | mriedem: the sriov claim. ill look into that and respond. after im done with the rest of artoms stuff | |