| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-20 | |||
| 15:21:13 | sean-k-mooney | am i thnk that should be fixed by the numa live migration changes | |
| 15:21:18 | artom | mriedem, not clear - they talk about policy, which I assume means hw:cpu_policy, which my thing *did* fix | |
| 15:21:19 | mriedem | you can track a todo however you want though | |
| 15:21:44 | sean-k-mooney | oh | |
| 15:21:51 | sean-k-mooney | this is the edgecase we removed | |
| 15:21:58 | artom | But... then they also talk about vcpuset | |
| 15:22:03 | sean-k-mooney | yes | |
| 15:22:03 | artom | So, it's confusing | |
| 15:22:17 | sean-k-mooney | this is migration of a non numa instance between host with vcpu_pin_set defined | |
| 15:22:27 | sean-k-mooney | so the thing we added then removed | |
| 15:22:35 | sean-k-mooney | so this is not fixed yet | |
| 15:22:45 | artom | I guess? In any case, that's what it can become, because we need something to track that | |
| 15:23:28 | mriedem | ok i left a comment https://bugs.launchpad.net/nova/+bug/1496135/comments/10 | |
| 15:23:29 | openstack | Launchpad bug 1496135 in OpenStack Compute (nova) "live-migration will not honor destination vcpu_pin_set config" [Medium,Confirmed] | |
| 15:24:19 | mriedem | artom: while you're here, are we going to try and get https://review.opendev.org/#/c/672595/ into train yet? | |
| 15:24:33 | mriedem | i'm not sure how much fun that would be to backport | |
| 15:24:35 | artom | mriedem, I'm always here man | |
| 15:24:38 | mriedem | though it's all test stuff | |
| 15:24:44 | mriedem | you weren't yesterday when i needed to bug you | |
| 15:24:57 | artom | PTO, apple picking with son's daycare | |
| 15:25:07 | mriedem | see, you're not *always* here | |
| 15:25:09 | artom | Bought some cider, good stuff | |
| 15:25:14 | artom | In my hear I am | |
| 15:25:18 | artom | *heart, even | |
| 15:25:19 | mriedem | heart | |
| 15:25:28 | artom | Anyways | |
| 15:25:28 | mriedem | meanwhile i'm just assing around | |
| 15:25:41 | artom | Yeah, the func test should land in Train | |
| 15:25:58 | artom | If you and dansmith are ready to dive back in, I can pick it up again | |
| 15:26:14 | artom | Although it conflicts with some of the PCPU stuff that's still in flight | |
| 15:26:43 | mriedem | i've added it to https://etherpad.openstack.org/p/nova-train-release-todo | |
| 15:26:48 | artom | https://review.opendev.org/#/c/681060/ specifically | |
| 15:26:49 | mriedem | i'm not going to review that today | |
| 15:27:00 | artom | ^^ needs to land first, anyways, then I rebase and continue | |
| 15:27:14 | mriedem | so likely rc2 at this point | |
| 15:28:19 | artom | I guess yeah | |
| 15:29:57 | mriedem | what is the end of the pcpu series? the reshaper patch? because i was thinking about rebasing https://review.opendev.org/#/c/683011/ on top of that so we can approve it to make sure it gets into rc1 | |
| 15:30:12 | sean-k-mooney | if its a test only patch we could technically backport it right | |
| 15:30:17 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert "Temporarily skip TestNovaMigrationsMySQL" https://review.opendev.org/683011 | |
| 15:30:18 | sean-k-mooney | so does it need to land now | |
| 15:30:21 | mriedem | sean-k-mooney: correct | |
| 15:30:49 | mriedem | sean-k-mooney: what i worry about is saying "sure we'll get to it in ussuri" and then 4 months go by and dansmith and i have lost all context on that stuff and then we don't want to land it and it just never lands | |
| 15:31:13 | mriedem | strike while the iron is hot and all that | |
| 15:31:22 | sean-k-mooney | i think we should time box any of this stuff to ensure it before m1 | |
| 15:31:28 | sean-k-mooney | ya | |
| 15:32:20 | sean-k-mooney | i was just suggesting that if did not make rc1/2 it could still end up in train | |
| 15:35:56 | mriedem | if anyone is looking for glance image properties docs to work on https://bugs.launchpad.net/nova/+bug/1763761 | |
| 15:35:57 | openstack | Launchpad bug 1763761 in Glance "CPU topologies in nova - doesn't mention numa specific image properties" [Medium,Triaged] | |
| 15:39:45 | mriedem | looking at old numa bugs, it looks like https://review.opendev.org/#/c/458848/ just needed some test, | |
| 15:39:51 | mriedem | might be something for a core to pick up | |
| 15:41:02 | mriedem | artom: i wonder if this is the same numa topology claim race failure thing described in the ML https://bugs.launchpad.net/nova/+bug/1829349 | |
| 15:41:03 | openstack | Launchpad bug 1829349 in OpenStack Compute (nova) "Resource Tracker failed to update usage in case numa topology conflict happen" [Undecided,In progress] - Assigned to leehom (feli5) | |
| 15:41:06 | artom | mriedem, not sure that's still applicable, stephenfin did a thing around that a long time ago | |
| 15:41:11 | mriedem | note they are enabling he workaround config | |
| 15:42:40 | artom | mriedem, I don't think that's related - their bug is "if two instances have the same pins, RT blows up" | |
| 15:43:00 | artom | But with NUMA LM we're not supposed to end up in that situation in the first place | |
| 15:43:21 | mriedem | "supposed to" | |
| 15:43:33 | mriedem | but if we have a race with claims not reporting correctly or something, we could still race and blow up right? | |
| 15:44:00 | artom | Right | |
| 15:44:34 | mriedem | anyway there are some details from the reporter on what they think the problem is and where, so maybe would help | |
| 15:45:02 | artom | mriedem, well they proposed https://review.opendev.org/#/c/661208/ | |
| 15:45:06 | artom | Which is... no | |
| 15:46:10 | artom | I need food | |
| 15:52:03 | sean-k-mooney | mriedem: if your looking for numa related bug we should fix sooner rather then later we should aim to merge and backport https://review.opendev.org/#/c/662522/ sonner rather then later | |
| 15:52:30 | sean-k-mooney | its alex_xu and stephenfin fix for https://bugs.launchpad.net/nova/+bug/1805767 | |
| 15:52:32 | openstack | Launchpad bug 1805767 in OpenStack Compute (nova) "The new numa topology in the new flavor extra specs weren't parsed when resize" [Medium,In progress] - Assigned to Stephen Finucane (stephenfinucane) | |
| 15:53:42 | sean-k-mooney | i think of think all move operation bar shelve are broken in some way if you have a numa toplogy | |
| 15:54:10 | sean-k-mooney | there are speic things that work but it feels like there are more edgecase that dont then do somethimes | |
| 15:54:29 | sean-k-mooney | that said we have a tone of fixes for them pending in general | |
| 15:55:08 | mriedem | this is where i make a generic statement about lack of integration test coverage with tempest for numa, | |
| 15:55:16 | mriedem | and then you say, "i'm working on a ci job for that" | |
| 15:55:17 | sean-k-mooney | i still thinks there are enogh latent bugs to keep us going for U before im going to be happy we have hardended it enough | |
| 15:55:37 | sean-k-mooney | well its true | |
| 15:56:17 | sean-k-mooney | but also we have been without any real ci coverage for like 18 months so :( | |
| 15:56:24 | mriedem | you've done more to try and get numa tempest integration testing on nova changes than anyone so i'm not faulting you | |
| 15:56:52 | sean-k-mooney | oh i know i just whish i had more time to spend on it | |
| 15:57:03 | mriedem | it's just that my eyes glaze over when i hear "we have a bug with x that's related to numa" | |
| 15:57:11 | sean-k-mooney | although for the next week or two im going to continue looking at testing and ci | |
| 15:58:34 | sean-k-mooney | mriedem: i just pushed this by the way https://review.opendev.org/#/c/683431/ | |
| 15:59:35 | sean-k-mooney | that will allow us to test cpu pinning hugepages and dpdk more or less reliably | |
| 16:00:32 | sean-k-mooney | im going to wait until after RC* to follow up with getting multi numa flavor on vexxhost and limestone | |
| 16:00:40 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a prelude for the Train release https://review.opendev.org/683327 | |
| 16:04:11 | mriedem | stephenfin: i'm going to rebase https://review.opendev.org/#/c/682267/4 on top of https://review.opendev.org/#/c/674895/44 since it sounds like we're holding back on merging the former so as to not interrupte the latter | |
| 16:04:48 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: Get the CPU model, not 'arch' from get_capabilities() https://review.opendev.org/682267 | |
| 16:05:06 | mriedem | there wasn't even a merge conflict so i'm not sure what gerrit was complaining about | |
| 16:06:13 | mriedem | bauzas: do you want to re-approve https://review.opendev.org/#/c/681750/ since it was rebased? | |
| 16:06:23 | mriedem | the rest of that stack is stuck behind it | |
| 16:08:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Update CPU topologies guide to reflect the new PCPU world https://review.opendev.org/683437 | |
| 16:08:37 | donnyd | sean-k-mooney: LGTM | |
| 16:08:42 | stephenfin | mriedem: Makes sense. Also, there the start of the doc ^ | |
| 16:08:46 | mriedem | yu[p | |
| 16:08:47 | donnyd | I gave it my whole +1 | |
| 16:09:43 | bauzas | mriedem: cool | |
| 16:09:52 | bauzas | and I call it a day | |
| 16:10:18 | mriedem | stephenfin: man i wish there wasn't a big refactor in that patch | |
| 16:10:32 | sean-k-mooney | donnyd: thanks the first thing i hope to use it for is ovs-dpdk testing with https://review.opendev.org/#/c/656580/ | |
| 16:10:42 | stephenfin | I literally just said I shouldn't have done that to sean-k-mooney /o\ | |
| 16:10:55 | stephenfin | (who's working up in the RH office in Dublin with me for the day) | |
| 16:11:00 | sean-k-mooney | yep you did | |
| 16:11:01 | stephenfin | mriedem: Gimme five to separate it out | |