| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-11 | |||
| 09:34:02 | alex_xu | stephenfin: sorry, I need to take a rest little bit. you can free to take over those two patches. I probably online after one or two hours. | |
| 09:34:11 | stephenfin | Sounds good (y) | |
| 09:35:21 | efried | luyao: btw, here's what I did to that provider tree unit test when I was messing around with it locally: http://paste.openstack.org/show/775098/ | |
| 09:35:40 | efried | that last assertTrue fails with copy.copy but succeeds with deepcopy. | |
| 09:38:33 | luyao | efried: get it, thanks. | |
| 09:39:21 | efried | luyao: note that https://review.opendev.org/#/c/678453/26 is in merge conflict (probably aspiers' fault :P ) so you'll need to rebase again. | |
| 09:40:21 | luyao | efried: haha :D | |
| 09:49:43 | luyao | efried: I check the unittests for get_allocations, it tests the function that we move the get_allocations_xx out of | |
| 09:49:53 | luyao | efried: so I have to remove it | |
| 09:50:17 | efried | luyao: then I say just leave that part of the patch as is | |
| 09:50:42 | efried | luyao: stephenfin and I have already reviewed it and accepted it this way, so it's not worth the trouble of changing. | |
| 09:52:26 | luyao | efried: Okay | |
| 09:54:40 | luyao | efried: sorry, confirm again, need try/except or drop? | |
| 09:56:10 | efried | Remove the try/except. I.e. keep this part of the patch as it appears in PS22 https://review.opendev.org/#/c/678450/22/nova/compute/manager.py@2215 | |
| 09:56:10 | efried | luyao: Sorry, this has gotten confusing. | |
| 09:57:04 | luyao | efried: Okay, thanks. | |
| 09:58:56 | efried | luyao: Okay, I've finished re-reviewing the series (except for https://review.opendev.org/#/c/678455/, which I'm still not going to touch) | |
| 09:59:19 | efried | stephenfin: Do you need my re-look at any of the cpu-resources patches at this point? | |
| 09:59:29 | stephenfin | efried: In about a half hour, yeah | |
| 09:59:36 | stephenfin | I think the quota problem is solved | |
| 09:59:53 | efried | sweet | |
| 09:59:56 | stephenfin | So it's just replacing the scheduler conf option with retry logic for placement left | |
| 10:00:02 | stephenfin | almost done with tests | |
| 10:03:35 | efried | luyao: note response on the defaultdict thing -- I still don't think you need it https://review.opendev.org/#/c/678449/21/nova/compute/provider_tree.py@70 | |
| 10:06:58 | luyao | efried: you're right. :) | |
| 10:07:13 | efried | phew. I like being right | |
| 10:08:26 | bauzas | stephenfin: I wonder something | |
| 10:08:54 | bauzas | stephenfin: if we ask for an o.vo field not existing, can we just do something like 'if field not in my_object:' ? | |
| 10:09:07 | bauzas | IIRC, we need to use a specific o.vo method | |
| 10:09:13 | bauzas | unless the field is set to None | |
| 10:09:14 | stephenfin | bauzas: You can do that, yeah | |
| 10:09:17 | stephenfin | For some time now | |
| 10:09:33 | stephenfin | you used to need to use is_attr_set or something like that | |
| 10:09:37 | bauzas | ok, I had concerns about the virt.hardware module in https://review.opendev.org/#/c/671800/ | |
| 10:09:44 | stephenfin | but __contains__ was implemented many moons ago | |
| 10:09:59 | stephenfin | anywhere is particular? | |
| 10:17:39 | bauzas | nah it's okay then | |
| 10:18:02 | bauzas | my o.vo skills are a bit rusty | |
| 10:18:07 | bauzas | but I could have tested it | |
| 10:19:04 | sean-k-mooney | efried: after 6 year of working on nova the main lesson i have learns is unlesss you have 2 cores from different compaines lined up to review the think you want to do before you start it will take 2 cycles to complete | |
| 10:19:39 | sean-k-mooney | the first cycle you get it working and people beging to become familar with and the second you finall land the thing | |
| 10:20:22 | efried | sean-k-mooney: I would love to be able to commit cores to review blueprints before/as they are approved. I just a) doubt cores would be willing to commit, and b) don't want to do that much PM type work | |
| 10:21:43 | sean-k-mooney | its partly a social thing. you need to explain why this thing you care about is imporant so that people will be interested enogh to review and demonstrate it works and is sane. | |
| 10:22:21 | sean-k-mooney | the non craze telco things i have worked on are less hard to enabel but im normlally in craze telco land | |
| 10:22:26 | efried | It sucks that "be interested" is a criterion | |
| 10:23:31 | sean-k-mooney | well it only is in terms of making it higher on list of other things you have to review | |
| 10:24:03 | sean-k-mooney | there are only a limit number of sme fore each system and they are overloaded frequently | |
| 10:25:14 | sean-k-mooney | if we had more subsystem maintainer or more reviewres it would help but we work with what we got. | |
| 10:25:31 | sean-k-mooney | we still get a lot done each cycle | |
| 10:25:42 | sean-k-mooney | the latency can just be a bit high sometimes | |
| 10:25:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: DNM: Try to fallback to PCPU request when VCPU failed https://review.opendev.org/681383 | |
| 10:25:56 | stephenfin | efried, bauzas: Can you sanity check before I waste more time fixing up tests ^ | |
| 10:26:06 | stephenfin | the approach in general, that is | |
| 10:26:42 | bauzas | stephenfin: I left a comment on https://review.opendev.org/#/c/671800/ | |
| 10:28:03 | bauzas | stephenfin: cool, I'll do it a bit later (needs to refuel my stomach) | |
| 10:32:46 | efried | stephenfin: sorry, looking at this patch for the first time, and... is this really the right approach? If we bounce with PCPUs, try VCPUs?? If they *have* cut over, and we're just *actually* out of PCPUs in the cloud, they're going to get *not* desired behavior, nah? | |
| 10:33:21 | openstackgerrit | Luyao Zhong proposed openstack/nova master: object: Introduce Resource and ResourceList objs https://review.opendev.org/678448 | |
| 10:33:21 | openstackgerrit | Luyao Zhong proposed openstack/nova master: db: Add resources column in instance_extra table https://review.opendev.org/678447 | |
| 10:33:22 | openstackgerrit | Luyao Zhong proposed openstack/nova master: Retrieve the allocations early https://review.opendev.org/678450 | |
| 10:33:22 | openstackgerrit | Luyao Zhong proposed openstack/nova master: Add resources dict into _Provider https://review.opendev.org/678449 | |
| 11:04:37 | sean-k-mooney | efried: the numa toplogy filter will handel the out of PCPU case when they have swapped over to using cpu_dedicted_set | |
| 11:05:10 | efried | eh? | |
| 11:05:38 | sean-k-mooney | the numa toplogy filter will still run | |
| 11:06:03 | sean-k-mooney | it can tell the difference between a pre train host and a train host with non pCPUs free | |
| 11:06:04 | efried | If I'm out of PCPU resource per placement, this code will try the alloc request again with VCPUs. All of that happens before we get to the filter, yah? | |
| 11:06:18 | sean-k-mooney | yep | |
| 11:06:24 | efried | so... that's bad | |
| 11:06:47 | sean-k-mooney | and the filter will reject all host that are train but have not pcpus left when we are given vcpu allocations | |
| 11:07:05 | efried | that's not the scenario I'm concerned with. | |
| 11:07:15 | sean-k-mooney | what is? | |
| 11:07:51 | efried | Let's say I'm fully upgraded, with my confs properly cut over to cpu_{shared|dedicated}_set. | |
| 11:07:58 | sean-k-mooney | yep | |
| 11:08:01 | efried | this means my cloud has both PCPU and VCPU resources scattered around it. | |
| 11:08:05 | sean-k-mooney | yep | |
| 11:08:09 | efried | sometimes on the same host, maybe sometimes not, whatever. | |
| 11:08:16 | sean-k-mooney | sure | |
| 11:08:21 | efried | I request a dedicated VM. | |
| 11:08:35 | efried | But I'm genuinely out of PCPU resources in my cloud. | |
| 11:08:41 | openstackgerrit | Stephen Finucane proposed openstack/nova master: fixup! Add support for translating CPU policy extra specs, image meta https://review.opendev.org/681445 | |
| 11:08:42 | sean-k-mooney | yep then we fall back | |
| 11:08:45 | efried | This code will bounce in GET /a_c and then *retry* with VCPUs instead. | |
| 11:08:53 | efried | So I'll wind up with my instance on VCPUs | |
| 11:08:58 | sean-k-mooney | no | |
| 11:09:00 | stephenfin | efried: nope | |
| 11:09:16 | stephenfin | scroll up and look at the conversation between alex_xu and I about two hours ago | |
| 11:09:20 | sean-k-mooney | you get allocation from placmenet that are for VCPUs | |
| 11:09:30 | sean-k-mooney | but then the numa toplogy filter will reject all the hosts | |
| 11:09:47 | sean-k-mooney | stephenfin: do you want to explain why | |
| 11:09:48 | efried | ...because the host knows the VCPUs aren't dedicated. | |
| 11:09:52 | efried | ? | |
| 11:10:02 | stephenfin | efried: Yeah | |
| 11:10:05 | efried | which it doesn't know pre-Train, but in Train it does | |
| 11:10:21 | stephenfin | Placement will try for PCPU, not get it, and then try for VCPU | |
| 11:10:23 | efried | now, this is the version of the filter that runs on the compute host, not the scheduler, right? | |
| 11:10:36 | stephenfin | For Stein hosts, that's fine | |
| 11:10:37 | sean-k-mooney | efried: not its in the schduler | |
| 11:10:50 | efried | oh | |
| 11:10:51 | stephenfin | For Train hosts, the NUMATopologyFilter will then reject the host | |
| 11:10:55 | sean-k-mooney | stepehn modifled the host numa toplogy object | |
| 11:11:07 | stephenfin | and if that's disabled, it'll late fail on the compute node | |