| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-12 | |||
| 11:46:57 | sean-k-mooney | *allocation_candiate | |
| 11:47:02 | stephenfin | a corner case, but there's no reason to give someone that loaded gun | |
| 11:47:38 | sean-k-mooney | i prefer not giving people firearms at all | |
| 11:48:06 | donnyd | sean-k-mooney: something is wrong with the mirror at FN | |
| 11:48:08 | sean-k-mooney | sowrd at noon is much more civalised then pistols at dawn | |
| 11:48:38 | donnyd | I am working on getting it fixed | |
| 11:48:55 | stephenfin | https://www.youtube.com/watch?v=vtnS3_bmtm8 | |
| 11:49:05 | sean-k-mooney | hurleys are a perfectly suitable alternitive | |
| 11:49:36 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow resizing server with port resource request https://review.opendev.org/679019 | |
| 11:49:36 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Do not query allocations twice in finish_revert_resize https://review.opendev.org/678827 | |
| 11:51:13 | sean-k-mooney | stephenfin: i havent seen that in a while | |
| 11:54:37 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Extract pf$N literals as constants from func test https://review.opendev.org/680991 | |
| 11:57:05 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Improve dest service level func tests https://review.opendev.org/680998 | |
| 11:57:06 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Follow up for Ib50b6b02208f5bd2972de8a6f8f685c19745514c https://review.opendev.org/681490 | |
| 11:57:15 | alex_xu | stephenfin: for sean-k-mooney pointed case both have VCPU and PCPU, then we only keep the PCPU allocation_candidtes override VCPu allocation_candidtes in alloc_reqs, then it will be ok. | |
| 11:57:51 | alex_xu | if both PCPU and VCPU on the same host, that VCPU must be for shared cpu | |
| 11:58:22 | sean-k-mooney | right | |
| 11:58:57 | sean-k-mooney | but i did not see code to do that in stephens code | |
| 11:59:08 | sean-k-mooney | that siad i only quickly looked at it | |
| 11:59:27 | alex_xu | yea, nice spot | |
| 11:59:35 | stephenfin | alex_xu: Yeah, we don't have that. Instead we're relying on ordering preventing that from happening | |
| 11:59:43 | stephenfin | I think I should add the filtering, to be safe | |
| 12:00:11 | sean-k-mooney | if we were building a dict of host to allocation candiate we coudl use set_default | |
| 12:00:14 | stephenfin | Scenario: we only have one host that provides both PCPU and VCPU | |
| 12:01:05 | stephenfin | actually, I was going to say the the NUMATopology filter fails the request with PCPUs but passes the VCPU request | |
| 12:01:07 | stephenfin | would that happen? | |
| 12:01:11 | stephenfin | ...though? | |
| 12:01:21 | sean-k-mooney | no | |
| 12:01:29 | stephenfin | I guess something could have changed in the time between the two tests | |
| 12:01:30 | sean-k-mooney | the filter does not operate on requests | |
| 12:01:34 | sean-k-mooney | they operate on hosts | |
| 12:01:50 | stephenfin | yeah, correct | |
| 12:02:16 | stephenfin | so if our order is preserved, we'll have hit that host already due to the allocation request for PCPUs | |
| 12:02:41 | stephenfin | if it failed once, it will fail if we hit it a second time due to the allocation request with VCPUs | |
| 12:02:43 | stephenfin | right? | |
| 12:03:00 | alex_xu | no, the scheduler filtered the hosts by allocation-candidtes first, then iterate the filtered hosts | |
| 12:03:03 | efried | "something could have changed" disregard this, already a window between GET /a_c and PUT /allocs | |
| 12:04:03 | stephenfin | efried: I mean we take the allocation request with PCPUs and try passing that through the NUMATopologyFilter or something and it fails | |
| 12:04:05 | sean-k-mooney | i though we got all the host form all allcoation candiate and looped over them once | |
| 12:04:14 | stephenfin | we then try a load of other hosts, which all fail | |
| 12:04:15 | sean-k-mooney | applying all fiters | |
| 12:04:28 | sean-k-mooney | then passed that to the weighers | |
| 12:04:34 | stephenfin | then we come back to that same original host only using the allocation candidate with VCPUs | |
| 12:04:42 | sean-k-mooney | and finally select a host then find the corresponing allcoaiton candiate | |
| 12:04:43 | alex_xu | stephenfin: maybe we ignore the allocation_request if the rp_uuid already in alloc_reqs_by_rp_uuid https://review.opendev.org/#/c/681719/1/nova/scheduler/manager.py@212 | |
| 12:05:06 | stephenfin | only in the time inbetween, something has changed on the host so the NUMATopologyFilter or whatever passes the request | |
| 12:05:24 | stephenfin | it's going to be such a tiny window but I wonder if it's worth worrying about? | |
| 12:05:47 | sean-k-mooney | stephenfin: if something changes the claim wil fial in placmeent | |
| 12:05:48 | alex_xu | ah, no, I'm wrong, we can't ignore directly, since we have same rp for child rp | |
| 12:05:53 | sean-k-mooney | then we will move on to the next host | |
| 12:06:09 | efried | I wouldn't have thought it was any different than what we're doing now for multiple candidates on one host. | |
| 12:06:27 | stephenfin | no, it's not really actually | |
| 12:06:38 | efried | sean-k-mooney: I think he's talking about if the second one spuriously succeeds | |
| 12:07:01 | efried | but yeah, I don't think this is worth worrying about. | |
| 12:07:05 | sean-k-mooney | my point is i dont think the numa toplogy filter will ever run twice for the same host | |
| 12:07:11 | sean-k-mooney | it will only run once | |
| 12:07:18 | efried | and i think that ^ is what we want | |
| 12:07:19 | alex_xu | ^ agree | |
| 12:07:32 | alex_xu | agree to sean-k-mooney... :) | |
| 12:07:33 | sean-k-mooney | because filter today have no knowadge of allcoation candates | |
| 12:08:14 | efried | because if there are available PCPUs for a host, we want to check that candidate only. But if there aren't any, there will only be (at most) VCPU candidates in the list. | |
| 12:08:47 | sean-k-mooney | if there are free pcpus and vcpus we will pass the host and have two candiates | |
| 12:09:00 | sean-k-mooney | and hopefully rely on orderr to pic the PCPU one | |
| 12:09:14 | efried | Yes exactly. Whole point of this is to make sure the first thing we look at is *either* a PCPU candidate *or* there are no PCPU candidates. | |
| 12:09:29 | sean-k-mooney | ya | |
| 12:09:49 | sean-k-mooney | stephenfin: rather then guessing | |
| 12:09:52 | sean-k-mooney | can you add a test | |
| 12:10:05 | stephenfin | sure. What should the test do? | |
| 12:10:06 | sean-k-mooney | create a host that has inventories fo both | |
| 12:10:15 | sean-k-mooney | and assrt the pcpu inventory was calimed | |
| 12:10:20 | sean-k-mooney | as a functional test | |
| 12:10:36 | stephenfin | I think I have that already | |
| 12:10:40 | sean-k-mooney | if it is we are all good | |
| 12:10:58 | efried | If ordering was random, that would be nondeterministic | |
| 12:11:07 | stephenfin | sean-k-mooney: Yeah, look at https://review.opendev.org/#/c/671801/46/nova/tests/functional/libvirt/test_numa_servers.py@364 | |
| 12:11:10 | sean-k-mooney | well we coudl do it in a loop | |
| 12:11:10 | alex_xu | If a host have both vcpu and pcpu, then we will get two allocation_request in alloc_reqs_by_rp_uuid https://review.opendev.org/#/c/681719/1/nova/scheduler/manager.py@212 | |
| 12:11:15 | sean-k-mooney | bug fair point | |
| 12:11:24 | stephenfin | It's a resize test but it does exactly this | |
| 12:11:25 | alex_xu | then we only claim the PCPU one | |
| 12:11:28 | stephenfin | Lemme run that in a loop | |
| 12:11:29 | efried | alex_xu: exactly | |
| 12:12:14 | alex_xu | so there will be check to found out there two allocation req, one for vcpu and another for pcpu, then only claim for pcpu | |
| 12:14:14 | sean-k-mooney | ok i think ye have that in hand to ill go back to the vpmem reviews | |
| 12:17:34 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: Update flavor-related metadata on resize https://review.opendev.org/681004 | |
| 12:18:08 | stephenfin | efried: What was the decision on '.update()' vs. the for-loop again? | |
| 12:21:18 | sean-k-mooney | stephenfin: update is fine | |
| 12:21:36 | sean-k-mooney | the sumaris will be the same | |
| 12:22:15 | sean-k-mooney | efried: the vpmem xml generation patch looks sane to me | |
| 12:22:29 | sean-k-mooney | im starting form the bottom of the seriese now and working my way up | |
| 12:25:00 | openstackgerrit | Merged openstack/nova master: Fix race in _test_live_migration_force_complete https://review.opendev.org/681540 | |
| 12:25:01 | sean-k-mooney | donnyd: thanks for the head up on the mirror issue | |
| 12:26:24 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: Update flavor-related metadata on resize https://review.opendev.org/681004 | |
| 12:28:03 | donnyd | sean-k-mooney: I think we have it narrowed down. should be back up in the next couple hours | |
| 12:28:30 | sean-k-mooney | it looks like its the same issue i was having from your conversation on infra | |
| 12:28:41 | sean-k-mooney | you are hitting MTU issue becasue of the tunnel | |
| 12:28:49 | sean-k-mooney | well similar | |
| 12:29:00 | sean-k-mooney | i needed to clamp it on HEs end | |
| 12:29:14 | sean-k-mooney | in your case you need to clamp the mtu in the neutron subnet | |
| 12:29:39 | sean-k-mooney | but same general symptoms of retransmits of large packets | |