| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-25 | |||
| 14:09:45 | johnthetubaguy | thats defiantly my preference, because of the release tags we are signed up to | |
| 14:10:14 | bauzas | edleafe: that depends on your driver | |
| 14:10:36 | cdent | Kevin_Zheng: I'll respond on the issue | |
| 14:10:40 | bauzas | edleafe: but whatever the situation is, nova *is* accepting that | |
| 14:10:51 | Kevin_Zheng | cdent: thanks | |
| 14:11:01 | johnthetubaguy | bauzas: you have to get an 8 VCPU vm, else it wouldn't live-migrate, as I understand it | |
| 14:11:03 | bauzas | in other words, folks, how can I make that functional test be working when using the placement API ? | |
| 14:11:24 | bauzas | I think there is a big misunderstanding between vcpus and pcpus | |
| 14:11:37 | cdent | bauzas: when you say "accepting that" what do you mean by "that", explicitly? | |
| 14:11:56 | bauzas | cdent: accepting to give you a destination that is not verifying the max unit | |
| 14:11:59 | johnthetubaguy | FWIW, I know rackspace cloud have never run the core filter, we only run the RAM filter | |
| 14:12:11 | bauzas | johnthetubaguy: then you could be in serious touble | |
| 14:12:21 | cdent | bauzas: sure but destinations don't matter, where vms are built does | |
| 14:12:30 | bauzas | johnthetubaguy: see https://review.openstack.org/#/c/417961/24/releasenotes/notes/resource_providers_scheduler_db_filters-16b2ed3da00c51dd.yaml | |
| 14:12:42 | johnthetubaguy | we would just have to set the over-comitt ratio correctly, but thats OK as a warning | |
| 14:12:49 | cdent | that's what I keep trying to clarify here: is the behavior of where vms built actually changing, or just the list of destinations? | |
| 14:12:52 | sahid | johnthetubaguy: i replied to your questions about the patch related to the feature emulator threads | |
| 14:13:11 | sahid | hope that is going to clarify your mind | |
| 14:13:49 | bauzas | johnthetubaguy: cdent: seriously, I'm done | |
| 14:14:09 | bauzas | I'm trying to spot issues that could come up in production and I'm only seeing a "meh, it's bonkers" | |
| 14:14:15 | cdent | bauzas: why? I'm really trying hard to understand the situation. | |
| 14:14:34 | bauzas | cdent: Jenkins will tell you the situation | |
| 14:14:43 | cdent | If it is the case that "the behavior of where vms built actually changing" then we have an issue. | |
| 14:15:08 | cdent | functional tests failing isn't necessarily telling us anything because we've demonstrated time and again (recently) that the functional tests have bad setups. | |
| 14:15:38 | bauzas | cdent: in particular, tbp, some functional tests in the test_server_group testcase won't give you the same exact result once we call the placement service, which is to me a good indicator that placement decisions are not feature-parity between Newton and Ocata | |
| 14:16:11 | cdent | that's a good indicator, but only if we verify that the tests are correct | |
| 14:16:23 | cdent | so I guess the next step is to wait and see what fails and go from there | |
| 14:17:21 | johnthetubaguy | so for folks with the core filter turned off the decisions will be different now we enforce that, it seems to me | |
| 14:17:37 | bauzas | johnthetubaguy: my point indeed | |
| 14:17:49 | johnthetubaguy | bauzas: yep, I am trying to agree with you here | |
| 14:17:49 | bauzas | johnthetubaguy: I spotted that problem in a ML thread | |
| 14:18:01 | johnthetubaguy | bauzas: I liked that first version of your code that dealt with that | |
| 14:18:15 | bauzas | johnthetubaguy: and the agreement we made with matt and jay was to rather ask the operators to set their ratios to an high value | |
| 14:18:32 | cdent | all along my position has been "bauzas' suggestion sounds sane but I defer to jay, dan and matt". I still do. | |
| 14:18:49 | edleafe | johnthetubaguy: I'm really not getting this. You're saying that it is expected behavior that Nova will build an 8 VCPU VM on a 2 VCPU host? | |
| 14:19:00 | bauzas | johnthetubaguy: but I'm trying to explain that because of an implementation detail and a design agreement that the placement engine should respect max units, we can't have the same | |
| 14:19:27 | bauzas | edleafe: a 2VCPU host doesn't mean anything to me | |
| 14:19:38 | bauzas | edleafe: hosts have physical threads | |
| 14:20:21 | sfinucan | sahid: Left a comment on https://review.openstack.org/#/c/400875. Could you take a look so I can continue with the rest of the series, please :) | |
| 14:20:21 | johnthetubaguy | edleafe: thats not my point here, my point is we are breaking folks existing configurations across upgrade | |
| 14:20:57 | johnthetubaguy | edleafe: if folks have the core filter off, they probably have not set the over commit raitio correctly | |
| 14:21:10 | edleafe | johnthetubaguy: I was using bauzas's example. | |
| 14:21:15 | johnthetubaguy | edleafe: when we change the default filters, it broke the XenAPI CI, for one example | |
| 14:21:29 | edleafe | johnthetubaguy: so our release notes tell them how to fix that | |
| 14:21:42 | bauzas | edleafe: but the releasenote I wrote is not *fixing* that | |
| 14:21:51 | bauzas | edleafe: I thought yesterday it was the case | |
| 14:21:55 | bauzas | but it's not | |
| 14:22:03 | bauzas | because *again* we cap on max_unit | |
| 14:22:25 | edleafe | johnthetubaguy: I understand it can break tests, but what is the effect in the real world? | |
| 14:22:27 | openstackgerrit | Vladik Romanovsky proposed openstack/nova: Adding vlans field to Device tagging metadata https://review.openstack.org/405951 | |
| 14:22:54 | bauzas | either way, we're 24 hours of the FF, and what we all discuss can possibly be a no-brainer if we are not able to amend grenade in time | |
| 14:23:07 | johnthetubaguy | aye | |
| 14:23:08 | cdent | bauzas: when a compute node reports inventory it reports a value from compute_node.vcpus. how is that calculated? | |
| 14:23:41 | cdent | (I'm trying to expand my understanding here, nothing else) | |
| 14:23:51 | bauzas | cdent: you mean the helper method, but how nova can know what cpus the node has ? | |
| 14:23:56 | bauzas | s/but/or | |
| 14:24:37 | cdent | I saying: there's a properly on the compute_node object called vcpus, which is set by update_from_virt_driver | |
| 14:24:39 | cdent | what does it mean? | |
| 14:27:03 | johnthetubaguy | bauzas: random thought, what if we change max unit to pCPU * allocation ratio? | |
| 14:27:22 | johnthetubaguy | edleafe: real world, feels like it only makes sense for test rigs and dev rigs | |
| 14:29:10 | edleafe | johnthetubaguy: what if we dump max_unit altogether? | |
| 14:29:21 | bauzas | johnthetubaguy: I'm pretty sure I would get a strong negative feedback on that approach | |
| 14:29:47 | johnthetubaguy | edleafe: it makes sense for some resources, just not all | |
| 14:30:12 | johnthetubaguy | edleafe: I think disk it was particularly useful for, from what I remember | |
| 14:31:48 | johnthetubaguy | hmm, I duno, it all feels very edge case, and there are work arounds if folks hit things | |
| 14:32:21 | bauzas | I'm feeling my patch is like a giant canary patch for spotting placement integration with nova | |
| 14:32:38 | bauzas | and being so close to FF makes me absolutely nervous | |
| 14:33:03 | bauzas | so we could spend yet another hours for discussing the good and the bad of operatoes | |
| 14:33:19 | bauzas | but the fact is that the landing window is reducing fast | |
| 14:34:12 | johnthetubaguy | bauzas: its also the most important patch to get all the lovelyness of placement exposed to our users, if that makes you feel better | |
| 14:34:24 | bauzas | i know | |
| 14:34:49 | bauzas | but that doesn't help if we're stuck | |
| 14:35:28 | johnthetubaguy | so the reality is, this is a big change, its going to be a bit bumpy | |
| 14:35:29 | mdbooth | Speaking of which, this is a not-insubstantial change which has been sitting a while: https://review.openstack.org/#/c/383859/ | |
| 14:35:47 | johnthetubaguy | the sooner we get this out there, the sooner we get to fix the stuff that really affects people | |
| 14:35:47 | mdbooth | I know it's a bugfix, but if it gets any additional exposure from landing before FF that would be great. | |
| 14:38:22 | edleafe | johnthetubaguy: bauzas: from the RP spec: "As an example, let us say that a particular compute node has two quad-core Xeon processors, providing 8 total physical cores. Even though the cloud administrator may have set the cpu_allocation_ratio to 16 (the default), the compute node cannot accept requests for instances needing more than 8 vCPUs. So, while there may be 128 total vCPUs available on the | |
| 14:38:28 | edleafe | compute node, the min_unit would be set to 1 and the max_unit would be set to 8 in order to prevent unacceptable matching of resources to requests." | |
| 14:39:01 | edleafe | is this reasoning wrong? | |
| 14:40:00 | johnthetubaguy | so it has 16 threads, with hyperthreading I guess, which feels a better match to vCPU | |
| 14:41:05 | cdent | what max_unit is set to for vcpu is dependent on what the virt driver reports. if it is hyperthreading aware, it probalby reports more. | |
| 14:41:11 | edleafe | johnthetubaguy: so you're saying that the reasoning in the spec is wrong? | |
| 14:42:45 | dansmith | johnthetubaguy or bauzas: easy --verbose patch that is at the bottom of all the remaining cells/devstack/devstack gate changes: https://review.openstack.org/#/c/421228/ | |
| 14:43:47 | bauzas | dansmith: aye | |
| 14:43:57 | bauzas | I'm stuck anyway, I have now plenty of time for reviews | |
| 14:44:26 | dansmith | let me catch up for the morning and then we can look | |
| 14:44:29 | dansmith | at your stuff | |
| 14:45:27 | bauzas | dansmith: I'll need to send an email | |
| 14:51:57 | BobBall | edleafe: Which spec are you talking about there? I've just been chatting to one of the Xen developers and they believe its dangerous to over-commit in a single VM. Linux slows to a crawl (and may crash) and Windows definitely crashes. Apparently. | |
| 14:56:31 | sfinucan | johnthetubaguy, sdague: Easy Mox-mock patch that holding back a bug fix (ye've both reviews Mox-mock patches before :)) | |
| 14:57:31 | openstackgerrit | Takashi NATSUME proposed openstack/nova: Use ListOfIntegersField in oslo.versionedobjects https://review.openstack.org/423912 | |
| 14:59:25 | johnthetubaguy | dansmith: I am wondering about this, do we get away without needing a context manager there? https://review.openstack.org/#/c/319379/78/nova/compute/api.py@1800 | |
| 15:00:06 | cdent | BobBall: that was the original resource providers spec: http://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/resource-providers.html | |
| 15:00:18 | BobBall | ah ok | |
| 15:01:04 | BobBall | Thanks. | |
| 15:04:18 | bauzas | johnthetubaguy: cdent: edleafe: dansmith: just made a summary in the ML thread about the problem | |
| 15:04:26 | bauzas | with a silver bullet in it | |
| 15:04:49 | dansmith | johnthetubaguy: yeah, i think you might be right | |
| 15:05:07 | dansmith | johnthetubaguy: I'm surprised melwitt didn't do that though, so I'm wondering if she knows better | |
| 15:05:18 | dansmith | johnthetubaguy: I will queue a change though and then ask her when she's aroune | |