| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-25 | |||
| 14:03:26 | bauzas | cdent: because once the scheduler upgraded to Ocata, it will not return the exact same nodes | |
| 14:03:44 | cdent | an operator could never place a host on a node that had fewer vcpus than requested | |
| 14:03:45 | bauzas | cdent: and hosts that were okay for Newton won't be available per Ocata | |
| 14:03:53 | cdent | but they are not okay | |
| 14:03:54 | bauzas | cdent: people did | |
| 14:04:03 | cdent | how does that even work? | |
| 14:04:14 | bauzas | cdent: because they disabled their filters | |
| 14:04:21 | cdent | can I run an 8vcpu server on a 2 vcpus host? | |
| 14:04:25 | gmann | johnthetubaguy: ok, let me push changes tomorrow and then we can see till where we can go with those | |
| 14:04:33 | bauzas | you can if you don't enable the CoreFilter | |
| 14:04:41 | cdent | that's bonkers | |
| 14:04:59 | bauzas | cdent: yet again, we can be pedantic and say that | |
| 14:05:12 | edleafe | bauzas: you could get that back from scheduler, but it would fail to build | |
| 14:05:13 | bauzas | or we can be pragmatic and find a way that doesn't trample operators | |
| 14:05:17 | alex_xu | gmann: really appreciate that | |
| 14:05:20 | openstack | Launchpad bug 1658868 in OpenStack Compute (nova) "Should raise retry limit exceed error instead of class exist in Placement API" [Undecided,New] - Assigned to Zhenyu Zheng (zhengzhenyu) | |
| 14:05:20 | Kevin_Zheng | cdent: hi, since you are here, is this an issue? https://bugs.launchpad.net/nova/+bug/1658868 | |
| 14:05:21 | cdent | Well a) clearly I don't know enough about what people are doing to have a straight opinion on this stuff, but b) I don't think we should allow bonker-ness, c) I'd defer to matt and dan on this | |
| 14:05:33 | bauzas | edleafe: the claim could | |
| 14:05:40 | cdent | will look in just a sec Kevin_Zheng | |
| 14:05:51 | edleafe | bauzas: the claim could... what? | |
| 14:05:52 | bauzas | edleafe: but the claim takes care of the ratio as a limit | |
| 14:06:12 | bauzas | edleafe: you know the process, right? | |
| 14:06:21 | bauzas | so | |
| 14:06:24 | edleafe | bauzas: yeah, I've seen it once or twice | |
| 14:06:41 | gmann | alex_xu: np! | |
| 14:06:55 | bauzas | okay, so the problem is that we suddently stop supporting something without a clear upgrade path just because we advocate it's bonkers | |
| 14:07:07 | bauzas | that doesn't look a correct answer to me | |
| 14:07:32 | bauzas | since we allowed for a while to opt-out filters | |
| 14:07:48 | edleafe | bauzas: so are you saying that today if you disable CoreFilter, you could successfully build a VM with 8 VCPUs on a 2 VCPU host? | |
| 14:08:15 | bauzas | edleafe: if you live-migrate, yes | |
| 14:08:19 | cdent | the "build a VM" part is the critical question, not what select_destionations does or does not return | |
| 14:08:42 | edleafe | I am unaware of any hypervisor that can do that magic | |
| 14:09:00 | bauzas | let's be clear, you can live-migrate a 8 VCPU VM to a 2 VCPU node | |
| 14:09:08 | johnthetubaguy | bauzas: we can deprecate something, saying its bonkers, and remove it the following cycle | |
| 14:09:08 | bauzas | at least Nova doesn't prevent you to do so | |
| 14:09:21 | bauzas | johnthetubaguy: that was the original approach | |
| 14:09:32 | edleafe | bauzas: but do you get an 8 VCPU vm? | |
| 14:09:42 | bauzas | johnthetubaguy: but I got strong consensus to not do that and rather straight not accepting | |
| 14:09:45 | johnthetubaguy | thats defiantly my preference, because of the release tags we are signed up to | |
| 14:10:14 | bauzas | edleafe: that depends on your driver | |
| 14:10:36 | cdent | Kevin_Zheng: I'll respond on the issue | |
| 14:10:40 | bauzas | edleafe: but whatever the situation is, nova *is* accepting that | |
| 14:10:51 | Kevin_Zheng | cdent: thanks | |
| 14:11:01 | johnthetubaguy | bauzas: you have to get an 8 VCPU vm, else it wouldn't live-migrate, as I understand it | |
| 14:11:03 | bauzas | in other words, folks, how can I make that functional test be working when using the placement API ? | |
| 14:11:24 | bauzas | I think there is a big misunderstanding between vcpus and pcpus | |
| 14:11:37 | cdent | bauzas: when you say "accepting that" what do you mean by "that", explicitly? | |
| 14:11:56 | bauzas | cdent: accepting to give you a destination that is not verifying the max unit | |
| 14:11:59 | johnthetubaguy | FWIW, I know rackspace cloud have never run the core filter, we only run the RAM filter | |
| 14:12:11 | bauzas | johnthetubaguy: then you could be in serious touble | |
| 14:12:21 | cdent | bauzas: sure but destinations don't matter, where vms are built does | |
| 14:12:30 | bauzas | johnthetubaguy: see https://review.openstack.org/#/c/417961/24/releasenotes/notes/resource_providers_scheduler_db_filters-16b2ed3da00c51dd.yaml | |
| 14:12:42 | johnthetubaguy | we would just have to set the over-comitt ratio correctly, but thats OK as a warning | |
| 14:12:49 | cdent | that's what I keep trying to clarify here: is the behavior of where vms built actually changing, or just the list of destinations? | |
| 14:12:52 | sahid | johnthetubaguy: i replied to your questions about the patch related to the feature emulator threads | |
| 14:13:11 | sahid | hope that is going to clarify your mind | |
| 14:13:49 | bauzas | johnthetubaguy: cdent: seriously, I'm done | |
| 14:14:09 | bauzas | I'm trying to spot issues that could come up in production and I'm only seeing a "meh, it's bonkers" | |
| 14:14:15 | cdent | bauzas: why? I'm really trying hard to understand the situation. | |
| 14:14:34 | bauzas | cdent: Jenkins will tell you the situation | |
| 14:14:43 | cdent | If it is the case that "the behavior of where vms built actually changing" then we have an issue. | |
| 14:15:08 | cdent | functional tests failing isn't necessarily telling us anything because we've demonstrated time and again (recently) that the functional tests have bad setups. | |
| 14:15:38 | bauzas | cdent: in particular, tbp, some functional tests in the test_server_group testcase won't give you the same exact result once we call the placement service, which is to me a good indicator that placement decisions are not feature-parity between Newton and Ocata | |
| 14:16:11 | cdent | that's a good indicator, but only if we verify that the tests are correct | |
| 14:16:23 | cdent | so I guess the next step is to wait and see what fails and go from there | |
| 14:17:21 | johnthetubaguy | so for folks with the core filter turned off the decisions will be different now we enforce that, it seems to me | |
| 14:17:37 | bauzas | johnthetubaguy: my point indeed | |
| 14:17:49 | bauzas | johnthetubaguy: I spotted that problem in a ML thread | |
| 14:17:49 | johnthetubaguy | bauzas: yep, I am trying to agree with you here | |
| 14:18:01 | johnthetubaguy | bauzas: I liked that first version of your code that dealt with that | |
| 14:18:15 | bauzas | johnthetubaguy: and the agreement we made with matt and jay was to rather ask the operators to set their ratios to an high value | |
| 14:18:32 | cdent | all along my position has been "bauzas' suggestion sounds sane but I defer to jay, dan and matt". I still do. | |
| 14:18:49 | edleafe | johnthetubaguy: I'm really not getting this. You're saying that it is expected behavior that Nova will build an 8 VCPU VM on a 2 VCPU host? | |
| 14:19:00 | bauzas | johnthetubaguy: but I'm trying to explain that because of an implementation detail and a design agreement that the placement engine should respect max units, we can't have the same | |
| 14:19:27 | bauzas | edleafe: a 2VCPU host doesn't mean anything to me | |
| 14:19:38 | bauzas | edleafe: hosts have physical threads | |
| 14:20:21 | johnthetubaguy | edleafe: thats not my point here, my point is we are breaking folks existing configurations across upgrade | |
| 14:20:21 | sfinucan | sahid: Left a comment on https://review.openstack.org/#/c/400875. Could you take a look so I can continue with the rest of the series, please :) | |
| 14:20:57 | johnthetubaguy | edleafe: if folks have the core filter off, they probably have not set the over commit raitio correctly | |
| 14:21:10 | edleafe | johnthetubaguy: I was using bauzas's example. | |
| 14:21:15 | johnthetubaguy | edleafe: when we change the default filters, it broke the XenAPI CI, for one example | |
| 14:21:29 | edleafe | johnthetubaguy: so our release notes tell them how to fix that | |
| 14:21:42 | bauzas | edleafe: but the releasenote I wrote is not *fixing* that | |
| 14:21:51 | bauzas | edleafe: I thought yesterday it was the case | |
| 14:21:55 | bauzas | but it's not | |
| 14:22:03 | bauzas | because *again* we cap on max_unit | |
| 14:22:25 | edleafe | johnthetubaguy: I understand it can break tests, but what is the effect in the real world? | |
| 14:22:27 | openstackgerrit | Vladik Romanovsky proposed openstack/nova: Adding vlans field to Device tagging metadata https://review.openstack.org/405951 | |
| 14:22:54 | bauzas | either way, we're 24 hours of the FF, and what we all discuss can possibly be a no-brainer if we are not able to amend grenade in time | |
| 14:23:07 | johnthetubaguy | aye | |
| 14:23:08 | cdent | bauzas: when a compute node reports inventory it reports a value from compute_node.vcpus. how is that calculated? | |
| 14:23:41 | cdent | (I'm trying to expand my understanding here, nothing else) | |
| 14:23:51 | bauzas | cdent: you mean the helper method, but how nova can know what cpus the node has ? | |
| 14:23:56 | bauzas | s/but/or | |
| 14:24:37 | cdent | I saying: there's a properly on the compute_node object called vcpus, which is set by update_from_virt_driver | |
| 14:24:39 | cdent | what does it mean? | |
| 14:27:03 | johnthetubaguy | bauzas: random thought, what if we change max unit to pCPU * allocation ratio? | |
| 14:27:22 | johnthetubaguy | edleafe: real world, feels like it only makes sense for test rigs and dev rigs | |
| 14:29:10 | edleafe | johnthetubaguy: what if we dump max_unit altogether? | |