| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-31 | |||
| 11:26:24 | efried | alex_xu The "numbered" ("non-sharing") group can only get its resources from a single RP. | |
| 11:27:15 | efried | alex_xu When several groups are in a single request, all the resources as a whole must come from the same nested tree + sharing RPs in the same aggregate. | |
| 11:27:57 | openstackgerrit | Merged openstack/nova master: api-ref: document caveats with scheduler hints https://review.openstack.org/516442 | |
| 11:28:11 | efried | alex_xu It's all in the spec, which of course you memorized :) https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/granular-resource-requests.html | |
| 11:28:51 | efried | alex_xu Specifically: https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/granular-resource-requests.html#semantics | |
| 11:42:36 | cdent | efried: one thing I’m a bit curious about is how much type 2 fun a flavor author is going to have trying to figure out how to properly use this syntax | |
| 11:42:53 | efried | "type 2 fun"? | |
| 11:42:59 | cdent | one moment please | |
| 11:43:17 | efried | no, I got it. | |
| 11:43:26 | cdent | k | |
| 11:43:38 | cdent | it might even be type 3 | |
| 11:44:45 | cdent | I mean, I guess once you’ve got it right you can forget about it | |
| 11:45:09 | efried | cdent That. And a guy could just use all numbered groups and forgedaboutit | |
| 11:46:01 | efried | Putting a single RC into each of several numbered groups will have the same effect as putting them all into the unnumbered group. | |
| 11:46:11 | efried | Then you just cluster together resources that really need to be clustered to gether. | |
| 11:46:17 | efried | s/ // | |
| 11:47:12 | efried | Given that we need to be able to do this, I'm not sure if there's a simpler syntax. | |
| 11:47:26 | efried | I would say I'm open to suggestions, but spec freeze and all that. | |
| 11:47:28 | cdent | I suppose these are power tools for power users, so it’s okay. I’m glad it’s not a common case for daily end users. | |
| 11:47:44 | cdent | Oh sorry, I’m not disagreeing with the syntax, at all | |
| 11:47:52 | cdent | It solves the problem admirably | |
| 11:47:58 | cdent | I’m disagreeing with the problem :) | |
| 11:51:07 | efried | cdent As in, you think we're solving a non-problem? Or it just sucks that it's a thing that has to be solved? | |
| 11:53:52 | amarao1 | Which nova component produces the least amount of RPC messages? (I want to test timeout settings for idle connection and I need some nova service which do not send messages but only wait for them). | |
| 11:53:53 | cdent | the latter | |
| 11:54:44 | cdent | efried: it’s necessary complexity for a certain set of complex things but I’d prefer a world that was somehow simpler. Or something? | |
| 11:55:28 | efried | cdent I can get behind a desire for a simpler world. | |
| 11:57:21 | openstackgerrit | Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766 | |
| 12:06:21 | jaypipes | alex_xu: hey bud, you up still? | |
| 12:08:03 | jaypipes | efried: you around yet? | |
| 12:08:08 | efried | jaypipes Yup | |
| 12:09:05 | jaypipes | efried: so I was thinking I might try and do a couple patches to clean up/standardize the AllocationCandidates._get_by_filters() mega-method | |
| 12:09:24 | jaypipes | efried: I know alex_xu already has a series where the *end* of the series refactors that method | |
| 12:09:44 | jaypipes | efried: but I'm thinking it might be more useful if I broke that method out into separate funcs before his series. | |
| 12:10:04 | openstackgerrit | Radoslav Gerganov proposed openstack/nova master: VMware: fix memory stats https://review.openstack.org/516634 | |
| 12:10:04 | openstackgerrit | Radoslav Gerganov proposed openstack/nova master: VMware: implement get_inventory() driver method https://review.openstack.org/506175 | |
| 12:10:05 | openstackgerrit | Radoslav Gerganov proposed openstack/nova master: VMware: expose max vCPUs and max memory per ESX host https://review.openstack.org/516635 | |
| 12:10:17 | jaypipes | efried: you've been reviewing alex's series (https://review.openstack.org/#/c/479766/ is the refactor megamethod patch). which is why I'm wanting to get your opinion. | |
| 12:12:10 | efried | jaypipes That was the intent behind what alex_xu and I have been working on in this series, for sure. Are you talking about doing some refactoring that leads into that series, or a parallel effort that approaches it a different way? | |
| 12:15:39 | jaypipes | efried: that leads *in* to that series. | |
| 12:16:11 | efried | jaypipes Sounds good to me. What do you have in mind? | |
| 12:16:33 | jaypipes | efried: I'm not a fan of the objectified approach in that series and would like to break that mega-method out into separate module-level functions that are appropriately named. | |
| 12:17:39 | efried | jaypipes As in, get rid of e.g. the ProviderSummary[List] and AggregatedProviderSummary[List] etc.? | |
| 12:17:49 | jaypipes | yes, exactly. | |
| 12:17:58 | jaypipes | I don't believe those objects are healthy. | |
| 12:18:06 | jaypipes | or necessary. | |
| 12:18:09 | efried | jaypipes Okay, so essentially supplanting the series with a different approach. | |
| 12:18:52 | jaypipes | efried: meh, in a way. more just laying a different foundation for the series to build from... | |
| 12:19:17 | efried | I'm less enthused about that, but not opposed to it. We need to be able to move forward here, and it's clear that the current monolith is untenable. But it's also clear we're having trouble constructing SQL that handles all the cases. | |
| 12:20:05 | efried | So on the one hand, having some jaypipes join-fu help would be great; but on the other hand, having something only jaypipes can understand... less than ideal. | |
| 12:20:15 | jaypipes | efried: and I've said in reviews on this series (and others before) that we don't need to try and do all the pieces in one giant SQL statement. | |
| 12:20:22 | efried | ++ | |
| 12:20:57 | efried | I'm not well versed enough to understand where we introduce huge inefficiencies by moving join-y stuff from the database to python. | |
| 12:21:05 | efried | But in principle, I love that idea. | |
| 12:21:06 | jaypipes | cdent: for the life of me, I cannot understand why I'm getting the failure (only on the gabbit in py35) in this particular patch: https://review.openstack.org/#/c/385693/. If you might take a looksie, that would be most appreciated... | |
| 12:21:22 | cdent | jaypipes: shore. grass still too wet to cut | |
| 12:22:38 | efried | jaypipes So like, would it be appropriate to have one method that pulls just inventory/allocation info for a single RP+RC? Or is that too small? | |
| 12:23:11 | jaypipes | efried: it's not that complicated actually. the basic idea is don't complicate the SQL unless there's a reason to do so. All lengthy and complex SQL statements, as you know, can be broken out into composite sub-statements and executed separately. Sometimes, actually, doing multiple queries that each are very efficient can be faster than trying to do a single SQL complex statement. | |
| 12:23:42 | jaypipes | efried: for the use_single_provider request groups, that is EXACTLY the right way to do things. | |
| 12:24:14 | efried | jaypipes Yeah, I've been thinking the SQL for the use_single_provider groups would be way simpler than what's happening now. | |
| 12:25:25 | efried | jaypipes Okay, so I'm behind the idea, but alex_xu and I ought to know what we should be doing with the existing series while you work that up. Probably best to quit messing with it, cause it'll either change drastically or be abandoned altogether? | |
| 12:25:39 | jaypipes | produce each of those sets in a way that can be reasoned about easily afterwards. | |
| 12:25:39 | jaypipes | efried: think in terms of sets of records. there are multiple sets of records we want to retrieve based on the problem we're attempting to solve. in the case of shared providers, that's one set of records. the set of providers that provider ALL parts of a request of resources, that's another set. The set of providers that provides *some* but not all resources, that's yet another set of records. the trick is make efficient SQL statements to | |
| 12:26:56 | efried | jaypipes And presumably to make sure the amount of data we're pulling from the database and then paring down from python isn't way too big. | |
| 12:27:04 | jaypipes | efried: right. | |
| 12:27:51 | priya | gibi:Could you please review https://review.openstack.org/#/c/508533/ | |
| 12:28:10 | efried | jaypipes Dunno if you remember in Austin the example I called out: they had a shell script that did "SELECT * FROM really_huge_database" | wc -l | |
| 12:28:27 | jaypipes | efried: who is "they"? | |
| 12:29:08 | jaypipes | priya: I *think* gibi may be on public holiday today... not entirely sure though. | |
| 12:29:10 | efried | jaypipes I don't remember. Could have been a cinder thing? To count up soft-deleted entries or something? And it wasn't production code, just a testing util. But it still grated. | |
| 12:29:22 | jaypipes | yeah... | |
| 12:30:22 | cdent | jaypipes: the gabbi problem is a combination of case sensitivity and dict ordering unpredictability. you didn’t introduce it, it just didn’t fail before in commit a9105b4904b that added the failing test. I’ll note the fix on the review, which we might want to do separately to yours and merge nowish | |
| 12:30:59 | jaypipes | cdent: hmm, intersting. is this a py35 vs py27 thing? or just a freak occaaion? | |
| 12:31:04 | jaypipes | occasion.. | |
| 12:31:19 | cdent | I suspect that py35’s dict ordering is a factor | |
| 12:31:32 | efried | jaypipes So let me know if I can help out here. Sounds to me like the intent is to write a bunch of separate independent functions and eventually refactor _get_by_filters to use 'em. So in that first stage, if you want to dole out "write me a function that does X" we can work in parallel. | |
| 12:31:35 | gibi | jaypipes, priya: I will be on public holiday tomorrow :) so I will look at it soon | |
| 12:31:41 | cdent | but there’s also freakery involved otherwise it would have failed on efried’s patch | |
| 12:31:56 | jaypipes | efried: zactly. | |
| 12:32:27 | jaypipes | efried: just like the current code currently first calls out to a separate function to get "root providers" and then gets shared providers, etc... | |
| 12:32:41 | jaypipes | efried: just needs to be cleaned up a bit. | |
| 12:32:45 | jaypipes | efried: and better doc'd. | |
| 12:40:16 | openstackgerrit | David Rabel proposed openstack/nova master: Fix format in live-migration-usage.rst https://review.openstack.org/516416 | |
| 12:45:35 | cdent | jaypipes: i’ll go ahead and fix that thing separately https://bugs.launchpad.net/nova/+bug/1728934 | |
| 12:45:37 | openstack | Launchpad bug 1728934 in OpenStack Compute (nova) "placement gabbi tests that manipulate microversion can intermittently fail" [Medium,Triaged] - Assigned to Chris Dent (cdent) | |
| 12:57:25 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] avoid case issues microversions in gabbits https://review.openstack.org/516662 | |
| 12:57:34 | efried | jaypipes Was about to respin the flavor-side granular series (https://review.openstack.org/#/c/515151/) - you got more to say there at the moment? | |
| 12:57:53 | cdent | jaypipes, efried, gibi ^^ gate flakiness bug fix | |
| 12:58:11 | efried | cdent Would that be officially remedied by making gabbi use case insensitive dicts (like requests does) for headers? | |
| 12:58:25 | cdent | efried: read the commit message for the answer to that | |
| 12:59:35 | cdent | efried: the idea is that gabbi needs to support the behaviors of http, not whatever clients happen to be available | |
| 12:59:40 | gibi | cdent: looking... | |
| 12:59:55 | efried | cdent I thought the http standard was for headers to be case-insensitive. But I'm no expert. | |
| 13:00:07 | efried | cdent Did you deliberately say "vagrancies" rather than "vagaries"? | |
| 13:00:37 | cdent | and because http header names are case insensitive it is possible to accidentally set the same header twice in some implementations | |
| 13:00:43 | cdent | no, but i’ll take it :) | |
| 13:01:22 | efried | I think even with case insensitivity you can set the same header multiple times. | |
| 13:01:45 | cdent | yes, that’s the “implementation” part of the commit message. | |
| 13:01:55 | cdent | it’s a yaml dict | |
| 13:02:29 | cdent | so _if_ we want to send a header twice, we have to use different case | |