| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-27 | |||
| 17:40:40 | mriedem | i'm pretty surprised at the improvements there | |
| 17:42:18 | cdent | mriedem: which job results on https://review.openstack.org/#/c/507918/ are my best target for pokage? | |
| 17:42:40 | dansmith | mriedem: hmm | |
| 17:43:01 | dansmith | mriedem: if you roll back to the other patch does it go back to the perf you measured before? | |
| 17:43:37 | dansmith | mriedem: with my patch we iterate the list fewer times | |
| 17:44:01 | dansmith | I'd be surprised if it made that much difference, but it should make some | |
| 17:44:25 | mriedem | can try that in a bit | |
| 17:44:44 | dansmith | my microversion survey results are interesting | |
| 17:44:46 | dansmith | and not good | |
| 17:44:57 | dansmith | will be done in a few minutes | |
| 17:45:16 | mriedem | cdent: i'd think just the normal tempest dsvm job http://logs.openstack.org/18/507918/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/d0a5723/ | |
| 17:45:27 | cdent | roger that | |
| 17:45:29 | mriedem | lots of copied bash in here so i likely screwed something up | |
| 17:46:36 | mriedem | hmm, didn't even get to my stuff | |
| 17:48:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove dest node allocations during live migration rollback https://review.openstack.org/507687 | |
| 17:49:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove dest node allocations during live migration rollback https://review.openstack.org/507687 | |
| 17:50:07 | dansmith | mriedem: check that out: https://imgur.com/a/2lmiw | |
| 17:50:11 | dansmith | sdague: you too ^ | |
| 17:50:29 | mriedem | jesus, graphs?! | |
| 17:51:04 | mriedem | well, looking at 2.46 and 2.47, i think it's 2.47 https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id41 | |
| 17:51:06 | dansmith | 2.26 was not free but 2.47 is killing us | |
| 17:51:07 | dansmith | yeah | |
| 17:51:41 | mriedem | this is 500 error/500 active right? | |
| 17:51:46 | mriedem | dansmith: want to report a bug with details? | |
| 17:51:51 | dansmith | mriedem: yes, 500/500 | |
| 17:53:04 | dansmith | so that was on my patch | |
| 17:53:19 | dansmith | I'm going to run it again on master but starting at maybe 2.24 to make sure the knees are in the same place | |
| 17:53:26 | mriedem | ok | |
| 17:53:28 | sdague | dansmith: interesting.... any idea why the display on the embedded data is causing things to go nuts | |
| 17:53:34 | mriedem | i did notice this w/o your change too though | |
| 17:53:50 | mriedem | i wonder if we're lazy-loading the flavor extra specs? | |
| 17:53:58 | dansmith | mriedem: yeah, I don't see differences in my numbers so I'm just doing it for completeness | |
| 17:54:02 | dansmith | sdague: I haven't looked yet | |
| 17:54:04 | sdague | mriedem: ah, right, that's probably it | |
| 17:54:20 | dansmith | we shouldn't be lazy-loading extra specs | |
| 17:54:28 | dansmith | they should be in the flavor in the instance | |
| 17:55:56 | sdague | I mean, it is a bunch more data. I guess it could just be serialization cost of more data, though it seems weird. | |
| 17:56:34 | mriedem | well, we were always getting the flavor | |
| 17:56:35 | dansmith | mriedem: https://bugs.launchpad.net/nova/+bug/1719966 | |
| 17:56:37 | openstack | Launchpad bug 1719966 in OpenStack Compute (nova) "Microversion 2.47 punches nova in its special place" [Undecided,New] | |
| 17:56:38 | mriedem | even before 2.27 | |
| 17:56:43 | mriedem | ha | |
| 17:56:57 | dansmith | sdague: right, no difference in what we're pulling from the db across that boundary, just what we do with it in the api | |
| 17:57:36 | dansmith | sdague: (I checked) | |
| 17:57:40 | mriedem | https://github.com/openstack/nova/blob/3174ee13a1541230a4b7b2a4737d679691fb14b3/nova/api/openstack/compute/views/servers.py#L269 | |
| 17:57:53 | mriedem | so we were always pulling it https://github.com/openstack/nova/blob/3174ee13a1541230a4b7b2a4737d679691fb14b3/nova/api/openstack/compute/views/servers.py#L263 | |
| 17:58:15 | mriedem | and we were always joining on it in the db https://github.com/openstack/nova/blob/3174ee13a1541230a4b7b2a4737d679691fb14b3/nova/api/openstack/compute/views/servers.py#L58 | |
| 17:58:26 | mriedem | so why is this so much slower? https://github.com/openstack/nova/blob/3174ee13a1541230a4b7b2a4737d679691fb14b3/nova/api/openstack/compute/views/servers.py#L248 | |
| 17:58:33 | mriedem | the policy check for each instance? | |
| 17:59:42 | dansmith | the only thing we can lazy-load from flavor isprojects, BTW | |
| 17:59:52 | dansmith | not extra_specs or anything else | |
| 18:00:07 | dansmith | https://github.com/openstack/nova/blob/master/nova/objects/flavor.py#L318-L319 | |
| 18:00:08 | mriedem | ok i thought that we always had extra_specs but didn't go back to check | |
| 18:00:13 | sdague | mriedem: yeh, the policy check is going to be per instance | |
| 18:00:29 | sdague | the policy check is a fs.stat as well | |
| 18:00:39 | dansmith | we should check it once and pass it to the per-instance flavor method right? | |
| 18:00:43 | mriedem | yes | |
| 18:00:50 | sdague | because policy file is dynamically reread | |
| 18:00:55 | mriedem | right | |
| 18:00:56 | mriedem | ... | |
| 18:00:58 | mriedem | jesus | |
| 18:00:58 | dansmith | that might explain why mriedem sees a bigger hit | |
| 18:01:15 | dansmith | you know what | |
| 18:01:19 | mriedem | where is cfriesen when it's time to talk about performance degradation? | |
| 18:01:19 | dansmith | I think we might want to backport this fix | |
| 18:01:27 | mriedem | we for sure do | |
| 18:01:33 | dansmith | I mean.. maybe | |
| 18:01:44 | mriedem | the policy thing is backportable | |
| 18:01:46 | mriedem | check once | |
| 18:01:54 | dansmith | we could leave it and just further relegate pike to the trashcan of releases | |
| 18:01:59 | mriedem | ha | |
| 18:02:02 | mriedem | but, | |
| 18:02:06 | mriedem | ocata is already in that trashcan | |
| 18:02:09 | dansmith | haha | |
| 18:02:36 | mriedem | i've literally been sending emails internally for weeks saying, "once you upgrade to pike, this should all be much better" | |
| 18:02:46 | mriedem | should* | |
| 18:02:57 | mriedem | *: not actual statement of fact backed up by any evidence | |
| 18:03:19 | melwitt | heh | |
| 18:05:23 | sdague | it would be nice if a context only evaluated a particular policy rule once | |
| 18:05:36 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Support traits in the Ironic driver https://review.openstack.org/507052 | |
| 18:06:29 | sdague | because it's going to be a little awkward to handle cases where you need to send down the preevaluted permission through a bunch of function calls | |
| 18:07:25 | mriedem | this one shoudn't be too terrible | |
| 18:07:28 | mriedem | dansmith: are you started on the fix? | |
| 18:07:29 | dansmith | right this one should be easy | |
| 18:07:37 | dansmith | mriedem: no but I can | |
| 18:07:57 | dansmith | I'd rather fix this and you finish reviewing my patch | |
| 18:08:08 | mriedem | which patch? the kahuna? | |
| 18:08:16 | mriedem | i can't finish what i haven't started | |
| 18:08:25 | dansmith | I'd rather fix this and you start reviewing my patch | |
| 18:08:49 | Tengu | hello! anyone can point me a valid doc for pike and host aggregation + flavor pinning? I'm stuck right now trying to get all working, and I find contradictory docs :/ | |
| 18:09:24 | mriedem | define "flavor pinning" | |
| 18:09:35 | mriedem | you can associate a host aggregate with a specific flavor via metadata / extra specs | |
| 18:09:46 | Tengu | mriedem: "m1.small must run on that aggregate, while m2.small must run on this aggregate" | |
| 18:09:48 | mriedem | however, any other aggregate which is not tied to that flavor can still use it | |
| 18:09:52 | mriedem | there is no exclusion | |
| 18:10:32 | mriedem | Tengu: i think you're looking for this then https://review.openstack.org/#/c/381912/ | |
| 18:10:32 | Tengu | what should I put in the metadata? | |
| 18:10:42 | Tengu | ah, will check that | |
| 18:11:13 | Tengu | 3 days ago? darn… pretty fresh | |
| 18:11:27 | mriedem | that spec has been around quite awhile | |