| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-31 | |||
| 18:06:43 | cfriesen | no, it's 3 secs just for "from nova.api.validation import parameter_types" | |
| 18:06:56 | sean-k-mooney | oh that is different | |
| 18:07:25 | leakypipes | sean-k-mooney: well, there's no state in the dunderinit file at least... just functions. | |
| 18:07:44 | sean-k-mooney | cfriesen: can you temporally comment out the file level suff just to see if it changes | |
| 18:09:07 | leakypipes | holy fucksticks, Batman. | |
| 18:09:08 | leakypipes | [jaypipes@uberbox nova]$ source .tox/py27/bin/activate | |
| 18:09:09 | leakypipes | sys0m0.201s | |
| 18:09:09 | leakypipes | user0m3.331s | |
| 18:09:09 | leakypipes | real0m5.653s | |
| 18:09:09 | leakypipes | (py27) [jaypipes@uberbox nova]$ time python -m nova.api.validation.parameter_types | |
| 18:09:34 | sean-k-mooney | cfriesen: the only thin that looks even remotely expensive is maybe _build_regex_range and even then the most expecive thing i see there is re.escape | |
| 18:10:08 | sean-k-mooney | i dont really like this https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L128-L142 | |
| 18:11:08 | sean-k-mooney | def _get_all_chars(): | |
| 18:11:10 | sean-k-mooney | for i in range(0xFFFF): | |
| 18:11:12 | sean-k-mooney | yield six.unichr(i) | |
| 18:11:50 | sean-k-mooney | so that is got to loop 65535 times | |
| 18:12:01 | sean-k-mooney | *going too | |
| 18:12:44 | sean-k-mooney | and we call the function 17 times | |
| 18:14:08 | sean-k-mooney | so that 1.1 million callse to re.escape every time we load that module | |
| 18:14:45 | sean-k-mooney | leakypipes: cfriesen shall i fix this or do one of ye wnat to do it | |
| 18:15:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Default AZ for instance if cross_az_attach=False and checking from API https://review.openstack.org/469675 | |
| 18:15:40 | cfriesen | sean-k-mooney: go for it. :) | |
| 18:16:00 | sean-k-mooney | cfriesen: good find by the way that is nuts | |
| 18:17:25 | openstack | Launchpad bug 1790195 in OpenStack Compute (nova) "performance problems starting up nova process due to regex code" [Undecided,New] | |
| 18:17:25 | cfriesen | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1790195 | |
| 18:18:17 | leakypipes | cfriesen: ++ | |
| 18:21:06 | cfriesen | sean-k-mooney: I think that's likely the culprit. Changing it to "for i in range(0x1):" made "python -m nova.api.validation.parameter_types" take 1.5 sec instead of 7. | |
| 18:21:19 | mriedem | hmm, so https://review.openstack.org/#/c/598366/ passed with https://review.openstack.org/#/c/598365/ but i'm not sure if that tells us much | |
| 18:21:48 | sean-k-mooney | cfriesen: ill fix this two ways. first ill cache te result of _build_regex_range wit a decorator and second ill probably convert the file level vars to propertys so they are lazy evaulted | |
| 18:22:04 | cfriesen | sean-k-mooney: sounds good | |
| 18:48:52 | openstackgerrit | Adam Harwell proposed openstack/nova stable/pike: Add apply_cells to nova-manage https://review.openstack.org/599050 | |
| 18:51:48 | openstackgerrit | Merged openstack/nova master: Report client: update_from_provider_tree w/reshape https://review.openstack.org/585049 | |
| 18:51:57 | openstackgerrit | Merged openstack/nova master: Compute: Handle reshaped provider trees https://review.openstack.org/576236 | |
| 18:52:07 | openstackgerrit | Merged openstack/nova master: Do test_reshape with an actual startup https://review.openstack.org/597218 | |
| 18:52:16 | openstackgerrit | Merged openstack/nova master: Fix reshaper report client functonal test nits https://review.openstack.org/598330 | |
| 18:55:28 | cfriesen | any chance of a second core looking at https://review.openstack.org/#/c/588657/ ? It's a fairly straightforward change related to image properties, claims, and evacuate. | |
| 18:58:53 | mriedem | that is an excellent patch | |
| 19:01:23 | leakypipes | cfriesen: ack, I'll review shortly. | |
| 19:07:30 | cfriesen | leakypipes: much appreciated | |
| 19:08:07 | leakypipes | cfriesen: you KNOW how much I love the instance migration/evacuate/shelve/funkychicken code paths. | |
| 19:08:42 | cfriesen | just the thing for a friday | |
| 19:11:15 | sean-k-mooney | cfriesen: so the decorator alone went from 0m3.535s to 1.128s | |
| 19:11:43 | cfriesen | cool | |
| 19:33:54 | leakypipes | cfriesen: +W | |
| 19:34:22 | cfriesen | leakypipes: sweet, thx | |
| 19:43:29 | openstack | Launchpad bug 1790204 in OpenStack Compute (nova) "Allocations are "doubled up" on same host resize even though there is only 1 server on the host" [Medium,Triaged] | |
| 19:43:29 | mriedem | fried_rice: you wanted to know when i reported this https://bugs.launchpad.net/nova/+bug/1790204 | |
| 19:43:40 | mriedem | cfriesen: ^ fyi | |
| 19:44:05 | dansmith | mriedem: shall we? https://review.openstack.org/#/c/598353 | |
| 19:44:26 | mriedem | dansmith: see my comment on the test? | |
| 19:44:42 | dansmith | oh, sure | |
| 19:44:48 | mriedem | iow, if i remove the fix the test would still pass i think | |
| 19:44:49 | fried_rice | mriedem: ack, thx | |
| 19:44:59 | dansmith | mriedem: yep probably | |
| 19:45:15 | dansmith | mriedem: the way the gate is that makes this probably tuesday before it's in | |
| 19:45:24 | dansmith | I dunno what the fail rate is | |
| 19:45:50 | mriedem | my gut says the pass rate is not great | |
| 19:46:09 | dansmith | fail rate? | |
| 19:46:22 | mriedem | but this has also been this way since queens i think when i changed devstack to use [upgrade_levels]compute=auto so what's a few more days | |
| 19:47:00 | mriedem | i don't actually know what the pass/fail rate is right now | |
| 19:47:32 | mriedem | heh http://grafana.openstack.org/d/QBHIN5Smk/tempest-failure-rate?orgId=1 | |
| 19:48:27 | mriedem | these dashboards are all using old job names | |
| 19:49:55 | dansmith | okay i thought this was spiking in the gate based on the discussion | |
| 19:50:07 | mriedem | not it's definitely not our worst failure | |
| 19:50:19 | dansmith | ack | |
| 19:50:25 | mriedem | http://status.openstack.org/elastic-recheck/#1789484 | |
| 19:50:27 | openstackgerrit | Dan Smith proposed openstack/nova master: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/598353 | |
| 19:50:31 | mriedem | it's not nothing either | |
| 20:05:47 | mriedem | dansmith: +2 | |
| 20:13:31 | openstack | Launchpad bug 1790204 in OpenStack Compute (nova) "Allocations are "doubled up" on same host resize even though there is only 1 server on the host" [Medium,Triaged] | |
| 20:13:31 | mriedem | fried_rice: more details https://bugs.launchpad.net/nova/+bug/1790204/comments/1 - i expect reschedules would make any solution for that bug even messier | |
| 20:13:59 | fried_rice | mriedem: ack. I'm subscribed. | |
| 20:17:20 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix image-defined numa claims during evacuate https://review.openstack.org/599062 | |
| 20:17:51 | mriedem | we've got an easy gate fix here https://review.openstack.org/598353 | |
| 20:17:58 | mriedem | would be good to get that in given the gate borkages of late | |
| 20:19:12 | fried_rice | mriedem, dansmith: +A | |
| 20:20:28 | mriedem | thanks | |
| 20:22:07 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/599064 | |
| 20:51:58 | openstackgerrit | Eric Fried proposed openstack/nova master: Other host allocs may appear in gafpt during evac https://review.openstack.org/597301 | |
| 20:55:10 | openstackgerrit | Eric Fried proposed openstack/nova master: Other host allocs may appear in gafpt during evac https://review.openstack.org/597301 | |
| 20:55:45 | openstackgerrit | sean mooney proposed openstack/nova master: add caching to _build_regex_range https://review.openstack.org/599071 | |
| 21:10:50 | openstack | Launchpad bug 1790221 in OpenStack Compute (nova) "shelved offloaded server still shows old AZ while shelved" [Low,Triaged] | |
| 21:10:50 | mriedem | that wily old shelve https://bugs.launchpad.net/nova/+bug/1790221 | |
| 21:12:06 | cfriesen | mriedem: why is that bad? | |
| 21:12:22 | cfriesen | the unshelve api doesn't say anything about selecting a new AZ | |
| 21:12:41 | mriedem | cfriesen: a shelved instance isn't on a host, | |
| 21:12:49 | mriedem | note that the server details don't show a host or node when it's shelved | |
| 21:13:00 | mriedem | if it's not on a host, it's clearly not in an AZ | |
| 21:13:00 | cfriesen | mriedem: sure, but the AZ was selected at boot time as part of the boot criteria | |
| 21:13:06 | mriedem | nope | |
| 21:13:13 | mriedem | i didn't create the server with an az | |
| 21:13:23 | mriedem | so when i unshelve it, it can go into any othre az | |
| 21:14:02 | mriedem | related https://review.openstack.org/#/c/567701/ | |
| 21:15:10 | cfriesen | I had thought that once we pick an AZ at boot it might have implications on which cinder backends are available, so you couldn't leave it afterwards | |
| 21:15:23 | mriedem | oh fuck no | |
| 21:15:36 | mriedem | related https://review.openstack.org/#/c/469675/ | |
| 21:15:37 | cfriesen | cool, learned something new | |
| 21:15:59 | mriedem | we only give a shit about volume az if cinder.cross_az_attach=False, which it's not by default | |
| 21:16:09 | mriedem | if it is, well, good luck friend | |
| 21:16:12 | mriedem | b/c nova sucks at handling that | |