| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-27 | |||
| 00:47:48 | mriedem | dansmith: time for your change, do i need https://review.openstack.org/#/c/505456/ or just the one below it? | |
| 00:48:17 | dansmith | mriedem: the one below it should orphan those so they're never called | |
| 00:48:25 | dansmith | so you shouldn't notice any difference afaik | |
| 00:48:44 | mriedem | ok | |
| 01:38:29 | mriedem | dansmith: ok i have results in https://etherpad.openstack.org/p/nova-instance-list | |
| 01:38:31 | mriedem | with your change | |
| 01:39:18 | dansmith | is that faster/same except for details? | |
| 01:39:25 | mriedem | compared to w/o your change, (1) GET /servers with microversion 2.1 is slightly faster | |
| 01:39:42 | mriedem | GET /servers/detail with microversion is about the same, a bit faster | |
| 01:39:45 | mriedem | 2.1 | |
| 01:39:56 | mriedem | but, GET /server/details with microversion 2.53 is slower | |
| 01:40:01 | mriedem | not a ton, but it's slower | |
| 01:40:10 | mriedem | 25.78 compared to 30.10 | |
| 01:40:28 | mriedem | but, it's not a huge different | |
| 01:40:30 | dansmith | oh only detail with the later microversion | |
| 01:40:33 | mriedem | right | |
| 01:40:43 | mriedem | something about >2.1 always makes listing with details slower | |
| 01:40:44 | dansmith | and there's some fault handling behavior difference? | |
| 01:40:53 | mriedem | at least because of the joins on the (1) services table and (2) tags table | |
| 01:41:17 | mriedem | i don't think there is any fault handling behavior differences with microversion >2.1 | |
| 01:41:22 | mriedem | if there was, that might explain it | |
| 01:41:27 | dansmith | okay I thought you were saying there was | |
| 01:41:41 | dansmith | I dunno why because I'm pre-joining it when we were loading them separate | |
| 01:41:48 | mriedem | the only other joins i can think of right now with microversion >2.1 is on the services table (2.16) and tags able (2.26) | |
| 01:42:26 | mriedem | still, it's a difference of about 4 seconds, which isn't huge here | |
| 01:42:35 | dansmith | so aside from fault, there's no difference in what I'm doing vs what we do currently, | |
| 01:42:42 | dansmith | other than we're not serializing the queries | |
| 01:43:18 | dansmith | without my change we issue the cell0 one and then the cell1 one, where now we're doing both at once | |
| 01:43:28 | dansmith | is this a devstack vm on your laptop or something better? | |
| 01:43:38 | mriedem | it's in a vexxhost vm | |
| 01:44:43 | mriedem | the fault stuff is the only major difference i can think of, since we'll be joining on fault all the time, rather than just for instances in ERROR state | |
| 01:45:09 | mriedem | maybe that is equaling things out somehow, idk, like if i had 1000 all in ERROR state before/after your change, that might be different in favor of yours | |
| 01:45:31 | dansmith | hmm, yeah, I guess maybe that might be it | |
| 01:46:08 | mriedem | i do have the numbers from yesterday before your change with 1000 ACTIVE, | |
| 01:46:20 | mriedem | so tomorrow i could run yours through with all active and see if there is a bigger difference because of the fault join | |
| 01:46:26 | dansmith | well, I guess we could go back to the not automatic loading of fault | |
| 01:46:48 | mriedem | i'll run that all active scenario tomorrow to see if it could be the fault stuff, | |
| 01:46:54 | mriedem | it's nearly 9pm so i'm not going to do it tonight | |
| 01:47:09 | dansmith | there was something the API was doing that made it seem way better to do this than what it was doing | |
| 01:47:19 | dansmith | but it's been a while now | |
| 01:48:34 | dansmith | we could also plumb the logic of when to load the fault into the lower layers | |
| 01:49:14 | mriedem | yup i was thinking that too | |
| 01:49:34 | mriedem | another thing that might be causing the microversion bloat, is maybe the microversion to pull the embedded flavor out of the instance | |
| 01:49:42 | mriedem | added in pike | |
| 01:50:01 | dansmith | you could run through each microversion and see where the spike is | |
| 01:50:33 | dansmith | the sorting layer on top of this really has nothing to do with what we're sorting though | |
| 01:50:42 | dansmith | it doesn't make any more copies of things, nor iterate the list more times | |
| 01:50:47 | mriedem | 2.47 | |
| 01:51:59 | dansmith | so, the change right before the switchover should do the fault loading but not the sorting, so you could run against that and see if it's more like the earlier or more like the later | |
| 01:53:49 | mriedem | https://review.openstack.org/#/c/506774/ ? | |
| 01:53:59 | mriedem | like, revert that on top of the change that uses the new code in the API | |
| 01:54:00 | mriedem | ? | |
| 01:54:14 | mriedem | oh, nvm, | |
| 01:54:15 | dansmith | oh, I guess you were running on master already? | |
| 01:54:23 | mriedem | yes, new devstack as of today | |
| 01:54:38 | dansmith | yeah, okay | |
| 01:55:19 | mriedem | so 2.16 makes us join on services, 2.26 makes us join on tags, 2.47 returns instance.flavor, and your change always joins on faults | |
| 01:55:25 | mriedem | 2.47 is suspicious | |
| 01:55:31 | mriedem | since that's from instance_extra | |
| 01:55:41 | dansmith | but again, it shouldn't be any different | |
| 01:57:41 | mriedem | yeah, nvm, we also didn't start loading that in the api as of 2.47, we already pulled out instance.flavor to get the link stuff | |
| 01:59:17 | mriedem | totally unrelated, but when we lazy-load instance.flavor, we're still joining on system_metadata now, we should be able to stop doing that | |
| 02:00:02 | dansmith | yeah | |
| 02:04:40 | mriedem | ok, i'll run through with 1000 active instances tomorrow with your change and see if that makes a big difference, and if so, it could be the fault thing | |
| 02:05:13 | dansmith | alright | |
| 02:29:52 | yushb | JOIN #openstack-karbor | |
| 03:01:31 | dansmith | mriedem: I don't auto-join fault until: https://review.openstack.org/#/c/505456/10/nova/api/openstack/compute/servers.py | |
| 03:01:36 | dansmith | so I don't think it's the fault | |
| 03:33:50 | openstackgerrit | Michael Still proposed openstack/nova master: Move ploop commands to privsep. https://review.openstack.org/492325 | |
| 03:33:50 | openstackgerrit | Michael Still proposed openstack/nova master: Read from console ptys using privsep. https://review.openstack.org/489486 | |
| 03:33:51 | openstackgerrit | Michael Still proposed openstack/nova master: Don't shell out to mkdir, use ensure_tree() https://review.openstack.org/492326 | |
| 03:33:51 | openstackgerrit | Michael Still proposed openstack/nova master: Cleanup mount / umount and associated rmdir calls https://review.openstack.org/494423 | |
| 03:33:52 | openstackgerrit | Michael Still proposed openstack/nova master: Move lvm handling to privsep. https://review.openstack.org/495516 | |
| 03:33:52 | openstackgerrit | Michael Still proposed openstack/nova master: Move shred to privsep. https://review.openstack.org/495537 | |
| 03:33:53 | openstackgerrit | Michael Still proposed openstack/nova master: Move xend existence probes to privsep. https://review.openstack.org/495538 | |
| 03:33:53 | openstackgerrit | Michael Still proposed openstack/nova master: Move the idmapshift binary into privsep. https://review.openstack.org/495541 | |
| 03:33:54 | openstackgerrit | Michael Still proposed openstack/nova master: Move loopback setup and removal to privsep. https://review.openstack.org/495664 | |
| 03:33:54 | openstackgerrit | Michael Still proposed openstack/nova master: Move nbd commands to privsep. https://review.openstack.org/500351 | |
| 03:33:55 | openstackgerrit | Michael Still proposed openstack/nova master: Move kpartx calls to privsep. https://review.openstack.org/500354 | |
| 03:33:55 | openstackgerrit | Michael Still proposed openstack/nova master: Move blkid calls to privsep. https://review.openstack.org/500398 | |
| 03:54:40 | openstackgerrit | Merged openstack/nova master: Add slowest command to tox.ini https://review.openstack.org/507657 | |
| 06:07:24 | openstackgerrit | jichenjc proposed openstack/nova master: check query param for used_limits function https://review.openstack.org/499091 | |
| 06:21:07 | lennyb | Hi, I am working on devstack master, and my n-cond-cell1.service got stucked during stop #link http://paste.openstack.org/show/622005/. it's log is empty, no exceptions in other logs. Any tips/ideas will be appreciated. | |
| 06:37:49 | openstackgerrit | jichenjc proposed openstack/nova master: check query param for server groups function https://review.openstack.org/500347 | |
| 06:39:06 | openstackgerrit | Yikun Jiang proposed openstack/nova master: Update Instance action's updated_at when action event updated. https://review.openstack.org/507473 | |
| 06:57:04 | openstackgerrit | Lajos Katona proposed openstack/nova master: Moving more utils to ServerResourceAllocationTestBase https://review.openstack.org/499539 | |
| 06:57:04 | openstackgerrit | Lajos Katona proposed openstack/nova master: factor out compute service start in ServerMovingTest https://review.openstack.org/503037 | |
| 06:57:05 | openstackgerrit | Lajos Katona proposed openstack/nova master: Test resource allocation during soft delete https://review.openstack.org/495159 | |
| 06:59:09 | openstackgerrit | Alex Xu proposed openstack/nova-specs master: Add trait support in the allocation candidates API https://review.openstack.org/497713 | |
| 07:10:27 | openstackgerrit | Yikun Jiang proposed openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API https://review.openstack.org/507762 | |
| 07:20:22 | openstackgerrit | Yikun Jiang proposed openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API https://review.openstack.org/507762 | |
| 07:25:09 | openstackgerrit | Zhenyu Zheng proposed openstack/nova stable/pike: Mention API behavior change when over quota limit https://review.openstack.org/507769 | |
| 07:58:42 | openstackgerrit | Zhenyu Zheng proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware https://review.openstack.org/507486 | |
| 08:30:57 | gibi | johnthetubaguy: hi! do you have any comments on http://lists.openstack.org/pipermail/openstack-dev/2017-September/122658.html ? mikal has already stated that rackspace private cloud is not using it http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-09-26.log.html#t2017-09-26T19:57:03 | |
| 08:49:54 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove dead code of api.fault notification sending https://review.openstack.org/505164 | |
| 08:56:37 | ralonsoh | dansmith: hi, can I talk about https://review.openstack.org/#/c/449257/42/nova/objects/instance_pci_requests.py? | |
| 08:58:32 | johnthetubaguy | gibi: I am very far removed from if that would be used now | |
| 08:59:05 | johnthetubaguy | gibi: I believe the context around that was providing an SLA that involved knowing about the 500 errors in the API, and doing bug fixes to reduce them | |
| 08:59:42 | johnthetubaguy | gibi: i.e. the SLA is related to the % of 5xx errors from the API, so errors in the API are really important to those folks | |
| 09:00:48 | johnthetubaguy | gibi: I don't remember using those notifications myself, mostly got that data from ELK when I was last looking at that stuff | |