| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-21 | |||
| 14:55:32 | mriedem | andreykurilin: hey i assume you like to do some scale and performance testing since you work on rally, right? | |
| 14:56:54 | dansmith | gibi: I updated the docstring... did I miss a reference? | |
| 14:57:38 | gibi | dansmith: https://review.openstack.org/#/c/504983/6/nova/compute/instance_list.py | |
| 14:57:43 | dansmith | what in the | |
| 14:57:48 | gibi | dansmith: L69 and L93 | |
| 14:57:58 | dansmith | craaap | |
| 14:58:06 | dansmith | I must have dumped it during a rebase | |
| 14:58:09 | dansmith | urgh | |
| 14:58:12 | dansmith | I _did_ update ;) | |
| 14:58:20 | gibi | I believe you :) | |
| 14:58:23 | openstackgerrit | Elod Illes proposed openstack/nova master: Add instance.interface_attach notification https://review.openstack.org/503089 | |
| 14:58:50 | dansmith | gibi: from my history: https://pastebin.com/u7nUYi38 | |
| 14:58:51 | dansmith | :P | |
| 14:59:20 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: Spec: Use keystoneauth1 Adapter for endpoints https://review.openstack.org/500190 | |
| 14:59:31 | efried | mriedem edmondsw ^ | |
| 14:59:52 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix hyperlinks in document https://review.openstack.org/506101 | |
| 14:59:54 | edmondsw | efried ack | |
| 15:01:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix hyperlinks in document https://review.openstack.org/506101 | |
| 15:01:53 | openstackgerrit | Dan Smith proposed openstack/nova master: Add base implementation for efficient cross-cell instance listing https://review.openstack.org/504983 | |
| 15:01:54 | openstackgerrit | Dan Smith proposed openstack/nova master: Add db.instance_get_by_sort_filters() https://review.openstack.org/504985 | |
| 15:01:54 | openstackgerrit | Dan Smith proposed openstack/nova master: Make instance_list honor global query limit https://review.openstack.org/504984 | |
| 15:01:55 | openstackgerrit | Dan Smith proposed openstack/nova master: Add fault-filling into instance_get_all_by_filters_sort() https://review.openstack.org/505391 | |
| 15:01:55 | openstackgerrit | Dan Smith proposed openstack/nova master: Support pagination in instance_list https://review.openstack.org/504986 | |
| 15:01:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Add get_instance_objects_sorted() https://review.openstack.org/505417 | |
| 15:01:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Add tests to validate instance_list handles faults correctly https://review.openstack.org/505392 | |
| 15:01:57 | openstackgerrit | Dan Smith proposed openstack/nova master: Fix a pagination logic bug in test_bug_1689692 https://review.openstack.org/505661 | |
| 15:01:57 | openstackgerrit | Dan Smith proposed openstack/nova master: Copy some tests to a cellsv1 mixin https://review.openstack.org/505442 | |
| 15:01:58 | openstackgerrit | Dan Smith proposed openstack/nova master: Remove legacy fault-loading routines https://review.openstack.org/505456 | |
| 15:01:58 | openstackgerrit | Dan Smith proposed openstack/nova master: Use improved instance_list module in compute API https://review.openstack.org/505418 | |
| 15:02:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: fix default sort key when listing servers https://review.openstack.org/506227 | |
| 15:02:09 | andreykurilin | mriedem: hi! basically, I like to do great things, so rally is my choice :) but yeah, scale and performance testing are somehow related :) | |
| 15:03:22 | gibi | dansmith: I don't need proofs I have faith :) | |
| 15:03:46 | dansmith | gibi: I need proof for my own sanity :) | |
| 15:03:55 | gibi | :) | |
| 15:04:25 | gibi | yeah, sanity is important | |
| 15:05:42 | mriedem | andreykurilin: ok some of us got talking at the PTG about how we need some scale testing done on nova, especially with multiple cells, and thought you might be interested in doing that, | |
| 15:06:01 | mriedem | especially since godaddy is using cells v1 and presumably has to investigate migrating to cells v2 at some point | |
| 15:06:40 | mriedem | andreykurilin: more specifically, dansmith is working on a series of changes to make listing instances across cells more efficient, | |
| 15:06:48 | mriedem | it would be nice if we could get benchmarks before/after | |
| 15:07:39 | mriedem | i'd also be interested in general in how long it takes to create instances in ocata vs pike, since pike is now doing quotas differently and doing resource claims in the scheduler rather than the computes | |
| 15:08:28 | mriedem | i could do this a bit sloppy with devstack and using the fake virt driver, but that doesn't give me multiple cells | |
| 15:10:06 | andreykurilin | mriedem: so basically, it is quite easy to add scenarios for cells v1/v2 (and generate the load), but the problem is in where to launch it. I'll talk to some folks to try find the place for such research | |
| 15:15:51 | mriedem | bauzas: a few comments inline https://review.openstack.org/#/c/506092/ | |
| 15:16:06 | mriedem | andreykurilin: great, thanks | |
| 15:16:26 | mriedem | we've been trying to get feedback from large cells v1 users for awhile now and so far we don't get much feedback | |
| 15:18:26 | efried | sdague You want me to do anything about https://review.openstack.org/#/c/488137/17/nova/utils.py@1309 ? (There or in a fup?) | |
| 15:22:19 | bauzas | mriedem: k, cool, reading | |
| 15:24:50 | bauzas | mriedem: just a question for just one comment, I'm not sure why we need to disable the source host | |
| 15:25:05 | bauzas | mriedem: because you'd like to make sure you end up in some specific host? | |
| 15:30:26 | dansmith | edleafe: I dunno why, but gerrit is being confusing about the stack of actual code patches for your selection/alternates bit | |
| 15:30:36 | dansmith | edleafe: is this the bottom? https://review.openstack.org/#/c/486215/7 | |
| 15:33:47 | mriedem | bauzas: that's what was reported in the bug for the recreate, | |
| 15:33:54 | mriedem | but i guess that's not really necessary to reproduce the issue | |
| 15:34:07 | mriedem | because the scheduler will compare 1 host to 2 instances and fail | |
| 15:34:14 | mriedem | bauzas: yeah so nevermind that part | |
| 15:34:33 | bauzas | okay | |
| 15:35:14 | edleafe | dansmith: yep | |
| 15:35:47 | edleafe | dansmith: it seems to be adding specs and code together | |
| 15:37:13 | dansmith | edleafe: well, it is also jumping around with the related-changes view of the code | |
| 15:37:51 | dansmith | I guess it's because the add-allocations patch has been orphaned for a while | |
| 15:39:44 | mriedem | sdague: https://review.openstack.org/#/c/500190/ is the "re-proposal" of the ksa adapters spec if you want to go over that. it's not so much a re-proposal as nearly a re-write though. | |
| 15:40:33 | mriedem | edleafe: do i need to review those alternative hosts specs in any order? there is the one about the object model and then the other one. | |
| 15:40:49 | mriedem | or should those be merged? | |
| 15:41:15 | edleafe | mriedem: alternate hosts was a concept we agreed to in Pike but never wrote up in a spec | |
| 15:41:31 | edleafe | mriedem: the Selection object is independent | |
| 15:44:02 | dansmith | edleafe: is the host.updated=True thing the flag we use to avoid choosing that same host for another instance of a multi-create? | |
| 15:44:12 | sdague | mriedem: will look | |
| 15:44:28 | sdague | mriedem: I'm trying to sort out a final set of unit tests fails on the QEMU_IMG version thing | |
| 15:45:27 | edleafe | dansmith: I think so. That host state stuff is still black magic to me | |
| 15:45:39 | dansmith | edleafe: oh actually it's host.updated=None.. how. strange. | |
| 15:45:55 | dansmith | edleafe: okay I will comment based on that assumption and hedge a bit | |
| 15:47:00 | drwahl | does placement-api (in newton) require keystone v3? | |
| 15:47:42 | bauzas | dansmith: that's because updated is the flag for saying "please refresh my state" | |
| 15:48:19 | dansmith | bauzas: okay I'm confused about why we need to do that in this context then | |
| 15:48:39 | bauzas | dansmith: what's the exact concern you have in mind ? | |
| 15:48:48 | dansmith | oohh, | |
| 15:48:51 | bauzas | I just jumped on your above question | |
| 15:49:02 | dansmith | maybe this is "I'm going to consume this host, so make sure to refresh it soon" ? | |
| 15:49:08 | dansmith | https://review.openstack.org/#/c/486215/7/nova/scheduler/filter_scheduler.py | |
| 15:49:09 | dansmith | L92 | |
| 15:50:05 | bauzas | dansmith: no | |
| 15:50:14 | bauzas | dansmith: that's because we consume_from_request() before | |
| 15:50:35 | bauzas | dansmith: so the in-memory HostState is modified | |
| 15:50:42 | mriedem | stvnoyes: the scsi volume live migration tempest test failed for the new style attach flows in a grenade job, meaning one of the computes in the live migration is pike and one is queens http://logs.openstack.org/90/481290/6/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/d8962cc/console.html#_2017-09-17_04_21_05_125937 | |
| 15:51:01 | bauzas | dansmith: in case the calculations were wrong, we need to unset those modifications and force a fresh new DB call | |
| 15:51:26 | dansmith | bauzas: meaning if we fail to do the consume, we want to trigger a refresh? | |
| 15:51:26 | bauzas | we generally keep in memory the HostStates once per request | |
| 15:51:44 | bauzas | but for a multi-schedule call, then we could iterate wrongly | |
| 15:51:46 | openstackgerrit | Merged openstack/os-vif master: Rehome OVO unit tests to tests.unit.test_object.py https://review.openstack.org/489922 | |
| 15:51:53 | bauzas | dansmith: yup that sort of | |
| 15:51:58 | dansmith | hrm | |
| 15:52:09 | dansmith | not sure I fully get it, but it's not the thing I thought, so that's fine | |
| 15:52:33 | mriedem | stvnoyes: looks like it fails here https://review.openstack.org/#/c/487884/7/tempest/api/compute/admin/test_live_migration.py@185 | |
| 15:52:38 | dansmith | edleafe: so, what follows is some rambling for improvements we can make later, but listen and nod (or not): | |
| 15:52:57 | bauzas | dansmith: see HostManager.consume_from_request() and you'll understand | |
| 15:53:21 | dansmith | edleafe: I think what you're doing here is going through the list of hosts like we do today, and picking the target and alternates for each one, | |
| 15:54:02 | dansmith | edleafe: which may mean your first alternate for instance 2 may be the the target for instance 3 and so on, such that all the alternates for early instances are almost definitely going to be burned by the later instances for any num_instances>max_retries | |
| 15:54:03 | dansmith | right? | |
| 15:54:29 | dansmith | so, hmm, yeah if that's right, that seems like a problem | |
| 15:54:31 | edleafe | dansmith: yes. Selecting an alternate doesn't consume anything | |