| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-17 | |||
| 13:42:11 | mdbooth | kashyap: But before that it'll raise a different error if the job doesn't exist | |
| 13:42:28 | mdbooth | Specifically the error that edleafe is seeing | |
| 13:42:54 | kashyap | mdbooth: You're referring to: | |
| 13:42:55 | kashyap | if (!disk->mirror) { | |
| 13:42:55 | kashyap | virReportError(VIR_ERR_OPERATION_INVALID, | |
| 13:42:55 | kashyap | _("pivot of disk '%s' requires an active copy job"), | |
| 13:42:58 | kashyap | disk->dst); | |
| 13:43:00 | mdbooth | Yes | |
| 13:43:01 | kashyap | ? | |
| 13:43:26 | openstackgerrit | Roman Dobosz proposed openstack/nova-specs master: Provide detailed error information for placement API https://review.openstack.org/418393 | |
| 13:44:57 | kashyap | mdbooth: In that case, either the job was already finished (maybe not gracefully); or it was not started. Hence, it whines "no block operation for me work on" | |
| 13:45:25 | kashyap | Let's wait for edleafe's logs (hopefully small, with that specific test) | |
| 13:45:30 | mdbooth | Either way, based on that code I belive the pivot will fail | |
| 14:10:45 | openstackgerrit | Michael Henkel proposed openstack/nova master: contrail: add vrouter VIF plugin type support https://review.openstack.org/441183 | |
| 14:11:04 | leakypipes | mhenkel: :) | |
| 14:11:26 | kashyap | mdbooth: So, if you want to hear the results (credit: Max from QEMU did the tests & reported) of earlier experiment with qcow2, I can write here. | |
| 14:11:32 | mhenkel | hey leakypipes | |
| 14:11:36 | mdbooth | kashyap: Shoot | |
| 14:11:41 | kashyap | mdbooth: When I asked about the format, he says it doesn't really matter: Detection whether the source contains zeroes -- That is independent of the format, the code actually checks the data read from the source whether it is zero. | |
| 14:11:42 | mdbooth | s/with/without/ | |
| 14:12:21 | kashyap | mdbooth: Okay, here we go: | |
| 14:12:30 | mhenkel | leakypipes: is the commit msg looking ok now? | |
| 14:12:46 | kashyap | The experiment being: build a backing chain, of depth one, of empty images, perform 'block-stream' and see whether something gets allocated. Three cases: | |
| 14:12:53 | kashyap | (1) With empty images nothing is allocated at all; | |
| 14:13:18 | kashyap | (2) With zeros (using 'qemu-io -c 'write -P 0 0 64M' base.qcow2') written to the base, the qcow2 file for some reason (there's one, but not sure what) grows by a cluster (64 kB). | |
| 14:13:26 | kashyap | (3) With non-zero data (using 'qemu-io -c 'write -P 42 0 64M' base.qcow2'), all data is written to the overlay. | |
| 14:13:45 | kashyap | * * * | |
| 14:13:47 | openstackgerrit | Dan Smith proposed openstack/python-novaclient master: Fix aggregate_update name and availability_zone clash https://review.openstack.org/447009 | |
| 14:14:17 | dansmith | so mriedem, we supposedly have tests for this ^ and they pass, but the two cases they check don't work in real life | |
| 14:14:21 | kashyap | mdbooth: Max tells (& I can confirm by testing as I have the QMP env. ready) running the same tests on a raw image should yield the same results. | |
| 14:14:23 | dansmith | mriedem: thoughts on how to test that change? | |
| 14:14:27 | openstackgerrit | Roman Dobosz proposed openstack/nova-specs master: Provide detailed error information for placement API https://review.openstack.org/418393 | |
| 14:14:47 | mdbooth | kashyap: Is this dependent on how the target disk is initially created? | |
| 14:15:29 | kashyap | mdbooth: Hmm, that is a good question. The target image in this specific test is a simple qcow2 overlay | |
| 14:16:00 | mdbooth | So, in the case of volume migration the source and destination are both block devices from the pov of qemu | |
| 14:16:16 | edmondsw | mriedem, what is the plan/status on setting send_service_user_token conf setting to True by default (i.e. no longer experimental)? | |
| 14:16:23 | mriedem | dansmith: ooo i smell a customer bug | |
| 14:16:56 | mriedem | dansmith: tests where? novaclient? | |
| 14:17:00 | mdbooth | kashyap: I'd be interested to see the result if you create 2 sparse files with loopback devices, and copy between the loopback devices | |
| 14:17:08 | dansmith | mriedem: hang on, linking | |
| 14:17:08 | mdbooth | Is the result sparse? | |
| 14:17:13 | mriedem | edmondsw: we need some scale testing first, | |
| 14:17:37 | kashyap | Always asks tricky questions on a Friday, mdbooth... | |
| 14:17:58 | mdbooth | kashyap: Hopefully it's just a little reconfiguration of the test you've already got | |
| 14:18:04 | mdbooth | I think you've already done the hard work :) | |
| 14:18:04 | kashyap | mdbooth: I will let you know, once I come back from my bike ride. My eyes are aching | |
| 14:18:05 | mriedem | edmondsw: raj_sing- is the lead on that stuff i believe. we have service user tokens enabled in the nova-next job, but as part of the original spec we had talked about the need for some scale testing to stress it to force the usage of the service user tokens, and to stress keystone since it has to do double duty token-wise per request now. | |
| 14:18:15 | dansmith | mriedem: added a comment with links | |
| 14:18:25 | mriedem | edmondsw: https://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/use-service-tokens.html#performance-impact | |
| 14:18:53 | mriedem | "Rally benchmark tests will be ran with and without the “service_user” config options set to compare the results for long running tasks like snapshot or live migration." | |
| 14:19:14 | mriedem | edmondsw: if you have some people familiar with scale testing and rally, it would be helpful to pitch in on the testing there | |
| 14:19:25 | mriedem | i'm not sure if raj_sing- or the osic people are focusing on that right now | |
| 14:19:27 | lbragstad | aunnam around? | |
| 14:20:07 | mriedem | dansmith: i wouldn't trust the unit tests at all, | |
| 14:20:20 | mriedem | dansmith: i'd probably just write a functional test in the novaclient tree that does what you need for how this is supposed to work | |
| 14:20:27 | edmondsw | mriedem that all makes sense. We don't use rally but we do some scale testing. I can ask about trying this there | |
| 14:20:30 | dansmith | mriedem: clearly not, the same tests pass before and after this | |
| 14:20:30 | mriedem | see it break on the bug, and then show it passing with the fix | |
| 14:20:32 | kashyap | mdbooth: Quick clarification, before I head out for a few: IIUC, you want the loopback device to be the overlay, yes? | |
| 14:20:45 | mriedem | dansmith: like what i've been doing with the functional regression tests in nova lately | |
| 14:21:09 | mdbooth | kashyap: I want both source and destination to be loopback devices backed by sparse files | |
| 14:21:22 | mdbooth | kashyap: They should both be the same size | |
| 14:21:34 | mdbooth | kashyap: The source should contain some data, but the dest should initially be blank | |
| 14:21:34 | mriedem | edmondsw: cool. i'm thinking a long-running volume-backed migration where the token will expire and have to be revalidated for cinder and neutron, | |
| 14:21:47 | dansmith | mriedem: okay I forgot about the functional ones in here.. looks like we don't have any for aggregates | |
| 14:21:50 | mriedem | edmondsw: or a long-running snapshot of a large instance (note you'll need the unmerged glance patches for that) | |
| 14:22:27 | mriedem | dansmith: at first glance i'm not seeing anyway | |
| 14:22:33 | mriedem | but they are pretty simple to add | |
| 14:22:56 | mriedem | like https://github.com/openstack/python-novaclient/blob/master/novaclient/tests/functional/v2/legacy/test_extended_attributes.py | |
| 14:22:59 | dansmith | yeah, I'll work on it | |
| 14:23:01 | mriedem | specify the base 2.1 version, | |
| 14:23:05 | edleafe | kashyap: mdbooth: still trying to get the repro for the failure. Didn't give up (yet) :) | |
| 14:23:07 | kashyap | mdbooth: Okay, noted. Just to restate: The backing file(s) must be sparse; and the overlays are loopback devices -- then try a copy among the overlays and see if sparseness is preserved | |
| 14:23:08 | mriedem | extend base class, and that gives you most of what you need | |
| 14:23:12 | mriedem | self.nova() runs the command | |
| 14:23:21 | mdbooth | kashyap: Bingo | |
| 14:23:29 | kashyap | edleafe: Don't give up! One simply does not give up in #openstack-nova channel! | |
| 14:23:38 | mdbooth | kashyap: I think that accurately represents the issue as described | |
| 14:23:57 | kashyap | Okay, with that clarity, bbiab | |
| 14:24:01 | edleafe | kashyap: heh, I figured it was taking me so long you might have thought that I just went back to bed or something | |
| 14:24:25 | ijw | The Nova team has beds now? | |
| 14:24:31 | ijw | Luxury. | |
| 14:24:37 | edleafe | Indeed! | |
| 14:24:51 | mriedem | i have a dugout in the dirt | |
| 14:24:55 | mriedem | it serves my needs | |
| 14:27:07 | markus_z | mriedem: AFAIK I provided all information for your concerns on https://review.openstack.org/#/c/406898/ Can I do anything else there? | |
| 14:30:30 | mriedem | markus_z: i've removed the -1 | |
| 14:30:37 | markus_z | thanks | |
| 14:31:56 | mriedem | markus_z: so basically there are some things backported, like the console refactor patches, which were not functionally needed, but made the target bug fix backport easier | |
| 14:32:25 | mriedem | and all of those groundwork backports are already merged by now anyway | |
| 14:33:19 | markus_z | mriedem: The refactor patches were not backported as they weren't functionally needed. That's why there are conflicts. | |
| 14:34:18 | markus_z | *if* we would have backported the "groundwork" patches, then there wouldn't have been conflicts. | |
| 14:34:35 | mriedem | sure, but that refactor series also introduced a regression in ocata, | |
| 14:34:40 | mriedem | the one about xen and libvirtd | |
| 14:34:46 | mriedem | *virtlogd | |
| 14:35:04 | markus_z | ah, damn, you're right | |
| 14:35:13 | mriedem | right, so we don't want those backports | |
| 14:35:15 | mriedem | for the refactor patches | |
| 14:35:21 | mriedem | or did we backport that to newton? | |
| 14:35:36 | markus_z | sec, I have to check the fix again, need a minute | |