| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-01-09 | |||
| 13:30:30 | mdbooth | lyarwood: Yeah, bug doesn't have that metadata, but I'd say it should qualify | |
| 13:30:42 | sean-k-mooney | stephenfin: ya i remember you telling me that just was not sure if you missed it or not | |
| 13:30:58 | lyarwood | mdbooth: yeah, had to read through the commit but yup IMHO it's fine, I'll review the pike version now | |
| 13:31:17 | mdbooth | lyarwood: Thanks | |
| 13:33:00 | sean-k-mooney | stephenfin: lyarwood: since we are talking about backport i would like to backport this to pike and ocata too its trival enough but its a latent bug that has been present since os-vif was first created | |
| 13:33:02 | claudiub | efried: yeay, gonna have to celebrate. :)) | |
| 13:33:25 | claudiub | the patches on nova still need reviews though. :D | |
| 13:33:41 | sean-k-mooney | stephenfin: lyarwood by the book would this qualify for backporting? | |
| 13:33:43 | stephenfin | sean-k-mooney: "this" being? | |
| 13:33:53 | sean-k-mooney | stephenfin: https://review.openstack.org/#/c/531465/ | |
| 13:39:05 | lyarwood | sean-k-mooney: yeah I'd say that would be valid for ocata, it's trivial but useful. I'm not sure if there are other considerations that need to be taken into account for libs however. | |
| 13:41:21 | sean-k-mooney | lyarwood: in normal execution i think the edgecase that causes the interface not to be present happens very infrequently however in the gate it was causing random failures hence why it has been latent for so long | |
| 13:45:34 | openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Use neutron port_list when filtering instance by ip https://review.openstack.org/525505 | |
| 13:53:49 | kashyap | lyarwood: So apparently there are different reasons for hitting that error you pointed out earlier. (The bug URL I was looking for was, which has a slightly different error): https://bugzilla.redhat.com/show_bug.cgi?id=1378788 (Race condition during virtio-blk dataplane stop triggers "Virtqueue size exceeded") | |
| 13:53:50 | openstack | bugzilla.redhat.com bug 1378788 in qemu-kvm-rhev "Race condition during virtio-blk dataplane stop triggers "Virtqueue size exceeded"" [High,Closed: errata] - Assigned to stefanha | |
| 13:55:14 | mriedem | dtantsur: the devstack dependency for https://review.openstack.org/#/c/527289/ is now merged | |
| 13:55:50 | dtantsur | mriedem: thanks, approved | |
| 13:55:54 | mriedem | thanks | |
| 13:56:28 | lyarwood | kashyap: cool thanks, /me reads | |
| 13:56:56 | kashyap | lyarwood: But we still don't have a root cause for the unique error you pointed out | |
| 13:57:03 | kashyap | The above bug was fixed in version QEMU 2.6 & above, though. | |
| 13:57:34 | lyarwood | kashyap: kk, this is with 2.8 FWIW | |
| 13:57:58 | lyarwood | kashyap: do we need a bug open somewhere for this in that case? | |
| 13:57:59 | kashyap | lyarwood: Hmm, okay, Stefan just responded: | |
| 13:58:15 | kashyap | "This issue probably requires debugging. This kind of problem can occur if the vring memory is not up-to-date (e.g. requests were processed after migration paused the guest)." | |
| 13:58:36 | kashyap | lyarwood: Do you see multiple hits for this? | |
| 13:58:49 | kashyap | lyarwood: Yeah, an open bug helps definitely, | |
| 13:59:00 | kashyap | ...as others hitting it may have more reliable reproducers. | |
| 13:59:10 | lyarwood | kashyap: I've not looked, I'll check now, if we do then I'll create a LP for it | |
| 14:07:08 | edmondsw | mriedem and sdague, the powervm OVS patch is fully ready for reviews now, passing CI | |
| 14:07:11 | edmondsw | https://review.openstack.org/#/c/422512/ | |
| 14:07:32 | edmondsw | we're working a CI issue for the SEA patch this morning, but as stated yesterday the code is ready for review | |
| 14:07:50 | edmondsw | will update on the CI for SEA when we have that fixed | |
| 14:09:03 | mriedem | i would expect efried to review that first | |
| 14:09:25 | efried | will do | |
| 14:10:09 | lyarwood | kashyap: nvm we already have a LP from mriedem for this here - https://bugs.launchpad.net/nova/+bug/1737625 | |
| 14:10:10 | openstack | Launchpad bug 1737625 in OpenStack Compute (nova) "live migration failed with "Failed to load virtio-blk:virtio" (libvirt/qemu)" [Undecided,Confirmed] | |
| 14:11:56 | kashyap | lyarwood: Ah, thanks. | |
| 14:19:11 | efried | mriedem Done | |
| 14:19:32 | efried | (OVS, that is) | |
| 14:28:11 | stephenfin | mriedem: Our Python 3 woes have been resolved https://review.openstack.org/#/c/530950/ | |
| 14:28:32 | stephenfin | mriedem: Fancy revisiting https://review.openstack.org/#/c/345399/ at some point this week? | |
| 14:40:02 | ildikov | mriedem: are you planning to do further tweaks on the multi-attach chain in Nova? | |
| 14:42:42 | mriedem | stephenfin: maybe | |
| 14:42:45 | mriedem | ildikov: maybe | |
| 14:43:47 | ildikov | mriedem: ok, I started to look into the unit test failures, the first half is an easy fix, but won't touch the code for now then | |
| 14:43:53 | stephenfin | mriedem: Good enough for me | |
| 14:44:38 | mriedem | ildikov: i posted it at 11pm and had fixed what i knew about, but haven't looked at the results yet this morning | |
| 14:45:12 | ildikov | mriedem: http://logs.openstack.org/47/271047/48/check/openstack-tox-py27/991a4d8/testr_results.html.gz | |
| 14:45:30 | mriedem | i'll look later, in the middle of something | |
| 14:45:51 | ildikov | mriedem: I can fix these, upload and then we can agree who does what onwards, if that helps | |
| 14:45:56 | mriedem | sure | |
| 14:46:04 | ildikov | mriedem: ok, will do that then, tnx | |
| 14:46:29 | ildikov | mriedem: I just wanted to ask to avoid overwriting each others' versions... | |
| 14:47:34 | mriedem | yup | |
| 14:53:02 | mriedem | mdbooth: clearly this has to be wrong right? https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1669 | |
| 14:53:06 | mriedem | and source_type not in ('lvm') | |
| 14:54:42 | mriedem | i guess it does work as a substring match | |
| 14:54:48 | mriedem | probably by accident | |
| 14:55:38 | mdbooth | mriedem: I hadn't even spotted the 'strings are a list' thing... | |
| 14:56:17 | openstackgerrit | Ameed Ashour proposed openstack/nova master: change instance_system_metadata column type https://review.openstack.org/526900 | |
| 14:56:40 | mdbooth | Yeah, looks like it probably works accidentally | |
| 14:57:45 | mdbooth | >>> 'foo' in ('foobar',) | |
| 14:57:46 | mdbooth | False | |
| 14:58:36 | mriedem | yeah, just getting another set of eyes, just noticed it while looking at something else | |
| 15:02:31 | alex_xu_ | sdague: as I remember we forze the '/os-migrations' API, then we want to add links in the '/os-migrations' API to the '/servers/{uuid}/migrations', is it right? https://review.openstack.org/#/c/459483/ | |
| 15:03:10 | mriedem | alex_xu_: yes, but that was pretty short sighted | |
| 15:03:24 | mriedem | as that change was specifically for the change to abort a running live migration for a given instance | |
| 15:03:33 | mriedem | there is still use in the os-migrations API | |
| 15:03:55 | mriedem | for example, it's the only one that you can use to list not in progress migrations | |
| 15:06:58 | openstackgerrit | Lee Yarwood proposed openstack/nova master: rbd: flatten images when creating/unshelving an instance https://review.openstack.org/457886 | |
| 15:11:27 | mriedem | dansmith: i'm +2 on the migration paging/filtering series https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/add-pagination-and-change-since-for-migration-list | |
| 15:11:34 | mriedem | that includes the cross-cell lister | |
| 15:11:43 | dansmith | mriedem: okay | |
| 15:19:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: don't attempt to live snapshot paused instances https://review.openstack.org/532214 | |
| 15:20:04 | mriedem | sdague: does this look familiar? ^ i now remember why i added the PAUSED condition to the patch to enable libvirt live snapshot by default | |
| 15:20:08 | mriedem | it doesn't work on older libvirt/qemu | |
| 15:25:20 | openstackgerrit | sahid proposed openstack/nova master: hardware: only take into account small pages https://review.openstack.org/532168 | |
| 15:29:59 | mriedem | stephenfin: simple reno update https://review.openstack.org/#/c/531963/ | |
| 15:30:10 | mriedem | i was reading the release notes last night and when you put them together, the context is lost on some of them | |
| 15:30:18 | kashyap | mriedem: A quick aside: Follow up from your test that the 'shareable' disk flag "doesn't work when using raw images either http://paste.openstack.org/show/639185/" | |
| 15:30:32 | kashyap | It was a real bug apparently and fixed in libvirt 3.10 and above. (All the messy dependent bugs are linked here: https://bugzilla.redhat.com/show_bug.cgi?id=1378242) (libvirt patch series: "[PATCH v2 00/11] qemu: Fix |
|
| 15:30:33 | openstack | bugzilla.redhat.com bug 1378242 in libvirt "QEMU image file locking (libvirt)" [Unspecified,On_qa] - Assigned to pkrempa | |
| 15:31:52 | mriedem | kashyap: ok just reading https://bugzilla.redhat.com/show_bug.cgi?id=1378242#c21 | |
| 15:31:53 | openstack | bugzilla.redhat.com bug 1378242 in libvirt "QEMU image file locking (libvirt)" [Unspecified,On_qa] - Assigned to pkrempa | |
| 15:32:12 | mriedem | kashyap: but yeah, for now the change for multiattach in the libvirt driver only supports multiattach if qemu<2.10 or libvirt>=3.10 | |
| 15:32:23 | kashyap | mriedem: BTW, about that snapshots and paused instances; I already thought it was fixed earlier, maybe I'm hallucinating | |
| 15:32:40 | mriedem | https://review.openstack.org/#/c/267587/83/nova/virt/libvirt/driver.py | |
| 15:32:44 | kashyap | But see the version dependancy you document in the commit message. | |
| 15:33:01 | kashyap | mriedem: Noted (about the multi-attach change) | |
| 15:33:37 | mriedem | as for this test https://review.openstack.org/#/c/267587/83/nova/tests/unit/virt/libvirt/test_driver.py@19520 - i can't sort out the global mock nature that makes it fail when running the entire test module | |
| 15:33:56 | mriedem | https://review.openstack.org/#/c/267587/83/nova/tests/unit/virt/libvirt/test_driver.py@19520 is a challenge for mdbooth | |
| 15:35:21 | mdbooth | mriedem: Have you done the bisect thing? | |
| 15:35:25 | mriedem | mdbooth: nope | |
| 15:35:45 | mdbooth | I believe the instructions were originally written by sdague | |
| 15:35:46 | mriedem | i thought that splitting the tests out into it's own class would help so they weren't running with the other tests in LibvirtConnTestCase but that didn't help | |
| 15:36:49 | mdbooth | mriedem: This one: https://wiki.openstack.org/wiki/Testr#Reproducing_Failures | |
| 15:37:39 | mriedem | oh cool, i'll try that | |
| 15:37:41 | mdbooth | The output of that process is a minimal test ordering which causes the failure. | |
| 15:37:49 | mdbooth | Then you just squint at it for a bit. | |