| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-24 | |||
| 02:31:51 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031 | |
| 02:46:25 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Add a guide to add a new microversion support https://review.opendev.org/667002 | |
| 03:53:31 | openstackgerrit | Merged openstack/nova master: extend libvirt video model support https://review.opendev.org/647733 | |
| 04:11:20 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299 | |
| 04:11:35 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344 | |
| 04:11:53 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673 | |
| 05:32:47 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | |
| 05:33:08 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | |
| 05:33:24 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | |
| 05:33:43 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | |
| 07:50:39 | openstackgerrit | Slawek Kaplonski proposed openstack/nova master: Revert "Revert resize: wait for events according to hybrid plug" https://review.opendev.org/667035 | |
| 08:36:28 | lyarwood | ~. | |
| 08:55:36 | bhagyashris | Hi all, I am trying to live migrate the instance but getting this error: http://paste.openstack.org/show/753291/ on nova compute side can any one have nay idea about the same? | |
| 08:57:12 | kashyap | bhagyashris: My first guess from the error is that you don't have enough free RAM. | |
| 08:57:44 | kashyap | Oh, wait. Thre's also a "length mismatch" error | |
| 09:00:14 | kashyap | bhagyashris: What distro is this, and what are you rversions of libvirt and QEMU? | |
| 09:03:53 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: flatten rbd images when unshelving an instance https://review.opendev.org/457886 | |
| 09:05:48 | bhagyashris | kashyap: Hi, I have pasted all details here : http://paste.openstack.org/show/753292/ please check and let me know. | |
| 09:08:23 | stephenfin | bhagyashris: This looks relevant https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1713490 | |
| 09:08:24 | openstack | Launchpad bug 1713490 in Ubuntu Cloud Archive "error migrating blocked on virtio-net-pci.rom" [Undecided,Confirmed] | |
| 09:08:47 | stephenfin | bhagyashris: As does this https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg01663.html | |
| 09:09:38 | kashyap | stephenfin: You wouldn't believe: that "Length mismatch" is the improved (unactionable) error from QEMU | |
| 09:09:50 | kashyap | s/improved/"improved"/ (missed the scare quotes :D) | |
| 09:10:04 | stephenfin | o rly? | |
| 09:10:18 | kashyap | Yeah, checking with Dave from QEMU, he says: | |
| 09:10:29 | kashyap | < dgilbert> kashyap: So, that's a packaging bug for their virtio rom; they need to pad | |
| 09:11:26 | kashyap | stephenfin: Reference for the "improvement" (it actually _is_, though, relative to the previous one): https://bugzilla.redhat.com/show_bug.cgi?id=1090093 | |
| 09:11:27 | openstack | bugzilla.redhat.com bug 1090093 in libvirt "Improve error reporting when QEMU fails with incoming migration" [Unspecified,Closed: errata] - Assigned to jdenemar | |
| 09:13:12 | kashyap | bhagyashris: So I was told it's a distribution bug, so please file with Ubuntu. (Where the packager needs to pad the ROM with 0 bytes, using `dd`.) thanks, Dave Gilbert for the suggestion. | |
| 09:18:45 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add database schema upgrade check https://review.opendev.org/667047 | |
| 09:32:25 | openstackgerrit | Sam Morrison proposed openstack/nova stable/rocky: Fix double word hacking test https://review.opendev.org/665788 | |
| 09:33:30 | openstackgerrit | Sam Morrison proposed openstack/nova stable/queens: Fix double word hacking test https://review.opendev.org/665790 | |
| 09:39:45 | bhagyashris | kashyap: ok thank you! So for workaround I will set the kvm version same for both the machine and will try . | |
| 09:40:02 | kashyap | bhagyashris: Also see the guidance here: https://wiki.qemu.org/Features/Migration/Troubleshooting#Length_mismatch:_....:_...._in_.21.3D_..... | |
| 09:40:07 | kashyap | It does require work from distros. | |
| 09:43:02 | bhagyashris | kashyap: ok you mean it should install through bistro? | |
| 09:43:09 | bhagyashris | distro* | |
| 09:43:40 | kashyap | bhagyashris: I mean, there is an action item for Ubuntu QEMU packager to "pad the ROM" (it's a low-level thing) | |
| 09:50:48 | lwq | Hi there, I would like to discuss possible improvement of nova resize/migrate process - I mean the SCP part … Anyone with experience with this topic? :-) | |
| 10:20:19 | openstackgerrit | Merged openstack/python-novaclient master: Fix duplicate object description error https://review.opendev.org/666203 | |
| 10:22:31 | bhagyashris | kashyap: ohh ok thank you ! | |
| 10:35:22 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve W503 (line break occurred before a binary operator) https://review.opendev.org/651555 | |
| 10:35:23 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve E741 (ambiguous variable name) https://review.opendev.org/652103 | |
| 10:35:23 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve W605 (invalid escape sequence) https://review.opendev.org/652104 | |
| 10:35:24 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Use consistent URL regex substitution https://review.opendev.org/665949 | |
| 10:35:44 | openstackgerrit | Alex Xu proposed openstack/nova master: Assign the request_spec's network_metadata after create https://review.opendev.org/667061 | |
| 10:36:04 | openstackgerrit | Slawek Kaplonski proposed openstack/nova master: Revert "Revert resize: wait for events according to hybrid plug" https://review.opendev.org/667035 | |
| 10:51:09 | alex_xu | stephenfin: something should be under your radar https://review.opendev.org/667061 | |
| 10:51:22 | stephenfin | alex_xu: Looking at it at the moment :) | |
| 11:45:31 | NewBruce3 | has anyone ever been able to live migrate VMs from CentOS (RDO) with SELinux to Ubuntu (OSA) with AppArmor? | |
| 11:50:04 | NewBruce | i am convinced there is a way to do it, reading into the libvirt docs, but nothing has worked so far | |
| 11:57:52 | kashyap | NewBruce: I don't think (read: "I'm pretty sure") you will find official libvirt docs for it :-) | |
| 11:58:28 | kashyap | If you're trying that, be aware that you might be left holding holding a bag of broken VM parts. | |
| 11:58:47 | kashyap | s/"holding holding"/holding/ | |
| 11:59:25 | kashyap | NewBruce: But ... if you _did_ succeed (or otherwise) might want to write a public blog post, with your exploits | |
| 12:10:06 | NewBruce | well you can ask mnaser how he did it ;) | |
| 12:11:13 | NewBruce | sorry, having a quick chat with some libvirt people who think im doing some horrible sorcery to get migration from centos to ubuntu in the first place and that is courtesy of mnaser but long story short seems… impossible? | |
| 12:13:15 | NewBruce | i expected to see a seclabel=selinux in the xml, but wasnt there but migrations fail…. | |
| 12:13:36 | NewBruce | Live Migration failure: unsupported configuration: Unable to find security driver for model selinux: libvirtError: unsupported configuration: Unable to find security driver for model selinux | |
| 12:19:55 | kashyap | NewBruce: Ah, I see you're on #virt, OFTC. (Haven't read the detail; buried in too many things) | |
| 12:20:53 | NewBruce | yeah, figured someone here might have tried before; but more likely the libvirt peoples would be up on it - seems this is a corner case; though surprising no one has ever moved from RDO to OSA before…. | |
| 12:21:00 | kashyap | Your machine types on source and destination won't match -- one of your main blockers... | |
| 12:21:32 | NewBruce | i beleive that is what mnaser has helped patch for us (wasnt around when he did that) | |
| 12:22:22 | NewBruce | but live migrating from RDO -> OSA definitely works for us; ive done it thousands of times now on another site - its just this setup im with now has SELinux on the RDO side which is a pain on the Ubuntu side as there are horrible bugs on Ubuntu with SELinux | |
| 12:22:54 | NewBruce | like not being able to create new users (which we can work around…. but still) | |
| 12:23:09 | NewBruce | running SELinux on Ubuntu is not a preferred long-term solution | |
| 12:24:04 | kashyap | Right, as I just noted on #virt; you do need some way of recompiling QEMU on Ubuntu, based on CentOS QEMU source (for the machine types) | |
| 12:24:16 | NewBruce | which it appears we have | |
| 12:24:32 | NewBruce | … should we move this over to #virt? :D | |
| 12:33:35 | kashyap | Yeah. | |
| 12:42:21 | mnaser | kashyap, NewBruce: pretty much deconstructed the srpm from redhat and added all the patches to the Deb which included all the machine typed etc to make it possible | |
| 12:46:17 | kashyap | mnaser: Wauw; have to admit, that's one really burdensome source you'll have to maintain | |
| 12:46:28 | NewBruce | according to the folks ok #virsh mnaser this makes you certifiable! :D | |
| 12:47:28 | mnaser | kashyap: I did tell the NewBruce and co that this will survive one single live migration and we reinstall the original binaries right after and it will block live migrations after | |
| 12:48:00 | mnaser | So the goal was just to get the one live migration done, reinstall original qemu and then on any full reboot they get the Ubuntu machine type | |
| 12:49:16 | kashyap | mnaser: If I may ask, is the goal to get off of CentOS altogether? | |
| 13:14:47 | efried_pto | sean-k-mooney: Is https://blueprints.launchpad.net/nova/+spec/libvirt-video-device-models code complete since https://review.opendev.org/#/c/647733/ has merged? | |
| 13:15:37 | efried_pto | sean-k-mooney: I see there's some minor cleanup, but that doesn't look critical. And something on the glance side? | |
| 13:25:13 | openstackgerrit | Merged openstack/os-traits master: add libvirt image metadata traits https://review.opendev.org/652996 | |
| 13:25:16 | openstackgerrit | Merged openstack/os-traits master: add GOP, NONE and VIRTIO grapics models https://review.opendev.org/665830 | |
| 13:33:44 | openstackgerrit | Merged openstack/nova master: Remove redundant group host setup https://review.opendev.org/665954 | |
| 13:33:53 | openstackgerrit | Merged openstack/nova master: [FUP] fix backleveling unit test for video models https://review.opendev.org/666919 | |
| 13:34:03 | openstackgerrit | Merged openstack/nova master: Mention [cinder]/cross_az_attach in the AZ docs https://review.opendev.org/650456 | |
| 13:41:01 | mriedem | sean-k-mooney: so same host revert resize huh, who know neutron had a gate job for ovs hybrid plug... | |
| 13:41:12 | mriedem | we should probably throw that into nova's experimental queue to be run on-demand | |
| 13:41:31 | mriedem | s/should/could/ since that means someone would have to remember to run it | |
| 13:44:00 | sean-k-mooney | ya we should | |
| 13:45:45 | efried_pto | sean-k-mooney: see question above about libvirt-video-device-models? | |
| 13:46:50 | efried_pto | oh | |
| 13:47:18 | sean-k-mooney | sorry give me 5 mins | |
| 13:47:35 | efried | sure, no hurry. Meanwhile, I'll hammer you in -placement as well. | |
| 13:49:56 | mriedem | sean is also working on pushing a patch to try and fix the neutron gate, | |
| 13:50:04 | mriedem | unless we just go with this revert https://review.opendev.org/#/c/667035/2 | |
| 13:50:18 | mriedem | slaweq: looks like ^ makes the job pass now, i checked zuul on your neutron change that depends on the nova revrt | |
| 13:50:21 | mriedem | *Revert | |
| 13:51:26 | efried | that's way more important than my blueprint paperwork | |
| 13:51:41 | mriedem | slaweq: what i'd like to know is do we still get a network-vif-plugged event for ovs hybrid plug ports during same host revert resize at the time of plugging the vif (because apparently we don't when binding the port and the host doesn't change - which is different in behavior from how non-hybrid plug ovs ports work apparently) | |
| 13:51:58 | efried | (</ptl-hat> for pretty much any value of $that) | |
| 13:52:05 | sean-k-mooney | mriedem: actully we had a production outage of our internal cloud and i just spent the last 2 hours helping them fix it | |
| 13:52:32 | mriedem | sean-k-mooney: please oh please say it was because of this ovs hybrid plug patch :) | |