| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-16 | |||
| 15:32:51 | kashyap | mriedem: Was looking up the scroll this morning; about increasing timeouts due to QEMU / libvirt updates | |
| 15:32:57 | kashyap | https://review.openstack.org/#/c/547676/ | |
| 15:33:18 | kashyap | Even clarkb (not on this channel now) also reported at PTG having to increase tests' timeouts | |
| 15:33:54 | kashyap | I think it must probably be the QEMU / libvirt slowdown after the Meltdown / Spectre fixes. /me goes to check w/ the libvirt folks | |
| 15:36:39 | openstackgerrit | Surya Seetharaman proposed openstack/nova stable/pike: Unmap compute nodes when deleting host mappings in delete cell operation https://review.openstack.org/553829 | |
| 15:37:46 | openstackgerrit | Surya Seetharaman proposed openstack/nova stable/pike: Unmap compute nodes when deleting host mappings in delete cell operation https://review.openstack.org/553829 | |
| 15:41:27 | mriedem | kashyap: yes mnaser already confirmed that | |
| 15:41:51 | kashyap | mriedem: I struggle to believe the libvirtd dameon _itself_ taking 1 _whole_ minute! | |
| 15:42:00 | kashyap | mriedem: I'm checking w/ the Red Hat perf team to see if they have any numbers on this | |
| 15:42:02 | mriedem | libvirt/qemu in the ocata UCA got updates and has to great a guest on first start to scan cpu features, which is not something newer libvirt in the pike UCA has to do | |
| 15:42:13 | mriedem | *create | |
| 15:42:22 | mriedem | https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/ocata-staging/+sourcepub/8774271/+listing-archive-extra | |
| 15:42:24 | kashyap | I know, they ship with new CPU models, all ending with "*-IBRS" | |
| 15:42:45 | openstackgerrit | Surya Seetharaman proposed openstack/nova stable/pike: Unmap compute nodes when deleting host mappings in delete cell operation https://review.openstack.org/553829 | |
| 15:42:59 | kashyap | (Yep, very familiar w/ those updates for a couple of months already. As I had to apply to a bunch of Fedora / RHEL machines) | |
| 15:45:02 | kashyap | This additional slow down even bolsters the case for the couple of Cloud Providers to enable nested virt in the Gate. | |
| 15:45:23 | kashyap | Upstream KVM maintainer said he's going to enable it for Intel by default in the upstream kernel. | |
| 15:46:19 | kashyap | ClarkB, myself and a bunch of others participated in a discussion with the Cloud Providers at PTG, some cloud providers (OVH) seemed amenable to that. | |
| 15:46:24 | kashyap | Let's see... | |
| 15:47:42 | fried_rice | figleaf: Reviewed. Nothing blocking :) | |
| 15:48:17 | fried_rice | figleaf: Are you still -1 on https://review.openstack.org/#/c/553476/ for the name of that article, having read my excellent excuses? | |
| 15:48:54 | mriedem | volume test issues in the gate should be flushed once https://review.openstack.org/#/c/553794/ merges | |
| 15:49:21 | jgwentworth | woot, thanks | |
| 15:49:25 | openstackgerrit | Dan Smith proposed openstack/nova master: Add placeholder migrations for Queens backports https://review.openstack.org/553831 | |
| 15:49:27 | superdan | mriedem: johnthetubaguy ^ | |
| 15:49:56 | mriedem | superdan: aren't there some tests that you need to modify for the placeholder range? | |
| 15:50:07 | superdan | mriedem: um, no, jeez | |
| 15:50:08 | superdan | (yes) | |
| 15:51:03 | cfriesen | kashyap: for what it's worth, I haven't had good experiences with stability of nested KVM with CentOS 7.3. | |
| 15:51:21 | figleaf | fried_rice: yes, 'upt' is much too cryptic | |
| 15:51:23 | kashyap | cfriesen: Got a link to the bugs you've filed? :-) | |
| 15:51:41 | kashyap | cfriesen: More seriously, some Cloud Providers in the room themselves said it was stable in their environments | |
| 15:51:52 | fried_rice | figleaf: Having noticed your # noqa hack, I guess I can make it work with a longer name, eh? | |
| 15:51:58 | cfriesen | kashyap: heh...nothing useful to report, just random intermittent crashes of the first-level guest. | |
| 15:52:01 | kashyap | cfriesen: The cardinal rule here is to try with latest CentOS (stable 7.4) | |
| 15:52:20 | kashyap | cfriesen: I used to track all the bugs I reported for nested virt upstream, a year or so ago: https://kashyapc.fedorapeople.org/kernel-kvm-bugs.txt | |
| 15:52:26 | kashyap | All of them fixed now | |
| 15:52:41 | cfriesen | cool, will have to give it a try with 7.4 | |
| 15:53:08 | kashyap | cfriesen: But, one of the blocking points is the migration of the L1 guest (with the mere presence of L2 in it; it doesn't even have to run). And upstream is working on it as we speak | |
| 15:53:11 | fried_rice | mriedem: What am I tracking for these volume failures to be resolved before I clog the gate with more rechecks? | |
| 15:53:18 | kashyap | cfriesen: s/one of the/the only/ | |
| 15:53:37 | cfriesen | kashyap: live migration I assume? | |
| 15:53:53 | fried_rice | mriedem: https://review.openstack.org/#/c/553794/ ? | |
| 15:53:59 | kashyap | cfriesen: Yep, live migrating L1 (while L2 is in it; online or offline) | |
| 15:54:32 | figleaf | fried_rice: you don't have to make it *that* long! | |
| 15:55:03 | fried_rice | figleaf: update-provider-tree will push me over 80. Anything else doesn't really make sense. upd-prov-tree? Ew. | |
| 15:55:33 | mriedem | fried_rice: yes, as noted above | |
| 15:55:39 | mriedem | "(10:48:59 AM) mriedem: volume test issues in the gate should be flushed once https://review.openstack.org/#/c/553794/ merges" | |
| 15:55:44 | fried_rice | mriedem: oh, see it now, thx | |
| 15:56:46 | openstackgerrit | Dan Smith proposed openstack/nova master: Add placeholder migrations for Queens backports https://review.openstack.org/553831 | |
| 15:57:36 | fried_rice | figleaf: Looks like your gabbi has some failures? | |
| 15:58:08 | fried_rice | figleaf: Oh, because now you have to send a dict-with-generation to set aggregates :) | |
| 15:58:43 | figleaf | fried_rice: saw that. Working on it | |
| 16:00:17 | fried_rice | figleaf: Added helpful link | |
| 16:00:30 | fried_rice | (not that you needed it) | |
| 16:06:45 | openstackgerrit | Eric Fried proposed openstack/nova master: update_provider_tree devref and docstring updates https://review.openstack.org/553476 | |
| 16:06:56 | fried_rice | figleaf: Done ^ | |
| 16:10:44 | superdan | fried_rice: okay, so I get that you're hitting your peak on review efficiency and finding issues, | |
| 16:10:56 | superdan | fried_rice: but that should fall short of finding issues in things I need to get merged | |
| 16:11:18 | fried_rice | superdan: Send me that list. See where it gets ya. | |
| 16:11:24 | superdan | heh | |
| 16:11:40 | fried_rice | superdan: Are you talking about the member_of thing? | |
| 16:11:44 | superdan | yeah :) | |
| 16:11:48 | fried_rice | zuul had me beat on that one. | |
| 16:12:00 | fried_rice | I'll have to try harder. | |
| 16:12:07 | superdan | that was already approved, but failed the gate, now -1d | |
| 16:12:17 | superdan | I'm not serious, of course, I'm glad you found stuff | |
| 16:12:32 | fried_rice | yeah, the rebase needed to take into account the functionality of the thing it got rebased against. | |
| 16:12:46 | fried_rice | TBH, I shoulda seen that on my first pass. | |
| 16:12:48 | superdan | is this the microversion you two were racing for? | |
| 16:13:47 | superdan | stackalytics should discount reviews where you're clearly taking shots at someone else to win a version race :P | |
| 16:15:18 | cdent | reviewers should discount stackalytics | |
| 16:15:46 | superdan | s/reviewers/everyone/ | |
| 16:16:45 | cdent | superdan: I was trying to (quickly) come up with a regexp on your statement but I got lazy and instead printed the output. (meaning, yes, I agree) | |
| 16:17:44 | superdan | heh | |
| 16:18:37 | figleaf | superdan: it was losing the microversion race that made the member_of patch fail. For once, can't blame fried_rice | |
| 16:18:48 | superdan | figleaf: ah | |
| 16:19:41 | superdan | 1.21 is a better microversion anyway | |
| 16:19:49 | fried_rice | It can drink | |
| 16:20:03 | fried_rice | (wheat, unfiltered IIRC) | |
| 16:32:28 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add 'member_of' param to GET /allocation_candidates https://review.openstack.org/552098 | |
| 16:32:36 | figleaf | leakypipes: superdan: fried_rice: ^^ func tests fixed | |
| 16:33:04 | figleaf | Had to resort to the gabbi HISTORY stuff, since I couldn't assume generation=0 | |
| 16:33:22 | figleaf | Besides, we're not supposed to assume that, right? :) | |
| 16:48:33 | fried_rice | figleaf: Ideally you should get it from the RP creation. I did that in a couple of places in my other patch. It's a PITA in gabbi. (Not such a problem in real code.) | |
| 16:48:51 | fried_rice | FWIW, I wouldn't -1 hardcoded generation numbers in a gabbi test. | |
| 16:49:03 | fried_rice | and would even write them myself :) | |
| 16:51:11 | cdent | fried_rice: it's not a pita in gabbi, just do what figleaf has done | |
| 16:52:12 | superdan | lyaaaaaaaaaaaaar: jgwentworth: looks like those reverts are failing some tests.. on two of the three patches at least | |
| 16:52:13 | fried_rice | cdent: Sorry, I just meant you have to use a pretty long string to get there. | |
| 16:52:21 | cdent | ow, my fingers | |
| 16:52:46 | cdent | is more explicit than e.g.. 'upt' | |
| 16:53:18 | superdan | cdent: was that an idiocracy reference? | |
| 16:53:23 | superdan | if so, kudos | |
| 16:53:24 | lyaaaaaaaaaaaaar | superdan: yup looking | |
| 16:53:44 | cdent | superdan: sadly no, entirely unintentional | |
| 16:54:23 | superdan | cdent: ah then no kudos for you | |
| 16:54:32 | cdent | /o\ | |
| 16:54:39 | superdan | cdent: https://www.youtube.com/watch?v=r_4jrMwvZ2A | |
| 16:56:25 | superdan | kashyap: speaking of all that, any ETA on finishing that cpu features patch? | |