| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-01-25 | |||
| 13:21:00 | mdbooth | Ok, yeah, I can see that would be annoying if all the tools default to 80. | |
| 13:21:17 | lyarwood | stephenfin: ack will do | |
| 13:39:34 | stephenfin | TIL the 79 character default is in flake8, not tox.ini/setup.cfg | |
| 13:40:10 | stephenfin | sean-k-mooney: Fancy looking at https://review.openstack.org/#/c/631053/ and predecessors again today/early next week? | |
| 13:40:54 | jangutter | And 79 is actually codified in pep8 itself https://www.python.org/dev/peps/pep-0008/#maximum-line-length | |
| 13:40:56 | sean-k-mooney | sure i was just looking at your chages for requiring the plugin atribute for os-vif and the noop plugin | |
| 13:42:00 | sean-k-mooney | jangutter: ya they choose 79 so that if you have emacs in whitespace-newline-mode where it add a $ to mark the end of the line it wont wrap | |
| 13:42:16 | jangutter | think if we type 79 enough, 80 is going to look weird to sean-k-mooney? | |
| 13:43:00 | sean-k-mooney | jangutter: but all modern tools use 80 as teh defulat so its annoying | |
| 13:52:23 | lyarwood | stephenfin: https://review.openstack.org/#/c/633197/1/nova/objects/numa.py - stupid o.vo question in there if you have a second, I assume it's fine just wanted to check. | |
| 13:53:38 | sean-k-mooney | lyarwood: it should not no | |
| 13:54:10 | sean-k-mooney | lyarwood: the function signitru is part of the class deffinion not the serialised object | |
| 13:55:03 | stephenfin | lyarwood: Yeah, what Sean says. The scheduler, for example, will use its own version of the object | |
| 13:55:06 | lyarwood | sean-k-mooney: right, wasn't sure so it was worth asking. Thanks. | |
| 13:55:09 | stephenfin | *definition | |
| 13:55:27 | sean-k-mooney | lyarwood: its worth asking but no we are fine | |
| 13:58:48 | sean-k-mooney | this would matter for signiture of rpc methods where we are invokeing a remote method but not in this case | |
| 13:59:01 | stephenfin | lyarwood: I want tooling to automatically add (and remove, if abandoned) patches submitted to internal Gerrit to their BZ | |
| 13:59:15 | stephenfin | ...and reject the patch if there is no BZ linked | |
| 13:59:38 | sean-k-mooney | stephenfin: sound like a nice proejct. to discuss downstream ... | |
| 13:59:45 | stephenfin | Goddddd damn it | |
| 14:00:24 | stephenfin | Good thing I don't work in finance | |
| 14:00:28 | sean-k-mooney | i mean if someone upstream whats to write that for use tye are wellcome. im sure jangutter has plenty of time to do it:) | |
| 14:03:28 | jangutter | sean-k-mooney: if someone writes that, EVERYONE's going to pile on stuff to the weirdest BZ's. | |
| 14:04:44 | jangutter | stephenfin, sean-k-mooney: the OpenContrail project has something like that. The unintended consequence is that BZ quality goes down :-( | |
| 14:05:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change InstanceFaultRollback handling in _error_out_instance_on_exception https://review.openstack.org/633212 | |
| 14:05:28 | mriedem | mdbooth: i think you're familiar with _error_out_instance_on_exception - ^ is something that's always bugged me | |
| 14:05:44 | openstackgerrit | Matthew Booth proposed openstack/nova master: WIP: Don't age swap backing files in the image cache by default https://review.openstack.org/633213 | |
| 14:06:02 | openstackgerrit | Merged openstack/nova master: Kill @safe_connect in _get_provider_traits https://review.openstack.org/613613 | |
| 14:06:05 | mdbooth | mriedem: Looking, but ^^^ while you're here (discussion in bug: thanks for that) | |
| 14:07:28 | mriedem | i didn't really understand what swap disks have to do with the image cache manager | |
| 14:07:37 | mdbooth | mriedem: Right. | |
| 14:07:52 | mriedem | it's blank right? so why does the *image* cache manager worry about it? | |
| 14:08:04 | mdbooth | mriedem: It's not blank, it's formatted. | |
| 14:08:20 | mriedem | i mean there is no glance image in swap disks yeah? | |
| 14:08:20 | mdbooth | So it's basically a cache of the output of mkswap. | |
| 14:08:31 | mdbooth | No, there's no glance image. | |
| 14:09:01 | mdbooth | It saves a completely inconsequential amount of time during creation, and is likely a performance hit at runtime. | |
| 14:09:07 | mdbooth | And will save essentially no disk space. | |
| 14:09:45 | mdbooth | So we really shouldn't do it all, but cleaning that up would be a significant task. We'd need an upgrade task to rebase disks in use which already use this. | |
| 14:10:24 | mdbooth | However, aging these cached formatted swap files is really unlikely to be of much use to anybody. | |
| 14:10:40 | mdbooth | This user has found it's really expensive, so my suggestion is just don't age them. | |
| 14:11:37 | mdbooth | mriedem: Anyway, gimme a shout if you think ^^^ is an approach you could get behind and I'll fix that patch up properly. | |
| 14:13:09 | mriedem | i just don't really understand the use case for why these are in the image cache | |
| 14:13:22 | mriedem | granted, i'm not super familiar with the image cache manager to begin with | |
| 14:13:48 | mriedem | beyond it's used to cache and (on a timer) age/invalidate images in the cache so new versions can be fetched | |
| 14:14:06 | mriedem | but i thought that was only for root disks | |
| 14:14:14 | mdbooth | mriedem: I don't know the origin, so I can only guess. However ephemeral filesystem are also in there, and there are certain filesystems which *are* expensive to initialise. | |
| 14:14:24 | mdbooth | My guess is swap was simply handled the same way. | |
| 14:14:55 | mdbooth | Although to be clear, I would remove ephemeral filesystems too. We're already not aging them. | |
| 14:17:13 | mriedem | i don't see where ephemerals are in here | |
| 14:18:00 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fix ComputeNode ovo compatibility code https://review.openstack.org/631199 | |
| 14:18:23 | mdbooth | ext3 vs ext4 fs creation comparison: http://paste.openstack.org/show/743462/ | |
| 14:19:05 | mdbooth | mriedem: I had to check closely, but I'm confident we don't age ephemeral backing files. | |
| 14:19:19 | mdbooth | reasonably confident :) That code is very hard to read. | |
| 14:20:10 | mdbooth | I believe ext3 was the default, might still be, and ext3 fs creation can be slow. | |
| 14:20:33 | mriedem | yeah i remember a ML thread from robert collins several years ago about changing the default to ext4 | |
| 14:20:41 | mdbooth | I suspect that's how this feature came about. But note that we still have to do this for other imagebackends anyway, and ext4 fs creation is really fast anyway. | |
| 14:22:27 | mriedem | looks like ext4 is the default now | |
| 14:22:30 | openstackgerrit | Merged openstack/nova master: Add missing ws seperator between words https://review.openstack.org/632606 | |
| 14:22:58 | mriedem | https://github.com/openstack/nova/blob/master/nova/privsep/fs.py#L258 | |
| 14:26:33 | kashyap | mdbooth: (Our of curiosity, I ran your same test on an NVMe: http://paste.openstack.org/show/743468/) | |
| 14:27:16 | mdbooth | kashyap: That's consistent, although I doubt either of our tests were actually hitting disk. | |
| 14:27:27 | mdbooth | kashyap: Likely due to RAM/CPU speed differences. | |
| 14:28:17 | kashyap | Oh, indeed. I always forget that point | |
| 14:28:42 | kashyap | (I recall that `fio` is the better way to do benchmarks) | |
| 14:29:03 | kashyap | (Reference: http://blog.vmsplice.net/2017/11/common-disk-benchmarking-mistakes.html) | |
| 14:33:23 | kashyap | mriedem: Completely unrelated, posted the initial investigation from last night here: https://bugs.launchpad.net/nova/+bug/1813192/comments/1 | |
| 14:33:25 | openstack | Launchpad bug 1813192 in OpenStack Compute (nova) "libvirt: instance delete fails with "Cannot destroy instance, operation time out: libvirt.libvirtError: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainBlockJobAbort)" in bionic nodes (libvirt 4.0.0, qemu 2.11)" [Medium,Confirmed] | |
| 14:33:44 | kashyap | I did some more digging this morning for an hour, will write it up today. | |
| 14:34:37 | kashyap | The unsolved problem for me here is: how to get tracebacks ("thread all apply bt") from the hung QEMU (or libvirtd) from the CI hosts. | |
| 14:35:13 | kashyap | The only current awful way for that is to abuse 'rootwrap' to elevate privileges and then upload a debug build of libvirt/QEMU. | |
| 14:35:30 | kashyap | Then you can reproduce the problem, because "Heisenbug". | |
| 14:38:59 | mriedem | :/ | |
| 14:39:22 | mriedem | it's only 3 hits in 7 days in the gate so it's not burning us down like some of the other known bugs in the gate | |
| 14:44:15 | kashyap | Yeah, it's not. But I'd like to track down these gnarly bugs. Because if you recall, we had the same problem with the fated "live snapshot" bug | |
| 14:44:23 | kashyap | ... that made us reap the whirlind. | |
| 14:45:02 | kashyap | (Typo earlier: "Then you can" --> "Then you can't") | |
| 14:45:26 | kashyap | mriedem: Mind pointing me to the "known bugs" in the gate? | |
| 14:46:09 | mriedem | kashyap: http://status.openstack.org/elastic-recheck/ | |
| 14:46:17 | mriedem | http://status.openstack.org/elastic-recheck/#1813198 | |
| 14:46:18 | kashyap | Yep, remembered the URL | |
| 14:46:28 | mriedem | http://status.openstack.org/elastic-recheck/#1813147 | |
| 14:46:35 | mriedem | those are 2 big new ones | |
| 14:47:10 | kashyap | Thanks | |
| 14:53:42 | openstackgerrit | Ralf Haferkamp proposed openstack/nova master: compute: reset instance events on reload https://review.openstack.org/633220 | |
| 14:56:40 | mriedem | melwitt: do you expect to start on https://blueprints.launchpad.net/nova/+spec/count-quota-usage-from-placement soon? given you're out next week and we've got 6 weeks left i'm a bit worried about that one | |
| 15:02:42 | melwitt | mriedem: yeah, I hope to have something WIP uploaded by late today | |
| 15:30:34 | openstackgerrit | sean mooney proposed openstack/nova master: Libvirt: do not set mac when unplugging macvtap vf https://review.openstack.org/624842 | |
| 15:30:35 | openstackgerrit | sean mooney proposed openstack/nova master: Allow per-port modification of vnic_type and profile https://review.openstack.org/607365 | |
| 15:30:35 | openstackgerrit | sean mooney proposed openstack/nova master: Add free for claimed, allocated devices https://review.openstack.org/616120 | |
| 15:30:36 | openstackgerrit | sean mooney proposed openstack/nova master: SR-IOV Live migration indirect port support https://review.openstack.org/620115 | |
| 15:30:36 | openstackgerrit | sean mooney proposed openstack/nova master: Add get_instance_pci_request_from_vif https://review.openstack.org/619929 | |
| 15:30:37 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: auto detach/attach sriov ports on migration https://review.openstack.org/629589 | |
| 15:35:37 | openstackgerrit | Jan Gutter proposed openstack/nova master: Add support for vrouter HW datapath offloads https://review.openstack.org/572082 | |
| 15:35:37 | openstackgerrit | Jan Gutter proposed openstack/nova master: Convert vrouter legacy plugging to os-vif https://review.openstack.org/571325 | |
| 15:58:32 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Raise InstanceFaultRollback for UnableToMigrateToSelf from _prep_resize https://review.openstack.org/633227 | |
| 16:00:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change InstanceFaultRollback handling in _error_out_instance_on_exception https://review.openstack.org/633212 | |
| 16:00:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Raise InstanceFaultRollback for UnableToMigrateToSelf from _prep_resize https://review.openstack.org/633227 | |
| 16:04:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change InstanceFaultRollback handling in _error_out_instance_on_exception https://review.openstack.org/633212 | |