| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-11-06 | |||
| 11:09:07 | gibi | lyarwood: looking... | |
| 11:10:10 | openstackgerrit | Hemanth Nakkina proposed openstack/nova stable/train: Update pci stat pools based on PCI device changes https://review.opendev.org/761725 | |
| 11:16:27 | openstackgerrit | Hemanth Nakkina proposed openstack/nova stable/train: Update pci stat pools based on PCI device changes https://review.opendev.org/761725 | |
| 11:19:34 | gibi | lyarwood: so with the new focal job we can avoid 1901739 on master, but we still hit it on stable/victoria isn't it? | |
| 11:25:23 | openstackgerrit | Hemanth Nakkina proposed openstack/nova stable/stein: Update pci stat pools based on PCI device changes https://review.opendev.org/761727 | |
| 11:48:03 | lyarwood | gibi: yes, unless we backport the job there | |
| 11:48:18 | lyarwood | gibi: that might not be a crazy idea | |
| 11:48:33 | gibi | yeah, for victoria it is OK as we had the focal goal for V | |
| 11:48:46 | gibi | does this bug hits older stable branches too? | |
| 11:49:03 | lyarwood | gibi: I think I've only seen it in master thus far | |
| 11:49:12 | lyarwood | gibi: but we haven't been landing much in stable/victoria | |
| 11:49:27 | lyarwood | gibi: vs master that is | |
| 11:49:47 | openstackgerrit | Hemanth Nakkina proposed openstack/nova stable/stein: Update pci stat pools based on PCI device changes https://review.opendev.org/761727 | |
| 12:01:51 | gibi | lyarwood: ack | |
| 12:03:31 | gibi | I totally support backporting this job changes to V | |
| 12:06:54 | lyarwood | gibi: ack thanks | |
| 12:58:03 | sean-k-mooney | elod: can you take a look at this os-vif backport and the proceeding patch when you have time https://review.opendev.org/#/c/759255/1 | |
| 13:04:06 | gibi | gmann: does this ring a bell to you? https://zuul.opendev.org/t/openstack/build/bf932888821345c1955cda68226b1a87/log/job-output.txt#28429 was there any policy default change in glance or policy config change in devstack recently? | |
| 13:19:12 | elod | sean-k-mooney: sure! (at first glance the 'refactor' topic tricked me, but as far as i see it's a bugfix :)) | |
| 13:30:23 | sean-k-mooney | ya kind of its pulling out some common code but its need for the follow up | |
| 13:57:00 | mnasiadka | good afternoon | |
| 13:57:19 | mnasiadka | does Nova support having ephemeral volumes from different backends? (e.g. one ceph, second from a local filesystem) | |
| 14:22:57 | lyarwood | mnasiadka: for the libvirt virt driver no, we only support a single images_type configurable | |
| 14:23:06 | lyarwood | mnasiadka: for multibackend support you should use cinder | |
| 14:23:27 | mnasiadka | lyarwood: that's what I considered as an alternative | |
| 14:23:30 | mnasiadka | lyarwood: thanks | |
| 14:23:32 | lyarwood | mnasiadka: np | |
| 14:25:53 | sean-k-mooney | stephenfin:.... https://github.com/openstack/nova/commit/b8695de6da56db42b83b9d9d4c330148766644be rewrote the pci generation that is why my fix is broken | |
| 14:26:49 | sean-k-mooney | i guess that needs to go back to queens too | |
| 14:44:18 | bauzas | gibi: others, some bugfix that could be backported https://review.opendev.org/#/c/758470/ | |
| 14:44:25 | bauzas | easy peasy IMHO | |
| 14:45:58 | gibi | bauzas: will check soonish | |
| 14:46:04 | bauzas | no rush | |
| 14:46:06 | bauzas | but ta | |
| 14:47:15 | gibi | it is Friday I run out of rush alreay ;) | |
| 14:49:08 | openstackgerrit | sean mooney proposed openstack/nova master: Fix unplugging VIF when migrate/resize VM https://review.opendev.org/751642 | |
| 14:49:08 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | |
| 14:52:36 | lyarwood | I'm not sure if anyone has been looking at this but I can't see an obvious issue with nova-ceph-multistore aside from instance spawns taking a long time overall | |
| 14:53:03 | lyarwood | I'll look at another example failure after a downstream call and file a bug to track these timeouts | |
| 14:54:13 | gmann | gibi: let me check. we did change one glance policy in nove ceph multistore job. | |
| 14:54:44 | gmann | gibi: ah this is same copy_image policy. checking why it is failing. | |
| 14:55:25 | gibi | gmann: thanks! | |
| 14:56:16 | bauzas | is it only me or you do you get a shit of exceptions when running functional and unittests locally with nova.storage.rbd_utils about the 'rados' and 'rbd' packages missing ? | |
| 14:56:24 | bauzas | or do* | |
| 14:56:27 | openstackgerrit | Ghanshyam Mann proposed openstack/nova stable/victoria: Modify glance's copy_image permission for nova-ceph-multistore https://review.opendev.org/761755 | |
| 14:56:30 | gibi | bauzas: I see it too | |
| 14:56:42 | gibi | and I remember seeing a patch fixing it | |
| 14:56:45 | bauzas | I tested with both py38 and py39 | |
| 14:56:54 | bauzas | but that's mostly annoying | |
| 14:56:55 | gmann | gibi: bauzas lyarwood backporting the nova-ceph-multistore fix https://review.opendev.org/#/c/761755/ | |
| 14:57:13 | gibi | bauzas: https://review.opendev.org/#/c/746873/ | |
| 14:57:15 | gmann | Tempest test merged before we backported this | |
| 14:58:08 | gibi | gmann: thanks! | |
| 14:58:13 | bauzas | gibi: meh, I should have it in my local repo | |
| 14:58:47 | gibi | bauzas: then maybe it is not the fix but it is the patch that generates the exception in the first place | |
| 14:58:58 | lyarwood | gmann: ah did that bork the job on stable? | |
| 14:59:21 | gmann | gibi: lyarwood nova-ceph-multistore job is from victoria so backporting until victoria is needed | |
| 14:59:29 | gmann | lyarwood: yeah - https://zuul.opendev.org/t/openstack/build/bf932888821345c1955cda68226b1a87/ | |
| 14:59:59 | gmann | tempest test which started running in nova-ceph-multistore on master as well on stable/victoria started running and needed this change on nova side | |
| 15:00:16 | bauzas | gibi: hah, that's maybe because the exception is not the same | |
| 15:00:52 | bauzas | ah no | |
| 15:00:54 | bauzas | nevermind | |
| 15:00:58 | bauzas | we log the exception | |
| 15:01:02 | lyarwood | gmann: ack I was going to look at that failure so thanks :) | |
| 15:01:11 | bauzas | hence the stracktrace | |
| 15:01:18 | bauzas | but that's horribly annoying | |
| 15:01:45 | bauzas | lyarwood: any ways for us to not getting a shitty number of exception lines anytime we use tox ? | |
| 15:01:58 | gmann | lyarwood: we added tempest test to copy-image for private images also (https://review.opendev.org/#/c/742546/23) which needed this policy to be open for everyone | |
| 15:02:03 | bauzas | because of rados and rbd missing | |
| 15:02:29 | bauzas | lyarwood: context being https://review.opendev.org/#/c/746873/4/nova/storage/rbd_utils.py | |
| 15:02:59 | lyarwood | gmann: ack | |
| 15:03:37 | lyarwood | bauzas: we could add some logic to skip the check based on an ENV var or something in test envs? | |
| 15:04:29 | lyarwood | bauzas: or we add them as test requirements? | |
| 15:04:36 | lyarwood | bauzas: even though they are virt driver specific | |
| 15:05:06 | bauzas | lyarwood: context is http://paste.openstack.org/show/799789/ | |
| 15:05:19 | lyarwood | bauzas: yeah I;ve seen it | |
| 15:05:32 | bauzas | lyarwood: IMHO we should lazy load the modules | |
| 15:05:42 | bauzas | for rdb and rados | |
| 15:06:03 | bauzas | or making them needed for all *nova* | |
| 15:06:11 | bauzas | but, definitely, the former | |
| 15:06:19 | bauzas | if they are optional | |
| 15:06:29 | bauzas | (and only for a specific driver) | |
| 15:06:39 | lyarwood | bauzas: we lazy loaded before silently and it was a PITA to debug issues when they failed to import | |
| 15:06:46 | lyarwood | bauzas: assuming that's what you mean | |
| 15:07:07 | lyarwood | bauzas: could we not just add them as test deps? | |
| 15:07:16 | bauzas | lyarwood: then, lazy load them by creating the libvirt object | |
| 15:07:26 | bauzas | when initting | |
| 15:07:43 | bauzas | gibi: ^ to be honestly, I'm against https://review.opendev.org/#/c/746873/4/nova/storage/rbd_utils.py | |
| 15:08:07 | bauzas | because we create more problems for folks that don't use ceph than we fix the needed ones | |
| 15:09:07 | openstackgerrit | Merged openstack/nova master: zuul: Merge nova-evacuate into nova-multinode-live-migration https://review.opendev.org/755309 | |
| 15:09:18 | dansmith | bauzas: me too, that seems crazy | |
| 15:10:36 | dansmith | surely we could check images_type and decide whether or not to explode there? | |
| 15:10:37 | gibi | bauzas, lyarwood, dansmith: can we move these import into one of the classes in the rbd_utils? | |
| 15:11:03 | gibi | dansmith: it is not exploding the driver, it is logging the import error | |
| 15:11:10 | dansmith | gibi: I think the point here was to make it explode before those get loaded, but that's not really okay, IMHO | |
| 15:11:13 | bauzas | gibi: the problem is that when using import directly, you *need* rdb | |
| 15:11:22 | bauzas | by any module | |
| 15:11:34 | bauzas | hence either lazy-loading it | |
| 15:11:55 | bauzas | or having other ways to support it like we did by telling none | |
| 15:12:06 | dansmith | yeah, go back to the old way and make something else check to see if the rbd import is None during early init, if we're in a dependent code path | |