| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-11-04 | |||
| 09:43:22 | brinzhang_ | jorhson: agree with lyarwood, add a microversion to remove the device, and in order to avoid the conflict, we also need to add the task_state as current description in your spec | |
| 09:43:47 | lyarwood | well we have the instance.uuid lock to avoid conflicts | |
| 09:44:03 | lyarwood | most compute operations should be taking that IMHO | |
| 09:44:15 | lyarwood | I know it's not the case for everything at the moment | |
| 09:46:33 | lyarwood | so what if we didn't reject the request if the task_state was attaching and queued it in the API somehow? | |
| 09:46:39 | lyarwood | Jorhson / brinzhang_ ; ^ | |
| 09:47:46 | Jorhson | lyarwood: yeah, for the users, there are less probability to attach multiple volumes, and i agree with your sheme, but i think it's important to solve the option's conflict | |
| 09:48:18 | lyarwood | Jorhson: it's a pretty common usecase with k8s running on OpenStack IMHO | |
| 09:54:38 | brinzhang_ | lyarwood: ok, I understand what you mean, agree | |
| 09:55:28 | brinzhang_ | lyarwood: Jorhson will update this spec later, this way it's better for k8s on OpenStack scenario | |
| 09:59:36 | lyarwood | brinzhang_: ack many thanks both, I'm going to be AFK again this afternoon UK time but I'll review the spec this evening once I'm back. | |
| 10:00:34 | brinzhang_ | lyarwood: ack, thanks too | |
| 10:01:52 | aarents | Hi nova | |
| 10:02:22 | openstack | Launchpad bug 1832248 in OpenStack Compute (nova) "tempest.api.volume.test_volumes_extend.VolumesExtendAttachedTest.test_extend_attached_volume failing when using the Q35 machine type" [Medium,Triaged] - Assigned to Alexandre arents (aarents) | |
| 10:02:22 | aarents | lyarwood: there is some progress on that q35 bug: https://bugs.launchpad.net/nova/+bug/1832248 , if it is a qemu bug, what is the suitable status for this bug Fixed_in_relsease or invalid? | |
| 10:07:26 | lyarwood | aarents: awesome sleuthing! :) | |
| 10:08:04 | lyarwood | aarents: erm, invalid I guess? gibi ^ any thoughts, looking to close a Nova bug that was actually a QEMU bug that has already been fixed | |
| 10:09:11 | gibi | lyarwood: if it is fixed in QEMU and no need for any nova change then Invliad it is | |
| 10:09:20 | gibi | Invalid even | |
| 10:09:25 | lyarwood | cool thanks | |
| 10:15:38 | aarents | cool, I just hope it will fix also my downstrem bug regarding interface attach! | |
| 11:56:55 | sean-k-mooney | do we need to enable the new qemu from the cloud arcive on ussuri and below to ensure we dont hit that bug | |
| 12:04:23 | openstackgerrit | Wenping Song proposed openstack/nova-specs master: Support vGPU management by Cyborg https://review.opendev.org/750116 | |
| 13:04:53 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Prevent starting services with older than N-1 computes https://review.opendev.org/738482 | |
| 13:04:54 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add upgrade check about old computes https://review.opendev.org/760520 | |
| 13:19:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Prevent starting services with older than N-1 computes https://review.opendev.org/738482 | |
| 13:21:15 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add upgrade check about old computes https://review.opendev.org/760520 | |
| 13:23:41 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use _evacuate_server helper in func test https://review.opendev.org/758210 | |
| 13:27:17 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ignore PCI devices with 32bit domain https://review.opendev.org/756697 | |
| 13:27:17 | openstack | bug 1897528 in OpenStack Compute (nova) "32bit pci domain number is not supported" [High,In progress] https://launchpad.net/bugs/1897528 - Assigned to Balazs Gibizer (balazs-gibizer) | |
| 13:27:17 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1897528 https://review.opendev.org/756696 | |
| 13:36:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Support interface attach with qos ports https://review.opendev.org/755477 | |
| 14:03:43 | openstack | bug 1896463 in OpenStack Compute (nova) rocky "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 | |
| 14:03:43 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1896463 in func env https://review.opendev.org/754100 | |
| 14:05:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Set instance host and drop migration under lock https://review.opendev.org/754815 | |
| 14:07:17 | openstack | bug 1896463 in OpenStack Compute (nova) rocky "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 | |
| 14:07:17 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1896463 in func env https://review.opendev.org/754100 | |
| 14:08:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Set instance host and drop migration under lock https://review.opendev.org/754815 | |
| 14:51:12 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add upgrade check about old computes https://review.opendev.org/760520 | |
| 15:02:45 | openstackgerrit | Takashi Natsume proposed openstack/nova stable/victoria: Fix a hacking test https://review.opendev.org/758112 | |
| 15:05:44 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: [doc]: Fix glance image_metadata link https://review.opendev.org/761423 | |
| 15:07:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Use cell targeted context to query BDMs for metadata https://review.opendev.org/761424 | |
| 15:29:19 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Bump the lowest eventlet version to 0.26.1 https://review.opendev.org/761427 | |
| 15:55:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: [trivial]: replace NUMNA with NUMA https://review.opendev.org/761436 | |
| 16:19:43 | openstackgerrit | Merged openstack/nova-specs master: [trivial]: replace NUMNA with NUMA https://review.opendev.org/761436 | |
| 16:26:56 | openstack | Launchpad bug 1902276 in OpenStack Compute (nova) "libvirtd going into a tight loop causing instances to not transition to ACTIVE" [Undecided,New] | |
| 16:26:56 | gibi | somebody with connection to the libvirt maintainer should look at this nova bug https://bugs.launchpad.net/nova/+bug/1902276 | |
| 16:28:04 | melwitt | kashyap: ^ | |
| 16:28:26 | kashyap | melwitt: Yeah, familar with it, as I worked with the reporter here the other day | |
| 16:28:39 | melwitt | ah ok, cool | |
| 16:28:43 | kashyap | melwitt: I asked one of the libvirt devs on Friday, but my timing wasn't right | |
| 16:28:52 | gibi | kashyap: thanks! | |
| 16:28:57 | kashyap | I'll check again | |
| 16:29:18 | kashyap | gibi: It looks fishy, as it's not 100% reproducible ... as the reporter says "a few minutes later things go back to normal" | |
| 16:29:26 | kashyap | But we don't know what changed :-( | |
| 16:31:47 | gibi | kashyap: the nova image download take ~ 300 seconds for the VM that then triggers the loop in libvirtd so it might be that the hypervisor host has high load | |
| 16:31:57 | kashyap | gibi: Yeah, just reading your report :) | |
| 16:31:58 | gibi | but I was not able to confirm it from the logs | |
| 16:33:13 | kashyap | I see. Hypervisor load sounds plausible - as we've hit load-related (CI) issues libvirt driver before. But still let me check w/ Dan or someone from upstream libvirt | |
| 16:34:25 | gibi | kashyap: thanks for taking this up with the libvirt maintainers | |
| 16:35:34 | kashyap | gibi: Just posted on #virt, OFTC network. | |
| 16:36:18 | kashyap | gibi: Is this blocking patch merges? | |
| 16:39:05 | melwitt | I've been struggling for a couple of days trying to get an approved patch through the gate, but I'm not sure whether that particular bug is involved. I would need to re-look at the logs to verify | |
| 16:39:25 | kashyap | (I've reposted the looping libvirtd log bits as a plain text, as the pastebins expire) | |
| 16:40:58 | kashyap | melwitt: Noted; Michael, the reporter, was saying on last Friday that it's "intermittent", which makes it a bit more difficult to debug | |
| 16:41:56 | melwitt | yeah, that's been the theme of all of the gate bugs I'm aware of. intermittent and thus hard to troubleshoot :( | |
| 16:42:06 | melwitt | *current gate bugs I'm aware of | |
| 16:43:01 | kashyap | Yeah, matches my past experience | |
| 16:43:53 | kashyap | melwitt: In the same vein as how Twitter (I'm not on it) seems to label Trump's tweets as misleading, wonder we should adapt that text for these intermittent bugs :D | |
| 16:44:26 | kashyap | - Some or all of the content shared in this Tweet is disputed and might be misleading about an election or other civic process. | |
| 16:44:29 | kashyap | + Some or all of the content shared in this bug is disputed and might be misleading due to intermittent failures. | |
| 16:46:02 | kashyap | gibi: melwitt: More seriously, can I "subscribe" (Cc) someone else to a LaunchPad, right? | |
| 16:46:21 | kashyap | IIRC, yes. /me tries | |
| 16:46:26 | melwitt | I think you can | |
| 16:48:11 | kashyap | melwitt: I can't :-( I wanted to Cc Michal from libvirt but it says "No items matched <email ID>" | |
| 16:48:44 | melwitt | do you know his launchpad id? | |
| 16:48:55 | kashyap | melwitt: Oh, having a Launchpad ID is mandatory? | |
| 16:49:19 | melwitt | it might be, that's the only way I've seen subscribing | |
| 16:50:44 | kashyap | Ah, noted. I don't think he has one - searching doesn't show up anything. | |
| 16:51:41 | kashyap | melwitt: I pointed to him on IRC; he's taking a look | |
| 16:53:00 | melwitt | thanks! | |
| 16:55:23 | kashyap | melwitt: gibi: That's quick -- Michal (Privoznik) says it looks like a genuine bug. I'll update the bug once we get more details | |
| 16:56:35 | melwitt | sounds great, thank you kashyap | |
| 17:06:24 | kashyap | melwitt: So, the libvirt version in the logs above is 5.4.0; but havne't we switche dalready to libvirt-6.0.0? | |
| 17:06:39 | kashyap | lyarwood: --^ (By "we", I mean upstream CI) | |
| 17:08:11 | kashyap | So Michal says, there were improvements in libvirt-6.1.0 release on this area of event loops. | |
| 17:13:52 | stephenfin | lyarwood: Could you cast an eye over https://review.opendev.org/#/c/631053/ this evening, please? | |
| 17:14:04 | stephenfin | It's been around for quite a while :-D | |
| 17:18:01 | mloza | hello, is it possible to update the video model to vmga of an existing instance? | |
| 17:19:11 | mloza | if i edit /etc/libvirt/qemu/instance-, it reverts to default when the instance is hard rebooted | |
| 17:20:04 | stephenfin | mloza: Outside of rebuilding to a new image, no. We don't support setting it via the flavor so resize isn't an option | |
| 17:21:47 | stephenfin | mloza: You'll have to modify the DB manually if you want to avoid the rebuild | |
| 17:22:18 | mloza | can you tell me which table do I need modify | |
| 17:22:23 | mloza | to modif* | |
| 17:24:03 | stephenfin | iirc, we persist image metadata properties for an image in the instance_system_metadata table | |
| 17:25:16 | stephenfin | in case it wasn't obvious, back up the DB first and note that any support guarantees are gone out the window if you modify the DB manually | |
| 17:37:27 | sean-k-mooney | i think the table name is system_metadata not instance_system metadata but yes we do | |
| 17:37:34 | sean-k-mooney | with an img_ prefix | |
| 17:38:04 | sean-k-mooney | os if it was hw_video_model it woudl be img_hw_video_model in the db | |
| 17:38:48 | sean-k-mooney | mloza: were you asking about this on the mailing list too? we basically said the same in our replies | |