| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-05 | |||
| 17:37:36 | gmann | sean-k-mooney1: ^^ | |
| 17:37:43 | gmann | smcginnis: ignore it | |
| 18:42:29 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Migrate default policy file from JSON to YAML https://review.opendev.org/748059 | |
| 22:35:17 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Run voting jobs in gate pipeline also https://review.opendev.org/750078 | |
| #openstack-nova - 2020-09-06 | |||
| 07:39:16 | openstackgerrit | Mamduh proposed openstack/os-vif master: Fix - os-vif fails to get the correct UpLink Representor https://review.opendev.org/745895 | |
| 11:42:51 | openstackgerrit | Takashi Natsume proposed openstack/nova master: doc: Fix note directives https://review.opendev.org/750094 | |
| #openstack-nova - 2020-09-07 | |||
| 01:42:24 | openstackgerrit | Wenping Song proposed openstack/nova-specs master: Support vGPU in nova and cyborg interaction https://review.opendev.org/750116 | |
| 07:35:39 | bauzas | good morning Nova | |
| 07:37:14 | gibi | bauzas: good morning | |
| 08:29:30 | brinzhang | bauzas, gibi: good morning | |
| 08:30:24 | brinzhang | bauzas, gibi: two backport patches hope you can review, trivial changes https://review.opendev.org/#/c/749681/ , https://review.opendev.org/#/c/749701/ | |
| 08:53:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support SRIOV interface attach and detach https://review.opendev.org/740995 | |
| 08:53:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use UUID as vif and network_id in vif tests https://review.opendev.org/748722 | |
| 09:02:35 | openstackgerrit | Liang Fang proposed openstack/nova master: Add volume local cache support https://review.opendev.org/663542 | |
| 09:04:16 | elod | gibi: if you will have time: https://review.opendev.org/#/c/750068/ :) (Stein release patch) | |
| 09:09:17 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Make PCI claim NUMA aware during live migration https://review.opendev.org/748453 | |
| 09:10:02 | gibi | elod: done | |
| 09:10:03 | gibi | thanks | |
| 09:12:41 | gibi | stephenfin: hi! the nova-multi-cell failure in the vtpm resize patch seems to be relevant | |
| 09:13:01 | stephenfin | ack, looking | |
| 09:13:19 | gibi | https://zuul.opendev.org/t/openstack/build/7326bc59092346b18fe9858774606321/log/compute1/logs/screen-n-cpu.txt?severity=4#7741 | |
| 09:18:41 | elod | gibi: thx! \o/ | |
| 09:24:46 | luyao | stephenfin: Hi, not sure whether you saw my message last week, just a kind remind for vpmem-enhencement https://review.opendev.org/#/q/topic:bp/vpmem-enhancement+(status:open+OR+status:merged) . I move the patch 'improve orphans tracking' to the last of the patch sequence (we don't have any big concern on the other 3 patches I think), I redefined those orphans in updated patch since previous version involved the | |
| 09:24:47 | openstack | bug 1879878 in OpenStack Compute (nova) "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Medium,In progress] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane) | |
| 09:24:47 | luyao | bug #1879878, and I didn't notice you have fixed it. | |
| 09:25:25 | stephenfin | luyao: ack, will take a look this afternoon | |
| 09:25:43 | luyao | stephenfin: Thank you in advance. :) | |
| 09:34:52 | bauzas | gosh, morning hell here | |
| 09:47:00 | openstackgerrit | Wenping Song proposed openstack/nova-specs master: Support vGPU in nova and cyborg interaction https://review.opendev.org/750116 | |
| 09:59:46 | nightmare_unreal | live-migration force node is not working with v2.67 , In v.268 force live migration was completely removed but it should work with v2.67 | |
| 10:10:53 | bauzas | why don't we have good api reference for https://docs.openstack.org/api-ref/network/v2/index.html#subnets ? | |
| 10:11:24 | noonedeadpunk | hi everyone. any chance you know why nova-compute may fail on centos7 for master and ussuri that way? https://zuul.opendev.org/t/openstack/build/6add842202c34390959d5fd0bd6fc83b/log/logs/host/nova-compute.service.journal-15-37-20.log.txt#2729-2768 | |
| 10:13:19 | noonedeadpunk | centos8, debian,ubuntu feel ok with the same setup path and configs | |
| 10:13:45 | bauzas | gibi: do you know how I can know the API reference for getting the subnets information ? | |
| 10:13:54 | bauzas | gibi: looking at https://docs.openstack.org/neutron/latest/admin/config-routed-networks.html#example | |
| 10:14:05 | bauzas | gibi: it looks to me you can get the related segment of a subnet | |
| 10:14:15 | bauzas | openstack subnet show my_subnet --c segment_id | |
| 10:14:56 | bauzas | but i guess it's a neutron extension | |
| 10:17:37 | bauzas | ahah, that's the reference which is confusing | |
| 10:17:56 | bauzas | https://docs.openstack.org/api-ref/network/v2/index.html?expanded=show-subnet-details-detail#show-subnet-details | |
| 10:33:24 | gibi | bauzas: sorry, I was afk | |
| 11:27:12 | lee1 | kashyap: morning, random question, have you had to debug device detach issues before through libvirt and the guestOS and if so do you have any tips? | |
| 11:27:29 | openstack | Launchpad bug 1882521 in OpenStack Compute (nova) "Failing device detachments on Focal" [Critical,Confirmed] - Assigned to Lee Yarwood (lyarwood) | |
| 11:27:29 | lyarwood | kashyap: context is https://bugs.launchpad.net/nova/+bug/1882521 | |
| 11:27:51 | kashyap | Ack; back here in a min :) | |
| 11:28:20 | lyarwood | kashyap: I can reproduce while running the full suite of tests and I'm pretty sure it's just an issue of the guestOS (cirros) not being able to process the request but I just want to prove it somehow | |
| 11:28:23 | lyarwood | ack np | |
| 11:29:34 | kashyap | lyarwood: lee1 was your nick too, I suppose? | |
| 11:32:05 | kashyap | lyarwood: I vaguely recall some triaging some device detach issues; but I forget the details. Gimme a few | |
| 11:32:22 | kashyap | lyarwood: Are you also implying that this is not reproducible with non-CirrOS guests? | |
| 11:34:03 | kashyap | Okay, you say as much in #5 | |
| 11:34:05 | kashyap | "Each time this has been hit however it appears that the Guest OS (cirros) isn't able to react to the ACPI request to detach the disk device. " | |
| 11:34:45 | lyarwood | kashyap: yeah that's my feeling at the moment, I'm looking to prove it now | |
| 11:35:07 | lyarwood | kashyap: just trying to work out how to capture the moment libvirt / QEMU signal the guestOS to detach the device | |
| 11:35:24 | kashyap | lyarwood: Right, configure the debug log filters, it should definitely give us some clues | |
| 11:35:25 | lyarwood | kashyap: and then work out how to capture that in the guest, AFAICT dmesg doesn't list it | |
| 11:35:40 | kashyap | `journalctl`? | |
| 11:37:01 | lyarwood | kashyap: cirros doesn't have systemd | |
| 11:37:08 | kashyap | Darn, I keep forgetting | |
| 11:37:44 | lyarwood | ah it's using acpid | |
| 11:40:43 | sean-k-mooney1 | i need to try and find time to test alpine as a cirros alternitive | |
| 11:40:47 | kashyap | lyarwood: Fedora doesn't do it? | |
| 11:40:56 | kashyap | (As in, 'acpid' daemon) | |
| 11:41:39 | sean-k-mooney1 | fedora uses systemd for udev im not cure if that will handel acpid too | |
| 11:41:50 | sean-k-mooney1 | *sure | |
| 11:45:35 | kashyap | sean-k-mooney: 'systemd' can handle some ACPI events; not all - https://wiki.archlinux.org/index.php/Power_management#ACPI_events | |
| 11:45:43 | kashyap | On my Fedora laptop I see: | |
| 11:46:00 | kashyap | $> systemctl | grep -i acpi sys-devices-platform-thinkpad_acpi-leds-tpacpi::kbd_backlight.device loaded active plugged /sys/devices/platform/thinkpad_acpi/leds/tpacpi::kbd_backlight | |
| 11:46:04 | kashyap | systemd-backlight@leds:tpacpi::kbd_backlight.service loaded active exited Load/Save Screen Backlight Brightness of leds:tpacpi::kbd_backlight | |
| 11:46:15 | sean-k-mooney | lyarwood: so i have been suggesting we should look into useing alpine instead of cirros going forward in the gate. its still does not use systemd but its one of the lightest weight distros i know of and unlike cirros its still maintained regurally | |
| 11:46:19 | kashyap | (So some Thinkpad-related ACPI events are handled) | |
| 11:46:41 | sean-k-mooney | kashyap: that sound like a lenovo extention | |
| 11:46:51 | sean-k-mooney | rather then generic support | |
| 11:47:02 | kashyap | lyarwood: Back to your original question - yeah, we need to find the "event" (IIRC, DEVICE_DELETED - need to double-check) that libvirtsends to the guest OS | |
| 11:47:22 | lyarwood | kashyap: do you know what that actually maps to in terms of what the guestOS sees? | |
| 11:47:41 | lyarwood | kashyap: an ACPI event right but any idea what type etc? | |
| 11:47:52 | kashyap | lyarwood: Not top off my head, perhaps Michal from libvirt might know; he worked on the 'udev' integration | |
| 11:48:25 | lyarwood | kashyap: could you ask and I'll work out a while of capturing that within the guestOS itself | |
| 11:48:43 | kashyap | lyarwood: Yeah, just asked; he's AFK. I'm checking w/ a couple of others | |
| 11:49:02 | lyarwood | sean-k-mooney: tbh it's a little silly that we are using it in CI and running nodes with such little resource as well tbh | |
| 11:49:36 | sean-k-mooney | lyarwood: well we dont have enogh disk/ram to use something much hevier | |
| 11:49:50 | sean-k-mooney | not without reducing concurancy at least | |
| 11:50:41 | sean-k-mooney | cirros made sense when it was activly maintained and updated | |
| 11:59:55 | kashyap | lyarwood: Do you have access to the guest? If so - is this present in it: /sys/module/pci_hotplug? | |
| 12:01:53 | sean-k-mooney | kashyap: cirrus uses a striped down ubuntu 18.04 kernel so it may not be | |
| 12:01:56 | lyarwood | kashyap: yeah that's there, I assume I can enable that | |
| 12:02:04 | lyarwood | kashyap: debug that is | |
| 12:03:09 | lyarwood | and yeah was just reading https://blog.chrishowie.com/2019/09/19/hot-swapping-virtio-disks-on-qemu/ so it's a PCI hot remove with virtio-blk that makes sense | |
| 12:03:27 | sean-k-mooney | yep it is | |
| 12:03:47 | sean-k-mooney | that why i was asserting that virtio-scsi or q35 might help | |
| 12:04:25 | kashyap | lyarwood: So I learn that's the part (the /sys/module/pci_hotplug) which is responsible for hotplug/hotunplug events | |
| 12:04:34 | sean-k-mooney | virtio-scsi woudl be the simplest thing to enable | |
| 12:04:50 | lyarwood | sean-k-mooney: well if it the guestOS can't process the request to detach I don't think changing the underlying bus is going to help tbh | |
| 12:05:05 | kashyap | lyarwood: So I just chatted w/ a couple of QEMU devs; and it seems notoriously difficult to detect this. Way too low-level ... | |
| 12:05:10 | sean-k-mooney | lyarwood: well it wont be a pci hotplug anymore | |
| 12:05:31 | sean-k-mooney | lyarwood: it will be a scsi detach | |
| 12:05:32 | lyarwood | sean-k-mooney: true but the guest would still need to handle the SCSI command (?) to detach | |
| 12:05:52 | gibi | stephenfin: fyi, I have a question in https://review.opendev.org/#/c/746945/6/nova/tests/functional/libvirt/test_pci_sriov_servers.py@a370 | |
| 12:05:57 | sean-k-mooney | yes proably but i think that would be more relyable | |
| 12:06:03 | kashyap | lyarwood: A snippet: | |