Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-02-11
11:05:29 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Record the machine_type of instances in system_metadata https://review.opendev.org/c/openstack/nova/+/767533
11:05:30 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-manage: Add machine_type update command https://review.opendev.org/c/openstack/nova/+/774896
11:05:30 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-manage: Add machine_type get command https://review.opendev.org/c/openstack/nova/+/769548
11:05:31 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-status: Add hw_machine_type check for libvirt instances https://review.opendev.org/c/openstack/nova/+/770643
11:05:31 openstackgerrit Lee Yarwood proposed openstack/nova master: WIP nova-manage: Add machine_type list_unset command https://review.opendev.org/c/openstack/nova/+/774897
11:05:32 openstackgerrit Lee Yarwood proposed openstack/nova master: docs: Add admin docs for configuring and updating machine types https://review.opendev.org/c/openstack/nova/+/774899
11:05:32 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Add a config update workflow test for [libvirt]hw_machine_type https://review.opendev.org/c/openstack/nova/+/774898
11:07:13 lyarwood kashyap: Didn't find service registered by hostname after 60 seconds
11:07:30 kashyap lyarwood: Yeah; figured out - actually it's a CPU model prob
11:07:40 kashyap I gave a model that is marked as "no" by libvirt
11:07:56 lyarwood kashyap: so n-cpu didn't start?
11:07:56 kashyap Thanks for looking :) sean-k-mooney gave the hint, while we're on a call
11:08:03 lyarwood k
11:08:33 kashyap lyarwood: Saw the exception in journalctl -u devstack@n-cpu
11:11:20 gibi sorry I was away
11:11:24 gibi but I see you solved it
11:12:06 kashyap Yes; no problem. I never count on instant responses :)
11:56:36 gibi sean-k-mooney: hi! here is a bug report https://bugs.launchpad.net/nova/+bug/1915255 based on the git log https://review.opendev.org/c/openstack/nova/+/739131 seems suspicious causing it. Could you please take a look?
11:56:37 openstack Launchpad bug 1915255 in OpenStack Compute (nova) "[Victoria] nova-compute won't start on aarch64 - raises PciDeviceNotFoundById" [Undecided,New]
11:57:43 sean-k-mooney am yes i can take a look
12:00:00 sean-k-mooney so the function that is failing is not one i added but one im started calling
12:00:28 sean-k-mooney that exception i think shoudl alreay be caught but maybe not in the current code path
12:00:57 sean-k-mooney its failing to look up the parent netdev name for badnwith based schduling
12:02:01 sean-k-mooney we could rewrite that function to call libvirt instead of sysfs possibly
12:14:36 gibi hm, then I will take a look too
12:15:16 gibi lyarwood: hi! Could this be another manifestation of the detach retry bug I'm trying to fix? https://bugs.launchpad.net/nova/+bug/1914664
12:15:17 openstack Launchpad bug 1914664 in OpenStack Compute (nova) "QEMU monitor read failure in ServerStableDeviceRescueTest" [Undecided,New]
12:18:01 sean-k-mooney gibi: apparently the Cavium ThunderX does not always have a PF assinged to each VF
12:18:16 sean-k-mooney from https://bugs.launchpad.net/charm-nova-compute/+bug/1771662
12:18:17 sean-k-mooney ibvirt has had the assumption that every VF (virtual function) will
12:18:17 openstack Launchpad bug 1771662 in libvirt (Ubuntu Disco) "[bionic] libvirtError: Node device not found: no node device with matching name" [Undecided,Fix released]
12:18:19 sean-k-mooney have a PF (physical function) assigned, but that does not hold true on
12:18:21 sean-k-mooney some special Hardware like the Cavium ThunderX
12:18:21 gibi sean-k-mooney: how that is possible?
12:18:40 sean-k-mooney i am not sure maybe they mean netdev
12:19:02 sean-k-mooney its possible for the PF to no actully work as a nic and only provide VFs
12:19:34 gibi if there is no parent PF then the whole qos stuff is just breaks down as we modell bw inventory on the PF
12:19:43 sean-k-mooney ya
12:20:01 sean-k-mooney so in any case we need to allow the compute agent to start
12:20:08 sean-k-mooney so we need to deal with the excpeiton
12:20:26 sean-k-mooney but i think this is jus ta case of this is strange hardware
12:20:34 gibi yes, we have to allow the compute to start but then this interface will not be ever selected for a port that has qos request
12:20:43 sean-k-mooney yep
12:20:53 gibi thanks for the triage on this bug
12:22:05 sean-k-mooney so there are two ways we can adress this it looks like the old logic just logged.
12:22:12 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/739131/12/nova/pci/utils.py
12:22:28 sean-k-mooney so we can do that
12:22:47 sean-k-mooney or we can also maybe stop using sysfs and use libvirt
12:22:55 sean-k-mooney or both in seperate patchs
12:23:09 lyarwood gibi: looking
12:23:18 gibi lyarwood: thanks
12:23:26 openstackgerrit Stephen Finucane proposed openstack/nova master: Uncap PrettyTable https://review.opendev.org/c/openstack/nova/+/775142
12:23:39 gibi sean-k-mooney: as a fix I would just log now
12:24:14 sean-k-mooney oh actully
12:24:27 sean-k-mooney this code is common its not allowed to talk to libvirt
12:24:28 gibi sean-k-mooney: as far as I understand your patch moved away from libvirt for certain reasons
12:24:34 sean-k-mooney that is why we are using sysfs
12:25:13 sean-k-mooney ya libvirt can get out os sync
12:25:28 sean-k-mooney for nodedev names
12:25:51 sean-k-mooney so ya using libvirt here would not be right
12:26:22 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: requirements: Remove simplejson https://review.opendev.org/c/openstack/python-novaclient/+/775144
12:26:22 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: Uncap PrettyTable https://review.opendev.org/c/openstack/python-novaclient/+/775143
12:26:41 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/739131/12/nova/virt/libvirt/driver.py#b6960
12:26:52 sean-k-mooney that might be the issue
12:28:08 sean-k-mooney ya so my note
12:28:10 sean-k-mooney # NOTE(sean-k-mooney): if the VF is a parent of a netdev
12:28:13 sean-k-mooney # the PF should also have a netdev.
12:28:20 sean-k-mooney i guess is not true for this hardware
12:28:36 sean-k-mooney so we jus tneed to readd that try except
12:29:50 sean-k-mooney ill triage it now
12:31:10 gibi thanks
12:36:21 lyarwood gibi / dansmith ; https://bugs.launchpad.net/nova/+bug/1914664 - the test failure looks like a volume quota issue in that job, we might want to track that in another bug while we look at the unrelated detach device trace
12:36:22 openstack Launchpad bug 1914664 in OpenStack Compute (nova) "QEMU monitor read failure in ServerStableDeviceRescueTest" [Undecided,New]
12:45:27 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: Add support for microversion v2.88 https://review.opendev.org/c/openstack/python-novaclient/+/770573
12:47:56 stephenfin dansmith: Now that most of the migration compaction series has merged for the main DB, could I ask that you look at the API DB equivalent? I employed the same testing strategy so it should all be very familiar https://review.opendev.org/q/topic:%2522bp/compact-db-migrations-wallaby%2522+status:open+NOT+label:workflow%253E%253D1
13:10:05 gibi lyarwood: thanks
14:01:46 dansmith lyarwood: I already had a fix up for that, I think we lost bug linkage recently
14:02:15 gibi dansmith: we don't have bug linkage since we upgraded the gerrit server
14:02:22 dansmith right
14:02:47 dansmith https://review.opendev.org/c/openstack/devstack/+/774172
14:03:26 dansmith I opened a different bug I guess I thought it was a separate issue
14:18:58 lyarwood dansmith: ah wonderful
14:32:15 openstackgerrit Merged openstack/os-vif master: Resolve dependency issues https://review.opendev.org/c/openstack/os-vif/+/774694
15:46:07 openstackgerrit Balazs Gibizer proposed openstack/nova master: Support interface attach with QoS ports https://review.opendev.org/c/openstack/nova/+/756530
15:47:37 gibi bauzas: fixed up based on your comments ^^
15:48:03 bauzas gibi: coolio, but working atm on stephenfin's nits with routed networks :p
15:48:13 gibi ack, no worries
15:51:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Remove unused Instance method https://review.opendev.org/c/openstack/nova/+/757615
15:51:48 openstackgerrit Balazs Gibizer proposed openstack/nova master: Enable mypy on scheduler/report.py https://review.opendev.org/c/openstack/nova/+/765152
15:52:18 gibi nova meeting starts in 8 minutes on #openstack-meeting-3
15:54:43 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Record the machine_type of instances in system_metadata https://review.opendev.org/c/openstack/nova/+/767533
15:54:44 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-manage: Add machine_type update command https://review.opendev.org/c/openstack/nova/+/774896
15:54:44 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-manage: Add machine_type get command https://review.opendev.org/c/openstack/nova/+/769548
15:54:45 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-status: Add hw_machine_type check for libvirt instances https://review.opendev.org/c/openstack/nova/+/770643
15:54:45 openstackgerrit Lee Yarwood proposed openstack/nova master: WIP nova-manage: Add machine_type list_unset command https://review.opendev.org/c/openstack/nova/+/774897
15:54:46 openstackgerrit Lee Yarwood proposed openstack/nova master: docs: Add admin docs for configuring and updating machine types https://review.opendev.org/c/openstack/nova/+/774899
15:54:46 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Add a config update workflow test for [libvirt]hw_machine_type https://review.opendev.org/c/openstack/nova/+/774898
15:54:47 openstackgerrit Lee Yarwood proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/c/openstack/nova/+/708701
16:25:05 supamatt nah don't need it
16:25:09 supamatt err sorry wrong chan

Earlier   Later