Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-02-14
17:42:14 efried gibi: nm, I think I see it now. And I am suitably shocked that this is the case.
17:42:25 cfriesen melwitt: it looks like the intent in LibvirtDriver.detach_volume() was to swallow the DeviceNotFound exception, but I seem to be hitting the libvirtError exception path which gets re-raised and handled in DriverVolumeBlockDevice.driver_detach
17:44:01 artom Weird question, but does oslo.config support *writing* config files? Sorta like python's configparser, but hopefully without getting rid of comments when writing (which configparser does :( )
17:44:02 melwitt cfriesen: right. it looks like you're hitting an unexpected combination where you have an libvirt 'internal error' with a 'not found' message, whereas we expect a 'operation failed' libvirt error to go with the 'not found' message
17:45:24 cfriesen got it...I hadn't clued in we were dealing with two separate exceptions
17:46:19 openstackgerrit Sylvain Bauza proposed openstack/nova master: WIP: Use the correct mdev allocated from the pGPU https://review.openstack.org/636591
17:46:20 openstackgerrit Sylvain Bauza proposed openstack/nova master: libvirt: implement reshaper for vgpu https://review.openstack.org/599208
17:46:20 openstackgerrit Sylvain Bauza proposed openstack/nova master: Add functional test for libvirt vgpu reshape https://review.openstack.org/631559
17:46:38 bauzas efried: updated based on our discussions ^
17:46:55 bauzas now the pGPU finding is a dependency for the reshape patch
17:47:21 bauzas because if not, it would mean we would need to trigger another reshape for allocations wrongly set
17:47:45 melwitt cfriesen: I expect the one you're getting is VIR_ERR_INTERNAL_ERROR and we're only handling VIR_ERR_OPERATION_FAILED and VIR_ERR_INVALID_ARG
17:48:41 cfriesen melwitt: thanks, that gives me something to dig into at least.
17:50:33 melwitt cfriesen: np. looks like we might need a 'if errcode in (libvirt.VIR_ERR_OPERATION_FAILED, libvirt.VIR_ERR_INTERNAL_ERROR):' on L409
17:50:52 cfriesen melwitt: it looks like we expect the detach from the persistent config to sometimes fail...what would cause that?
17:51:47 melwitt cfriesen: it's only expected if the device is not there (already detached). that is, libvirt will raise an error if it's already detached
17:52:40 cfriesen melwitt: so why would it be already detached? a previous iteration of the loop?
17:52:43 melwitt that's why we specifically look for 'not found' (in the case of persistent) or 'no target device' (in the case of live)
17:53:54 melwitt cfriesen: we've seen things like, someone requests a detach, the detach from the libvirt domain succeeds, but the detach from the compute host fails, so a fail is returned up to the user
17:54:29 melwitt then later, the user tries to detach the volume again, the device was detached from the libvirt domain last time, so we expect 'not found' at that point, but we still need to detach from the compute host
17:55:15 cfriesen got it, thanks
17:55:20 melwitt np
18:08:19 efried mriedem: I'm +2 now on https://review.openstack.org/#/c/616239/
18:08:48 efried bauzas: Yes, I agree. Looking...
18:09:52 mriedem efried: ack cool
18:10:01 mriedem will try to hit that later today or tomorrow morning
18:10:28 efried My confusion is probably because I missed the bulk of the spec discussion and the bottom of the series.
18:10:55 efried Trying to catch up seemed futile, so I'm trying to jump in in the middle, and suffering the consequences.
18:11:03 mriedem i haven't gone through the comments, but looks like you were expecting us to track vcpu/disk/ram in the request group?
18:11:26 mriedem we still pull those values off the request_spec.flavor during scheduling prior to calling GET /a_c
18:11:40 efried mriedem: Yes, I expected that the request spec's list of request groups would contain the request groups.
18:11:48 efried Not a subset of the request groups
18:12:10 efried afaict it actually only contains the bw-related rgs.
18:12:22 mriedem yup
18:12:25 mriedem it's new as of this series
18:12:37 efried so like even if you requested a VGPU in a numbered group, and your VCPU in another numbered group, and...
18:12:43 efried it would still only have the bw groups
18:13:31 efried it's eventually going to need to contain everything
18:13:52 mriedem did he put a comment about that in the code or going to document the limitation in a follow up?
18:13:54 efried or if not "need", it'll soon get to a point where the things that are being excluded won't make any sense.
18:14:25 efried Well, for someone who has been following along, I'm sure it's not a thang.
18:14:31 efried But no, he just responded in the comments.
18:14:54 efried If the rgs member in the request spec was called bandwidth_request_groups, I wouldn't have batted an eye
18:14:56 mriedem i don't disagree that the 10 roads to get to our GET /a_c is a problem
18:15:02 efried but then we would have had to change it later
18:15:10 efried and changing things on an ovo is suck
18:16:06 mriedem i think it was VGPUs that made me think at one point about how we now have resources we pull from the explicit flavor attributes (ram/vcpu/disk) and VGPU is not a top-level attribute on the flavor, it's an extra_spec
18:16:09 efried I guess what would help is if RequestSpec.requested_resources had a big NOTE on it saying "currently only houses bandwidth groups"
18:16:29 mriedem efried: right - that's good for a follow up
18:16:35 mriedem fup 3 at this point
18:17:30 efried I'm adding a note.
18:18:17 openstackgerrit Adrian Chiris proposed openstack/nova master: Libvirt: do not set MAC when unplugging macvtap VF https://review.openstack.org/624842
18:18:17 openstackgerrit Adrian Chiris proposed openstack/nova master: Add free for claimed, allocated devices https://review.openstack.org/616120
18:18:18 openstackgerrit Adrian Chiris proposed openstack/nova master: Allow per-port modification of vnic_type and profile https://review.openstack.org/607365
18:18:18 openstackgerrit Adrian Chiris proposed openstack/nova master: Add get_instance_pci_request_from_vif https://review.openstack.org/619929
18:18:19 openstackgerrit Adrian Chiris proposed openstack/nova master: SR-IOV Live migration indirect port support https://review.openstack.org/620115
18:18:19 openstackgerrit Adrian Chiris proposed openstack/nova master: libvirt: auto detach/attach sriov ports on migration https://review.openstack.org/629589
18:18:45 mriedem like, flavor at this point doesn't even really need to have vcpu/memory_mb/root_gb at top level fields - root_gb=0 was already problematic since people used it to denote volume-backed server flavors
18:19:34 mriedem which reminds me https://review.openstack.org/#/c/603910/
18:20:56 prometheanfire is it possible to request the status of pci devices on hypervisors?
18:21:04 prometheanfire https://wiki.openstack.org/wiki/Pci-api-support#List_and_show_PCI_devices_on_VMs doesn't seem to be used
18:21:20 mriedem prometheanfire: not in the api
18:22:04 mriedem prometheanfire: the review for https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/show-server-numa-topology.html talked about that since it was originally munging that in with NUMA topology for a server
18:22:20 mriedem but that would be allocated pci devices per instance anyway
18:22:31 mriedem if you're looking for inventory, it has to be retrieved outside of the compute API right now
18:22:43 mriedem if/when pci device inventory is tracked in placement you'd use the placement API
18:22:57 prometheanfire ya, placement makes sense
18:23:11 mriedem oh, and because of the [pci]passthrough_whitelist config option per compute the actual pci devices on the host might not be used by nova
18:23:48 openstackgerrit Matt Riedemann proposed openstack/nova master: Stub out port binding create/delete in NeutronFixture https://review.openstack.org/636413
18:23:49 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012
18:23:49 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035
18:23:50 openstackgerrit Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036
18:23:50 openstackgerrit Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037
18:23:51 openstackgerrit Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353
18:23:51 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.openstack.org/631123
18:23:52 openstackgerrit Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892
18:23:52 openstackgerrit Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581
18:23:53 openstackgerrit Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853
18:23:53 openstackgerrit Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313
18:23:54 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293
18:23:54 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtDestTask https://review.openstack.org/627890
18:23:55 openstackgerrit Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832
18:23:55 openstackgerrit Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.openstack.org/627891
18:23:56 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add finish_snapshot_based_resize_at_dest compute method https://review.openstack.org/635080
18:23:56 openstackgerrit Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.openstack.org/635646
18:23:57 openstackgerrit Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.openstack.org/635668
18:27:10 prometheanfire right, what I want to know is the status/usage of the passthrough_whitelist and alias for the compute hosts
18:27:28 prometheanfire having that in hv show would be nice
18:28:17 mriedem throw it in the old train ptg etherpad!
18:28:41 mriedem but the answer is likely going to be "it will be available in placement (eventually, in a couple of years maybe)"
18:29:11 mriedem you could write a tool that mines that data from the database....
18:29:20 prometheanfire meh, not that important
18:29:25 mriedem the pci_devices table is both inventory and allocations
18:31:31 efried soon you will be able to query this from the cyborg API :P
18:32:15 efried using incantations formulated from the charred remains of pci.passthrough_whitelist
18:45:39 openstackgerrit melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350
18:45:40 openstackgerrit melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351
18:52:21 openstackgerrit melanie witt proposed openstack/nova master: Add user_id column to the instance_mappings table https://review.openstack.org/633349
18:52:21 openstackgerrit melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350
18:52:22 openstackgerrit melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351

Earlier   Later