Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-06-12
15:43:35 Sundar What would be the success metric for the test?
15:46:43 mriedem Sundar: tempest isn't going to be doing virsh list anyway
15:46:57 mriedem tempest tests should be hypervisor agnostic, like the compute api should be hypervisor agnostic
15:47:27 mriedem Sundar: the basic success metric for tempest tests are going to be that the API behaves as expected,
15:47:49 mriedem i.e. if i create a server with a flavor configured for an ARQ device profile, that once the server is ACTIVE there is an attached ARQ in cyborg
15:48:02 mriedem and when the server is deleted, the corresponding ARQ resource is also deleted
15:49:08 mriedem Sundar: whitebox integration testing is something that would likely be done in a tempest plugin / tests that are configured to only run when there is real hardware in a 3rd party CI env
15:50:32 Sundar mriedem: "once the server is ACTIVE" -- with a fake device, it never becomes active. It will go to error state.
15:51:20 mriedem Sundar: because of the event not being sent from cyborg, or because libvirt will fail to start the domain?
15:51:26 Sundar We need to ensure the device gets attached to the VM
15:51:28 sean-k-mooney Sundar: not if we add a bus type for dummy devices and handel that in nova
15:51:54 Sundar mridem: latter - libvirt will fail it
15:52:13 mriedem then i agree with sean-k-mooney that we'd need to stub something out if we know we're using a fake device,
15:52:20 sean-k-mooney Sundar: we dont we jsut need to pass info to nova to let it know its a fake device and libivirt can just not try to atach it
15:52:22 mriedem which is gross, but i'd rather have that than no api integration testing
15:52:33 Sundar sean-k-mooney: 'bus type for dumy devices' -- is this a new thing? It is not there today, right?
15:52:54 mriedem wouldn't there be some metadata on the ARQ resource that can tell us (nova) that it's a fake device?
15:52:55 sean-k-mooney sure but its useful for testing
15:53:03 mriedem similar to vif type on a port
15:53:10 mriedem or type on a volume
15:54:06 sean-k-mooney Sundar: we have many thing we can do like have the fake driver create mdev or other software device like a loop device and tell nova to use those too
15:54:30 sean-k-mooney or we can have a sentenel that just tells nova its a fake device for testing
15:55:18 Sundar mriedem, sean-k-mooney: The attach handle in the ARQ, returned by Cyborg to Nova virt driver, will identify the type of the handle, e.g. 'PCI'. I could return a special type, say, 'testPCI', and modify Nova virt driver to ignore that
15:55:37 Sundar That would be used solely for testing
15:55:47 sean-k-mooney Sundar: yes that is basically what i was thinking wew would do
15:55:48 mriedem i'd just calle it "fake" or something like that but yeah
15:56:28 Sundar Ah ok. Got it. Thanks. :)
15:56:49 sean-k-mooney Sundar: the real thing we want to test is teh end to end workflow
15:57:08 mriedem s/real/main/ for now
15:57:21 sean-k-mooney :) right
15:57:28 mriedem real low-level whitebox integration testing can be done with 3rd party CI and real hardware,
15:57:34 mriedem but let's not get the cart before the horse
15:57:38 sean-k-mooney in a third party ci we can then also validate it with real hardware
15:57:56 sean-k-mooney yep
16:01:18 stephenfin mriedem: Cool. I spotted some other things when rebasing. Comments left and I've a follow-up patch I'll post. Feel free to squash it into yours if you want
16:01:56 openstackgerrit Stephen Finucane proposed openstack/nova master: Drop pre-cinder 3.44 version compatibility https://review.opendev.org/621061
16:01:57 openstackgerrit Stephen Finucane proposed openstack/nova master: Follow-up for I6a777b4b7a5729488f939df8c40e49bd40aec3dd https://review.opendev.org/664967
16:01:57 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove 'MultiattachNotSupportedByVirtDriver' https://review.opendev.org/651315
16:03:48 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove 'MultiattachSupportNotYetAvailable' exception https://review.opendev.org/651315
16:07:48 openstackgerrit Balazs Gibizer proposed openstack/nova master: func test for migrate server with ports having resource request https://review.opendev.org/655113
16:07:48 openstackgerrit Balazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth https://review.opendev.org/655109
16:14:03 Nick_A Do you guys have any tips for what we should look at to determine why cloud-init is working properly for us on lxd but not kvm?
16:18:50 sean-k-mooney if its working on lxd then the nova metadata service is working properly
16:19:41 sean-k-mooney it might be a diffrence in how netwroking works
16:20:18 sean-k-mooney you really need to try and trace the request
16:21:04 sean-k-mooney so if you can log into one of your kvm instnce try an curl the metadata api via the 169... adress
16:22:03 sean-k-mooney if you dont have a vm image with a password set for debuging then you can enable config drive to initally be able to log in to the vm and debug the issue
16:24:33 Nick_A thank you
16:34:42 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: flatten rbd images when unshelving an instance https://review.opendev.org/457886
16:37:58 openstackgerrit Adam Spiers proposed openstack/nova master: Add extra spec parameter and image property for memory encryption https://review.opendev.org/664420
16:42:45 mriedem stephenfin: this shouldn't be stable-only i don't think but you might want to verify if the bug is legit https://review.opendev.org/#/c/664838/1
16:48:59 aspiers efried: Slightly out of my depth here. So shall I go ahead and change spawn() to persist allocations as a new field in the Instance object?
16:50:42 efried aspiers: I wasn't following whatever conversation led up to this
16:51:00 efried Is something we're doing with SEV requiring us to be able to look at the allocation some time after spawn?
16:51:10 efried or is this related to some other topic?
16:51:11 aspiers Yes
16:51:18 efried Do tell
16:51:42 aspiers http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-06-12.log.html#t2019-06-12T12:50:55
16:52:14 aspiers There are multiple places scattered through the code for building the guest config XML which need to know whether SEV is required
16:52:40 aspiers but to perform the check in the way you requested, allocations need to be accessible at those points
16:53:00 efried say wha now?
16:53:13 efried you're telling me that every time libvirt attaches a volume, it rebuilds the whole XML from scratch?
16:53:20 aspiers no
16:53:31 aspiers it needs to know whether to use iommu
16:53:32 efried you're telling me that attaching a volume needs to be SEV aware?
16:53:45 aspiers not necessarily even attaching a volume
16:54:00 efried okay, I see.
16:54:02 aspiers just block devices and stuff at launch-time
16:54:18 aspiers and also to decide whether to include <locked />
16:54:30 aspiers the latest 3 commits at the top of the series
16:55:00 efried However, in this scenario, you could also solve by commonizing (libvirt/utils?) the code that inspects the flavor/image meta.
16:55:00 efried I think long term, having allocations on the instance object is a great idea. mriedem, dansmith, how do y'all feel about that?
16:55:19 dansmith efried: why?
16:55:25 aspiers Yes, that possibility occurred to me too
16:56:42 efried dansmith: example here is SEV. We were considering doing the flavor/image parsing and validation just once, in the request filter, to add resources=MEM_ENCRYPTION_CONTEXT. Then after that, e.g. in spawn(), just looking at the allocation for MEM_ENCRYPTION_CONTEXT so we don't have to redo that parsing/validation.
16:57:12 aspiers here's sean-k-mooney's take: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-06-12.log.html#t2019-06-12T13:13:16
16:57:18 efried In spawn specifically, we are getting the allocation as a param, so that's fine. But there are (aspiers tells me) other places where we need the same information but don't (currently) have access to the allocations.
16:57:21 dansmith spawn having access to the allocations makes sense, but storing them on/with the instance doesn't really
16:57:52 aspiers here's an example: https://review.opendev.org/#/c/662558/5/nova/virt/libvirt/driver.py
16:58:08 dansmith I imagine bauzas would want access to allocations for gpu stuff as well, and I'm sure we discussed making that a param to spawn in the past
16:58:15 aspiers currently this is broken because it's insufficient to pass only the flavor to _sev_required()
16:58:23 efried allocations are already passed to spawn
16:58:33 dansmith ack, okay I was sure we had discussed
16:58:36 aspiers right, the problem is that they are not passed much further on
16:58:58 aspiers sean-k-mooney's other suggestion was to pass from spawn() to _get_guest_xml() and so on
16:59:21 dansmith yeah, pass them further if you want, but storing them .. no so much
16:59:33 aspiers but that doesn't solve things like attaching volumes to an SEV instance
16:59:39 efried you could cache a map of instance:allocations on the virt driver
16:59:47 dansmith efried: dude
16:59:53 efried but if you restart the compute driver you'd have to rebuild that by querying placement
16:59:57 dansmith aspiers: so look them up and pass them in if you need
17:00:09 efried we've said we don't like virt drivers talking to placement
17:00:19 dansmith yeah
17:00:31 efried though I guess `read` may be acceptable
17:00:32 dansmith and caching placement data is also smelly, IMHO
17:00:46 dansmith efried: no, make compute manager look them up and pass them in
17:01:04 efried dansmith: You mean augment every ComputeDriver method to accept allocations?
17:01:30 dansmith surely we don't need them on every call
17:01:40 efried creeping death of a thousand cuts
17:02:15 efried we already store other (pre-placement) allocation-like things on the Instance, don't we? PCI devices and such?

Earlier   Later