| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-14 | |||
| 12:18:13 | gibi | bauzas: I saw in the debug log of the test run that the VM has an mdev in the xml | |
| 12:18:16 | gibi | <hostdev mode="subsystem" type="mdev" managed="no" model="vfio-pci"> | |
| 12:18:19 | gibi | <source> | |
| 12:18:22 | gibi | <address uuid="4b20d080-1b54-4048-85b3-a6a62d165c01"/> | |
| 12:18:24 | gibi | </source> | |
| 12:18:27 | gibi | </hostdev> | |
| 12:19:00 | gibi | bauzas: I guess that the problem can be that the VGPU inventory is not added by the compute itself to the root RP but added by the functional test manually. | |
| 12:23:59 | gibi | bauzas: somehow if not the compute adds the VGPU to the root RP then the compute does not see it later when checking if reshape is needed. | |
| 12:25:03 | gibi | bauzas: it seems like the provider_tree cache is not updated after the VGPU is added to the root RP by the functional test, even if the functional test simulated the compute restart | |
| 12:25:43 | gibi | bauzas: I will dig into this. It can be an artifact of the way the functional test a) adds the VGPU inventory to the root RP or b) the way the tests simulates the compute service restart | |
| 12:33:23 | alex_xu | stephenfin: thanks for take care https://blueprints.launchpad.net/nova/+spec/remove-force-flag-from-live-migrate-and-evacuate | |
| 12:48:37 | openstackgerrit | Andrey Volkov proposed openstack/nova master: AZ list performance optimiztion: avoid double service list DB fetch https://review.openstack.org/636947 | |
| 12:50:06 | gibi | bauzas: yepp, it seems that when the func test simulates the compute restart the provider tree kept in the the report client is not dropped and therefore during the restart it is not refreshed | |
| 12:50:18 | gibi | bauzas: does the reportclient went to become a singleton? | |
| 12:58:12 | gibi | bauzas: nvm | |
| 12:58:17 | gibi | bauzas: it doesn't | |
| 13:02:15 | openstackgerrit | Andrey Volkov proposed openstack/nova master: AZ list performance optimization: avoid double service list DB fetch https://review.openstack.org/636947 | |
| 13:07:07 | gibi | bauzas: I managed to find the root case | |
| 13:07:14 | gibi | bauzas: it is in the test env | |
| 13:08:01 | gibi | bauzas: the service.stop service.start in the functional test does not properly simulate a real service restart, a lot of state are kept, like the resource tracker and the reportclient provider_tree cache. | |
| 13:08:25 | gibi | bauzas: I manually dropped the provider_tree cache between stop and start and now the test passes | |
| 13:10:18 | gibi | bauzas: pushing the fix to the test soon | |
| 13:11:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add functional test for libvirt vgpu reshape https://review.openstack.org/631559 | |
| 13:11:27 | gibi | bauzas: ^^ | |
| 13:18:19 | bauzas | gibi: yeah, I was about telling you that I didn't had a problem with the reshape when testing it in some hardware | |
| 13:19:19 | bauzas | gibi: see http://paste.openstack.org/show/745093/ | |
| 13:21:15 | openstackgerrit | Silvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts https://review.openstack.org/522245 | |
| 13:21:18 | bauzas | gibi: I just removed the -2 | |
| 13:29:55 | gibi | bauzas: the whole test problem boils down to the fact that it is not possible to create an old compute RP inventory directly from the compute code as that code was replaced by the new vgpu code. So the test should simulate that there is an old RP inventory with VGPU on the root | |
| 13:36:33 | bauzas | gibi: if you wanna test the real reshape, I agree | |
| 13:37:02 | bauzas | gibi: what does the test atm is making sure that on greenfields, you start with VGPU resources to be nested | |
| 13:37:04 | kaisers | stephenfin: Hi! FYI, i just ran an update to https://review.openstack.org/#/c/522245 based on yesterdays feedback :) | |
| 13:37:13 | mdbooth | Anybody got experience configuring devstack to use an https endpoint with a self-signed cert? | |
| 13:38:14 | mdbooth | To the best of my knowledge the cert is correctly installed in the system-wide ca-bundle, and certainly curl will now connect without --insecure. | |
| 13:38:19 | mdbooth | But openstackcli will not. | |
| 13:38:25 | mdbooth | And neither will Nova. | |
| 13:40:06 | mdbooth | '$ openstack --insecure volume list' *does* work, but afaik there's no similar option to disable nova's certificate validation when using cinderclient. | |
| 13:41:33 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Ignore VolumeAttachmentNotFound exception in compute.manager https://review.openstack.org/636954 | |
| 13:41:56 | mdbooth | But... is there some alternate ca-bundle somewhere? | |
| 13:42:07 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Provide a useful error message when trying to update non-compute services https://review.openstack.org/636955 | |
| 13:45:55 | bauzas | gibi: I'm not super happy with https://review.openstack.org/#/c/631559/5..6/nova/tests/functional/libvirt/test_reshape.py | |
| 13:48:53 | stephenfin | kaisers: Still got comments, unfortunately https://review.openstack.org/#/c/522245 | |
| 13:49:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change sqlalchemy warnings filter to an error https://review.openstack.org/636957 | |
| 13:55:17 | openstackgerrit | Mark Goddard proposed openstack/nova stable/queens: Don't emit warning when ironic properties are zero https://review.openstack.org/608611 | |
| 14:04:29 | mdbooth | Hmm, this is a difference in behaviour between urllib.requests (urllib2) and requests, which appears to use urllib3 | |
| 14:05:13 | mdbooth | The former appears to use the system ca bundle, the latter does not | |
| 14:06:37 | gibi | efried: I answered your questions / comments in https://review.openstack.org/#/c/616239/24 | |
| 14:07:16 | mdbooth | certifi, j'accuse | |
| 14:07:23 | gibi | bauzas: how can I make you happier about that reshape functional test? | |
| 14:08:25 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params https://review.openstack.org/636678 | |
| 14:10:34 | bauzas | gibi: self.compute is generated by nova.test.py which returns a ServiceFixture | |
| 14:11:23 | bauzas | gibi: if we really want to mock a real compute stop, we need to update the fixture | |
| 14:12:06 | bauzas | gibi: https://github.com/openstack/nova/blob/master/nova/test.py#L399 | |
| 14:13:06 | bauzas | gibi: somehow, you say in your comment that https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L83 doesn't work correctly | |
| 14:13:13 | bauzas | because we still have a cache | |
| 14:13:27 | bauzas | so we need to invalidate the cache there ^ | |
| 14:14:02 | gibi | bauzas: yeah, it would be nice to improve the fixture to support a real service restart | |
| 14:14:08 | gibi | bauzas: let me dig a bit about that | |
| 14:14:42 | gibi | bauzas: I remember I had issues with the restart in the past | |
| 14:14:53 | bauzas | for placement I guess ? | |
| 14:15:04 | bauzas | because I don't see any problems with the existing fixture | |
| 14:15:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: Add descriptions for vol-backed snapshots https://review.openstack.org/615084 | |
| 14:15:51 | bauzas | gibi: AFAICS, we kill the service | |
| 14:16:15 | bauzas | so I guess the problem is more about the python module | |
| 14:16:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Address nits from I9e30a24a4c0640f282f507d0a96640d3cdefe43c https://review.openstack.org/636967 | |
| 14:17:19 | bauzas | gibi: actually, I think I voiced about this when I was reviewing the scheduler reportclient cut | |
| 14:17:28 | bauzas | we shouldn't cache it | |
| 14:18:50 | gibi | bauzas: the fixture.stop() does not kill the rela service all the module level things are in place. fixture.kill and a starting a new compute service through the fixture would be better | |
| 14:23:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Drop the integrated-gate (py27) template https://review.openstack.org/634949 | |
| 14:26:12 | matoef1 | Hi guys. After enable SSL on DevStack I got the following message from nova.. ..devstack@n-api.service[30584]: DEBUG nova.api.openstack.wsgi [None req-00203afa-89e4-4c28-bed8-cbbc985ee67c demo demo] Returning 400 to user: Invalid input for field/attribute user_data. Value: Q29udGVud... | |
| 14:26:57 | matoef1 | But my user_data value does not have more then 64k. | |
| 14:28:07 | matoef1 | But the error message is : Invalid input for field/attribute user_data .. is too long | |
| 14:28:37 | matoef1 | Is there any way how to investigate this error ? | |
| 14:28:41 | matoef1 | Many thanks | |
| 14:34:59 | mdbooth | lyarwood: So... after *finally* getting a python3 devstack up and running with nova talking to cinder over ssl on F29... I don't get the recursion error. | |
| 14:35:15 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params https://review.openstack.org/636678 | |
| 14:35:34 | mdbooth | lyarwood: I downgraded eventlet to 0.21.0 | |
| 14:35:43 | mdbooth | Nada. | |
| 14:47:25 | gibi | bauzas: simply killing the compute service and starting a new compute service leads to a HostMapping issue which reminds me of https://review.openstack.org/#/c/512552/ | |
| 14:51:09 | openstackgerrit | Jack Ding proposed openstack/nova master: Flavor extra spec and image properties validation https://review.openstack.org/620706 | |
| 14:53:00 | gibi | bauzas: I will refresh https://review.openstack.org/#/c/512552/ and try to combine that with the reshape test to see if that helps | |
| 14:57:23 | gibi | bauzas: I don't actually have to complicate that much | |
| 14:57:35 | gibi | bauzas: there is already a restart_compute_service() call that does the trick we need | |
| 14:58:59 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add functional test for libvirt vgpu reshape https://review.openstack.org/631559 | |
| 14:59:03 | gibi | bauzas: what about this ^^ | |
| 15:00:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove deprecated 'os-flavor-manage' policy https://review.openstack.org/633656 | |
| 15:01:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix a missing policy in test policy data https://review.openstack.org/633686 | |
| 15:06:58 | openstackgerrit | Elod Illes proposed openstack/nova master: DNM: test pip 19 - pip freeze with git editable https://review.openstack.org/636980 | |
| 15:09:40 | bauzas | gibi: looking | |
| 15:09:44 | bauzas | gibi: sorry was afk | |
| 15:10:20 | bauzas | gibi: FWIW, I checked and we still need to have https://review.openstack.org/#/c/636591/1 | |
| 15:10:29 | bauzas | gibi: that said, looks like it's not a problem | |
| 15:10:57 | bauzas | gibi: I'll use a specific type on my node, and see whether we have problems | |
| 15:11:04 | gibi | bauzas: OK | |
| 15:11:45 | bauzas | gibi: http://paste.openstack.org/show/745109/ | |
| 15:12:24 | bauzas | gibi: the fdfd8f9b-e645-4a6f-aa60-477b00ba32b7 instance is having the mdev_27840b52_31e3_4fb7_a391_ec529e73be93 | |
| 15:12:54 | bauzas | gibi: placement said it should be in virtlab606.virt.lab.eng.bos.redhat.com_pci_0000_84_00_0 which is RP #3 | |
| 15:13:16 | bauzas | gibi: but when looking at libvirt, the mdev parent is pci_0000_87_00_0 | |
| 15:13:24 | bauzas | ie. not the same pGPU | |
| 15:13:45 | bauzas | given all pGPUs have the same type, I don't think it's a problem, but I'll test it | |