| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-05-07 | |||
| 09:57:26 | bauzas | but we don't create | |
| 10:05:26 | Yumeng | bauzas: emm IMHO, cyborg doesn't create the port in neutron during the boot. When Cyborg discovers SRIOV device and stores the device's info, user need to use Cyborg create a port/bandwith request, i.e. device_profile | |
| 10:05:56 | bauzas | yeah I was confused, my apologies | |
| 10:06:03 | bauzas | so, there is a pre-creation step | |
| 10:06:14 | bauzas | and then nova gets the port at boot time | |
| 10:06:25 | Yumeng | yes | |
| 10:10:03 | bauzas | OK, I think I still need to go thru the etherpad to understand the reasoning but that's a good first step | |
| 10:10:49 | Yumeng | gibi,bauzas: Thanks for your time! I am going off line now. my time is UTC+8. You can either ping me in IRC at #openstack-cyborg channel if you have more question. or just leave comments at ehterpad(topic 13):https://etherpad.opendev.org/p/cyborg-victoria-goals https://etherpad.opendev.org/p/cyborg-victoria-goals | |
| 10:11:04 | Yumeng | bauzas: thanks for taking time reading it! | |
| 10:11:10 | gibi | Yumeng: thanks I tried to add some not the nova etherpad. | |
| 10:11:17 | bauzas | gibi: fwiw, playing with stestr run --until-failure and then stestr --last helps me reproduce the failure everytime | |
| 10:11:52 | Yumeng | gibi: ok, thanks. I will go back to nova ehterpad later. | |
| 10:12:12 | gibi | bauzas: meantime I was able to reproduce the problem only runnig the single test case test_multiple_instance_create$ so it is not test case interference | |
| 10:12:35 | gibi | (it took 356 runs though) | |
| 10:12:51 | gibi | Yumeng: OK. have a nice evening | |
| 10:12:54 | bauzas | you mean, the single test method() without nova.tests.functional.libvirt.test_vgpu.VGPUTests.test_multiple_instance_create_filling_up_capacity ? | |
| 10:13:02 | bauzas | Yumeng: ack, will do | |
| 10:13:23 | gibi | bauzas: yes a single test method nova.tests.functional.libvirt.test_vgpu.VGPUTests.test_multiple_instance_create [7.793940s] ... FAILED | |
| 10:13:31 | bauzas | gibi: if so, the race is probably because we need to hold for a few things | |
| 10:14:05 | gibi | bauzas: yeah it is most likele a missing wait_for_<something> at some point in the test | |
| 10:14:07 | bauzas | gibi: lemme see if adding a pause of 1 sec will just solve the problem | |
| 10:14:18 | gibi | ack | |
| 10:14:24 | bauzas | gibi: because what's failing is that we lookup the mdevs | |
| 10:14:34 | bauzas | gibi: we should potentially wait a bit | |
| 10:15:02 | bauzas | this being said, we wait for the ACTIVE status | |
| 10:15:22 | bauzas | oh wait | |
| 10:15:30 | bauzas | this is a multiple instance creation | |
| 10:15:43 | bauzas | so we only check the first instance creation | |
| 10:16:05 | gibi | that could explain it | |
| 10:16:19 | bauzas | lemme try | |
| 10:16:38 | bauzas | --until-failure allows me to run the test easily | |
| 10:18:07 | bauzas | ahah, interesting | |
| 10:18:11 | bauzas | we don't hold | |
| 10:26:21 | bauzas | gibi: okay, changed my test to wait for all servers, running it | |
| 10:26:28 | gibi | cool | |
| 10:26:28 | bauzas | gibi: will leave it run for one hour or so | |
| 10:26:44 | gibi | if you push the patch up I can do the same while I have lunch | |
| 10:26:54 | bauzas | and if I no longer see any issues after, say, 200 runs, then we will know it's fixed | |
| 10:27:01 | bauzas | k | |
| 10:27:28 | bauzas | gibi: just writing a bug and uploading then | |
| 10:28:09 | gibi | ack | |
| 10:29:36 | bauzas | ah shit, they started it :p | |
| 10:30:44 | bauzas | https://bugs.launchpad.net/nova/+bug/1877281 | |
| 10:30:44 | openstack | Launchpad bug 1877281 in OpenStack Compute (nova) "vGPU multiple instance creation test is racey" [High,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza) | |
| 10:31:27 | bauzas | gibi: worth thinking about RC3 for this one ? :( | |
| 10:31:50 | bauzas | or, maybe we just release and then backport later | |
| 10:31:59 | bauzas | the issue isn't that bad | |
| 10:32:12 | jazeltq | hello gibi | |
| 10:32:41 | jazeltq | I have one question about nova-scheduler | |
| 10:35:56 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Wait for all servers to be active when testing vGPUs https://review.opendev.org/726087 | |
| 10:36:19 | bauzas | gibi: ^ | |
| 10:37:01 | bauzas | gibi: FYI (functional) [sbauza@sbauza nova]$ stestr --test-path=./nova/tests/functional run --until-failure nova.tests.functional.libvirt.test_vgpu.VGPUTests.test_multiple_instance_create | |
| 10:42:21 | stephenfin | melwitt: When you're about, can you take a look at https://review.opendev.org/#/c/677444/ please? | |
| 10:48:39 | gibi | bauzas: I don't think we need an RC for a racy func test. It is OK to merge it after GA | |
| 12:00:36 | bauzas | gibi: okay, after 4560 seconds, no errors were found | |
| 12:00:57 | gibi | bauzas: no error seen in 412 runs | |
| 12:01:02 | gibi | so I agree you fixed it! | |
| 12:01:28 | gibi | nice work | |
| 12:27:54 | efried | stephenfin: go ahead and update that commit message. I'm here to re+1, for all the good it'll do ya. | |
| 12:35:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: config: Explicitly register 'remote_debug' CLI opts https://review.opendev.org/677444 | |
| 12:35:31 | bauzas | stephenfin: https://review.opendev.org/#/c/726087/1 closing https://bugs.launchpad.net/nova/+bug/1877281 | |
| 12:35:31 | openstack | Launchpad bug 1877281 in OpenStack Compute (nova) "vGPU multiple instance creation test is racey" [High,In progress] - Assigned to Sylvain Bauza (sylvain-bauza) | |
| 12:35:32 | stephenfin | efried: ack, thanks | |
| 12:35:39 | stephenfin | bauzas++ | |
| 12:37:55 | efried | stephenfin: done (with bonus) | |
| 12:39:48 | stephenfin | bauzas: I have an ask on https://review.opendev.org/#/c/726087/ | |
| 12:40:01 | stephenfin | if you don't want to, I can live with it as-is | |
| 12:43:59 | bauzas | stephenfin: no worries, I can write a new rev | |
| 12:44:06 | bauzas | there is no urgency to fix this race | |
| 12:44:18 | bauzas | we only get it for around 0.5% | |
| 12:45:55 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Wait for all servers to be active when testing vGPUs https://review.opendev.org/726087 | |
| 12:46:10 | bauzas | stephenfin: fer your plizure ^ | |
| 12:46:47 | stephenfin | +" | |
| 12:46:49 | stephenfin | *2 | |
| 13:18:57 | francoisp | bauzas, salut, when you have time: https://review.opendev.org/#/c/725272/ (this time for Rocky) - thanks | |
| 14:27:17 | bauzas | francoisp: ack, will try to do it asap | |
| 14:27:43 | francoisp | bauzas, thank you! | |
| 14:32:53 | openstackgerrit | Jiri Suchomel proposed openstack/nova-specs master: Add spec for downloading images via RBD https://review.opendev.org/572805 | |
| 14:36:19 | openstackgerrit | Takashi Kajinami proposed openstack/nova master: Check py3 fix in python-barbicanclient https://review.opendev.org/726164 | |
| 14:57:54 | openstackgerrit | Sasha Andonov proposed openstack/nova master: rbd_utils: increase _destroy_volume timeout https://review.opendev.org/705764 | |
| 15:45:49 | gibi | nova meeting starts in 15 minutes on #openstack-meeting-3 | |
| 16:15:01 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Rewrite 'nova-manage db' docs to use proper directives https://review.opendev.org/677509 | |
| 16:15:01 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Use proper headers in nova-manage man page https://review.opendev.org/726191 | |
| 16:15:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Rewrite 'nova-manage api_db', 'cell_v2' docs https://review.opendev.org/726192 | |
| 16:15:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Rewrite 'nova-manage placement' docs https://review.opendev.org/726193 | |
| 16:15:03 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Only use section headers for HTML https://review.opendev.org/726194 | |
| 16:15:03 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Document options for nova-*proxy https://review.opendev.org/726195 | |
| 16:44:23 | openstackgerrit | Shi Yan proposed openstack/nova master: Fix aggregate placement sync issue https://review.opendev.org/726201 | |
| 16:47:07 | openstackgerrit | Merged openstack/os-vif master: Remove six https://review.opendev.org/724966 | |
| 16:47:08 | openstackgerrit | Merged openstack/os-vif master: Update master for stable/ussuri https://review.opendev.org/724714 | |
| 16:47:09 | openstackgerrit | Merged openstack/os-vif master: Add Python3 victoria unit tests https://review.opendev.org/724715 | |
| 17:01:56 | artom | gmann, continuing from #-meeting, yeah, sounds like more discussion would have been nice | |
| 17:02:26 | gmann | stephenfin: gibi py3.8 functional job as voting on check and gate pipeline - https://review.opendev.org/#/c/725372/ | |
| 17:02:27 | artom | I just... I don't get the opposition. Like, yeah, the name could have been better chosen, and yeah, we may want to improve it in the future | |
| 17:02:56 | artom | But as long as we document what it does for the former, and make it evolve-able for the latter, why not do it? | |
| 17:03:14 | stephenfin | gmann: Oh, it works now? I thought there were issues with mock from Python 3.8 | |
| 17:03:50 | melwitt | artom: well, I think we need to know whether it's extendable right? we don't fully understand that yet, do we? | |
| 17:04:07 | gmann | artom: i will say if old things have limit and need more clarity then documenting those is fine but any future thing and we can better design so better design is my preference than going with document-limits | |
| 17:04:44 | artom | gmann, agreed, but we were pre-empted by projects with lesser standards than ours ;) | |
| 17:05:08 | gmann | stephenfin: i think it worked. let me again check log that it is correctly doing py3.8 | |