| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2026-07-17 | |||
| 11:51:47 | chandankumar | https://zuul.opendev.org/t/openstack/builds?change=997365&result=FAILURE&skip=0 , one time py3 and ipv6 failed job ran on rax-ord-main . | |
| 11:52:05 | chandankumar | yesterday | |
| 11:53:54 | chandankumar | https://zuul.opendev.org/t/openstack/builds?job_name=cyborg-tempest-ipv6&change=997365&skip=0 - rest of the passing ipv6 ran on different provider. | |
| 11:54:13 | chandankumar | I am not sure it is linked with provider also. | |
| 12:00:09 | opendevreview | Takashi Kajinami proposed openstack/cyborg master: Add cors middleware defaults https://review.opendev.org/c/openstack/cyborg/+/997712 | |
| 12:00:25 | jgilaber | today it's been running in different providers, will recheck a few more times to see | |
| 13:11:02 | opendevreview | Takashi Kajinami proposed openstack/cyborg master: Add cors middleware defaults https://review.opendev.org/c/openstack/cyborg/+/997712 | |
| 13:31:16 | jgilaber | chandankumar, I think you might be onto something, the ipv6 failed now and it ran on rax-ord-main https://zuul.opendev.org/t/openstack/build/87e90ed8ebbc4b7cad73c3d69d76c6a0 | |
| 13:43:35 | sean-k-mooney | jgilaber: it could be timeing related but i think the lspci approch is going to more robust anyway | |
| 13:43:52 | sean-k-mooney | so ya if its still stable ill approve it in an hour or two | |
| 13:44:34 | sean-k-mooney | i was surpsied when you menteiond it sicne i had never seen it fail personally before | |
| 13:45:16 | sean-k-mooney | but again the ci can show issue we dont see on our local system so im not that surpised either | |
| 13:45:30 | jgilaber | yes, it did seem stable before that point | |
| 13:45:52 | jgilaber | it's super stable in my dev env as well, could not get it to fail there | |
| 13:47:10 | chandankumar | Do we want to add some wait condition https://github.com/openstack/cyborg-tempest-plugin/blob/master/cyborg_tempest_plugin/tests/scenario/test_pci_driver_lifecycle.py#L194 after reboot to make sure server is ip and call _assert_server_pci_ready ? | |
| 13:47:48 | chandankumar | from multiple runs, it is failing on rax-ord-main provider only | |
| 14:10:56 | sean-k-mooney | we shodul rewrite that test i think | |
| 14:11:09 | sean-k-mooney | well | |
| 14:11:33 | sean-k-mooney | we can keep it but yes we need a retry loop | |
| 14:11:53 | sean-k-mooney | i was thinkin gif we wanted to delete that test and just rely on teh nova api reboot tests | |
| 14:12:59 | sean-k-mooney | but for the gues one we need to wait for the vm to be back up before calling _assert_server_pci_ready | |
| 14:13:53 | sean-k-mooney | if we really want this to be more roboust https://github.com/openstack/cyborg-tempest-plugin/blob/master/cyborg_tempest_plugin/tests/scenario/test_pci_driver_lifecycle.py#L161 shoudl be SSHABLE instead of active as well | |
| 14:15:24 | sean-k-mooney | since we are going to try adn ssh into it waiting for SSHABLE which is a tempest concept is more accurate as ACTIVE just means nova thinks its running | |
| 14:15:39 | sean-k-mooney | it does nto mean the guest os has fully booted | |
| 14:52:43 | jgilaber | sean-k-mooney, sorry I had to step out for a bit | |
| 14:53:02 | sean-k-mooney | its fine | |
| 14:53:05 | jgilaber | I can change the state there for 'SSHABLE' but I don't think that is called in the test in question | |
| 14:53:22 | sean-k-mooney | lets revisit that | |
| 14:53:35 | sean-k-mooney | we can look at doign that speratly | |
| 14:53:58 | sean-k-mooney | beasue we likely shoudl do that consitently not just in those tests | |
| 14:54:07 | sean-k-mooney | also when waiting for the server to create | |
| 14:54:14 | sean-k-mooney | not just reboot ectra | |
| 14:54:46 | sean-k-mooney | jgilaber: are you happy enough to proceed with the currnt improment | |
| 14:55:19 | sean-k-mooney | if so i can appove the test change and we can then look at appoving the job change patch | |
| 14:55:41 | sean-k-mooney | ill be around for anouther hour or two then ill be finishing up | |
| 14:58:52 | jgilaber | we could go ahead, I think it is slightly better, although it does not completely solve the problem | |
| 14:59:59 | jgilaber | i.e if we enable the tests we might get gate failures which will be annoying | |
| 15:00:17 | sean-k-mooney | yes but i wont be here next week | |
| 15:00:35 | sean-k-mooney | so we wont be merging anything anyway and ye can tell me how it goes when i get back :P | |
| 15:00:45 | sean-k-mooney | so that sounds like a you problem to me :) | |
| 15:01:45 | jgilaber | yep, that's about right | |
| 15:01:51 | sean-k-mooney | on a seious not however i think i woudl prefer having the test run for a week even if they fail | |
| 15:02:04 | sean-k-mooney | just to actullly flesh out any opther issues with them | |
| 15:02:14 | jgilaber | we can go ahead since it won't be blocking anything and we can collect more data from other patches | |
| 15:47:41 | opendevreview | Merged openstack/cyborg-tempest-plugin master: Prefer lspci over sysfs for PCI device detection https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997684 | |
| 15:48:53 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Add Cyborg MDEV driver lifecycle scenario tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997352 | |
| 15:48:54 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Verify guest reboot started in PCI lifecycle test https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997751 | |
| 15:49:36 | jgilaber | I'm not convinced this will fix the problem but I'll try during the week | |
| 15:50:44 | jgilaber | sean-k-mooney, do you want to merge https://review.opendev.org/c/openstack/cyborg/+/997365 before you leave? | |
| 15:59:58 | sean-k-mooney | i will yes on a call | |
| 16:47:55 | sean-k-mooney | ok its reappoved | |
| 16:48:17 | sean-k-mooney | assuming it passes in gate that shoudl merged | |
| 16:48:47 | sean-k-mooney | im goign to finish up in the next 15 minutes or so | |
| 16:49:01 | sean-k-mooney | but i can take a look ifthere is anythin else pressing | |
| 16:49:29 | sean-k-mooney | i will have some conenctiviy while traveling next week but im not sure how much so i will check in when possible | |
| 16:59:36 | jgilaber | ack thanks don't worry | |
| 16:59:59 | jgilaber | I'll continue working on it | |
| 17:27:25 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Verify guest reboot started in PCI lifecycle test https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997751 | |
| 17:27:26 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Add Cyborg MDEV driver lifecycle scenario tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997352 | |
| 17:28:28 | sean-k-mooney | jgilaber: os tempest has some conenctivty checks that we coudl reuse | |
| 17:28:35 | sean-k-mooney | im not againt doing it our selves | |
| 17:28:42 | sean-k-mooney | but just pointing that out | |
| 17:29:16 | jgilaber | I'll look into that next week if the current version works | |
| 17:29:35 | jgilaber | can you add +2 to https://review.opendev.org/c/openstack/cyborg/+/997365? It looks like the gate pipeline has not started | |
| 17:29:59 | sean-k-mooney | oh lol | |
| 17:30:02 | sean-k-mooney | i tough i already had | |
| 17:30:12 | sean-k-mooney | done | |
| 17:31:12 | jgilaber | it got deleted in one of the updates | |
| 17:33:32 | sean-k-mooney | i left a comment on your patch but this is the helper i think you shoudl use https://github.com/openstack/tempest/blob/master/tempest/common/compute.py#L113-L153 | |
| 17:34:22 | sean-k-mooney | or use https://github.com/openstack/tempest/blob/master/tempest/common/waiters.py#L727-L736 or https://github.com/openstack/tempest/blob/master/tempest/common/waiters.py#L678-L686 | |
| 17:34:24 | sean-k-mooney | directly | |
| 17:35:57 | sean-k-mooney | wait_for_server_status dose not actully supporort SSHABLE as it not actully a nova status | |
| 17:36:45 | jgilaber | yes, I've changed that I saw it failed in CI | |
| 17:38:11 | jgilaber | I'm leaving know I'll change the patch on Monday to use the tempest functions, thanks for looking and safe travels next week! | |
| 17:38:50 | sean-k-mooney | no worries | |
| 18:34:46 | opendevreview | Merged openstack/cyborg master: Configure pci lifecycle tempest settings via devstack plugin https://review.opendev.org/c/openstack/cyborg/+/997365 | |
| 19:14:55 | opendevreview | melanie witt proposed openstack/cyborg master: Add Intel XXV710 25GbE backplane NIC to known NIC list https://review.opendev.org/c/openstack/cyborg/+/997782 | |
| #openstack-cyborg - 2026-07-20 | |||
| 07:28:41 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Refactor PCI lifecycle tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997936 | |
| 07:28:42 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Add Cyborg MDEV driver lifecycle scenario tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997937 | |
| 07:29:52 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Add Cyborg MDEV driver lifecycle scenario tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997352 | |
| 09:22:24 | opendevreview | chandan kumar proposed openstack/cyborg-tempest-plugin master: Add Cyborg-to-Placement integration scenario test https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/992210 | |
| 09:23:00 | opendevreview | chandan kumar proposed openstack/cyborg master: Allow non-CUSTOM traits in device profile groups https://review.opendev.org/c/openstack/cyborg/+/997945 | |
| 10:06:23 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Verify guest reboot started in PCI lifecycle test https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997751 | |
| 10:06:24 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Refactor PCI lifecycle tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997936 | |
| 10:06:24 | opendevreview | Joan Gilabert proposed openstack/cyborg-tempest-plugin master: Add Cyborg MDEV driver lifecycle scenario tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/997352 | |
| 10:11:49 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add MDEV and PCI device type constants https://review.opendev.org/c/openstack/cyborg/+/997047 | |
| 10:11:49 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add MdevBusManager for mdev bus discovery https://review.opendev.org/c/openstack/cyborg/+/997048 | |
| 10:11:50 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add [mdev] configuration section https://review.opendev.org/c/openstack/cyborg/+/997049 | |
| 10:11:50 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add generic mdev driver https://review.opendev.org/c/openstack/cyborg/+/997050 | |
| 10:11:51 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add duplicate device warning in conductor https://review.opendev.org/c/openstack/cyborg/+/997051 | |
| 10:11:51 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Refactor GPU driver to use MdevBusManager https://review.opendev.org/c/openstack/cyborg/+/997052 | |
| 10:11:52 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add generic mdev driver documentation https://review.opendev.org/c/openstack/cyborg/+/997053 | |
| 10:11:52 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add mdev support to pci-sim kernel module https://review.opendev.org/c/openstack/cyborg/+/997054 | |
| 10:11:54 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add mdev support to pci-sim DevStack plugin https://review.opendev.org/c/openstack/cyborg/+/997055 | |
| 10:11:54 | opendevreview | Joan Gilabert proposed openstack/cyborg master: [WIP] Configure mdev driver lifecycle tests https://review.opendev.org/c/openstack/cyborg/+/997364 | |
| 10:13:43 | opendevreview | Joan Gilabert proposed openstack/cyborg master: [WIP] Configure mdev driver lifecycle tests https://review.opendev.org/c/openstack/cyborg/+/997364 | |
| 12:17:15 | opendevreview | chandan kumar proposed openstack/cyborg-tempest-plugin master: Add Cyborg-to-Placement integration scenario test https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/992210 | |
| 12:17:34 | opendevreview | chandan kumar proposed openstack/cyborg master: Allow non-CUSTOM traits in device profile groups https://review.opendev.org/c/openstack/cyborg/+/997945 | |
| 12:39:16 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Enable database migration tests with mysql https://review.opendev.org/c/openstack/cyborg/+/997979 | |
| 12:50:22 | opendevreview | Joan Gilabert proposed openstack/cyborg master: Add MDEV and PCI device type constants https://review.opendev.org/c/openstack/cyborg/+/997047 | |