| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2026-06-02 | |||
| 14:05:17 | chandankumar | Earlier in previous patch iteration, I went with cleaning and cleaning_failed status message but there are too many state. | |
| 14:05:23 | chandankumar | So I sticked with | |
| 14:05:25 | chandankumar | default maintaining state and cleanup_failed flag to keep the flow simpler. | |
| 14:05:36 | chandankumar | Do we want to add a cleaning state to know the exact state? and if cleanup failed, we will stick with maintaining? | |
| 14:05:37 | sean-k-mooney | so im not ok with reusing disabled for cleaning failed | |
| 14:05:51 | chandankumar | How do we want to handle that? | |
| 14:05:52 | sean-k-mooney | that was one of the feedack i gave on irc when you pushed the sepc initally | |
| 14:05:56 | amoralej | o/ | |
| 14:06:21 | chandankumar | sean-k-mooney: sorry I am not getting | |
| 14:06:51 | sean-k-mooney | i want to intoduce a sperate device_state filed which will taransation between avilable -> allcoated -> cleaning and hten to eitehr error or aviabel dependign on if cleanign succeeded | |
| 14:07:34 | chandankumar | ok | |
| 14:07:49 | sean-k-mooney | we also need to set reserved=total durign arq bind not durign unbind | |
| 14:08:18 | sean-k-mooney | so on bind we woudl transiation the device to allcoated and set reserved=total | |
| 14:08:55 | sean-k-mooney | on unbind it moved to cleaning keeping reserved=total | |
| 14:09:14 | chandankumar | above device state sounds good, I was focus earloer on cleanup | |
| 14:09:16 | sean-k-mooney | and ether end in error (reserved=total) or aviable (reseting reserved=0) | |
| 14:10:01 | chandankumar | yes that approach seems better | |
| 14:10:33 | chandankumar | One more thing, Does operator can toggle the device state manually? once clenaup finishes successfully | |
| 14:10:42 | sean-k-mooney | im also wondering if we want to add a /device/<uuid>/clean endpoint ot manually trigger cleaning | |
| 14:10:51 | sean-k-mooney | that would be an admin operator similar to program | |
| 14:11:03 | sean-k-mooney | chandankumar: no | |
| 14:11:07 | sean-k-mooney | the admin cannot | |
| 14:11:22 | sean-k-mooney | if we add clean as a deivce action | |
| 14:11:30 | sean-k-mooney | that is how they would recover it | |
| 14:11:47 | sean-k-mooney | we coudl perhaps consider a way to force it | |
| 14:12:02 | jgilaber | that endpoint would only work for devices that are in error? | |
| 14:12:04 | sean-k-mooney | but this si really internal stant that an admin shoudl not have to change in a normal workflow | |
| 14:12:31 | sean-k-mooney | jgilaber: i would say error or aviableale woudl be ok | |
| 14:12:47 | sean-k-mooney | but a 409 for allcoated/cleaning | |
| 14:13:06 | jgilaber | right, available would be fine, but redundant | |
| 14:13:17 | sean-k-mooney | we coudl dicusssi fi cleanign an allcoated fpga before repogrammign it shoudl eb allowed or not in the spec | |
| 14:14:12 | sean-k-mooney | the other approch here for erroed device woudl eb a cyborg-manage command or simialr for operator to manualy update the state | |
| 14:14:28 | sean-k-mooney | im a little reluctant to mirror nova's reset-state api | |
| 14:14:31 | jgilaber | that is currently proposed in the spec, right chandankumar? | |
| 14:14:48 | sean-k-mooney | nova and cinder allows admin to reset the state of instnace after you have fixed the instance/volume | |
| 14:15:12 | chandankumar | currently If a nvme device with cleanup_failed to true, we need to run cleanup manually | |
| 14:15:23 | sean-k-mooney | but we have had a lot fo issues with that in the past with customers or support causing more damabge by using it then if we didnt provide it | |
| 14:16:12 | sean-k-mooney | chandankumar: jgilaber lets follow up in the spec on the exact mechanics | |
| 14:16:30 | sean-k-mooney | i do want to capature both the happy and error paths and ensure we have a documented workflow for both | |
| 14:17:44 | jgilaber | ack | |
| 14:17:53 | chandankumar | I alsoanual recovery:** Operators retry failed cleanups using | |
| 14:17:55 | chandankumar | ``cyborg-nvme-cleanup --device <uuid>`` (requires admin credentials). The CLI | |
| 14:17:57 | chandankumar | tool re-triggers cleanup and resets ``cleanup_failed=False`` on success. | |
| 14:18:06 | chandankumar | that I have proposed in the spec | |
| 14:18:31 | chandankumar | anyway let me address the current comments and new design based on above discussion | |
| 14:18:38 | chandankumar | we can follow up on spec | |
| 14:19:41 | sean-k-mooney | im not really a fan of provideing a cil for the cleaning | |
| 14:20:10 | sean-k-mooney | but ack i have some pednign coeemt form when i first lookd but i stoped at the problem description after my inital skim pass | |
| 14:20:30 | sean-k-mooney | ill review it in detail this week ideally today or tomrorow | |
| 14:20:54 | sean-k-mooney | to be clear im not really a fan of having any dirver speciric clis | |
| 14:21:18 | chandankumar | sean-k-mooney: I will ping you tomorrow for review once I update it based on above design, | |
| 14:22:02 | sean-k-mooney | cyborgs core role is too provied a hardware indepent common api over the acclerator it manges so you shoudl nto need nvme specific clis but a generic way to triger cleaning or programmign is ok | |
| 14:22:43 | sean-k-mooney | ack | |
| 14:23:48 | jgilaber | ack, is that all for this topic? | |
| 14:23:49 | chandankumar | sure | |
| 14:23:54 | chandankumar | thank you jgilaber sean-k-mooney ! | |
| 14:24:10 | jgilaber | thanks Chandan, let's move to reviews | |
| 14:24:17 | jgilaber | #topic Reviews | |
| 14:24:23 | jgilaber | we have one | |
| 14:24:29 | jgilaber | #link https://review.opendev.org/c/openstack/cyborg/+/991027 | |
| 14:24:37 | jgilaber | with tempest tests https://review.opendev.org/c/openstack/cyborg-tempest-plugin/+/991081 | |
| 14:24:56 | sean-k-mooney | i have 2 to add later | |
| 14:24:57 | chandankumar | I was working on dropping the image verification code for device program functionality | |
| 14:25:52 | sean-k-mooney | so the fake driver already supprot program | |
| 14:25:53 | chandankumar | It is dropping the code and marking existing verify_glance_signatures config as deprecated | |
| 14:26:00 | sean-k-mooney | https://review.opendev.org/c/openstack/cyborg/+/991027/2/cyborg/accelerator/drivers/fake.py#127 | |
| 14:26:03 | chandankumar | it has a update method not program one | |
| 14:26:04 | sean-k-mooney | that what update is | |
| 14:26:14 | chandankumar | will I rename that? | |
| 14:26:17 | sean-k-mooney | no update is what prgram is called in teh drier interface | |
| 14:26:24 | chandankumar | ok | |
| 14:26:36 | sean-k-mooney | so that a sperat question | |
| 14:26:45 | sean-k-mooney | we could perhas rename it but that is a large change | |
| 14:26:56 | sean-k-mooney | sicne it woudl be changing the public api of the drivers | |
| 14:27:02 | chandankumar | program interface is currently used in fpga driver only not in generic one | |
| 14:27:16 | sean-k-mooney | again that is but true and untrue | |
| 14:27:41 | sean-k-mooney | so form an api perspective we shoudl not have any driver specifc apis | |
| 14:28:11 | sean-k-mooney | so its more correct to say that the update api si a noop for driver other then the fpg driver | |
| 14:28:24 | sean-k-mooney | rahter then teh progarm api is only for fpga | |
| 14:28:43 | sean-k-mooney | the ablity to program or update a device with a glance image | |
| 14:29:05 | sean-k-mooney | is a genereic capablity that is not used by other drivers | |
| 14:29:18 | sean-k-mooney | but the nvme driver coudl supprot it as we have dicussed in the past | |
| 14:29:24 | chandankumar | yes | |
| 14:30:27 | chandankumar | https://github.com/openstack/cyborg/blob/master/cyborg/accelerator/drivers/driver.py#L27 with respect to current patch, How we do want to proceed? Reusing update at all places a seperate patch and calling update flow via profram cli? | |
| 14:30:30 | jgilaber | I'm looking quickly but the program api can't call the fake driver right? | |
| 14:30:42 | chandankumar | jgilaber: yes | |
| 14:31:28 | sean-k-mooney | so the current patch is incorrect | |
| 14:31:30 | sean-k-mooney | https://github.com/openstack/cyborg/blob/master/cyborg/accelerator/drivers/driver.py#L26-L35 | |
| 14:31:31 | jgilaber | and is there any endpoint for update? | |
| 14:31:38 | sean-k-mooney | udpate is part of the base driver interface | |
| 14:31:44 | sean-k-mooney | and that is how programing shoudl be invoked | |
| 14:32:09 | sean-k-mooney | at least that is muy current understandign but ill need to review that in more detail to confirm | |
| 14:32:19 | chandankumar | there is no endpoint for update, we currently invoke program https://bugs.launchpad.net/openstack-cyborg/+bug/2144308/comments/1 | |
| 14:32:29 | jgilaber | so the fpga implemented it incorrectly? | |
| 14:32:41 | sean-k-mooney | so there are 2 ways to program the ptgs | |
| 14:32:42 | chandankumar | $CYBORG_URL/deployables/$DEPLOYABLE_UUID/program - was the api interface | |
| 14:32:54 | sean-k-mooney | you an refence the image via the device profile | |
| 14:33:01 | sean-k-mooney | or you can manually do it via the deployable api | |
| 14:33:54 | jgilaber | but to test chandankumar's patch we need to call the api iirc | |
| 14:34:03 | sean-k-mooney | if we look at progrma its curenly using PATCH which i want to fix in the future but PATCH is an "update" the same way put is | |