| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-29 | |||
| 13:35:39 | efried | Well, I'm not sure how we can stop 'em. | |
| 13:35:43 | edleafe | dansmith: cyborg can set all the traits they want. The problem is expecting Nova to act on them outside of the normal flow of placement/scheduler | |
| 13:35:53 | alex_xu_ | I only think about burn that image, and don't change it anymore... | |
| 13:35:53 | jaypipes | efried: by not encouraging it. | |
| 13:36:16 | edleafe | jaypipes: and not supporting it | |
| 13:36:17 | dansmith | edleafe: right, we control how much of nova tries to obsess over the traits being set | |
| 13:36:29 | jaypipes | traits != status changes | |
| 13:36:57 | efried | jaypipes: Would you mind expostulating a bit on the reasoning behind that in the ML thread? | |
| 13:37:18 | efried | jaypipes: "[nova] [cyborg] Race condition in the Cyborg/Nova flow" | |
| 13:37:46 | efried | jaypipes: Though it seems like this conversation happens at least once a quarter, so maybe a permadoc somewhere would be expedient. | |
| 13:37:56 | jaypipes | alex_xu_: I'm fine with a trait that means "this FPGA is capable of doing X". What I don't want to see is a precedence set for traits that mean "this foo is on at this moment or off at this moment" and have that trait be used by virt drivers or external agents as a poor-man's servicegroup API. | |
| 13:38:40 | alex_xu_ | jaypipes: I see that, I think we can use trait for the pre-programmed fpga device | |
| 13:38:41 | jaypipes | efried: yes, I will comment on the ML thread. not sure about expostulating, though. | |
| 13:38:51 | edleafe | efried: another problem is we spent a lot of time in Dublin discussing these things and agreeing on boundaries. The current discussion seems to ignore all of that | |
| 13:38:57 | alex_xu_ | and pre-programmed fpga device doesn't allow to re-program | |
| 13:39:10 | alex_xu_ | jaypipes: in that case, does make sense? | |
| 13:39:11 | dansmith | alex_xu_: jaypipes: let me restate what I think is a legit use, again using cyborg | |
| 13:39:16 | edleafe | alex_xu_: pre-programmed functions are resource classes | |
| 13:39:17 | efried | edleafe: How ignoring? I've been saying it ain't gonna fly. | |
| 13:39:23 | edleafe | efried: not you | |
| 13:39:32 | edleafe | sundar primarily | |
| 13:39:37 | alex_xu_ | edleafe: I also think about that... | |
| 13:39:57 | alex_xu_ | pre-programmed is normal accelerator device... | |
| 13:40:01 | dansmith | I think it's legit for blazar to pre-provision images into FPGAs via cyborg according to upcoming schedule, set traits on them for the image, and de-provision/trait them at the end of the 24-hour reservation window | |
| 13:40:14 | edleafe | if it's programmed for X and can't be changed, then it has an inventory on that function. That's a resource class | |
| 13:40:35 | jaypipes | hold up, let dansmith finish his synopsis for a sec | |
| 13:40:59 | dansmith | that's it.. just another example of pre-provisioned images getting traits so we can schedule to them by function | |
| 13:41:10 | jaypipes | ok | |
| 13:41:22 | edleafe | dansmith: that sounds like using traits as resource class proxies | |
| 13:41:25 | alex_xu_ | dansmith: what's different with cyborg cache manager... | |
| 13:41:33 | jaypipes | and frankly, that's what I *thought* the agreement was in Dublin, but granted, I only made the tail end of the discussion. | |
| 13:41:34 | dansmith | I don't think that programming in a dynamic device needs to be resource class | |
| 13:41:46 | dansmith | jaypipes: same | |
| 13:42:04 | dansmith | alex_xu_: it's not, as I said above in the earlier example | |
| 13:42:58 | jaypipes | dansmith: it *could* be done via a custom resource class, but then again, I find a nice simplicity in just having a standard resource class called FPGA_CONTEXT (or something like that) and decorating those pre-programmed slots (providers) with particular traits | |
| 13:43:10 | dansmith | sure it could, I said "need" | |
| 13:43:11 | efried | edleafe: The RC-vs-trait discussion is a tangent. The issue is whether/when it's okay to modify RPs "on the fly". And how on-the-fly is "on the fly"? | |
| 13:43:23 | dansmith | jaypipes: it needs to be class per type of device, but yeah | |
| 13:43:49 | edleafe | efried: I don't agree it's a tangent. If you model your resources correctly, the whole discussion goes away | |
| 13:44:05 | dansmith | efried: if you have a preferred trait of HAS_SSL, you get one without it, you claim it in scheduling, it's cool IMHO for cyborg to trait it up when it does the programming | |
| 13:44:26 | alex_xu_ | dansmith: I must misunderstand this words "it's not great, and it could certainly be abused past my tolerance level," It means yes... or no.. | |
| 13:44:51 | dansmith | alex_xu_: that was referring to people doing other things, not this | |
| 13:44:51 | efried | dansmith: That would be cool. jaypipes are you on board with that? | |
| 13:45:39 | jaypipes | I'm cool with what dansmith just said, yes. | |
| 13:45:41 | edleafe | dansmith: that's a totally legit use case | |
| 13:45:48 | jaypipes | here are the things I do *not* want to see: | |
| 13:46:04 | jaypipes | 1) agents, including the virt driver, using the traits API as a servicegroup/status API. | |
| 13:46:05 | dansmith | alex_xu_: sorry I see why that was confusing I mixed that into other thoughts when I re-read | |
| 13:46:21 | jaypipes | 2) *Consumer requests* causing traits to be set against a provider. | |
| 13:47:04 | dansmith | jaypipes: 2 meaning something like nova setting traits on providers of has-instances or has-10-instances, or something like that yeah? | |
| 13:47:26 | dansmith | or has-some-volumes-attached-to-host-right-now-yo | |
| 13:47:55 | mriedem | bottom 6 patches in the placement-req-filter series have a +2 https://review.openstack.org/#/c/544730/ | |
| 13:47:57 | jaypipes | It is that latter point that I am afraid of, since it means we essentially throw out all the scheduling heuristics that are now made in placement (since we would no longer be able to filter on providers having a trait but would need to add some mechanism to detect providers that *could have* a trait on them, if certain conditions apply) | |
| 13:48:08 | efried | jaypipes: #2 is a bit sticky, though, innit? The consumer requests an instance with FPGA with function X, cyborg finds a previously-unfettered device suitable for use and flashes it with function X, then sets a trait saying it's got function X on it now. Is that "consumer requests causing traits to be set"? | |
| 13:48:08 | openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Proposes Multiple GPU types https://review.openstack.org/557065 | |
| 13:48:12 | dansmith | jaypipes: yup | |
| 13:48:29 | dansmith | efried: it has given that device capability | |
| 13:48:48 | bauzas | jaypipes: dansmith: very simple spec about providing a config option for defining a list of pGPUs a specific type would be supported https://review.openstack.org/557065 | |
| 13:48:51 | dansmith | it has changed the device in some meaningful way that means it can do things | |
| 13:48:58 | efried | dansmith: tbc, *I* am fine with that scenario. I'm asking if it violates jaypipes' #2 | |
| 13:49:12 | jaypipes | efried: it would, yes. | |
| 13:49:16 | bauzas | dansmith: jaypipes: if you can review it quickly, that would unblock me from using n-r-p for libvirt vGPUs | |
| 13:49:27 | dansmith | jaypipes: o.O | |
| 13:50:02 | bauzas | mriedem: dansmith: I'll sponsor https://review.openstack.org/#/c/544730/ and above | |
| 13:50:14 | dansmith | jaypipes: now I'm confused :) | |
| 13:50:16 | jaypipes | efried: if cyborg configures its inventory/traits dynamically *at the time and due to a specific request from a consumer*, I have an issue with that | |
| 13:50:35 | dansmith | jaypipes: heh, that's what you just said you were okay with | |
| 13:50:45 | mriedem | alex_xu_: interesting idea to deprecate the backup API in https://review.openstack.org/#/c/511825/ - you should probably start a ML thread on that if you want wider input | |
| 13:50:46 | jaypipes | dansmith: it's a difference in timing. | |
| 13:51:06 | dansmith | jaypipes: no, I said above "sets the trait when programming" | |
| 13:51:19 | alex_xu_ | mriedem: will do that later | |
| 13:51:22 | jaypipes | dansmith: when is the programming done. | |
| 13:51:44 | jaypipes | dansmith: that was a question, sorry.. when is the programming of the slots done? | |
| 13:52:03 | jaypipes | dansmith: if it's done after GET /allocation_candidates, then that won't work. | |
| 13:52:13 | dansmith | jaypipes: sure it will | |
| 13:52:23 | jaypipes | dansmith: since clearly the trait won't be useful in filtering out destination providers. | |
| 13:52:29 | alex_xu_ | jaypipes: dansmith note, for programmable fpga, it inventory always 1 | |
| 13:52:32 | dansmith | jaypipes: it would be preferred | |
| 13:52:56 | alex_xu_ | there is no race like we tweak traits in the fly for gpu | |
| 13:53:07 | dansmith | jaypipes: hangout? | |
| 13:53:11 | jaypipes | dansmith: sdure | |
| 13:53:12 | jaypipes | sure | |
| 13:53:20 | jaypipes | set it up. I will coffee up. | |
| 13:53:35 | jaypipes | let me grab a pug for emotional support. | |
| 13:53:39 | dansmith | https://hangouts.google.com/call/vBep88lVuQCIThVzSqUZAAEE | |
| 13:54:06 | mriedem | powervm interface attach/detach bp patch needs another +2 https://review.openstack.org/#/c/546813/ | |
| 13:54:15 | dansmith | e'yone's invited of course | |
| 13:56:43 | openstackgerrit | Claudiu Belu proposed openstack/nova master: tests: autospecs all the mock.patch usages https://review.openstack.org/470775 | |
| 13:59:06 | bauzas | dansmith: just for sanity, left some comments on https://review.openstack.org/#/c/544730/16 but +W'd it | |
| 14:10:26 | openstackgerrit | Claudiu Belu proposed openstack/nova master: tests: autospecs all the mock.patch usages https://review.openstack.org/470775 | |
| 14:10:27 | openstackgerrit | Claudiu Belu proposed openstack/nova master: WIP: replace spec with autospec https://review.openstack.org/557299 | |
| 14:11:28 | mriedem | bhagyashris: at least one more update in here i think https://review.openstack.org/#/c/511825/ | |
| 14:17:26 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Fix bad management of _TRAITS_SYNCED flag https://review.openstack.org/557722 | |
| 14:17:33 | cdent | mriedem: that ^ fixes it in my tests | |
| 14:18:34 | mriedem | let's recheck it a few times | |
| 14:20:02 | bhagyashris | mriedem: ok thank you for review. you mean to depricate the create backup api and newly introduce deleteBackup api will delete the backup using the rotation parameter | |
| 14:21:16 | mriedem | claudiub: can you update this quick? https://review.openstack.org/#/c/557377/ | |
| 14:21:46 | mriedem | bhagyashris: no, the alternative is to simply deprecate the createBackup API, and not introduce a new deleteBackups API | |
| 14:22:07 | mriedem | since createBackup can all be done using orchestration of existing nova/glance APIs | |
| 14:22:33 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Filter resource providers by forbidden traits in db https://review.openstack.org/556472 | |