| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-11-13 | |||
| 12:51:47 | jkulik | Would that be acceptable, I mean? | |
| 12:51:52 | sean-k-mooney | honestly i dont think so | |
| 12:52:46 | jkulik | so we would be left with trying to maintain an out-of-tree driver, which is not supported, and keeping all the to-be-removed code in sync with upstream in our downstream fork ... :/ | |
| 12:52:48 | sean-k-mooney | at this point i think we woudl neeed to see more then a token effort | |
| 12:53:01 | sean-k-mooney | jkulik: out of tree dirvers wont work | |
| 12:53:08 | jkulik | yeah, I read that | |
| 12:53:35 | stephenfin | sean-k-mooney: I don't see any reason why another third party couldn't maintain it. As you say though, it'd have to be more than a token effort | |
| 12:53:40 | sean-k-mooney | the core of vmware is opensouce right (EXSI( | |
| 12:54:05 | sean-k-mooney | stephenfin: someone else could | |
| 12:54:25 | jkulik | no, ESXi is not open source. not sure if that was sarcasm | |
| 12:54:42 | sean-k-mooney | i tought ther ewas an opensouce core | |
| 12:54:45 | sean-k-mooney | exi is not | |
| 12:54:53 | sean-k-mooney | but isnt esxi based on linux | |
| 12:55:02 | jkulik | I haven't heard of any. The things we use are not open source. | |
| 12:55:24 | jkulik | what would "more than a token effort" entail? | |
| 12:56:20 | sean-k-mooney | actully continutigi to the maintaicne of vmaware as well as first/tridpary ci | |
| 12:56:48 | sean-k-mooney | jkulik: i might of been thinking fo the fact there is a free version of esxi | |
| 12:57:07 | sean-k-mooney | its not opensouce but there are free lisnces | |
| 12:57:19 | stephenfin | At a minimum, CI coverage. Practically speaking though, there's mountains of tech debt in the VMWare driver that needs addressing if we're keeping it around long-term | |
| 12:57:30 | sean-k-mooney | basically i was considing if instead of third party ci we coudl do this first party somehow | |
| 12:58:37 | sean-k-mooney | right there is a lot of techdebt( lack of functional tests, old apis/deps) and large feature gaps | |
| 12:59:05 | jkulik | sounds like it boils down to us (SAP) working more with upstream, cleaning the driver and by that recognizing CI going missing, if we want that driver to stay | |
| 12:59:09 | stephenfin | that cost compares to maintaining an out-of-tree driver would be interesting to study | |
| 12:59:09 | stephenfin | jkulik: Really it would be a near-full-time job. The obvious answer here is that SAP put forward someone to maintain CI and work to get your downstream fork synced back upstream (or at least work to get equivalent functionality added upstream) but I don't know how SAP are for upstream development. It's definitely something that has to developed and nurtured be in the org/company culture. There's also a cost to it, though how | |
| 12:59:17 | stephenfin | jkulik: exactly | |
| 13:02:56 | jkulik | we're open to upstreaming, but given that we need the features downstream first, they often don't end up upstream due to time constraints and us not actively including ourselves in the upstream community. I feel that pain every time when we port our patches to a newer release. I'll bring that topic up internally with management. | |
| 13:03:00 | jkulik | thank you both | |
| 13:05:40 | stephenfin | jkulik: np. I'll also add that we're generally very open to adding additional cores, particularly when those cores are focused on a particular area and have competency there, but obviously that competency needs to be demonstrated for some time first | |
| 13:06:00 | jkulik | :+1: | |
| 13:06:30 | sean-k-mooney | well keep in mind the job of a core is primarly to review code not write it | |
| 13:07:30 | sean-k-mooney | so while stephenfin is correct we would be open to adding cores the copendncy we woudl use to make that determination is there review output not there code output | |
| 13:08:07 | jkulik | review output in Nova in general, I guess, not only the vmwareapi driver then? As I guess, that one doesn't see too many patches. | |
| 13:09:17 | sean-k-mooney | sortof both. in general in the past we have expect a general knowladge of the entire codebase but most core have areas or specialisation where they spend most of there time | |
| 13:10:16 | stephenfin | That would be an interesting discussion to have as a community. As you say, that area doesn't see too many patches and there is very little expertise in that area among existing cores. Ensuring we had decent review throughput would be an interesting thing to figure out | |
| 13:10:45 | sean-k-mooney | well part of the expction would be that they would enforece the same feature design crita as we do for the rest of nova | |
| 13:11:06 | sean-k-mooney | in the past vmware have added funcitonlity in the out of tree drive and then tried to upstream it | |
| 13:11:34 | sean-k-mooney | or made changes that normaly woudl requrie spec or blueprints that got little or no discussion | |
| 13:12:08 | sean-k-mooney | so if we were to add cores spcialised in vmware i woudl expoect the process to addign vmware features to be the saem as libvirt/ironic | |
| 13:12:30 | jkulik | hm ... yeah, I'd expect upstream to work like upstream not like downstream :D | |
| 13:12:57 | sean-k-mooney | the problem we have had historically with driver that have out of tree vairiant | |
| 13:13:21 | sean-k-mooney | is they and just enouch funcitonality in the intree driver to enabel them to create the feature they want to have in teh out of tree one | |
| 13:14:03 | jkulik | hm ... having a lot of patches downstream, I can see the point. the VMware driver manages a whole cluster, so things might work quite differently than in the reest of the drivers / Nova in general | |
| 13:14:19 | sean-k-mooney | i was plannign to bring this up next cycle but once the vmware/hyperv dirers were removed i was gong to propose removoing the ablit y to have out of tree dirvers entirly | |
| 13:15:04 | sean-k-mooney | jkulik: we dont offically support out of tree driers in nova but we dont explictly block it either | |
| 13:15:36 | jkulik | I mean, we're having a fork basically, so our patches are not necessarily "out of tree" - they are on top of tree :D | |
| 13:16:00 | stephenfin | yeah, that's how cloudbase were handling things with the hyper-v driver | |
| 13:16:57 | stephenfin | through a combination of (I suspect) lack of upstream review throughput but also lack of investment in the time needed to drive consensus on larger design changes | |
| 13:17:37 | jkulik | yeah, same here. whenever we try to upstream common code (as a basis or start to upstream more), those reviews hang around for a long time. | |
| 13:17:50 | stephenfin | the zVM driver takes a different tack, where the driver is essentially a shim over an external library where all the "real" functionality lives | |
| 13:18:05 | jkulik | since there's noone invovled with the vmware driver upstream, that's fully understandable, but doesn't help in reducing the technical debt either | |
| 13:18:42 | stephenfin | jkulik: then that would be on us. If someone were to start maintaining a CI and the codebase it would be up to us to ensure things aren't hanging around for months | |
| 13:20:32 | jkulik | point being no CI anymore for nearly 2 years. got it. | |
| 13:20:45 | sean-k-mooney | ya so zvm is interesting because i also have not see it run as a third party ci | |
| 13:21:05 | sean-k-mooney | but we did nto mark it as experimental so we need to give them time to correct taht | |
| 13:21:17 | sean-k-mooney | the IBM PowerKVM CI does not test zvm | |
| 13:21:26 | sean-k-mooney | its thesting livbirt on power | |
| 13:22:53 | sean-k-mooney | that siad it may only run wehn you make chagnes to the zvm dirver | |
| 13:23:09 | sean-k-mooney | so we need to submit a test patch and determin if its still functional | |
| 13:25:09 | jkulik | Does it make sense to submit monthly patches just to check the different CIs still running? | |
| 13:25:43 | stephenfin | CIs can be configured to run periodic jobs. That would probably be a better approach | |
| 13:26:00 | stephenfin | and publish the URL where those reports can be found | |
| 13:50:47 | tkajinam | o/ May I ask for reviews to merge https://review.opendev.org/c/openstack/nova/+/900538 to make sqlalchemy-2.x job voting again ? | |
| 14:22:21 | zigo | Also, would be cool to have this one merged: https://review.opendev.org/c/openstack/oslo.cache/+/900158 (ie: oslo.cache leaving sockets to memcached in CLOSE_WAIT state...) | |
| 14:26:48 | opendevreview | Danylo Vodopianov proposed openstack/nova master: Packed virtqueue support was added. https://review.opendev.org/c/openstack/nova/+/876075 | |
| 14:32:02 | sean-k-mooney | zigo: wrong channel | |
| 14:32:29 | zigo | ok :) | |
| 14:32:35 | sean-k-mooney | im not sure about hte while true: | |
| 14:32:43 | zigo | I am. | |
| 14:32:51 | sean-k-mooney | that might never exit | |
| 14:32:53 | zigo | There's a break ... | |
| 14:33:02 | sean-k-mooney | there is only if you get an indexError | |
| 14:33:16 | zigo | Which is going to happen when all connections have been closed. | |
| 14:33:31 | sean-k-mooney | no | |
| 14:33:39 | zigo | queue.pop() will return nothing... | |
| 14:33:49 | zigo | I mean, will do an IndexError no? | |
| 14:33:50 | sean-k-mooney | it returns None by default no? | |
| 14:34:02 | sean-k-mooney | which will cause an AtributeError when we defernce the connection | |
| 14:34:05 | zigo | tkajinam wrote it, he may tell. | |
| 14:34:26 | zigo | while ( conn = self.queue.pop().connection ) != None: | |
| 14:34:26 | zigo | At first, I wrote: | |
| 14:35:05 | sean-k-mooney | https://docs.python.org/3/library/collections.html#collections.deque.pop | |
| 14:35:22 | sean-k-mooney | ok pop does raise indexerror if its empty | |
| 14:35:50 | sean-k-mooney | i was expeciting it to reutrn a default like get on a dict | |
| 14:37:05 | sean-k-mooney | zigo: i still dont partically like useign excptions as contolflow but i see why this is working | |
| 14:38:02 | zigo | I was surprised at first too. Probably this only deserves some added code comments ... | |
| 14:38:16 | sean-k-mooney | zigo: i still feel like it might be better to have a timeout but i see why you want it to wait | |
| 14:38:32 | zigo | A timeout ?!? | |
| 14:38:39 | zigo | You mean wait until the CLOSE_WAIT times out ? | |
| 14:38:57 | zigo | That is a crazy idea. One of our server doesn't even have source port available because of this. | |
| 14:39:08 | zigo | Also, we reach the memcached max open sockets very fast. | |
| 14:39:12 | sean-k-mooney | no | |
| 14:39:26 | sean-k-mooney | i mean perhasp we shoudl not try to clsoe the connections indefintly | |
| 14:40:02 | zigo | If we don't, then there's some remaining sockets in CLOSE_WAIT state. | |
| 14:40:03 | tkajinam | the logic loops over all connections in the queue but it does not retry for a single one | |
| 14:40:16 | sean-k-mooney | oh your right | |
| 14:40:19 | tkajinam | in case a connection closure fails then it leaves that one and process the next | |
| 14:40:24 | sean-k-mooney | if we cant close it its already poped | |
| 14:40:31 | tkajinam | yeah | |
| 14:40:32 | sean-k-mooney | so we will at most try each connection once | |
| 14:40:36 | sean-k-mooney | ok yep | |