| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-11 | |||
| 14:16:55 | dansmith | gibi: thoughts on the vmware deprecation patch? I'm glad it got some attention raised, but we've been a long time without vmware ci and despite some response, I feel like we probably need a deadline or something to make sure something happens | |
| 14:17:50 | dansmith | I'm not even sure when the last time the vmware ci commented on a nova change, but I feel like it's been a *long* time | |
| 14:23:58 | sean-k-mooney | dansmith: we could merge the patch propose a revert and hold it until m3. if the ci is running by m3 we merge the revert if not then leave it | |
| 14:24:31 | dansmith | yep, we could also convert this to just a quality warning, merge that now, and hold the deprecation warning on the ci | |
| 14:24:35 | dansmith | which i think is what we did for xen | |
| 14:25:47 | gmann | brinzhang: i will check after coffee. | |
| 14:26:10 | brinzhang | gmann: cool, thanks | |
| 14:30:22 | mriedem | we merged the deprecation warning for xen in train, it wasn't contingent | |
| 14:33:36 | dansmith | okay when I was looking for examples, I found a "merge quality warning" for one of the drivers before a deprecation | |
| 14:33:46 | gibi | dansmith: we can merge a deprecation warning without causing extra problem to hemna and undeprecate if vmware ci magicly shows up | |
| 14:33:47 | mriedem | looks like dan's patch has at least gotten the necessary attention, | |
| 14:33:53 | mriedem | whether that holds or not i guess we'll see | |
| 14:34:15 | dansmith | gibi: ack, well, that's what I've got up, so maybe go vote on that :) | |
| 14:34:16 | gibi | the real break happens if we delete something | |
| 14:34:25 | gibi | dansmith: will do | |
| 14:34:32 | mriedem | yeah the people -1ing the change are saying "please don't delete it" | |
| 14:34:35 | mriedem | which isn't what the patch does | |
| 14:34:52 | dansmith | there has still been no progress in finding a maintainer for" | |
| 14:34:52 | dansmith | mriedem: I quote you: " After three months since the quality warning change merged [1] | |
| 14:34:55 | mriedem | they can -1 the patch to remove the driver in a later release if it gets to that point | |
| 14:35:19 | dansmith | https://github.com/openstack/nova/commit/af280ffe3098b84123eae218989ea056e9935bf1 | |
| 14:35:47 | mriedem | good thing i was so verbose in my commit messages :) | |
| 14:35:47 | dansmith | but I'd rather go straight to deprecation, | |
| 14:36:06 | dansmith | and then we can avoid removing it if CI shows up, and un-deprecate if it looks like CI will stick around | |
| 14:36:18 | mriedem | i think there is more than enough ammo for vmware deprecation based on repeated attempts in the ML to figure out what's going on with CI over the last several years, | |
| 14:36:58 | dansmith | agree, and like I said, I'd be kinda surprised if it really works in U after all the provider stuff | |
| 14:37:06 | dansmith | working in queens I totally believe | |
| 14:37:58 | jkulik | fyi, we (SAP) reached out to VMware to get a statement on their upstream approach. they weren't ready for one, yet. | |
| 14:38:05 | mriedem | i forget her name but i was sending emails about the lack of CI back when the old project manager was involved | |
| 14:38:10 | dansmith | IMHO, to keep it we need more than just CI too.. we need support for devstack problems, like the one referenced on the mailing list and general confidence that it's working for more than just trivial CI configurations | |
| 14:38:42 | mriedem | jkulik: as far as i know vmware hasn't had an "upstream" team in awhile | |
| 14:38:49 | mriedem | cdent was the last closest person to that for nova at least | |
| 14:39:01 | dansmith | and honestly, it has always been a fight to keep them involved, which is pretty exhausting | |
| 14:39:33 | mriedem | i'm sure that team would say it was exhausting dealing with us as well :) | |
| 14:39:45 | mriedem | "what do you mean my bug fix needs tests?!" | |
| 14:41:39 | jkulik | as far as I heard: yes, they found it exhausting. | |
| 14:41:48 | mriedem | heh | |
| 14:41:57 | dansmith | yeah I have no doubt | |
| 14:42:03 | mriedem | i mean, quality control, who needs it | |
| 14:42:04 | mriedem | psh | |
| 14:42:17 | jkulik | main problem seems to be, that they have their own product based on openstack which works fine for them | |
| 14:42:36 | mriedem | VIO | |
| 14:42:39 | mriedem | right? | |
| 14:42:40 | jkulik | yes | |
| 14:42:45 | mriedem | sure, with patches | |
| 14:42:46 | dansmith | and that's cool, but we don't need to keep a broken driver in our tree, especially with people asking on the ML why it doesn't work | |
| 14:42:53 | mriedem | so you buy that thing and you're stuck with them for support b/c of the patches | |
| 14:43:06 | jkulik | sure, they need to make money off it | |
| 14:43:12 | jkulik | otherwise everybody could just install openstack ;) | |
| 14:43:40 | sean-k-mooney | jkulik: our they could enforce an upstream first poicy for all freatres and backports | |
| 14:43:47 | sean-k-mooney | *or | |
| 14:43:57 | jkulik | which they obviously still make money from, because you'd still have VMware hypervisors and stuff | |
| 14:44:17 | jkulik | tbh, upstream first is hard. | |
| 14:44:50 | sean-k-mooney | it can be but its generally worth it. i thnik the quality of the final solution is typically better | |
| 14:45:24 | sean-k-mooney | but it is a higher barrier to entry for enableing a feature | |
| 14:45:26 | jkulik | I agree. But some things you can't upstream and once you're down that road, you're not that willing to upstream the rest. | |
| 14:45:29 | mriedem | depends on where you want to invest time and money, up front or on the backside dealing with maintaining a fork | |
| 14:46:28 | dansmith | jkulik: we're not arguing that they shouldn't have a downstream.. we're arguing that if they want their base in the upstream, there's a minimum bar and we're not going to do their maintenance for free, that's all | |
| 14:46:29 | mriedem | and getting the team culture in place for people that aren't used to having their code reviewed | |
| 14:46:29 | dansmith | we all make money from openstack one way or the other, nobody here doesn't recognize that | |
| 14:46:49 | mriedem | i make $0 from openstack now | |
| 14:46:58 | mriedem | wtf am i even doing in this conversation? :) | |
| 14:47:07 | jkulik | sorry :D | |
| 14:47:17 | dansmith | mriedem: well, the 5th year senior that still hangs around campus being the exception :) | |
| 14:47:41 | mriedem | i get older, nova stays the same....wait | |
| 14:47:46 | lyarwood | haha | |
| 14:47:47 | dansmith | LOLOL | |
| 14:47:54 | jkulik | I think my main problem is, that I can't convince VMware to do the maintenance upstream and need that driver to work ;) | |
| 14:48:15 | mriedem | jkulik: so can SAP have one or two developers that start working on it? | |
| 14:48:28 | mriedem | it's not like vmware is the only company that can work on that driver | |
| 14:48:30 | dansmith | jkulik: I want lots of things for free too | |
| 14:48:33 | mriedem | it's been on life support for years | |
| 14:49:25 | sean-k-mooney | jkulik: do you consume the driver directly form upstream or via the vio product | |
| 14:49:26 | jkulik | I can talk to my managers about it, but given our team size, it'll basically still be life-support. | |
| 14:49:34 | jkulik | from upstream directly | |
| 14:49:54 | sean-k-mooney | ok makes sense | |
| 14:51:07 | dansmith | jkulik: you know every time we make a change to the virt drivers, we have to make a change to the vmware one, and with no tests, we don't know if it works or not right? that's a lot of burden for us, and if you look at all the changes to that driver in the last two years, it's just that.. guesses. | |
| 14:51:31 | jkulik | dansmith: yes. I totally get that. | |
| 14:51:50 | jkulik | in addition, VMware doesn't work like libvirt at all in too many cases. | |
| 14:52:02 | dansmith | yup :/ | |
| 14:53:25 | sean-k-mooney | the current vmware driver talks to vspher too right rather then to esxi | |
| 14:53:34 | mriedem | at least we don't have the 1:M host:node thing with that driver anymore | |
| 14:53:44 | mriedem | doesn't mean a single node isn't hosting 1K instances which crash the RT | |
| 14:54:01 | sean-k-mooney | so while esxi can be used without a licnce vsphere cant so we cant do first party ci even if we and the capastity to maintain it | |
| 14:54:13 | mriedem | which has been a major pain point with the vmware team over the years, trying to change the nova architecture to handle that type of scale on a single node | |
| 14:54:13 | jkulik | the driver talks to vsphere, one nova node is a cluster (multiple hypervisors), but nova doesn't know about it | |
| 14:54:19 | mriedem | jkulik: yeah i know | |
| 14:54:31 | mriedem | to nova it's a node with several thousand VMs potentially | |
| 14:54:39 | jkulik | yes | |
| 14:54:41 | mriedem | unlike libvirt with maybe a couple dozen | |
| 14:54:44 | dansmith | ...which is why I'm dubious about all the placement stuff really working these days | |
| 14:55:07 | jkulik | we recently ran into a problem with that concept, trying to deploy really big VMs | |
| 14:55:17 | sean-k-mooney | dansmith: well we added the same host migrate thing last cycle right? | |
| 14:55:22 | hemna | so the driver won't work in U ? | |
| 14:55:25 | sean-k-mooney | so at least at that point it worked | |
| 14:55:38 | sean-k-mooney | but ya not sure placment will imporve performance or schduling in anyway | |
| 14:55:46 | dansmith | hemna: we have no idea | |
| 14:55:55 | dansmith | sean-k-mooney: which? | |
| 14:56:19 | jkulik | which is the main pain point ... not knowing if stuff even works | |
| 14:56:33 | hemna | sorry I haven't been following nova as much lately, but has there been a change that would cause the driver to not work? | |