I think these debates come back to the fact that visibility is one of the least well designed aspects of the entire OO paradigm. I posted some suggestion for a rethink of visibility a few weeks ago. When you try and figure out what the specific goals are and how JVM visibility solves those goals, it’s hard to be satisfied.
The “best practices” from Effective Java were written by people whose primary experience of Java was developing the standard libraries. Not surprisingly, they think a lot about making minimal APIs that can last a long time and which can operate in a sandboxed environment, and thus minimising accessibility is going to be one of their recommendations. That doesn’t mean their experience generalises to everyone.
Many Java developers are not writing long term APIs. They’re simply writing apps that use them. For these people, the entire concept of visibility is a big net loss:
- Makes it harder to monkey-patch libraries when they almost but not quite do what is needed.
- Makes it harder to write white box unit tests
- Often results in useful code in a library that the developer didn’t quite want to support for the long term being non-reusable, resulting in the worst case in copy/paste coding with all the well known downsides.
I have been on both sides of this, both designing and maintaining a largeish Java API for many years, and also being a user of other people’s big APIs.
When I first made the library I maintained (bitcoinj), I tried to design for a long term stable API that could have a tight and supportable design. So a lot of stuff was private or package private. Some things were final. I pretty quickly noticed that the bulk of the “contributions” I was getting to the library were in fact just raising the visibility of things so they could be reused or overridden. Often these were classes that I was pretty sure I wanted to change, but the developers submitting the patches were adamant that they wanted to use the functionality anyway. I didn’t want to pick fights with my own users, so I tended to accept such patches, with the caveat that the API wasn’t stable yet and so they used these things at their own risk.
Eventually it became clear that actually freezing the API was a fools errand and would never happen, because developers overwhelmingly cared more about features and performance than having a stable API. They wanted an API that was stable enough which did not imply completely frozen, and they were absolutely happy to override/patch/reuse stuff at will even if they knew that in future they’d have to change things because of it.
What’s more, sometimes I found that APIs I had felt sure I’d change ended up not really changing, or APIs that I was sure were done ended up needing to be altered. Making a Bitcoin library, especially back then, was not like making an HTTP library or whatever where the problem is well understood and doesn’t change. BitcoinJ was basically the first big OO Bitcoin API so it was hard to know what it should look like.
So virtually everything ended up being public. And whenever I gave into temptation and attempted to use the “best practice” of making something private or final, without fail this would result in a bunch of annoyed users sending me patches to expose it.
The JDK team have gone through the same process. The sun.misc.Unsafe class was meant to be private to the JDK but ended up being widely used because the functionality it gave was available in no other way. It’s now unofficially public API. But they never learn! Now they’re making a new Unsafe class and saying that this time it’s gonna be really really private with super-duper encapsulation via Jigsaw. Except, oh, there’ll be a flag to disable the encapsulation for when you really really need it. Guess which flag many big apps will end up requiring over the next ten years?
The concept of encapsulation has over the years become something close to religious dogma. It creates massive timesinks where developers waste hours and hours fucking about with reflection, uploading patched libraries to Maven repositories, debugging why their super-duper web framework is silently ignoring database writes because some bytecode weaving expected open classes but found a final class, etc.
If I was designing my own OO language, I’d probably chuck the entire concept of visibility down a black hole. I’d just burn it to the ground. Instead I’d have three things:
- Annotations or modifiers reflecting API stability: stable, beta, unsupported, deprecated etc
- Annotations or modifiers labelling methods or properties as “impl” (implementation detail) so they don’t show up in JavaDocs or code completion. But they’d still be usable if you typed out the name manually.
- Finally, a (rare) annotation called @Secure that makes “impl” methods/properties completely inaccessible from sandboxed code. This would correspond to private today, but impervious even to reflection (from within the sandbox).
Then if you’re a developer and you end up using code marked as “impl”, you upgrade your code and it breaks … ok, sucks to be you. Can’t say you weren’t warned. Ditto for if you were using beta/unsupported API. You want seamless upgrades in perpetuity? Gotta stick to the stable stuff.