Kotlin to support package protected visibility

Package visibility for placing private helper classes inside API-modules like in java is just a limited way. So helper classes cannot be placed into a deeper nested package, but have to be placed beside the main-API class to archive protection from the outside world.

Much more nicer will be the “friend” concept.

https://en.cppreference.com/w/cpp/language/friend

2 Likes

The amount of activity on this thread should be enough noise to signal that this problem needs to be addressed somehow

10 Likes

Make more noise - package private access is the most demanded language feature for me.
I want a tool to split my deliverables to submodules.

4 Likes

I completely agree,
Absence of package visibility is the thing that prevents us from using Kotlin, as modularity of our Java codebase is based on that.
Default or not, package visibility of at least classes is an important language feature!

1 Like

That’s true, but to make a class package private is an explicit decision.
For example, I make my implementation in a “impl” module package private to allow an assembly module (such as a Spring auto config) within the same package only to access it.
Public accessibility would destroy modulaity and internal won’t work.
It’s a weaker protection, but it’s explicit with very useful usecases.
This feature is the number one on my wishlist!

3 Likes

I find the (lack of) reasons for the inclusion of package private in Kotlin saddening. I thought of replying directly here, but it would’ve been too big to provide both arguments and counterarguments, so I spent several hours writing an article instead. (No, I’m not advertising the blog. There are neither ads nor my name on it.) Please read it, and let me know why you still want package privacy; or if you think any of my points are incorrect.

3 Likes

Nice summary and some good arguments. I really appreciate a lengthy write-up and a conclusion that tries to draw from a larger picture of the discussion up to this point. :smile:

Some meta comments about this topic

Consuming a long forum topic like this one is cumbersome and often falls into a feedback-loop of people petitioning for their position instead of contributing new or meaningful discussion. I think organizing the discussion is important to avoid this topic becoming a mega-topic.

I’d like to encourage anyone who skimmed through this topic and gets an urge to petition for or against this feature, please don’t “make more noise” for the sake of noise :slight_smile:

Has this topic become a mega-topic? Is it worth summarizing the major points (and pro/cons) and then closing to help the discussion progress? In order to keep the meta-discussion away from this topic, feel free to reply to this specific post here instead: Please close the mega topics (ternary and companion objects)

3 Likes

Where can I read more about this “KEEP” thing you mention?

IMO Kotlin is at two features away from perfection:

  • Ternary Operator (I know, throw rocks at me, but I just can’t stop loving it)
  • Package level visibility

Cheers!

You should be aware that KEEP is quite a slow process, especially if you don’t have any people from the kotlin team supporting it.

1 Like

What about Friend classes like C++. Isn’t that a nicer way? could that be implemented with jvm?

Really good article, thanks.

I want to add to the discussion that modules are superior to packages for a number of reasons:

  • They can depend on other modules and external dependencies as well, which encourages better dependency management.
  • One can choose custom language features, compiler configurations and plugins, etc. for a single module, while not affecting other modules.
  • A build tool like Gradle can parallelize module compilation and build caches to prevent redundant task executions like running tests of modules that were not changed.
  • You can compile and distribute every module of your application, allowing developers to modify specific modules and only compile the dependent modules, which is great for open-source projects where developers often make small changes.
  • You could publish JARs of every module so developers don’t need to even compile the dependent modules.
  • You can apply build tool plugins to specific modules.
  • You can apply annotation processing to only specific modules.

These are just some of the benefits I could come up quickly, but I’m sure there are many more.

I think people who want package-private are not used to working with modules and just want the same Java experience.

And for people complaining that “we will have 100 modules”, it’s 100 folders just like packages, but with way more flexibility.

2 Likes

I don’t think this is about better or worse. (Just as you wouldn’t say that arms are ‘better’ or ‘worse’ than legs.) Modules and packages are both tools. And while they have different aims and different uses, both can be helpful in organising code.

1 Like

Agreed. I could have been more explicit in my comment. To be clear, I think packages are useful, but for organization, not encapsulation. For encapsulation, modules are superior for the reasons I pointed.

A very good example of this is how many libraries have an internal package with public classes, which are then exposed to library consumers. This happens because often it’s necessary to have classes that are implementation details in different packages and then consume them in packages at higher levels of abstraction. This can’t be done with package-private, so internal here is much more interesting.

Hello all,

I am for bringing package-private visibility, but I think we should carefully decide how to solve the problem. I think that an elegant solution could be :

bringing current visibility modifiers to package declaration.

Explanation :

As visibility on a class impact all its members, I think that declaring visibility on a package could affect the default visibility of all the members of the file declaring the package:

  • default or public: By default, all defined class/function/variable is public
  • internal: All members in this file are internal, unless specified otherwise
  • private: And there is the trick for package private visibility: In this specific context the private keyword does not define privacy to the file, but to the package (putting all members private to the file would not make any sense anyway : how would use them ?).
  • protected: To discuss. Maybe package private + visible by inheritors by default. Or maybe not allow on packages. Or something else.

What are the drawbacks of such an approach:

  • First and foremost, it change semantic of private keyword when applied to package
  • You cannot assume anymore default visibility is public. You would have to check package declaration of the file

Now, the advantages I see (might be seen as limitation by others, but I hope we can discuss it further :slight_smile:):

  • Better control over access visibility: You can now easily organize your source files to group public / internal / package-private because you specify visibility rules of the file in its header. It let decide developer what default visibility rule best match its need
  • Avoid repeating information: You can avoid repeating internal keyword on each class / function.
  • Avoid introducing a new keyword: Instead of putting 5 visibility keywords, which could make discovery of the language harder, you extend current ones with new possibilities.

What do you guys think ? Is it a bad idea ? Is there problems that I didn’t see that this solution would add or not take care of ?

Hello Everyone,

I do not want to repeat any of the great arguments for package-private visibility.
However, it is really strange to me that Kotlin 1.4 introduced the Explicit API mode which solves very similar problem that the package visibility would do. Java 9 modules + package visibiliy, would make the internal visibility an inferior tool for the JVM.

My main point is to propose a possible solution for us old farts who find package-private visibility a very handy tool in our toolbox.
Introducing this feature to Kotlin will take a long time if it happens at all.
In the meantime, we can build a temporary (permanent?) solution to this issue using existing tools.

With Kotlin 1.4 it should be possible to write a compiler plugin that would change the default visibility of Kotlin classes from public to package.
This option should be opt-in and probably on a package by package basis.
To opt-in into this behaviour you would add annotation on the package (in package-info.java?).
The annotation could be something like @JavaPackageVisibility.
Classes and free methods in such packages without explicit visibility modifier would be compiled with package visibility.

This could relatively easily solve the problem without the need to change the language.

Just an idea. Happy to hear comments. Maybe similar tools already exist?

Daniel

1 Like

As developers we need a package scope visibility. This discussion clearly shows it.
Please see a interesting talk on how to implement hexagonal arch in Java
Jakub Nabrdalik - Mid-sized Building Blocks & Hexagonal Architecture

There is now (officially experimental) support for OptIn annotations. Using that you can create your own scopes in a way that you can also very easily audit using grep or static analysis. If you combine it with internal your scope can be quite narrow. I know it is not the same, but it gets very very close.

3 Likes

@pdvrieze In my opinion does not make sense. I prefer simple mechanism provided by lang spec like in Java :wink:

1 Like

@kamil.jedrzejuk Let’s look at when you’d want to use package visibility. Package visibility limits access only to code within the same package (in Jigsaw this would also be module-internal). It is used when private is not sufficient. Protected is a public/api level visibility so wouldn’t be satisfactory. A common use for package visibility is to provide access for testing. In any case, the intent of package visibility is to limit access to the symbol to a narrow set of code. The intent of the access normally is much narrower than the entire package, but in any case both provider (the thing being package visible) and the consumer can be edited at the same time.In other languages the effect could be achieved with a friend declaration.

There use of package private thus provides overly broad access to symbols, but not as broad as internal does. At the same time, package private means that users are forced into the same package, even when not optimal (for example for tests, that you might otherwise want to put in a test subpackage). What @RequiresOptIn does is allow you to create an annotation that creates an accessibility scope. Any symbol with the annotation that you create will only be accessible (if in error mode rather than warning mode) if it either is also annotated with the same annotation, or if it is annotated with @OptIn(<the annotation name>). This differs from friend in that “friendship” is provided at use rather than declaration (hence the auditability statement). I don’t see this as a serious problem, visibility isn’t a security measure, it is a programming measure. In many cases the scoping annotation could be declared internal, so not be escaping the module either.

1 Like

@pdvrieze The idea of custom OptIn visibility scopes based on annotations is interesting. One could create their own @PackagePrivate scope if they loved the word “package” for some reason-- but more powerfully they could name refined scopes to cover specific areas of their code.

As you mention, the use of package visibility and OptIn really provide the same kind of control by forcing the developer to opt in to use some functionality (either by putting their code in the same package, or by using an OptIn annotation).

Maybe I’m missing some cons of OptIn compared to package-private because from my current understanding, OptIn solves the problem strictly better than package-private could.

(I guess one con is that Java familiarized us to use “package” as a visibility scope, which feels comfortable. It might take some time breaking out of old habits to start defining our own scopes instead of limiting or expanding them to packages)