Feature request: Make it possible to break through everything from /test

Language Design
1

I love nested functions. When b() is called only from a(), it should, in my opinion, be defined inside a(), like

fun a(){
  fun b(){
    //blablabla
  }
  b()
  //blablabla
}

This is great encapsulation: You won’t see b() from outside of a(), and you easily find b() in a() when you are looking for it.

I like to write functions within functions within functions. A mess, you say? Not at all, as long as you always define the nested functions at the top of a function and don’t use local variables, it works really nice. I think this is great design! The problem is just testing: I can’t test the inner functions

I know this old saying that “you shouldn’t test private methods because they are implementation details, you should rather test functionality”. Well, I guess then you should just test only the main-function manually, as that’s the only thing that is public in the end… Also implementation details matter. You want to test one single unit of the code. Often it’s not possible to test just a public interface and at the same time spot what exactly goes wrong. This is especially the case if you write like me: Functions within functions within classes… (And this is a brilliant design, but that’s another discussion).

Why can’t it be possible to just break through everything as long as you are in the /test directory? It could, for example, be possible to do something like this:

€a(){ b() }

The € is used to break through a(). Then we are in a()'s scope, and can call b(). If a() takes an argument, and this argument is used in b(), we would need to do something like:

€a(10){ b() }

The same should of course be allowed for private methods:

€myInstance.myPrivateMethod()

What do you think? Why can’t we just break through everything from the test directory?

1 Like
2

I’m against testing private methods and local functions. You should only test effects of those through public API usage. Private methods are private for a reason, because they hide and aggregate the implementation details. If you approach your code from TDD perspective you don’t design your tests assuming each potential private method, but only the public contract the API should offer.

The same goes for exposing your methods only for testing reasons. If there is a need to do so it just indicates a bad code design.

6 Likes
3

Then I won’t be able to test since I write everything within the main-function :wink:

I think it’s pretty great design to write code like this:

fun a(){
  class Hey() {
    fun c(){
       fun bla(){
          class Other {
            // I'm used only from bla() so why can't I stay here?
          }
          // ...
       }
       fun bla2(){
          // ...
       }
       // yeah
    }
  }
}

Great encapsulation, or what do you think @madmax1f028 ? I kinda see the point with TDD, but wouldn’t it be great to be able to test properly all the details. I can only test a small part of my code when I follow my brilliant design shown above. I don’t want to mess up the design in order to write tests. The tests should be made so that they don’t mess up the design.

4

@Kotlinius Would you mind editing your post and removing the language? The code of conduct is to keep it family-friendly.

Your example design is not something commonly considered desirable. In fact, IMO if you see that kind of structure it’s likely a code-smell indicating there’s another better design. Nested functions and classes are great for making your code more clear, but if you find you’re using them a lot, you probably want to move them outside of their enclosing function.

Since there already is a solution to the problem, moving the local function outside of it’s enclosing function, there isn’t much benefit to adding an extra solution that provides only minimal benefit.

It’s good to remember the clean code guides that functions should usually be less than 20 lines and files should be less than 200 lines.

5 Likes
5

Thanks for this great reply @arocnies. I removed the language.

“This is not best practice”, end of discussion? I don’t think so :wink: Best practices are decided by the community (I guess), and should be changed. This is a new idea that maybe not too many have been thinking of. Of course I don’t write radical code like this at work. People haven’t seen the style before an then get confused. But at home, I write this radical clean code.

Nested functions and classes are great for making your code more clear, but if you find you’re using them a lot, you probably want to move them outside of their enclosing function.

Why? And don’t answer “it’s best practice” please.

It’s good to remember the clean code guides that functions should usually be less than 20 lines

I agree. My functions are normally not longer than 20 lines if you ignore the nested functions/classes, and they are very easy to ignore. You can collapse them by default. My system is like this:

fun a(){
  // Nested interfaces
  // Nested classes
  // Nested functions
  // The actual code
}

The actual code follow the clean code guides.

and files should be less than 200 lines.

I kind of agree within today’s language features of Kotlin (and other languages), but I don’t think “file” should be a concept in code, as it doesn’t tell anything. You can put one or many things within a file, and it doesn’t really say anything structural. Files are not necessarily structural (although they can be). Because of this, it’s not bad design to write 5000 lines in one file. You can just navigate through the code structure with the IDE’s structure view. There is a problem though: Version control. But that can be solved in the future (another feature request) by auto-generation of structural files based on the code structure. In the IDE, you only need to care about the code structure, while files are generated in the background based on these changes. When you write a new function, a new file is created etc.

But I agree that files shouldn’t be too long within today’s constraints. So I suggest to not yet go all-in on my function within function approach.

6

Thanks for the edit! :slight_smile:

I think the most effective way to win over support for accessing private scopes does not focus on nested scopes. I suspect it’s far too easy to simply move those functions out.
Local functions are basically properties within that function. Would you also request being able to access variables within a function? And what if that local function accesses other local properties, how would it deal with the state of the parent function?

Focusing on the testing aspect might be more effective. Still, it’s a difficult request because those functions can be marked internal pretty easily. This argument might be undermined by those requesting another scoping mechanism (or package-private scope like in Java).

Local functions can include the scope of their parent–meaning they are not always stateless. Example:

fun greetPeople(greeting: String, people: List<String>) {
    var totalNamesLength = 0

    fun greet(name: String) {
        println("$greeting $name")
        totalNameLength += name.size // accessing our parent's state
    }

    people.forEach { greet(it) }
}

The above example is a bit odd but the goal is to highlight that local functions either are “stateful” by using their parent functions scope, or they are “stateless” and can be moved out of their parent.
Any function you could test must be stateless and can be moved out of its parent. If the function isn’t stateless, you can’t test it (even if you could access it) and you can’t move it outside its parent.

You might still like the design of making your stateless helper functions local for the goal of shrinking their scope. IMO it does not provide much benefit over having them at the class or file scope. For readabilities sake, I prefer private stateless functions sorted by which is called first so the parent reads like a high-level summary instead of diving deep into details first–when I do use local functions, I they’re usually stateful and use the parent’s variables.

4 Likes
7

@arocnies Yes, this is right, a nested function can be stateful or stateless. I keep them stateless, except that I might use params from the parent function (not sure if you will call that stateless or stateful). Anyways, these functions are super easy to test. If I use one param from the parent function, I would do like

€a(10){ b(20) }

(This is valid Kotlin, but example on my new feature).
I break through a() with 10 as argument, then call the inner function b() with 20 is argument. This is 100% testable.

Pretty nice to not have to search around for classes and functions. Searching up and down in a class or in a whole workspace: Where did I put this class. Or search for the name in the IDE, but you don’t always remember the name.

I would never do like in your example. I always define the nested functions before the actual code: First interfaces, then classes, then functions, then the actual code.

8

To add a general consideration to the topic: one goal of software architecture is to reduce coupling for better maintainability. One such coupling is between test and implementation. You don’t want (or do you?) to change the test, if you change an implementation detail like extracting a function or merging two private functions. While this might work in small applications, it is to costly in larger applications.

3 Likes
9

@medium I think coupling is nice when it’s needed. Of course when using the functions-within-functions approach, my inner class/function would be at the same “detail level” as the public interface would be with traditional approach. I don’t write tests for every little detail, and you don’t do either if you only test the public interface. You write test when they provide a value, and you need to have coupling at that level. I don’t want the tests to mess up my code. If I follow the public interface approach, the encapsulation would be worse. I don’t want to design the code with tests in mind, but rather be able to break through in order to perform the tests. I want to be able to right click within an inner function, choose “Generate test”. And write a test for that specific functionality, if it makes sense to test.

10

The term “coupling” in the classical sense is something you don’t want. You want “high cohesion and low coupling”. I can imagine an inner function with high cohesion, after all, the only reason you’d want to make a function inner is that it’s super cohesive with its parent.

I agree this is somewhat of a new idea. Instead of making classes and functions separate, tying them into a parent’s scope (coupling them more) is not something I’ve seen recommended.

I would be surprised if you could find a book, blog, programming language, or coder with more than 5 years experience that would recommend a design where you nest details that would normally live on their own into a deep scope where those details can be easily coupled to many unrelated parents.

If your functions and classes truly stand-alone, they should be separated. Using imports, files, packages, and libraries is how this is normally done.

A common example you may run into is working on a project and you realize that a subset of the project should really become its own library. Or maybe you realize that part of a single class should be broken off into its own class. Or maybe you notice a function is really doing two separate things and should be split. Basically, you’ve noticed there are really two separate things and they would be better separated logically.
What you’re suggesting is to make the split but still leave the project/class/function entangled with the other half of the split–so much so that there is no protection or guarantee they are separated at all (i.e. if a function is stateless and/or pure compared to its parent function).

I agree that encapsulation is nice but making a function local does more than just encapsulation, it ties it to the parents state–there’s actually less information hidden compared to a separate function.

EDIT: Although it’s fun to talk about programming concepts I do feel like I’ve failed to address your original proposal: Accessing local functions. From my understanding, local functions are compiled into a function variable and aren’t even created unless the function execution inits them (I’d have to double-check bytecode to confirm). This is the same problem as trying to access a variable inside a function and there are a number of issues that prevent accessing them.

5 Likes
11

Here I am.

I am not asking about what “is recommended”, what the majority thinks etc. The majority voted Trump as president in the US. I’m interested in what really is the best solution. I don’t see why it’s tighter coupling to have b() within a() when it’s used only from a(). “Details that would live on their own”? Are you against private methods then? They are details, but don’t live on their own. Private methods give encapsulation, as inner functions do.

Let’s say we have a class called Person that is used only from one single function. You would define it publicly so that you could test the functionality, while I would define it only within the function it is used. Is Person any more “implementation details” for me than for you? The class is the same, which means it provides the same level of implementation detail.

Explain this please? The only coupling to its parents would be the use of the parameters from its parents. As I suggested: Don’t use any local vars in the nested functions, only use the function’s params and params from its parents. But maybe we should be even stricter and don’t use any params from its parents. Then the coupling-problem doesn’t exist anymore.

fun a(name: String){
  fun b(name: String){
    // Here we don't use the parent's params
  }
  b(name)
}

This way we avoid the coupling completely, it’s not tied to its parent’s state anymore. Here we have no coupling and full encapsulation. Or am I wrong?

Thanks for the clarification. I think somehow this should be fixed, though, so that I can do proper testing of my great code :wink:

12

What do you think of this now, @arocnies (or others)? Is there any “coupling” if we don’t use params or instance vars from parent functions? I have just been more and more fan of this new style. It works super great for me! (except that I can’t test it)

1 Like
13

I still would advise not using this style.

It’s true that you can write local functions in a way that is independent of its parent. But there is still coupling.

Here’s an analogy that may help show how I see it:

Imagine if I wrote two stories and combined them in one book. Maybe I interweave each page so that even number pages are one story while odd number pages are another story. Just like local functions not using any params from the parent, I make sure that the stories don’t use any characters or plots from one another.

Now, imagine an editor working on this bundled book. The first thing she see that both stories are independent of one another. She also sees that it’s confusing to interweave the stories. The first thing this editor recommends is to group all of the stories pages together–kind of like writing local functions only at the top and moving all of the parent’s logic after.

While reviewing the books, the editor ponders if there was some way to separate the books even more? After all, they are both independent stories, and combining them may lead readers to not understand that there they are really independent–in fact, the only way a reader would even know they were fully independent is to read both stories entirely! If only there was some way to break these two stories up so that it was immediately clear they are independent of each other.

Publishing the stories as two separate books is the solution.
The next book I send the editor is a dozen stories bundled as one book. I say to the editor, “I think this style is great! I think we should bundle all of our stories!”

What you’re doing by preferring to write local functions is similar to preferring to write your stories in one large document/book. It’s true that you gain some encapsulation by hidden the function within its parent–but you’re also gaining increased conceptual coupling between the local function and its parent
(and that parent’s parent, and that parent’s parent’s parent, all the way up like writing an entire library into a single giant bundle of a book).

You might promise to not use the coupling, and you might even succeed. Instead of having to trust your word and that you won’t make a mistake, I prefer to let the compiler guarantee it.
Yes I know I’m losing the extra encapsulation, but I don’t need it. I can achieve the same encapsulation using the existing visibility modifiers.

So back to the original quote where I said there is still coupling. Yes, I know there could be no dependency on the parent’s params (or other state) in practice because you promise not to do that. The conceptual coupling of the local function now having the added opportunity to make that dependency means we can no longer pull the function out without reading it to make sure you kept your promise.

There’s plenty of times where local functions and classes are perfectly fine for stateless functions–I’m happy to use and recommend it for those cases too. These are cases where we get more benefit to reading clarity for little cost in coupling. What you’re suggesting by most of your functions and classes is that this cost/benefit favors the nesting–it does not.

TL:DR, the code is often harder to read when you nest. Nesting everything is certainly harder to read.

2 Likes
14

I do like the idea of nesting functions.
This is basically the step-down of Uncle bob, which I referred to before.

It will be a bit of a pain, but did you look at optin requirements?
https://kotlinlang.org/docs/reference/opt-in-requirements.html

You can use it a bit to flatten the classes.
This however isn’t as well enforced as your current syntax

15

Love Uncle Bob. I order my functions by his top-down strategy as well. Here’s the link from that forum to explain it: The Stepdown Rule - DZone Java

(Although I can’t say I like the idea of the nested scope controls the way you propose :man_shrugging: but I won’t get off topic :slight_smile: )

We also haven’t considered the future possibilities of namespace { /* ... */ }. Decorators could also change things by adding receivers containing functions.

16

I think it should be possible for a team to make these types of rules. There could be warning when an inner function has coupling, and that could be analysed in the IDE.

When it comes to the book analogy: Why not use a package as an example instead: In a package you have also lots of different books together. Some of them are private, and these private functions and classes are as much “separate stories” as two inner functions/classes would be? Also, the inner functions do have something in common: They are used only within that specific function. In most cases, the meaning of an inner function is very related to its parent function, and therefor it makes sense to have them in the same book.

17

I think decorators and namespaces can indeed solve there problems.
Decorators which would provide private access to a scope would be great.
(You wouldn’t get access to such a function using with)

1 Like
18

He is basically saying that you present it in a top-down manner from more general down to more concrete details.

My problem with that is that assumes that this is the way that everyone learns things best. I had an experience years ago that clearly showed me that people grasp things differently. There are top-down people that learn best by starting at the overview and narrowing down to the specifics and there are bottom-up people that learn best by starting with the details and then combining these details to higher levels of understanding. If you are presented with things in the opposite way, it hinders learning.

I definitely fall in the bottom up learning style and when presented things in top-down style I cannot keep track of the higher levels of abstraction without concrete details to anchor them. A top down person gets lost in the details if presented ideas in the bottom up approach without the higher level view to anchor them.

So the step down rule assumes everyone is a top-down learner, because it was written by a top-down learner. It is often difficult for someone with one learning style to understand that the other style exists and is preferable by others.

So I agree that ordering should not be random, but not with the assumption that top-down is universally the best approach for all readers. When reading code I often find myself reading code backwards by starting at the bottom and scrolling up.

Given this dichotomy I think the best approach is a hybrid. Think of it as you are presenting a tree of information. Do you present things in a breadth-first manner or a depth-first manner. The pre-order depth first manner seems to be the best compromise as it anchors the top down people with where this detail fits into the big picture but keeps bottom up people engaged with the details of one aspect of the tree before bombarding them with abstract details of other parts of the tree.

19

Oh yes, people definitely learn differently. But the way you talk about top-down makes me think we may be talking about the same thing. Step-down is depth-first. (I’ll use “step-down” instead of “top-down” to avoid confusion from here out)

Like you I also feel I learn best quickly by diving into the details instead of trying to swallow all of the abstractions first. I agree a hybrid is also my favorite. The step-down ordering is a hybrid. The ordering places the detail functions directly after the higher abstraction functions so that you read it in a depth-first manner.

I like this because it I’m not reading all of the higher abstraction functions first, instead I’m diving into the details each abstraction one at a time. As you mention, reading from high abstraction entirely before detail is not universal–I would say its not even common.

Sometimes I also read upwards in step-down ordering. The nice thing here is that we get another hybrid of building up to one higher abstraction at a time (again, not requiring you go over all details but instead only the ones relevant to your abstraction). If you read downwards in step-down, you start with the abstraction and break it down one at a time into the details.

20

It is hard to say from the example in the referenced article since it is not a deep tree. There are essentially only 2 levels in the tree. So perhaps I am misinterpreting the step down rule. I interpreted it to mean that you have all functions at the each level before the deeper level. But this statement from another source seems to agree with what I am saying:

Every public function call private children functions, which again call their children functions. They are ordered in the order they are called and in the order of the herachy.