Hoisting of Local Functions

Language Design
1

Could the Local Functions be hoisted so that we can use them before they’re defined in the code. This would promote readability, and/or void the need for classes to have too many private helper methods.

Currently this doesn’t work:

fun doComplexTask() {
    if (complexCondition())
        // do something

    fun complexCondition(): Boolean {
        // calculate complex condition value...
            
        return conditionValue
    }
}

yet putting all the helper methods before the business part of the function makes it less readable. This can especially become messy either way (too many private helper methods, or too many helper local funcs before the business part of the func) in Android, where we have predefined funcs like onCreate() in which we may have helper methods/local funcs such as initLayout(), animateUI(), processStartingIntent(), … In turn those helper funcs may be refactored to their helper funcs if waranted, to make them short and readable… Yet all those helper methods/funcs wouldn’t live in class scope as private methods, but be located where they’re used…

7 Likes
2

I don’t think this is true at all, that’s where I prefer them. If it’s a problem you should fix it in your IDE or text editor (using text folding or whatever). Where the static type of the function is needed, this would require two passes over the code or extensive backpatching. Basically this would slow down the compiler, and make it a more complex.

3

That’s your preference. Most of us tend to read code from higher abstraction down, and to see what the function or method does before 100 lines of detail how it does it. Most of programming is in maintaining code, and language design should make it easier not harder. Defending it for a few milliseconds of compile time on developer’s computer is lost in the late 20th century…

This is a non-solution to a language design post. Language design should make readability/maintainability of the code easy, not delegate it to IDE or code editor.

Correct… That’s why we have compiled languages like Kotlin, which don’t have cost each time the program is run. Moreover trade off of few milliseconds of compile time vs readability/maintainability of the code is very much in favor of the later. Especially since Kotlin does the same thing anyway on a top-level or class level.

This is a failed try to describe why this now works as it does. Further it shows inconsistencies even in Kotlin since class methods can call methods not defined before them, and top-level functions do the same. Only Local Functions require definition prior to their usage. Third time:

// works
fun a() = b()
fun b() = println("b")

fun c() {
   fun d() = e() // doesn't work
   fun e() = println("e")
}

works for non Local Functions. Thus having Local Functions behave like they do now is inconsistent on a language level, with added failure of hindering readability and maintainability of the code.

2 Likes
4

Except local functions capture a closure, while instance methods don’t. So it makes sense that they’d behave differently.

3 Likes
5

I don’t see how hoisting functions voids their scope… or why would closure usage of nested functions limit them for other purposes. Currently we’re limited to defining nested functions prior to use, while if that was resolved you could do whatever is more readable even continue using it as now.

Moreover closures hide internally scoped vars/funcs much like instance methods hide state/internal behavior of an object. Thus it could be argued that they’re more similar then not…

Could you elaborate more on that limitation, since that would make much more sense then compile time decision…

6

In JavaScript, if you have something like this:

function printFoo() {
    console.log("log: " + foo);
}
printFoo();
var foo = "foo";
printFoo();

It’ll print

log: undefined
log: foo

This makes sense, as with hoisting, it is essentially rewritten to:

var foo; // hoisted
function printFoo() {
    console.log("log: " + foo);
}
printFoo();
foo = "foo"; // assigned
printFoo();

I’m trying to envision how this would work in Kotlin, and…I can’t.

If you have:

fun printFoo() {
    println("log: $foo")
}
printFoo();
val foo = "foo"
printFoo()

Then…that doesn’t make sense, because at the point printFoo is defined, there’s no foo variable yet. And even if we did try hoisting the var a la JavaScript, then…you’d be able to access foo in its pre-initialized state, since you don’t move the assignment up, only the declaration.

I know you were asking about hoisting functions, and it appears I’m talking about hoisting variables, but they’re tightly coupled.

7

Actually, most compilers will just first parse everything into an AST. At that point you don’t need to go back because of file order. Nowadays compile speed is not because of parsing speed so much as for various other issues.

8

You have rightly noted that I’m not talking about hoisting variables. They preserve their scope and are defined from position they’re assigned onwards. What I’m talking is hoisting (nested) functions.

printFoo();
var foo = "foo";
printFoo();

function printFoo() {
    console.log("log: " + foo);
}

Instead of:

function printFoo() {
    console.log("log: " + foo);
}

printFoo();
var foo = "foo";
printFoo();

Either works in Javascript, but the top version doesn’t work in Kotlin for nested functions. Linter gives error because foo is undefined, and that’s ok, but not the topic.

Also functions are variables in Javascript, and your “they’re tightly coupled” explanation doesn’t hold even for Kotlin, since it is already done in top-level scope as shown with:

fun a() = b()
fun b() = println("sss")

Hence it’s not only doable, but already done, but not for Nested Functions…

9

Initially, I was very excited about Kolin’s having local functions. I thought I finally had a programming language that would allow me to follow what Uncle Bob’s suggestion of “step-down” rule in his “Clean Code” book in an explicit and natural way. Unfortunately, Kotlin’s local function feature is only half-baked. It doesn’t support forward referencing, and thus it is not possible to adopt the “step-down” rule to organize functions. Uncle Bob’s suggestion is same as what Invisible is saying that functions should be organized starting from higher abstraction and downward. This is the most natural way people understand complex ideas.

I later found out that Scala allows forward referencing for local functions. If Scala can do it, I don’t think Kotlin cannot be made so. Note, Scala allows forward referencing for local functions but not for local variables. This is fine because placing declaration of variables before they are used is a good practice.

I hope the Kotlin development team would seriously consider adding this feature to Kotlin.

2 Likes
10

The problem that I think is having a difficult time coming across from the naysayers is the fact hoisting the function isn’t a simple solution. If you hoist it to the top, and the function closes around a variable that isn’t defined yet, that fails. Okay, so you try to hoist to the first point after all the enclosed variables are defined. That works fine for times when all the variables used are effectively final, but if a variable isn’t effectively final for a while but eventually stops being assigned to and could be considered effectively final after that, where do you hoist the local function to? The midst of the variable changing? After it’s done changing? The most efficient (saving the creation of a Ref variable behind the scenes) would be to put it after all the changes, but could that change the intended semantics? Not as simple of a problem as you might think.

Sure, you’d probably intend to do everything the good, safe way of everything being effectively final and pure, but that doesn’t mean everyone will.

If you’re confused, let’s look at it like this:
Let’s use a=b+c as an analog to creating a function that encloses b and c, and print(a) is like calling that function.
// 1
b=1
c=2
… //2
c=3
… //3
print(a)
a=b+c // to be hoisted
If the line is hoisted to (1), then it simply doesn’t work because b and c aren’t defined. If it’s hoisted to (2), it equals 3. If it’s hoisted to (3), it equals 4.

If we just followed javascript-style hoisting, things would actually get worse. You see, hoisting doesn’t give the variable a useful value. It’s undefined. So this wouldn’t solve your problem at all, since your hoisted function would be undefined at the point of use, making its later definition useless and unused.

Within a block of runnable code, dependencies need to be defined before the dependers and after their own dependencies. There’s no getting around this. Method definitions can mix their order around because the body of a class is not a block of runnable code, it doesn’t have time-based dependencies (not completely true; you can mess things up pretty badly by calling a method inside the constructor that uses a field that hasn’t been instantiated yet, but even that can’t be fixed by changing the order of which methods are defined first).

1 Like
11

Hmm, no… hoisting or forward referencing is not that difficult concept to wrap your head around it, and it certainly isn’t new thingy on the block… It has known scope and the only thing hoisting does is allow your code to call (local) function which is defined later further down in the scope of the parent func().

As you’ve noted:classes have the same issue, especially with lateinit keyword, and the program can crash if you’re doing silly things. Yet, quite differently to Local Functions, it is allowed. By arguing against hoisting you note that it is alive, well and not misused in other parts of Kotlin!

Moreover your negating that it’s easily done has been refuted a couple of times by noting that:

fun proxyFunc() = showA()
fun showA() = println(a)

works on toplevel, just not while they’re Local Functions.

Language design can, and maybe should, handhold beginners to warn them about weird usage, but it definitively shouldn’t limit non-beginners to do what they know is safe. This is why Linting tools have “suppress” functionality in them…

@vngantk understood what is missing from Local Functions way before this topic, as did I and probably others. Allowing hoisting of Local Functions enables writing cleaner code by allowing programmer to use concept of functional decomposition. To define more detailed code after more generic code, so that it reads naturally when someone reads/maintains it.

We can mention Javascript and Scala and other who have implemented it… we can reminisce on Pascal flaws related to this, but I don’t find it worthwhile to ask for something because others have it. I’m asking for Local Functions to behave just as Kotlin’s toplevel functions which may or may not use toplevel val/vars, or just as class methods which may or may not use class properties. If that was done, they can be called before their definition, which in turn allows us not to have a bunch of private methods in the class scope which are only used to make one method more readable/maintainable. They should not be visible outside their parent function and their only definition place is right in that parent func. We want our code to be hierarchical by their complexity as shown in Wiki, we don’t want all helper functions on a flat class scope with only hint that they’re helper as KotlinDoc comment and private keyword. We don’t want them to mix with helper methods from other funcs… They don’t even warant for usage of “Extract Class” refactoring pattern, since they don’t break SRP principle, they’re just detailed implementation of the parent func().

Using Local Functions as helper methods does not void using them for closures, that was a weak argument against them. Yet limiting forward referencing by language design because someone may not understand their scope is equally weak argument against them. Especially, for the n-th time, since the same “problem” exists for toplevel functions and in classes but hoisting is not only allowed, but done, in Kotlin…

5 Likes
12

I second this. I really would like to be able to put local functions at the end of their outer function’s scope.

2 Likes
13

I’ll add my support for this feature. I find Uncle Bob’s “stepdown” technique (hoisted local functions) compelling. I use it in my Kotlin code but elide the code with IntelliJ to make it more readable, a barely acceptable workaround. A first class treatment making local functions usable anywhere within the parent function is a very reasonable request. Dumbing down the implementation using a rationale of protecting new Users is unreasonable in the face of training that encourages the use of clean code (stepdown technique). New developers should be encouraged and empowered to use techniques to write clean code.

1 Like
14

I still hope that Kotlin would support hoisted local functions. Scala’s local functions are hoisted by nature (Javascript of course, because of its dynamic nature), so there is no reason why Kotlin cannot do it. I know that if we just change the local functions to behave as hoisted there would be backward compatibility issues. So, I would suggest adding a keyword “hoist” to indicate local functions that need to be hoisted. Local functions without this keyword remain unhoisted. For example,

fun outerFunction() {
    bar() // okay: forward reference to an hoisted function
    foo() // error: forward reference to an unhoisted function

    fun foo() {
        bar() // okay: forward reference to an hoisted function
    }

    hoist fun bar() {
        bar() // okay: backward reference to an unhoisted function
        baz() // error: forward reference to an unhoisted function
    }

    fun baz() {
    }
}

The rule is simple: local functions marked with the “hoist” keyword are hoisted to the current enclosing scope, so that all functions in that scope can reference them regardless to their positions. Local functions without the “hoist” keyword remain the same with the forward referencing restriction.

15

I would prefer that the compiler recognize that a local function has been defined without a body and declare an error if the body is not found within the outer function scope. This as an alternative to implicit forwarding. Which kind of says I prefer not to add a keyword like “hoist” or anything else.

16

The old solution to this problem was a forward declaration. Does Kotlin support this? (I’m still learning the language.)

17

Not for local functions.

19

Well, you could do this, if you really need this functionality. It’s a bit stupid that I have to assign dummy functions, and another alternative would be to use null instead, but then one would need foo!!() at the call site.

fun topLevelFn() {
    // Forward declarations
    var foo: () -> Unit = {}
    var bar: () -> Unit = {}

    foo = {
        bar() // This call works
    }
    bar = {
        bar() // Call to self is fine
    }

    // The functions can be called normally
    foo()
    bar()
}
20

This doesn’t work. What @fatjoe79 wants (and me as well) is something like this

fun topLevelFn() {
    // some code
    utilityFn()
    // some more code 

   fun utilityFn() {
   }
}

The idea is that the less important utility functions can be hidden at the end of the function code. Right now you have to put them at the top where they just take up screen space.

Your system with forward declarations won’t work

fun topLevelFn() {
    var foo: () -> Unit = {}
    // some Code
    foo() // This call works but doesn't do anything
    // some more code
    foo = {  // totally useless assignment, since foo has been used before and won't be used again.
        //  do something
    }
}
21

That is true. I started working on a version which would handle it, which was a huge mess of a DSL which really didn’t work out in any nice way at all.

As someone who is normally a Lisp programmer, this made me really want a good macro system. Every time I program Kotlin and I implement DSL-like things, I keep being annoyed by the fact that it’s so limiting. Sure, it’s much more flexible than Java, but compared to what it could be, it’s really quite limited.

I wish Jetbrains would be in favour of implementing stronger metaprogramming facilities, but it would seem the concern is that people would abuse it. I don’t agree, but that’s just my opinion.