Proposal: syntax to condense class initialization via DSLs

Language Design
1

The ability to define one or more implicit receivers during initialization of an object would make DSLs even more useful.

Consider the following case, currently possible with contracts:

class Foo {
    val bar: Header
    var jaz: Body

    init {
        html {
            bar = header()
            jaz = body()
        }
    }
}

Compared to Kotlin’s regular constructor parameter and val declaration syntax, this is still pretty cumbersome, I think. (though at least we don’t need “lateinit vars” any more.) My proposal is to allow something like this instead:

class Foo = html {
    val bar = header()
    val jaz = body()
}

Or if we wanna get crazy:

class Foo = html {
    val bar = header()
    val jaz = body {
        outer val text = paragraph("Hello!")
    }
}

This assumes each DSL method has a contract of “callsInPlace(EXACTLY_ONCE)”. Alternatively, we could define the syntax like this (even at top-level scope), though it could be ambiguous in the case of inner classes:

html {
    class Foo {
        val bar = header()
        val jaz = body()
    }
}

Thoughts, ideas, improvements, or suggestions? Any hard reasons why this couldn’t work?

1 Like
2

So it’s been a year and a half, and I would still absolutely love this feature. This syntax in particular:

class Foo = html {
    val bar = header()
    val jaz = body()
}

…would be a godsend! Especially for larger classes, where halving the line count would really improve legibility.

Just to add, I’ve recently discovered you can do something similar with anonymous classes:

val myWebpage = html {
    object {
        val bar = header()
        val jaz = body()
    }
}

…which is great for some use cases, but then you lose the numerous advantages of having a named class with a constructor. This syntax surprised me, however, since it’s probably even more complex compiler-wise than what I’m asking for (just wrapping the initializers and constructor with the inlined method, and introducing the methods/properties in scope.)

This feature would seriously improve the libraries I’m working on - I regard it as high importance given the frequency with which I’m invoking DSLs during initialization and setting up properties. If any of you feel the same, speak now or forever hold your peace!

3

How would the compiler know whether val bar = should create a local variable or a property? What about functions and other features of the class?

Also, this feature seems strange to me, because what it does basically is to help defining multiple similar classes, according to some pattern. Isn’t OOP designed specifically for this? What is your specific use case?

4

EDIT: I think this code might help to clarify what I’m talking about. In the original example, html is a Kotlin DSL, defined as you normally would:

inline fun <T> html(block: HtmlBuilder.() -> T): T {
    contract { callsInPlace(block, InvocationKind.EXACTLY_ONCE) }
    return HtmlBuilder().block()
}

@DslMarker
annotation class HtmlDsl

@HtmlDsl
class HtmlBuilder {
    fun header(): HtmlHeader = ...
    fun body(): HtmlBody = ...
    fun paragraph(text: String): HtmlParagraph = ...
}

I just want the ability to invoke the “DSL” functions of HtmlBuilder in property initializers of any given class. The only way to currently achieve this is via an anonymous object inside the builder (see above.) You cannot currently achieve this in a named class, the alternatives with identical effect are either too verbose or brittle imo.

Hopefully that helps, but if still not clear, read on.

How would the compiler know whether val bar = should create a local variable or a property?

The same way it currently does. You cannot define local variables inside a class, only properties.

What about functions and other features of the class?

Functions work the same as before - their scope is unaffected. Only initializers and possibly constructors would have access to DSL functions/methods. Companion objects are also unaffected. Not sure what other features there are to consider.

what it does basically is to help defining multiple similar classes, according to some pattern.

Similar how? Classes can have whatever properties you create for them. They can be arbitrarily different, as can a normal class. The only difference here is that the initializers and possibly constructors have access to DSL methods. I don’t see how OOP is related here.

Regarding OOP, I suppose you could define your DSL as an abstract class that your “container” classes could inherit from. But I feel this is lacking for a few reasons -

  1. You can only inherit from one abstract class, meaning its a highly contended resource. Also, while you could use protected methods in this case, you couldn’t reuse the “DSL” outside of this context.
  2. Interface default methods otoh are always public facing in Kotlin, and other JVM languages I’m aware of. Presumably you wouldn’t want clients of the class to see the DSL methods used to instantiate it.

Ultimately it feels like the wrong tool for the job - if one of the two points were to change, I’d be more amenable to it, however.

What is your specific use case?

I provided one with html elements that I think would apply to a lot of people, but I have several that I actively use - mainly UI definition, and various OpenGL constructs - render pipeline, vertex layouts, shader compilation into programs, etc.

Here’s an example class with some ugliness that could be fixed by this feature:

class Programs(
        device: GLDevice,
        private val paths: NebRaxPaths,
) {
    val manager = ProgramManager(device, paths.shaderLogsDir, ::getShaderReader)
    private val programList = ArrayList<Program>(64)

    private fun getShaderReader(resourceName: String): Reader {
        return paths.shaderDirs.firstNotNullOfOrNull { it.resolve(resourceName).takeIf { it.isRegularFile() } }?.reader()
                ?: error("Couldn't find shader resource with name '$resourceName'!")
    }

    // I'd like to eliminate the following "DSL redefinition":
    private fun program(vararg shaders: Shader) = manager.makeProgram(*shaders).also { programList += it.acquire() }
    private fun program(vararg shaders: String) = manager.makeProgram(*shaders).also { programList += it.acquire() }
    private fun shader(shaderKey: String) = manager.makeShader(shaderKey)
    private fun shader(shaderKey: String, constParams: Map<String, String>) = manager.makeShader(shaderKey, constParams)
    private fun shader(shaderKey: String, vararg constParams: String) = manager.makeShader(shaderKey, *constParams)
    private fun constParams(vararg keyValues: String) = ProgramManager.makeConstParams(*keyValues)


    //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
    // Generic

    val simpleTextured = program("simple/textured.glvs", "simple/textured.glfs")
    val simpleDepth = program("simple/textureToDepth.glvs", "simple/textureToDepth.glfs")
    val textureToDepth = program("util/texture_to_depth.glvs", "util/texture_to_depth.glfs")
    
    val color = program("simple/color.glvs", "simple/color.glfs")
    val colorTextured = program("simple/color_textured.glvs", "simple/color_textured.glfs")
    val colorAlphaTextured = program("simple/color_textured.glvs", "simple/color_alpha_textured.glfs")
    val colorLayerTextured = program("simple/color_layer_textured.glvs", "simple/color_layer_textured.glfs")
    val colorTextureMasked = program("simple/color_textured.glvs", "simple/color_texture_masked.glfs")
    val colorIntTextured = program("simple/color_textured.glvs", "simple/color_int_textured.glfs")
    val colorDepthTextured = program("simple/color_textured.glvs", "simple/color_depth_textured.glfs")

    val blockDetailed = program("block/detailed.glvs", "block/block.glgs", "gbuffer/write_tex2D.glfs")
    val blockTiny = program("block/tiny.glvs", "block/block.glgs", "gbuffer/write_tex2D.glfs")
    val blockMarchinglyCubed = program("block/marchingly_cubed.glvs", "block/marchingly_cubed.glgs", "gbuffer/write_multi_mat.glfs")
    val blockMarchinglyCubedIT = program("block/marchingly_cubed_instanced_triangles.glvs", "gbuffer/write_multi_mat.glfs")

    val blockMarchinglyCubedFeedback = program(
            "block/marchingly_cubed.glvs",
            "block/marchingly_cubed_feedback.glgs"
    ).setTransformFeedbackVaryings(*MarchingCubesCompressedAttributeStrategy.BLOCK_MESH_TRANSFORM_FEEDBACK_VARYINGS)

    val blockMarchinglyCubedDepth = program("block/marchingly_cubed.glvs", "block/marchingly_cubed.glgs", "simple/depth.glfs")

    val light = program(
        shader("lighting/light.glvs"),
        shader("lighting/light.glfs", constParams("DISABLE_MATERIAL_UNIFORM", Config.DISABLE_MATERIAL_UNIFORM.toString())))

    val smaaEdge = program("anti_aliasing/smaa_edge.glvs", "anti_aliasing/smaa_edge.glfs")
    val smaaWeight = program("anti_aliasing/smaa_weight.glvs", "anti_aliasing/smaa_weight.glfs")
    val smaaBlend = program("anti_aliasing/smaa_blend.glvs", "anti_aliasing/smaa_blend.glfs")
    val blackHole = program("blackhole.glvs", "blackhole.glfs")
    val nanInfPass = program("simple/color_textured.glvs", "util/nan_inf_pass.glfs")

    //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

    init {
        manager.compilePending()
        device.exitOnGLError("programs")
    }

    fun onApplicationClosed() {
        programList.forEachNoGC { it.release() }
        programList.clear()
    }
}

Having to redefine methods like that for every application is brittle and far less than ideal, but the alternative is predeclaring every single program variable before assigning to it in an initializer block. This is a fortunate case in that the DSL is simple enough to be redefined in five lines, but for many DSLs this would not be a reasonable solution. Here’s a UI definition demonstrating how bad this can be:

class GuiHud(
        builder: GuiBuilder,
        private val pivotProvider: PRenderPivotProvider

) : PanelComponent(), IInGameWindow, IHudIndicatorProvider {

    // I'd like to eliminate all of these variable "pre-declarations":

    private val lblPlayerHeartRate: LabelComponent
    private val lblPlayerRadioactives: LabelComponent
    private val lblPlayerSelectedBlockType: LabelComponent
    private val lblPlayerShapePlaceMode: LabelComponent
    private val lblPlayerShapePlaceType: LabelComponent
    private val lblPlayerToolSnapping: LabelComponent
    private val mtrPlayerStamina: GuiMeter
    private val mtrPlayerBlood: GuiMeter
    private val mtrMultiRadioactives: GuiMeterMulti
    private val lblMessageCentered: LabelComponent
    private val lblMessageBottom: LabelComponent

    init { builder.applyAndBuild(this) {
        dock = FILL
        backgroundColor = 0

        panel {
            dock = LEFT
            preferredWidth = 300
            backgroundColor = 0

            panel {
                dock = TOP
                setMargin(10, 0, 10, 0)
                backgroundColor = 0

                lblPlayerHeartRate = label("BPM: 60") {
                    dock = Dock.RIGHT
                    marginX0 = 2
                    preferredWidth =  70
                    contentAlignment = CENTER
                    backgroundColor = 0x000000ff
                }
                mtrPlayerStamina = add(GuiMeter()) {
                    dock = TOP
                    preferredHeight = 16
                    foregroundColor = STAMINA_COLOR_RGBA
                    backgroundColor = STAMINA_BG_COLOR_RGBA
                }
                mtrPlayerBlood = add(GuiMeter()) {
                    dock = TOP
                    preferredHeight = 16
                    marginY0 = 2
                    value1Color = BLOOD_LOSS_COLOR_RGBA
                    foregroundColor = BLOOD_COLOR_RGBA
                    backgroundColor = 0x000000ff
                }
            }
        }
        val pnlRadioactivesWidth = 24
        panelFlowUp {
            dock = Dock.RIGHT
            preferredWidth = pnlRadioactivesWidth
            backgroundColor = 0

            lblPlayerRadioactives = label("???") {
                preferredHeight = 20
                contentAlignment = CENTER
                foregroundColor = -1
                backgroundColor = 0
            }
            mtrMultiRadioactives = add(GuiMeterMulti()) {
                setMargin(2)
                isFilledReverse = true
                isVertical = true
                overflowBehavior = FIT
                foregroundColor = -1
                backgroundColor = 0x000000ff
            }
        }
        panelFlowDown {
            setAnchorSrcDst(BOTTOM_RIGHT, BOTTOM_RIGHT)
            preferredWidth = 400
            backgroundColor = 0

            childConfig { preferredHeight = 20; backgroundColor = 0; isHidden = true }
            lblPlayerShapePlaceType = label("Placement Shape: Sphere") { contentAlignment = RIGHT }
            lblPlayerShapePlaceMode = label("Placement Mode: Normal") { contentAlignment = RIGHT }
            lblPlayerSelectedBlockType = label("Selected Material: None") { contentAlignment = RIGHT }
            lblPlayerToolSnapping = label("Voxel Snapping: Off") { contentAlignment = RIGHT }
        }
        withChildConfig({
            marginX1 = 300 - pnlRadioactivesWidth
            backgroundColor = 0
            isHidden = true
        }) {
            lblMessageBottom = label("") {
                dock = BOTTOM
                preferredHeight = 20
                contentAlignment = CENTER
            }
            lblMessageCentered = label("") {
                dock = FILL
                marginY0 = 20
                contentAlignment = CENTER
            }
        }
    }}

    // Insert class behavior here....
}

It’s just a lot of boilerplate. Not as many lines as the Programs class would require, but still not great. Having to look in two places to modify one thing is also not ideal. I imagine a lot of UI or client-side web code would benefit from this feature, given the prevalence of DSLs there in defining basic components.

5

Ok, I was initially confused about some things.

I asked about local variables and functions, because DSLs work by providing a lambda and one of the most important features of DSLs is that we can use any regular code inside them. So my concern was about something like this:

class Foo = html {
    val isProd = getConfig().isProd // local var
    val body = if (isProd) bodyForProd() else bodyForDev() // property
}

Or even this:

class Foo = html {
    if (condition) {
        val body = body() // property created conditionally
    }
}

Now I see that you don’t really think about DSLs as they are designed in Kotlin, but about something much different. You just need a standard class definition block, but with an implicit receiver. Which I think makes sense and could be useful. But your html() function example is misleading, because it clearly receives a lambda and I don’t think it can be done with lambdas.

Second thing that I was confused about was that html() is an util that is invoked statically, each instance of Foo just invokes html() and gets some objects from it. So I was wondering how is it better than just this:

class Foo {
    val header = header()
    val body = body()
}

fun header() { ... }
fun body() { ... }

But I guess the main difference is that by using html() the header and body are interconnected, which is not possible with separate functions. Ok, that makes sense.

Third confusing part was the OOP I asked. I imagine this feature would not make too much sense if we need to create a single Foo class. It makes sense only if we need to create several classes according to some pattern, but we don’t want to use OOP for this. And this is what concerns me, because it doesn’t look like just a small syntactic feature of the language, but more like an entirely new OOP concept. Probably somehow related to metaclasses.

Generally, your idea seems interesting and intriguing, but I’m not sure if this is enough to implement it. The most important thing are use cases. I still need to read through your examples.

6

Just to go ahead and address something - there are currently two features I’m thinking of:

  1. Inline lambda surrounding class initialization (or just implicit receiver)
  2. outer val to declare properties in nested scope during initialization.

While we’ve only discussed feature 1, feature 2 would be necessary to improve the UI code I listed. But let’s set aside feature 2 until we’ve settled feature 1 - there are plenty of use cases I can see for it without the other feature (which is less likely to be accepted anyway, imo.)

Replies

You just need a standard class definition block, but with an implicit receiver

Well, not exactly. Just an implicit receiver would be useful, but the html example is probably oversimplified, whereas inline functions in my code have both “setup” and “tear down” portions. In my UI code especially, I have methods like this:

inline fun <T> html(block: HtmlBuilder.() -> T): T {
    contract { callsInPlace(block, InvocationKind.EXACTLY_ONCE) }
    val builder = setupHtmlBuilder()
    try { return builder.block() }
    finally { builder.finishAndLaunch() }
}

This allows to accomplish things like childConfig as in the final example of my last post. The concept of “before and after” code is something that inline lambdas accomplish quite elegantly, and it would be great to reuse that functionality for class initialization.

Here’s a useful example for error logging, no DSL involved:

class Foo = logErrors {
    val prop1 = codeThatCanThrow()
    val prop2 = moreCodeThatCanThrow()
    ...
}

inline fun logErrors(block: () -> Unit) {
    contract { callsInPlace(block, InvocationKind.EXACTLY_ONCE) }
    try { block() }
    catch (t: Throwable) { myComplexLoggingMethod(t); throw t }
}

Again, this saves us the trouble having to separate property initialization into an init block, with declaration beforehand, just to have error logging.

I think the key benefit here is overcoming a fundamental limitation of property initialization - that there can be no code constructs surrounding them (e.g. try/catch/finally, local variables, etc.) There may be a better solution to this problem than what I’m describing (e.g. feature 2 above), but lambdas already get first-class treatment in Kotlin (e.g. apply, let, run, also), so it seems natural to extend them in this way. This is how the language designers decided to introduce implicit-receivers to us, after all, so it makes sense to prefer lambdas over a limited subset, imo.

And this is what concerns me, because it doesn’t look like just a small syntactic feature of the language, but more like an entirely new OOP concept. Probably somehow related to metaclasses.

I’m not sure how that would tie in, but in my mind, it really is as simple as “wrapping initialization with inline-lambda”, which ultimately makes initialization more convenient. DSLs are almost an added bonus, now that I’m writing things out. Classes using the same inline-lambda wouldn’t have any polymorphic relation, or any other change to their public API.

Feasibility

This is probably the biggest issue for this proposal. To accomplish this, code would have to be injected in two places - before and after all initializers and constructors. The inline function would need to be callsInPlace(EXACTLY_ONCE), and try/catch/finally inside the function would need to be respected by the final, inlined code. If this can all be accomplished, then I think we’re in the clear.

Just to be explicit, here’s what I think the ideal initialization order would be for a class hierarchy using this syntax:

  • Child constructor default arguments
  • Parent constructor default arguments
  • Child inline-lambda begin *
  • Parent inline-lambda begin *
  • Parent initializers
  • Parent constructors
  • Parent inline-lambda finish *
  • Child initializers
  • Child constructors
  • Child inline-lambda finish *

The point of contention here is probably Child inline-lambda begin happening before Parent inline-lambda begin. I think it’s possible to do, but if not, it could be moved to just after Parent inline-lambda finish. We’d have to go through use cases to really decide which is best, but the way I’ve written seems more intuitive to me.

There is another issue I thought about - whether the inline method can have more than one call-site for its lambda parameter (but is still called exactly once.) E.g. if (a) block(X) else block(Y). I think this can still be supported, but would inflate the bytecode quite badly, as it already does in other contexts. But it’s possible there’s some interaction here I’m not considering. Perhaps swapping the order as I stated above would ameliorate this. Fwiw, this scenario already trips up the debugger in my experience.

7

Yes, but these limitations were put there for a reason. They were intentional.

What is your expected value of val foo = Foo() if property initialization thrown an exception, then it was caught and logged?

8

Yes, but these limitations were put there for a reason. They were intentional.

I’m not suggesting that we should have those specific constructs - I’m saying this is an elegant solution to deal with the drawbacks of not having them. And in some sense, they actually do give us those constructs, just inside init blocks. Your argument can also be made about every existing language feature.

What is your expected value of val foo = Foo() if property initialization thrown an exception, then it was caught and logged?

Actually I meant that logger function to rethrow t (whoops), but in the case that you don’t, you live with a half initialized object. This can be avoided by simply not doing that - just like you shouldn’t invoke functions that reference a property before it’s been initialized, which is similarly undefined. Both Java and Kotlin part with that safety for the sake of ease of use.

EDIT: If we really want safety, the compiler could add a private flag to each class using this pattern, that gets checked before exiting the construction sequence. The last instruction in the sequence toggles the flag. If we throw to escape the sequence, then we don’t check the flag (as desired), but if we swallow an exception, then the flag will be checked while in the wrong state, and a language-level exception can be thrown.

9

This is exactly my point. Class definitions were designed this way specifically to make guarantees about the class structure and its data consistency. In dynamic languages like Python or JavaScript, we initialize an object with just a regular code. Then we can decide at runtime whether to create a property or not. In strongly typed languages like C++, Java or Kotlin all properties have to be known at the compile time. It should be clear for the compiler what these properties are and ideally, it should be also clear to a human. Also, the language guarantees that the object is either fully initialized or the whole operation fails.

Keeping these guarantees while allowing any code in the class definition, including if, for, try, etc. and even inlining some external code seems like a feature that is very hard to implement, if at all possible.

And allowing half-initialized objects seems like a very bad idea. Also, I believe technically impossible while targeting Java.

1 Like
10

Expanding on my earlier edit, I think that even a local variable in bytecode would be sufficient to detect the try/catch/finally swallow case and generate an error. for/while shouldn’t be a concern assuming the required EXACTLY_ONCE contract is upheld. (I actually don’t know how Kotlin handles it if the contract is incorrect, which is interesting.)

All properties should be known at compile-time, even if the contract is violated in some horrifically unsafe way - the only thing that might be affected is the generated <init> method, which assigns to them. If this code isn’t run, or run multiple times, then it’s actually no different than when this already occurs in an initializer block - Kotlin trusts contracts to determine whether a property is correctly initialized:

class Foo {
    val bar: Int
    init {
        0.let { bar = it }
        // Compiler determines that bar is assigned
        // on all paths, due to let's contract { callsInPlace(EXACTLY_ONCE) }
        // Otherwise there'd be a compile-time error!
    }
}

So the only concern is if we’re okay allowing constructors to be called a variable number of times, in the case of contracts being abused. If the answer is “no,” then I think we’d have to preclude the constructors from the inline-lambda, like this:

  • Child constructor default arguments
  • Parent constructor default arguments
  • Parent inline-lambda begin *
  • Parent initializers
  • Parent inline-lambda finish *
  • Parent constructors
  • Child inline-lambda begin *
  • Child initializers
  • Child inline-lambda finish *
  • Child constructors

We then lose access to implicit-receiver methods in the constructor, or wrapping parent code at all, but I think it’s an acceptable compromise. However, if we’re already okay with contract abuse during initializer blocks resulting in null properties, I have to wonder if messing up the constructor could result in much worse. Perhaps its only truly bad invoking the parent constructor variable times (since it’s not necessarily “our” code and thus a security issue, whereas we can take responsibility for errors involving our own constructor.) Thus we’d have:

  • Child constructor default arguments
  • Parent constructor default arguments
  • Parent inline-lambda begin *
  • Parent initializers
  • Parent constructors
  • Parent inline-lambda finish *
  • Child inline-lambda begin *
  • Child initializers
  • Child constructors
  • Child inline-lambda finish *

This would work better in the case of inline-lambda finish doing something meaningful, that might require the class to be fully initialized.

Walking back a bit, I went ahead and implemented an abstract ProgramSet in my code and found it to be more acceptable than a DSL would, including in places where I had wrapped anonymous objects (they now just inherit from ProgramSet.) This is mainly because the class can automatically manage the lifecycle of generated shaders, whereas a DSL makes that more involved. For a “container class” pattern, appropriating the parent class for this purpose seems reasonable, whereas in other contexts (e.g. UI code), it wouldn’t be possible.

I still have a DSL for specifying programs “inline” with the render pipeline, but using the container class was a definite improvement in a few places. I’ll have to think about use cases a bit more… that outer val feature would really help the UI code more than anything. Now that we’ve gone into this feature, I’m thinking the other one may actually be simpler - the syntax is just more surprising, I think.

11

Sorry, I don’t follow you. I seriously have no idea what even the simplest code samples are expected to do:

class Foo = whatever {
    if (condition) {
        val prop = 5
    } else {
        val prop = "5"
    }
}

class Foo = whatever {
    for (i in 0..10) {
        val foo = i
    }
}

And what if it would be var foo?

class Foo = whatever {
    run {
        val foo = 5 // local variable or property?
    }
}

None of these cases exist in the language right now. All of them and probably more would have to be described in the language specification and implemented just for this single feature.

12

None of what you just posted is valid under my proposal. “Feature 1” has absolutely no new syntax besides = whatever {. It just inlines whatever around the entire initialization code and makes the implicit receiver available to it, if one is defined by whatever. Code that has access to the implicit receiver includes expressions in property initializers, init blocks, and possibly constructors. Please refer back to the html example (minus the outer val thing.)

EDIT: iow, the curly braces are denoting a standard class definition, not a function body. Sorry, I can see why that’d be confusing now!

We haven’t talked about “feature 2” yet, I basically just said an outer val syntax would be nice to define properties inside nested scopes. Of course branching wouldn’t make sense there, each nested scope would have to be effectively contract { callsInPlace(EXACTLY_ONCE) }. But I’m still thinking about it - I was hoping to just discuss “feature 1” for now.

13

Ok, so that solves most of above problems and this was actually my initial understanding: :wink:

Still, representing class definition block as a lambda seems like a huge hack to me, so I don’t know if we can re-use the same syntax as in regular DSLs. But the concept generally seems fine to me.

1 Like