Base class parameter pass through

Language Design
1

The Problem.
Consider the Django Fieldclass . This class serves as a base class for several other classes, such as TextField, TimeField, BinaryField etc. To code examples like these cleanly, some type of language mechanism is needed which effectively achieves: "include all fields from the base class constructor (or other method from the base class) as also parameters to this class unless specifically declared in this class".

Python uses the *args,**kwargs system, which for several reasons is not an appropriate solution for Kotlin, but is at least a solution. There are some more elegant solutions possible for Kotlin, and can be just compiler syntactic sugar.

In Python the code for defining the BinaryField Class is as follows (not exact code for simplicity):

class BinaryField(Field):
     def __init__(self, *args, **kwargs): 
              kwargs['editable'] = False
              super().__init__(*args, **kwargs) // other init code goes here

while in Kotlin (slightly changed for name conventions and simplicity) the code becomes:

class BinaryField(
      verboseName:String?=null, name:String?=null, primaryKey:Boolean=false,
      maxLength:Int?=null, unique:Boolean=false, blank:Boolean=false, nulled:Boolean=false,
      dbIndex:Boolean=false, rel:String?=null, default:Any?=null, 
      //editable:Boolean=true, - force 'false' for editable serialize:Boolean=true,
      uniqueForYear:Int?=null, choices:String?=null, helpText:String="",
      dbColumn:Int?=null, dbTablespace:Int?=null, autoCreated:Boolean=false,
      validators:List?=null, errorMessages:String?=null 
    ):Field(
        verboseName=verboseName, name=name, primaryKey=primaryKey, 
        maxLength=maxLength, unique=unique, blank=blank, nulled=nulled,
        dbIndex=dbIndex, rel=rel, default=default, 
        editable=false,
       serialize=serialize, uniqueForYear=uniqueForYear, choices=choices,
       helpText=helpText, dbColumn=dbColumn, dbTablespace=dbTablespace,
       autoCreated=autoCreated, validators=validators, errorMessages=errorMessages
   ) { // class code here }

Clearly, the call to the base constructor will be much shorter if not using named parameters, which is a choice, but in a list this long I would use named parameters.

The code (or almost identical code) will be repeated TextField, TimeField and the over 12 other fields that inherit from Field. Any update to the parameter list for the base Field class is tedious to say the least.
This is a case where Kotlin requires boilerplate that is not required in python. What is needed is some way to say “accept all parameters to the base default constructor not specifically named in the base call, and pass these parameters through”. Given this problem will not occur in Java libraries which have no default parameters, perhaps solving a problem of this nature is of less problem.

However, given pragmatic nature of the language in avoiding boilerplate, it would be great to have solution. I can think of ways that would be very workable, but I would imagine the Kotlin designers can do even better than I.

Does anyone else see this as something worth considering?

2 Likes
2

Yes, I agree it’d be nice to have something as concise as the Python example, but typed


I mean, you can do something like this:

data class FieldProps(
    var verboseName: String? = null,
    var name: String? = null,
    var blank: Boolean = false,
    var editable: Boolean = true
)

open class Field(props: FieldProps) {
    val verboseName: String? = props.verboseName
    val name: String? = props.name
    val blank: Boolean = props.blank
    val editable: Boolean = props.editable

    companion object {
        operator fun invoke(optsBuilder: FieldProps.() -> Unit): Field {
            val props = FieldProps()
            props.optsBuilder()
            return Field(props)
        }
    }
}

class BinaryField(opts: FieldProps) : Field(opts.copy(editable=false)) { // <-- editable=false override here
    companion object {
        operator fun invoke(optsBuilder: FieldProps.() -> Unit): BinaryField {
            val props = FieldProps()
            props.optsBuilder()
            return BinaryField(props)
        }
    }
}

fun test() {
    val field = Field {
        name = "def"
    }
    
    val binaryField = BinaryField {
        verboseName = "abc"
    }
}

But it’s not DRY and still requires lots of boilerplate code


3

For all your Python lovers this issue contains a Python-inspired proposal to reduce the boilerplate in the above solution: https://youtrack.jetbrains.com/issue/KT-15471

2 Likes
4

Brilliant. !!

To quote the issue

(The whole concept is shamelessly lifted from Python, but it is imbued with an appopriate Kotlin-style type-safety. The whole idea is that it enables quite rich Python-style DSLs in a type-safe way).

(the ‘r’ had already been appopriated from the original :slight_smile: )

The addition of a class definition raises the implementation beyond just lifting an idea from python into "lifting and then greatly enhancing an idea from python".

But consideration is need to successfully deliver the enhancement. I have used this functionality extensively when writing python packages, so have experience with use cases. In the python code most often the class for **kwargs as received does not exactly match the class for **kwargs as expanded for calling the base class. As in the BinaryField example above, where kwargs is modified by adding editable= false in kwargs before use with the base constructor. Another common example would be where a parameter has type Any in the base class but may be Int or String or other in the sub-class. It is desirable to not need to also subclass the kwargs substitute class each time resulting in two sub-classes in place of one each time. Generally, these cases can be solved by allowing parameters specified outside the ‘kwargs’ to override definitions with a matching name inside the kwargs class.

This means that the for the BinaryField example, the code could become:

class BinaryField(dataargs kwargs: FieldData): Field(**kwargs, editable=false)

The editable=false would need to be the parameter to Field replacing any value in kwargs. This does not prevent the caller specifying a value for editable, even though it would not be used, however this same problem is present with the python code. The FieldData class could specifically exclude editable, but this would be an overhead for most classes which do pass editable through, and by design requires the baseclass to anticipate all use cases of sub classes in order to exclude those fields.

The second example is where type changes, like for example:

class IntField(dataargs kwargs: FieldData, val value:Int = 0): Field(**kwargs)

where value in FieldData has type Any, but in different subclasses it has more a useful class.
In this case with IntField class there would be separate value property with type Int as well as the value within the kwargs property with type Any.

5

Yeah, like @innov8ian pointed out, the class-based solution seems quite rigid and can lose information
 I’m thinking it’d be nice (maintenance-friendly) to just support this parameter reuse/tweaking directly, perhaps something like this:

// primary constructors:
class BinaryField(**args of super except [ editable ]) : Field(**args, editable = false)

// functions in general (if several, generate all of them?):
override fun someFunc(**args of override) {
    println("Override!")
    super(**args)
}

// as help for composition:
fun bubu(**args of bubuImpl::bubu) {
    bubuImpl.bubu(**args)
}

// or even
fun baba(**args of member::func, **moreArgs of otherMember::otherFunc) {
    member.func(**args)
    otherMember.otherFunc(**moreArgs)
}
6

Maybe it is easier to introduce an object which is essentially a map with annotated keys (annotation shows possible values and description), which could be used by IDE to show hints? This solution does not require new language constructs, only minor IDE tweaks. Also it in fact have much broader usage than just parameter pass through. For example it could be used for all kinds of schema in XML or any other markup language. Also it could be easily connected to groovy maps and map constructors.

7

Not sure I know what you mean, @darksnake
 In what way are these new objects that you’d introduce different from the objects that already exist?

8

The thing is that you want to for a function to take a set of arguments and pass it through to another function possibly slightly changing them in process. The obvious solution is to use a single Map<String, Any> as an argument, modify it if needed and pass though. This approach is used a lot in Groovy. The problem is that this map is not type safe and does not have any hints about what parameters could be there and what types and values are allowed.
It is not that hard to create a structure which stores this information (keys, types, default values etc) either dynamically or statically. In fact one already have this information in Method reflection object. The only thing one needs is a way to define these rules.
I use this approach to pass tree-like configuration structures (it is called Meta id DataForge) to any object (which is in fact much more complicated than simple maps). Everything works just fine and the only problem is that I do not have IDE hints. Since in most cases the description for this tree is available in compile-time, it is possible to create a compile-time inspection to add those hints (I just don’t have time to study IDEA internal workings).

9

Thanks for the longer explanation, though I’m still not sure what the difference is supposed to be? Isn’t a Map<String, Any> in which keys and associated types are pre-defined, exactly equivalent to an object/class?

10

There are few differences:

  1. The map could be transformed, so it is possible to pass part of the map to one function, and the other part to another one or make some complex operation on it.
  2. It allows in some cases to work without declaration, meaning that parsing and interpretation of parameters is delegated to underlying function. Usually it is bad pattern, but sometimes it greatly simplifies convention-over-configuration.
  3. The most important. It does not require additional language structures!
11

A map would seem to be less typesafe than a class/object and it would diminish optimization possibilities in the compiler.

12

Indeed, it will.
But on the over hand, there are cases when map is easier. Consider some interface, say multivariate function integrator, and multiple implementations, each of them requiring its own set of mandatory and optional parameters. The only way to pass parameters to any of these implementations in a type-safe way is to make this implementation generic and pass configuration type as a type-parameter. If some of implementations share a basic set of parameters, but have different optional parameters, one will have to build the whole generic architecture. If one would have a way to specify a declaration for map-parameter, it could be done without generics at all.
The map-parameters are a part of dynamic language feature set, so they are not very native to kotlin. On the other hand, as I already said, implementing them does not require new language constructs.
I can’t avoid it since most of the type I work with user configurations which are not known at compile time and I implemented it myself. The only problem is IDE support.

13

But how would you “specify a declaration for map-parameter” without also getting all the problems a class-based solution has?

14

For example by Annotations. Like that:

@ValueDefs(
        ValueDef(name = "normalize", type = arrayOf(ValueType.BOOLEAN), def = "true", info = "Normalize t0 dependencies"),
        ValueDef(name = "t0", type = arrayOf(ValueType.NUMBER), def = "30e3", info = "The default t0 in nanoseconds"),
        ValueDef(name = "window.lo", type = arrayOf(ValueType.NUMBER), def = "500", info = "Lower boundary for amplitude window"),
        ValueDef(name = "window.up", type = arrayOf(ValueType.NUMBER), def = "10000", info = "Upper boundary for amplitude window"),
        ValueDef(name = "binNum", type = arrayOf(ValueType.NUMBER), def = "1000", info = "Number of bins for time histogram"),
        ValueDef(name = "binSize", type = arrayOf(ValueType.NUMBER), info = "Size of bin for time histogram. By default is defined automatically")
)
@NodeDefs(
        NodeDef(name = "histogram", info = "Configuration for  histogram plots"),
        NodeDef(name = "plot", info = "Configuration for stat plots")
)

Of course, you will have to check consistency of actually passed parameters dynamically. On the bad side, it is dynamic, on the good side, you do not need to bother with configuration parsing, it is parsed automatically where it is consumed. Also it is quite easy to transform map (or tree as in my example) into actual parameters via reflections (in my case I don’t need it, because I use this Meta everywhere).

15

I like the except idea.

Generally, what is needed is for each case is:

baseParameterSet + additions - subtractions

This addition/subtraction type of ‘tweaking’ is needed for reuse as you said.

The except provides the minus, while providing the plus through specifying other parameters is a well know solution.

The alternative idea of using a map allows storing the tweaked parameter set, but personally, having heavily used this type of feature a lot, i strongly prefer the class with the tweaks specifically and clearly stated. Maps can provide a very flexible solution where an altered map could be reused- but usually each sub-class will will have its own tweaks, and a mechanism to explicitly convey the tweaks will I believe produce better code.

The proposal on youtrack by @elizarov (I which I think every one should vote for :slight_smile: ) is to effectively declare an instance of the ‘parameters’ class as a var parameter to class, capturing all parameters with matching names into instancing that ‘parameters’ object. This is really the way to go, the complexity raised by the except idea is that a class member excepted need not have a default value. I think the solution here is to leave that member uninitialised, like an open val with an override during base class init.

As proposed, and with the correct provision for additions and subtractions (subtractions from the except), this is a huge step forward over what is in python. Standing on the shoulders of others to reach new heights. Plus, it solves other specific limitations of kotlin.