If you have defined them nullable just because they do not have initial value (and not because it’s valid for them to have null value) lateinit properties may be of help.
Thanks. I don’t think lateinit properties are appropriate for my cases (DTO beans).
Typically it is valid that these properties have a null value. These “beans” are typically marshalled/unmarshalled from JSON. After the beans have be marshalled from JSON the application may check “application validation” (i.e. javax bean validation) including required/not null (but it is valid compiler wise that they are null).
In a similar manor I’d like to be able to do:
// non-null Int ... so primative int (and hence initialised to 0)
var statusCode: Int
rather than having to specify the 0 value:
var statusCode: Int = 0
I’m being picky regarding the syntax but it stands out to me as the weird time where Java syntax is arguably nicer (ignoring the getter/setter lack of properties).
We plan to natively support JSON marshalling/unmarshalling for such “beans”. So here is a question about your particular use-case. Do you really want to have “null” as the default there or “application validation” is going to reject those nulls anyway?
From my point of view, but I might be mistaken, what @rbygrave is suggesting is basically to avoid obvious initializations of nullable variables to null. Since they’re nullable, if no other value is specified would be safe to assume that the variable has a start value of null, similarly as how Java works.
What I’ll try to add is that at “system boundaries” like REST API’s there is a reasonable tendency to allow null values (and LOTs of them). These “DTO beans” frequently support multiple use cases and each use case has different requirements for what properties are allowed to be null. In short … there is a lot of " = null" and me I think that is ugly and unnecessary.
natively support JSON marshalling/unmarshalling for such “beans”.
That is a whole subject in itself and I’m hesitant to get into too much in this thread/subject but my personal experience suggests this is not a simple easy space (think Jackson core, Jackson ObjectMapper, handling bi-directional object graphs, @JsonIgnore, plugins, odd things (invalid JSON) we need to do for ElasticSearch bulk API etc. I’d suggest you talk to Tatu Saloranta a bit when you are doing that. My personal view is that the Java JSONP spec falls short in many real world cases. I myself integrate Jackson core into Ebean ORM so I’m reasonably aware of some of these issues - all the types Ebean supports have JSON support, integration with DB JSON types (like Postgres JSONB) and ElasticSearch.
Native JSON marshalling/unmarshalling is a very honorable goal - ping me if you think I can help or want an opinion.
Thank you for your offer to help with serialization. I’ll post all details when serialization prototype becomes avialable for preview with some kind of Kotlin EAP version without having to build your separate compiler. I do understand that serialization is not easy, and we are ready to take our time to get it right. It would be especially helpful to get feedback from people who have encountered various issues with other solutions and can give particular examples and use-cases.
With respect to this particular topic, I truly want to understand how nulls are used at system boundaries / REST APIs. Don’t get me wrong here, please. I’m not advocating here for or against this particular syntax change. It is just that in the kind of code I’m typically writing nulls are usually used only as a temporarily value during object deserialization from some kind of other representation. I usually don’t want my vals to be actually nullable. I would have those null default values only transiently, and will quickly replace them with values that were read from some kind of database, configuration file, or some other external representation. E.g. mine are all deserialization use-cases. Deserialization we plan to solve. But what are other use-cases for such nulls?
The decision to require explicit initializers for all properties was made in order to make it easier to reason about the code: to make it clear where and how each property is initialized. We were of course aware that it would require to put explicit initializers in certain cases where Java doesn’t require them, making the Java syntax arguably nicer for those cases. To revisit this decision, we’d need some fairly strong new arguments explaining why the existing design doesn’t work well enough. So far, it seems that there are only a few cases (yours included) where the added verbosity is noticeable.
I’m going to go with an example to see if that helps clarify things. Lets say we have a bean with a String name property and consider Marshalling and “Application validation”. In short I’m guessing the difference is that a null value for a “required” property is treated as a “Application validation issue” or a “Marshalling issue” depending on the approach.
For this name property lets say is is @NotNull@Size(min=3, max=100)
var name: String? = null
Marshalling will result in “Bad request” if the type is not a String, but null is treated as valid. Application validation runs and validates the name property. If the property fails validation (for not null, or characters between 3 and 100 … or any other validation rules) the validation framework creates a response that provides a nice error message for each property. The error message could include i8n translation, and the response could be in the form such that clients can automatically relate the error messages but to user interface fields (e.g. return a map of property path to validation error message).
var lateinit name: String
In this case I suspect it becomes a “Bad request” / marshalling error if no name is supplied by the request.
If this is the case then the difference with 1) above boils down to treating missing/null properties as marshaling errors as opposed to application validation errors.
Note that for “Application validation” I’m pretty much referring to “Java bean validation”.
Note that “Java bean validation” also includes support for validation groups. That is, different validation can be run for different uses cases. Is the cases of @NotNull we might have a use case that says name is allowed to be null (PATCH vs POST etc).
Hopefully the above helps clarify things. I suspect it comes down to if we treat null (missing value) as a marshaling error or a “validation error”.
Ok. For me personally I think it is reasonably intuitive a nullable type would default to null.
To paint the picture a bit more. In a Java8 + Compile static Groovy environment … where all the “DTO beans” are written in compile static Groovy - the Kotlin equivalent immediately looks odd/inelegant/unsatisfying and for me that is a real shame because it really should be “awesome” here.
That is, faced with an existing “Compile static Groovy fan base” it isn’t easy to push for another language (when Groovy/Spock is also used for testing). To my mind Kotlin should not “lose” to compile static Groovy anywhere (but this syntax makes it a loss for these DTO beans (which is one of the main areas compile static Groovy is used).
Thank for a detailed answer. So, my current thought is that we will be able to provide a Kotlin serialization framework that would natively understand Kotlin non-null types. For example, if you try to deserialize the the following data class
data class Person(val name: String)
from this JSON {} string or this JSON string {"name":null}, then you get “Bad request” just like it in the case when the property had bad type, because the name here is non-nullable String and it has no default value to use when it is not specified. Moreover, we will make deserialization easily customisable, so that you can have i8n for all the corresponding messages.
I’ll see how we can integrate it with bean validation for all the messages and to check additional constraints (beyond nullability) directly during deserialization. Would it be desirable to treat all kinds of errors uniformly? What bean validation implementation you’d recommend to play with for that?
PATCH vs POST is also a very important observation. I’ve completely overlooked PATCH needs so far. Could you point me to some example code, please, that would show a typical pattern for working with a PATCH method?
Note that a property can be initialized either inline or in the init block of a class. Right now if you see a property without an initializer, you know that it’s initialized in init. If we allowed to omit initializers, a property without an initializer can either have the default value or be initialized in init, and you don’t know which.
Can you clarify for me how it is handled when one use case has the property as nullable and a second use case has the same property as not null?
Are you talking about PATCH vs PUT? For PATCH vs PUT I think the solution is to support PATCH natively in a serialization framework. E.g. for PUT you’d do something similar to your regular JSON-maniupation code, but in a type-safe way:
val myObject = JSON.parse<MyClass>(request)
For PATCH we can implement something like this:
val originalObject = somehowGetMyOriginalObjectFromStorage()
val myObject = JSON.patch(request, originalObject)
So, you’ll use just one MyClass for both PUT and PATCH and you will not need any nullable properties, unless they are really nullable in your data model. Are there other use-cases?
No. In one use case it really is nullable, and in another it is not. i.e. “Validation groups”.
Personally I’m not keen on that. We can do a patch without doing the typically expensive … somehowGetMyOriginalObjectFromStorage(). That is, this mechanism is roughly doubling the cost because of the desire for the strict non-null rule at marshaling time (as I see it anyway). For myself no, I would not buy into that idea.
I’m going to guess there is not a plan for partial rendering (something like Jackson views). The scenario where we want to limit the properties rendered (excluding what you might otherwise by defining as non-nullable properties). There is a pretty good presentation from LinkedIn on the DSL they have for this (from 2009 if I recall correctly).
Man, this is starting to go off topic from the … “please can we exclude the = null syntax requirement”
Can you, please, share some more details about partial rendering in serialization thread here: Kotlin Serialization - #21 by mikehearn
We are still very early with it to say that we have or don’t have any plans with respect to any specific feature of serailzation. The more use-cases we gather, the better we’ll be able to support them. P.S.: right now I am prototyping interface serialization that could be used for some instances of partial rendering like jackson views, but I don’t know if that is going to cover your specific use-cases or not.
One particular aspect with serialization and null values is multiple format versions. In many such cases you would want to distinguish an “unspecified” value from a value that happens to be the default value. When using the value you want the null to be substituted by the default. When writing the value you don’t want to store it, or store it as null.
We definitely plan to support multiple format versions in Kotlin Serialization, but how exactly is still to be figured out. Nothing is even prototyped yet in that direction and I’d be really glad for any help with real-life use-cases, so that we can anchor our design process around something. Reading “older version” and replacing missing values with defaults seems to be easy and will definitely be supported. Writing a value and omitting defaults (protobuf-style) is also a no brainer and we’ll have to support it, because native protobuf support is high on our wish list (protobuf and JSON will likely be two serialization formats supported “out of the box” in the standard library).
However, things like “distinguish whether this default value was actually specified or was just plugged in because it was missing” require a substantial amount of extra design. I have some ideas on how to natively and seamlessly integrate it (having a language in your hands gives you much more freedom compared to a typical library writer), but it still has to be substantiated with use-cases to justify extra effort and extra complexity of the result.
From my perspective these aspects are things that would mainly provided by the user, not the standard library (or tooling/code generators). What I’d like to see from the serialization framework is sufficient flexibility and extensibility that allows this to be done (by users). Of course there are other reasons to support null (such as truly optional values). Null values may also be easier to support than union (C style) types where one serialized element can have multiple representations, depending on attribute values (eg. the object either has attributes a and b or c, d and e and in any case f). The policy to deal with that should probably be in the developer (not library writer) hands as there is no general “right” policy.
