Kotlinx serialization: How to migrate map from 0.20.0 to 1.0-M1-1.4.0-rc

1

Not sure this is the right forum but here goes:

We use kotlinx serialization 0.20.0, and just tried migrating to 1.0-M1-1.4.0-rc.

We have a map like this

@Serializable
class TripStorageData {
    val activeTripMap = mutableMapOf<TripId, TripStateInfo>()
}

@Parcelize
@Serializable
data class TripId(val serverId: Int, val id: Long, val type: TripType) : Parcelable {
    override fun toString(): String =
            """TripId(serverId=$serverId, id=$id, type=TripType.$type)"""
}

The new serialization version does not support may keys except as enums or primitives. I cannot see that change in the change log, but I suppose that is a changelog omission rather than a bug: kotlinx.serialization/CHANGELOG.md at master · Kotlin/kotlinx.serialization · GitHub

How do I migrate my code, and do I really have to use a string? Perhaps I need a custom serialization? I’m not quite sure where to start digging. Any help is appreciated.

Best Alex

2

Can you provide a better reproducer and explain what is the expected result?

3

Hi fvasco,

That question makes zero sense for me, if it is a feature that map keys can only be enums or primitives. GIven that the compiler says so to me, it is not just a random bug. Why are you asking for a sample? Do you know whether or not it is the case that map keys must now follow that rule?

Best Alex

4

It is not clear to me what format are using, what is the error output and what is your goal.

Yes, I might.

5

Hi Francesco, Thank for you for your time and effort. I’ve created a sample project containing the error, attached as zip.

EDIT: Man that’s bloody embarrasing. Apparently I didn’t read the error-message carefully and completely, and thus missed the nice detail that they added that told me exactly what to do. So all I have to do is enable structed keys, and I’m done. so problem solved.

I’ll include the below, in case other’s drop by later, though.

==========

My goal is to store my data on disk, and I have stored is as a map. In hindsight I see that I could have just stored is list, and moved to map after deserializing, but this was easier, and fit the bill at the time. I already have apps ‘out there’ with the old data format so in order to migrate to newer kotlin I have to somehow handle it.

I get the following error

Exception in thread "main" kotlinx.serialization.json.internal.JsonEncodingException: Value of type 'dk.MyTripId' can't be used in JSON as a key in the map. It should have either primitive or enum kind, but its kind is 'CLASS'.
Use 'allowStructuredMapKeys = true' in 'Json {}' builder to convert such maps to [key1, value1, key2, value2,...] arrays.
	at kotlinx.serialization.json.internal.JsonExceptionsKt.InvalidKeyKindException(JsonExceptions.kt:67)
	at kotlinx.serialization.json.internal.WriteModeKt.switchMode(WriteMode.kt:58)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.beginStructure(StreamingJsonEncoder.kt:63)
	at kotlinx.serialization.encoding.Encoder$DefaultImpls.beginCollection(Encoding.kt:268)
	at kotlinx.serialization.json.JsonEncoder$DefaultImpls.beginCollection(JsonEncoder.kt)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.beginCollection(StreamingJsonEncoder.kt:15)
	at kotlinx.serialization.encoding.Encoder$DefaultImpls.beginCollection(Encoding.kt:257)
	at kotlinx.serialization.json.JsonEncoder$DefaultImpls.beginCollection(JsonEncoder.kt)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.beginCollection(StreamingJsonEncoder.kt:15)
	at kotlinx.serialization.internal.MapLikeSerializer.serialize(CollectionSerializers.kt:119)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.encodeSerializableValue(StreamingJsonEncoder.kt:223)
	at kotlinx.serialization.encoding.AbstractEncoder.encodeSerializableElement(AbstractEncoder.kt:84)
	at dk.MyTripStorageData.write$Self(KotlinMigrationMain.kt:9)
	at dk.MyTripStorageData$$serializer.serialize(KotlinMigrationMain.kt)
	at dk.MyTripStorageData$$serializer.serialize(KotlinMigrationMain.kt:9)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.encodeSerializableValue(StreamingJsonEncoder.kt:223)
	at kotlinx.serialization.json.Json.encodeToString(Json.kt:73)
	at dk.KotlinMigrationMainKt.main(KotlinMigrationMain.kt:40)

This is my source code, the 1.3.72 version is outcommented below the 1.4.0 version:


import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonConfiguration

@Serializable
data class MyTripStorageData(val activeTripMap: MutableMap<MyTripId, MyTripStateInfo> = mutableMapOf<MyTripId, MyTripStateInfo>()) {

    fun add(trip: MyTripStateInfo) {
        activeTripMap[trip.tripId] = trip
    }

    operator fun get(tripId: MyTripId) = activeTripMap[tripId]
}

@Serializable
data class MyTripId(val serverId: Int, val id: Long)

@Serializable
data class MyTripStateInfo(
        val tripId: MyTripId,
        val bookingName: String
)

fun main(args: Array<String>) {
    val tripId = MyTripId(1, 12L)
    val tripId2 = MyTripId(2, 14L)
    val data = MyTripStorageData()
    data.add(MyTripStateInfo(tripId, "Foo"))
    data.add(MyTripStateInfo(tripId2, "Bar"))

    // example lookup
    println(data[tripId])

    // Simulate store my data on disk and restore on app-load
    // Kotlin 1.4 Version
    val ksonStorage: Json = Json { ignoreUnknownKeys = true; prettyPrint = true }
    val serializedData = ksonStorage.encodeToString(MyTripStorageData.serializer(), data)
    val decodedData = ksonStorage.decodeFromString(MyTripStorageData.serializer(), serializedData)

    // Kotlin 1.3.72 Version
//    val ksonStorage: Json = Json(JsonConfiguration.Stable.copy(ignoreUnknownKeys = true, prettyPrint = true))
//    val serializedData = ksonStorage.stringify(MyTripStorageData.serializer(), data)
//    val decodedData = ksonStorage.parse(MyTripStorageData.serializer(), serializedData)

    println(serializedData)

    println(decodedData[tripId])
}

The serialized format by v 1.3.72 is

{
    "activeTripMap": [
        {
            "serverId": 1,
            "id": 12
        },
        {
            "tripId": {
                "serverId": 1,
                "id": 12
            },
            "bookingName": "Foo"
        },
        {
            "serverId": 2,
            "id": 14
        },
        {
            "tripId": {
                "serverId": 2,
                "id": 14
            },
            "bookingName": "Bar"
        }
    ]
}

kotlin-test.zip (57.5 KB)

6

Ok @arberg,
so you are using the Json format and you should use the allowStructuredMapKeys.
You miss this issue and this pull request.

As suggested, you should set the allowStructuredMapKeys flag, ie:

    val ksonStorage: Json = Json { allowStructuredMapKeys = true; ignoreUnknownKeys = true; prettyPrint = true }
7

thank you for the links.