Kotlinx serialization: How to migrate map from 0.20.0 to 1.0-M1-1.4.0-rc

Not sure this is the right forum but here goes:

We use kotlinx serialization 0.20.0, and just tried migrating to 1.0-M1-1.4.0-rc.

We have a map like this

@Serializable
class TripStorageData {
    val activeTripMap = mutableMapOf<TripId, TripStateInfo>()
}

@Parcelize
@Serializable
data class TripId(val serverId: Int, val id: Long, val type: TripType) : Parcelable {
    override fun toString(): String =
            """TripId(serverId=$serverId, id=$id, type=TripType.$type)"""
}

The new serialization version does not support may keys except as enums or primitives. I cannot see that change in the change log, but I suppose that is a changelog omission rather than a bug: kotlinx.serialization/CHANGELOG.md at master · Kotlin/kotlinx.serialization · GitHub

How do I migrate my code, and do I really have to use a string? Perhaps I need a custom serialization? I’m not quite sure where to start digging. Any help is appreciated.

Best Alex

Can you provide a better reproducer and explain what is the expected result?

Hi fvasco,

That question makes zero sense for me, if it is a feature that map keys can only be enums or primitives. GIven that the compiler says so to me, it is not just a random bug. Why are you asking for a sample? Do you know whether or not it is the case that map keys must now follow that rule?

Best Alex

It is not clear to me what format are using, what is the error output and what is your goal.

Yes, I might.

Hi Francesco, Thank for you for your time and effort. I’ve created a sample project containing the error, attached as zip.

EDIT: Man that’s bloody embarrasing. Apparently I didn’t read the error-message carefully and completely, and thus missed the nice detail that they added that told me exactly what to do. So all I have to do is enable structed keys, and I’m done. so problem solved.

I’ll include the below, in case other’s drop by later, though.

==========

My goal is to store my data on disk, and I have stored is as a map. In hindsight I see that I could have just stored is list, and moved to map after deserializing, but this was easier, and fit the bill at the time. I already have apps ‘out there’ with the old data format so in order to migrate to newer kotlin I have to somehow handle it.

I get the following error

Exception in thread "main" kotlinx.serialization.json.internal.JsonEncodingException: Value of type 'dk.MyTripId' can't be used in JSON as a key in the map. It should have either primitive or enum kind, but its kind is 'CLASS'.
Use 'allowStructuredMapKeys = true' in 'Json {}' builder to convert such maps to [key1, value1, key2, value2,...] arrays.
	at kotlinx.serialization.json.internal.JsonExceptionsKt.InvalidKeyKindException(JsonExceptions.kt:67)
	at kotlinx.serialization.json.internal.WriteModeKt.switchMode(WriteMode.kt:58)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.beginStructure(StreamingJsonEncoder.kt:63)
	at kotlinx.serialization.encoding.Encoder$DefaultImpls.beginCollection(Encoding.kt:268)
	at kotlinx.serialization.json.JsonEncoder$DefaultImpls.beginCollection(JsonEncoder.kt)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.beginCollection(StreamingJsonEncoder.kt:15)
	at kotlinx.serialization.encoding.Encoder$DefaultImpls.beginCollection(Encoding.kt:257)
	at kotlinx.serialization.json.JsonEncoder$DefaultImpls.beginCollection(JsonEncoder.kt)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.beginCollection(StreamingJsonEncoder.kt:15)
	at kotlinx.serialization.internal.MapLikeSerializer.serialize(CollectionSerializers.kt:119)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.encodeSerializableValue(StreamingJsonEncoder.kt:223)
	at kotlinx.serialization.encoding.AbstractEncoder.encodeSerializableElement(AbstractEncoder.kt:84)
	at dk.MyTripStorageData.write$Self(KotlinMigrationMain.kt:9)
	at dk.MyTripStorageData$$serializer.serialize(KotlinMigrationMain.kt)
	at dk.MyTripStorageData$$serializer.serialize(KotlinMigrationMain.kt:9)
	at kotlinx.serialization.json.internal.StreamingJsonEncoder.encodeSerializableValue(StreamingJsonEncoder.kt:223)
	at kotlinx.serialization.json.Json.encodeToString(Json.kt:73)
	at dk.KotlinMigrationMainKt.main(KotlinMigrationMain.kt:40)

This is my source code, the 1.3.72 version is outcommented below the 1.4.0 version:


import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonConfiguration

@Serializable
data class MyTripStorageData(val activeTripMap: MutableMap<MyTripId, MyTripStateInfo> = mutableMapOf<MyTripId, MyTripStateInfo>()) {

    fun add(trip: MyTripStateInfo) {
        activeTripMap[trip.tripId] = trip
    }

    operator fun get(tripId: MyTripId) = activeTripMap[tripId]
}

@Serializable
data class MyTripId(val serverId: Int, val id: Long)

@Serializable
data class MyTripStateInfo(
        val tripId: MyTripId,
        val bookingName: String
)

fun main(args: Array<String>) {
    val tripId = MyTripId(1, 12L)
    val tripId2 = MyTripId(2, 14L)
    val data = MyTripStorageData()
    data.add(MyTripStateInfo(tripId, "Foo"))
    data.add(MyTripStateInfo(tripId2, "Bar"))

    // example lookup
    println(data[tripId])

    // Simulate store my data on disk and restore on app-load
    // Kotlin 1.4 Version
    val ksonStorage: Json = Json { ignoreUnknownKeys = true; prettyPrint = true }
    val serializedData = ksonStorage.encodeToString(MyTripStorageData.serializer(), data)
    val decodedData = ksonStorage.decodeFromString(MyTripStorageData.serializer(), serializedData)

    // Kotlin 1.3.72 Version
//    val ksonStorage: Json = Json(JsonConfiguration.Stable.copy(ignoreUnknownKeys = true, prettyPrint = true))
//    val serializedData = ksonStorage.stringify(MyTripStorageData.serializer(), data)
//    val decodedData = ksonStorage.parse(MyTripStorageData.serializer(), serializedData)

    println(serializedData)

    println(decodedData[tripId])
}

The serialized format by v 1.3.72 is

{
    "activeTripMap": [
        {
            "serverId": 1,
            "id": 12
        },
        {
            "tripId": {
                "serverId": 1,
                "id": 12
            },
            "bookingName": "Foo"
        },
        {
            "serverId": 2,
            "id": 14
        },
        {
            "tripId": {
                "serverId": 2,
                "id": 14
            },
            "bookingName": "Bar"
        }
    ]
}

kotlin-test.zip (57.5 KB)

Ok @arberg,
so you are using the Json format and you should use the allowStructuredMapKeys.
You miss this issue and this pull request.

As suggested, you should set the allowStructuredMapKeys flag, ie:

    val ksonStorage: Json = Json { allowStructuredMapKeys = true; ignoreUnknownKeys = true; prettyPrint = true }

thank you for the links.