Announcing kotlinx-metadata-jvm library for reading/modifying metadata of Kotlin/JVM class files

udalov · November 23, 2021, 10:37am

kotlinx-metadata-jvm 0.4.0 has been released.

Changelog:

Update to Kotlin 1.6 with metadata version 1.6, support reading/writing metadata of version 1.7 which will be used in Kotlin 1.7.
Add JvmPropertyExtensionVisitor.visitSyntheticMethodForDelegate for optimized delegated properties (KT-39055).
Add JVM-specific class flags:
- JvmClassExtensionVisitor.visitJvmFlags
- JvmFlag.Class.HAS_METHOD_BODIES_IN_INTERFACE
- JvmFlag.Class.IS_COMPILED_IN_COMPATIBILITY_MODE
KT-48965 Make the type of KmValueParameter.type non-null KmType
Remove unused JvmTypeAliasExtensionVisitor and JvmValueParameterExtensionVisitor
Fix type flags (suspend, definitely non-null) on underlying type of inline class available via KmClass.inlineClassUnderlyingType

udalov · December 1, 2021, 7:01pm

kotlinx-metadata-jvm 0.4.1 has been released.

Changelog:

Add KmProperty.syntheticMethodForDelegate for optimized delegated properties (KT-39055).

udalov · June 17, 2022, 10:56am

kotlinx-metadata-jvm 0.5.0 has been released.

Changelog:

Update to Kotlin 1.7 with metadata version 1.7, support reading/writing metadata of version 1.8 which will be used in Kotlin 1.8.
kotlinx-metadata-jvm can no longer be used on JVM 1.6, and now requires JVM 1.8 or later.
Add Flag.Type.IS_DEFINITELY_NON_NULL.
Add KmClass.contextReceiverTypes, KmFunction.contextReceiverTypes, KmProperty.contextReceiverTypes
- The API is experimental and requires @ExperimentalContextReceivers on the usages.

tmm1 · August 10, 2022, 11:02pm

Given a kotlinx.metadata.KmProperty, is it possible to tell if the property has a backing java field or not?

udalov · August 15, 2022, 11:06am

@tmm1 Yes, use KmProperty.fieldSignature: kotlin/jvmExtensions.kt at c9e9ee90a306e2dca52d9873528d8091cbc33428 · JetBrains/kotlin · GitHub

udalov · January 10, 2023, 6:14pm

kotlinx-metadata-jvm 0.6.0 has been released.

This release features several significant API changes. To help with migration, we’ve prepared a special guide.

Update to Kotlin 1.8 with metadata version 1.8, support reading/writing metadata of version 1.9 which will be used in Kotlin 1.9
Deprecate Visitors API
Replace usages of KotlinClassHeader with direct usage of kotlin.Metadata annotation. Former reserved exclusively for use from Java clients.
impl package renamed to internal
Writers are deprecated. Special function family KotlinClassMetadata.write is introduced instead.

Leonid.Startsev · July 12, 2023, 4:49pm

kotlinx-metadata-jvm 0.7.0 has been released.

This release also featured several significant API changes. To help with migration, we’ve prepared a special guide.

Update to Kotlin 1.9 with metadata version 1.9, support reading/writing metadata of version 2.0 which will be used in Kotlin 2.0
Rework flags API (see Migration from Flags API to Attributes API).
Restructure KotlinClass(Module)Metadata.write/read (see Changes in reading and writing API).
Add @JvmStatic + @JvmOverloads to writing functions in KotlinClassMetadata
Deprecate KmModule.annotations for removal because it is always empty and should not be used.
Move KmModuleFragment to an kotlinx.metadata.internal.common package. This class is intended for internal use only. If you have use-cases for it, please report an issue to YouTrack.
Improve toString() for KmAnnotationArgument
Add missing deprecation for KmExtensionType and experimentality for KmConstantValue.
Enhance kotlinx-metadata-jvm KDoc and set up Dokka.

tamirda · July 12, 2023, 6:28pm

Hello!
Thank you very much for the release!

I have few questions if I may in order to understand the backward/forward compatibility of the library.

In case I have an app with maximum metadata version, let’s say 1.5, if I change the metadata version to 1.9, will the kotlin engine that comes with the app be able to handle it? Or I need to use metadata version of max 1.6? (which requires me to use old metadata jvm library)
In case I have old library which uses kotlin metadata 1.4 (or 1.1.x using kotlin 1.3), do I need to use old metadata jvm library to handle it(read/modify/write)? or can I use the latest and greatest?

Thank you very much!

Leonid.Startsev · July 14, 2023, 2:48pm

will the kotlin engine that comes with the app be able to handle it

Sorry, I do not know what you mean by ‘kotlin engine’. If your class files have metadata version = 1.9 (compiled by Kotlin 1.9 or written with this version by metadata library), Kotlin 1.5 won’t be able to read them. Kotlin compiler can write metadata v1.5 using languageVersion setting, kotlinx-metadata-jvm library has metadataVersion parameter in KotlinClassMetadata.write to specify that.

In case I have old library which uses kotlin metadata 1.4 (or 1.1.x using kotlin 1.3), do I need to use old metadata jvm library to handle it(read/modify/write)

No, you can use the latest one to read and write metadata 1.4-2.0. IIRC, we can’t write metadata with versions 1.3 and lower because it was entirely different format.

tamirda · July 15, 2023, 4:36am

Hi @Leonid.Startsev ,
You actually helped me a lot.
Just to clarify what I meant by kotlin engine, if I take a class file from an app that was compiled with kotlin 1.5, I call it kotlin engine 1.5 since the app includes all relevant kotlin files of kotlin compiler 1.5. Sometimes I need to be more clear

Regarding metadata of kotlin 1.3, correct me if I am wrong, I can read it using the latest library and write version 1.4.2 instead. Correct?

This means I can always use the latest library and write the correct version according to the metadataVersion field, right?

Thank you again!

Leonid.Startsev · July 17, 2023, 1:31pm

if I take a class file from an app that was compiled with kotlin 1.5

It means it has metadataVersion = 1.5. You can read it and write back with this library. You can specify 1.5 version in KotlinClassMetadata.write so old compiler will still be able to read it after your transformations

Regarding metadata of kotlin 1.3, correct me if I am wrong, I can read it using the latest library and write version 1.4.2 instead. Correct?

Yes, I think so.

This means I can always use the latest library and write the correct version according to the metadataVersion field, right?

Yes, this is how it is intended to be used.

You are welcome!

tamirda · August 1, 2023, 1:02pm

Hello again!
Thank you for the hard work.

Using the recent release (0.7), on some packages I am getting some weird issues when reading their metadata.
I assume the errors are due to old metadata but is there a way to support reading those metadata?
for example, using this package:

I am getting the following exceptions on some files:

io/fotoapparat/hardware/CameraDevice$setFocalPoint$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/hardware/Device$getCameraParameters$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/hardware/CameraDevice$updateParameters$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/hardware/CameraDevice$updateFocusingAreas$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/camera/UpdateConfigurationRoutineKt$updateCameraConfiguration$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/capability/GetCapabilitiesRoutineKt$getCapabilities$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/focus/FocusRoutineKt$focus$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/focus/FocusOnPointRoutineKt$focusOnPoint$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/zoom/UpdateZoomLevelRoutineKt$updateZoomLevel$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/parameter/GetParametersRoutineKt$getCurrentParameters$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/routine/photo/TakePhotoRoutineKt$takePhoto$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table
io/fotoapparat/coroutines/AwaitBroadcastChannel$getValue$1.smali , error is kotlinx.metadata.InconsistentKotlinMetadataException: No VersionRequirement with the given id in the table

Example Metadata from these files:
from io/fotoapparat/hardware/CameraDevice$setFocalPoint$1.:

.annotation runtime Lkotlin/Metadata;
    bv = {
        0x1,
        0x0,
        0x3
    }
    d1 = {
        "\u0000\u0016\n\u0000\n\u0002\u0010\u0000\n\u0000\n\u0002\u0018\u0002\n\u0000\n\u0002\u0018\u0002\n\u0002\u0010\u0002\u0010\u0000\u001a\u0004\u0018\u00010\u00012\u0006\u0010\u0002\u001a\u00020\u00032\u000c\u0010\u0004\u001a\u0008\u0012\u0004\u0012\u00020\u00060\u0005H\u0096@\u00f8\u0001\u0000"
    }
    d2 = {
        "setFocalPoint",
        "",
        "focalRequest",
        "Lio/fotoapparat/hardware/metering/FocalRequest;",
        "continuation",
        "Lkotlin/coroutines/Continuation;",
        ""
    }
    k = 0x3
    mv = {
        0x1,
        0x1,
        0xd
    }
.end annotation

from io/fotoapparat/hardware/Device$getCameraParameters$1:

.annotation runtime Lkotlin/Metadata;
    bv = {
        0x1,
        0x0,
        0x3
    }
    d1 = {
        "\u0000\u0016\n\u0000\n\u0002\u0010\u0000\n\u0000\n\u0002\u0018\u0002\n\u0000\n\u0002\u0018\u0002\n\u0002\u0018\u0002\u0010\u0000\u001a\u0004\u0018\u00010\u00012\u0006\u0010\u0002\u001a\u00020\u00032\u000c\u0010\u0004\u001a\u0008\u0012\u0004\u0012\u00020\u00060\u0005H\u0096@\u00f8\u0001\u0000"
    }
    d2 = {
        "getCameraParameters",
        "",
        "cameraDevice",
        "Lio/fotoapparat/hardware/CameraDevice;",
        "continuation",
        "Lkotlin/coroutines/Continuation;",
        "Lio/fotoapparat/parameter/camera/CameraParameters;"
    }
    k = 0x3
    mv = {
        0x1,
        0x1,
        0xd
    }
.end annotation

What are these exceptions? how can we read & use those metadatas?

Thank you!

udalov · August 1, 2023, 2:09pm

@tamirda Please create an issue at https://kotl.in/issue and we’ll investigate. Also, please re-check if everything worked correctly with kotlinx-metadata-jvm 0.6.0 without any other changes.

tamirda · August 1, 2023, 2:58pm

@udalov
Thank you for the fast answer.
Opened here:
https://youtrack.jetbrains.com/issue/KT-60870/kotlinx.metadata.InconsistentKotlinMetadataException-No-VersionRequirement-with-the-given-id-in-the-table-In-kotlinx-metadata

This issue appears on all kotlinx-metadata-jvm versions from 0.2 until 0.7.
Thanks again

Topic		Replies	Views
Kotlinx metadata, get Metadata Annotation processor Support	5	2336	July 19, 2018
Custom compiler annotations	6	3009	July 3, 2015
Kotlinx-metadata, how to get function annotation? Libraries	1	1313	June 4, 2019
Annotation Processor Function Suspend Modifier? Support	3	940	May 1, 2021
Kotlin support for metadata added with javac's -parameters option Language Design	0	27	August 15, 2024

Announcing kotlinx-metadata-jvm library for reading/modifying metadata of Kotlin/JVM class files

Related topics