Announcing kotlinx-metadata-jvm library for reading/modifying metadata of Kotlin/JVM class files

@xdenss Could you please describe process of removing @Metadata annotation with R8?
Just added in gradle.properties entries to support R8 android.enableR8=true.
Also tried with android.enableR8.fullMode=true, but metadata annotation is still visible in decompiled code.

kotlinx-metadata-jvm 0.1.0 has been published on Bintray and will be available in Maven Central soon.

The highlight of this release is the new value-based API (KT-26602) which is now the preferred way to work with the metadata instead of visitors. Take a look at an updated ReadMe and/or an example test case to learn more.

For example, here’s an excerpt from that test case that creates class metadata from scratch:

    val klass = KmClass().apply {
        name = "Hello"
        flags = flagsOf(Flag.IS_PUBLIC)
        constructors += KmConstructor(flagsOf(Flag.IS_PUBLIC, Flag.Constructor.IS_PRIMARY)).apply {
            signature = JvmMethodSignature("<init>", "()V")
        }
        functions += KmFunction(flagsOf(Flag.IS_PUBLIC, Flag.Function.IS_DECLARATION), "hello").apply {
            returnType = KmType(flagsOf()).apply {
                classifier = KmClassifier.Class("kotlin/String")
            }
            signature = JvmMethodSignature("hello", "()Ljava/lang/String;")
        }
    }
1 Like

@udalov the new API is great, but I was wondering why the approach of all simple classes with mutable variables was taken rather than making them immutable data classes? Taking that same example snippet

val klass = KmClass(
    name = "Hello",
    flags = flagsOf(Flag.IS_PUBLIC),
    constructors = listOf(
        KmConstructor(
            flags = flagsOf(Flag.IS_PUBLIC, Flag.Constructor.IS_PRIMARY),
            signature = JvmMethodSignature("<init>", "()V")
        )
    ),
    functions = listOf(
        KmFunction(
            flags = flagsOf(Flag.IS_PUBLIC, Flag.Function.IS_DECLARATION),
            name = "hello",
            returnType = KmType(
                flags = flagsOf(),
                classifier = KmClassifier.Class("kotlin/String")
            )
            signature = JvmMethodSignature("hello", "()Ljava/lang/String;")
        )
}

Later mutating could be done safely with data class copy functions instead then

A separate question - would you be open to a PR with a non-ServiceLoader initialization option? As the library is not stable yet, it’s not safe to ship as a direct dependency yet in its current form. I’ve been testing out shading it here, but have had enough issues with getting service rewriting to work correctly that I’m hoping you’d be open to a non-service solution instead.

One more separate question (sorry if this the wrong place, since this isn’t a separate kotlinx repo I can’t file these as separate issues)

Is it possible to distinguish between receiver types and non-receiver types? For example: I don’t currently see a way to differentiate between String.() -> Unit and (String) -> Unit as they’re both just Function1<String, Unit> as far as metadata is concerned

Hi! Sorry for a late response.
To reproduce the R8 issue I implemented Kotlin class with reflection.
For example:

This example works fine with Proguard but crashed with R8 with access exception. Without Metadata there is no information about getters for vals.
In R8 sources I found line where metadata is used to collect additional info for desugaring (and maybe obfuscation) and then just remove it. There was following code (at least month ago):
snippet from src/main/java/com/android/tools/r8/naming/ClassNameMinifier.java

1 Like

@hzsweers Thanks for the feedback!

I was wondering why the approach of all simple classes with mutable variables was taken rather than making them immutable data classes?

One of the obstacles is the fact that we plan to extend the library to other platforms in the future, thus the API of common classes cannot refer to JVM specifics (such as signature). Currently all JVM extensions are done via Kotlin extension properties and this design seems to scale well in the presence of other platforms.

Another is performance. I agree that data classes seem nicer, but not that much nicer to justify the additional cost of all the allocations caused by mutations, and all the extra equals/hashCode/toString/component methods in the bytecode. We’re still aiming for kotlinx-metadata-jvm to be used in tools such as bytecode obfuscators and optimizers that are supposed to work fast on big codebases.

A separate question - would you be open to a PR with a non-ServiceLoader initialization option?

Yes, although I’m wondering what exactly do you have in mind. ServiceLoader seems to be the preferred way of simple DI (especially in light of the Java module system) and it should be handled properly by the Shadow plugin. In any case, let’s not discuss the details here – if you have an idea, please send a PR and I’ll take a look!

Is it possible to distinguish between receiver types and non-receiver types? For example: I don’t currently see a way to differentiate between String.() -> Unit and (String) -> Unit as they’re both just Function1<String, Unit> as far as metadata is concerned

The way this is handled in the language is by a type annotation ExtensionFunctionType. So the type String.() -> Unit is actually the same as @ExtensionFunctionType (String) -> Unit. You can find type annotations via the KmType.annotations extension property.

@udalov Ahhh perfect, didn’t know about the extension properties. Worked like a charm, and I’ve been tinkering with writing an immutable layer over the mutable one for IR use. The mutability/visitor reason makes sense with that context :+1:.

I’ll think on the serviceloader init alternative idea and see what I can do!

Is it possible to read file annotations from metadata?

No, annotations are not stored in the metadata and kotlinx-metadata-jvm doesn’t provide a way to read them. The only workaround is to read them directly by external means, such as from the bytecode, via reflection or annotation processing API. File annotations specifically are generated on the corresponding file facade class, if it exists.

We do plan to provide access to annotations on classes and members (see the comment above and KT-31857), but likely not as a part of the core kotilnx-metadata-jvm library.

Hello, (sorry if question isn’t directly relevant)

  • I would Like to get KmClass or kClass or just kotlin full name of
    the kclass parameter in my annotation inside my annotation processor
    I read above 2 comments This & This that kotlin.Metadata don’t hold annotations info
    but this can be done by external means So can you please tell me how to achieve
    my goal by external means because I don’t understand.

  • For more clarification take a look at the code below

// Another developer’s code using my library
@MyAnn(kClass = List::class)

// My annotation processor code
val myAnn = element.getAnnotationsByType(MyAnn::class.java).first()

// how to get KmClass or KClass or just String of full kotlin class name
// using myAnn.kClass

// ALSO Note what i currently can do is get it in a TypeMirror instance
// but that has a problem for ex. it shows above example as
// java.util.List instead of kotlin.collections.List

Thanks in Advance.

@MohamedAlaaEldin636 In your example, myAnn.kClass should return an instance of KClass, which has a qualified name and lots of other info. So, myAnn.kClass.qualifiedName should return kotlin.collections.List.

If you try this and still have troubles, please create a new topic on this forum instead of this one, since it’s not directly related to kotlinx-metadata-jvm.

1 Like

Hey @udalov would it be possible to have a description of what the metadata needs in order to peform reflection? Im writing an obfuscator and its great to have this library, but I think it needs a guide accompanying it. From my brief glances at it, it seems that I should simply change data2 strings to their mapped equivalents. Are there any other fields in the annotation I need to change?

@cookiedragon234 With an exception of Kotlin standard library, data1 and data2 are the only properties used meaningfully in kotlin-reflect. However, and maybe I’m not interpreting your question correctly, but it’s not recommended to just change strings in data2 (you wouldn’t need the kotlinx-metadata-jvm library for that anyway). It’s just a string table with all the names used in the metadata, so the same string may refer to multiple names, some of which are going to be obfuscated but some aren’t.

For example, if you have a class A and a function A declared or referenced in the same file, there will be one entry "A" in data2. If the obfuscator then decides to rename the class (but not the function!) to B, it’s incorrect to just change the entry in data2 to "B" since the function would be renamed too as a side effect.

kotlinx-metadata-jvm abstracts that away; you shouldn’t deal with the string table anymore. If you read such class to a KmClass instance, change its name and serialize it back to a .class file, you’ll observe that only the class is renamed. Under the hood, the library will transform the string table so that there’s an entry for the class name "B" and the entry for the unchanged function name "A".

1 Like