Please welcome a new library kotlinx-metadata-jvm designed for tools that need to read or modify the Kotlin metadata, such as bytecode obfuscators and annotation processors. With kotlinx-metadata-jvm, it’s possible to make sense of, and modify the data written by the Kotlin compiler in the @kotlin.Metadata annotation on the .class file, and in the .kotlin_module files.
The library is published under the kotlinx project and the version number is 0.* because we’re not yet sure that the API is optimal and won’t need major refinements. Until the library is released as stable, we reserve the right to perform both source- and binary-breaking changes; however of course, we’ll do our best to minimize migration issues when these occur.
I’ll use this forum thread to notify about new versions of kotlinx-metadata-jvm and will be happy to answer any questions about the API or future goals of the library here.
Hey @udalov! I’ve been trying this out in Moshi as well as a prototype for something, but in both cases I’m caught by a strange serviceloader issue where it can’t find any MetadataExtensions. They are visible in the Moshi kotlin/codegen compiler tests, but not when used in another artifact as a kapt dependency. I don’t see anything overtly wrong so not sure if I should file a bug. The crux of the issue is that it seems the only classpath available during kapt processing is the embedded compiler classpath, and I don’t see a way to manually add (or we we should need to) add any custom classpath arguments. Any thoughts?
Hi @hzsweers, this looks like an artifact of using maven-shade-plugin; I’ve commented in the moshi issue. If it doesn’t work out anyway, please try disabling the relocation of kotlinx-metadata altogether and see if it works. Otherwise, please report it to our issue tracker and I’ll have a closer look. Thanks!
Thanks for all this great work!
I was wondering why is this not published in mavenCentral as I’m using this in a kapt processor and each client that use the processor it force those to add them the maven repository:
maven { url “https://kotlin.bintray.com/kotlinx/” }
This is really annoying as I have to leave as requirement for all libs that use this processor to add this other repository. Any idea how to overtake this?
I’m trying to use this kotlinx-metadata-jvm library to edit @kotlin.Metadata annotations after a library has had its bytecode modified using ProGuard. My plan was to read the metadata in, filter out the unwanted elements using KmXxxVisitor objects and then write my updated metadata out via the ASM library. However, these KmXxxVisitor objects don’t appear well-suited to such filtering.
For example, consider KmConstructorVisitor: I need accept() to invoke the visitExtensions() method in order to obtain the constructor’s JvmMethodSignature object, and hence learn which constructor’s metadata I am actually dealing with. However, the visitValueParameters() method has already been invoked by this time, which means that I am no longer in a position to affect any value parameters’ metadata.
Similarly with KmFunctionVisitor, because KmClassVisitor.visitFunction() will only tell me the function’s name and I can only learn the function’s descriptor once KmFunctionVisitor.visitExtensions() has provided me with the JvmMethodSignature. (Function names obviously not being unique due to overloading.)
I could still make this filtering work, provided all visitExtensions() methods were guaranteed to be visited first! However, this clearly isn’t the case at the moment.
Have I missed something fundamental about this kotlinx-metadata-jvm library please? I have been proceeding mainly by reading the KDocs and by trial-and-error, so this is entirely possible.
You’re right, the existing visitors are a bit inconvenient for that purpose. We’re planning to provide a more convenient value-based API where you’ll be able to simply obtain all necessary information as a mutable object model (KT-26602).
Could you please clarify though, what exactly do you want to do with the metadata? There could be a way to make this work, for example by splitting the visitor into two visitors, one that does filtering and another that does transformation. The key observation is that all writer visitors in kotlinx-metadata-jvm only write everything they’re supposed to write in visitEnd methods. Therefore, if you collect all the necessary info before visitEnd is called, you’ll be able to use that in visitEnd to prevent the writer’s visitEnd from being called if needed.
In regards to making visitExtensions being called first – this could help in your case, but I’m not sure if it’s the correct solution in general. I’ve described it in more detail in the issue you’ve reported (KT-29140).
What I’m trying to do is “fix up” a Kotlin library that has been shrunk using ProGuard (and other tools) so that it’s safe to compile against. The Kotlin compiler seems to use the @kotlin.Metadata annotations as the Source of Truth for which APIs exist in the bytecode, and so it will happily create references to methods and fields that ProGuard has discarded, which then generate NoSuchMethod and NoSuchField errors at runtime.
I currently have a working solution that involves using the metadata’s ProtoBuf classes directly, but would prefer to “future proof” my code by using a public API rather than an internal one.
Hi @udalov,
I have almost the same problem as @chrisr3
I’m trying to use this library to modify metadata but got problem with following case:
I have 2 methods with same name - getData
I rename these methods in bytecode for Android. For example to ‘Ar’ and ‘Br’
d2 now should contain 2 strings instead of single getData and d1 should be recreated but there is to many different options and I cannot find any documentation how to create new metadata for existing bytecode class.
As a solution I found that Google’s r8 simply removes metadata during desugaring. But still I am not sure about solution.
Could you help me to find documentation about metadata generation please? Or about why metadata is required?
I want to get the annotation for overloaded function as below:
class A{
@debug
fun functionA(){
}
@debug
fun functionA(a:Int){
}
}
But kotlinx-metadata-jvm doesn’t seem to provide any utility function to do this. Function annotations can only retrieved from ExecutableElement, but I can’t relate the ExecutableElement to the function that is visited in FunctionVisitor. Will you provide an easy way to get function annotation from FunctionVisitor or an easy way to relate function to ExecutableElement which contains annotations?
@xdenss We’re actively working on a new API for kotlinx-metadata-jvm (KT-26602) at the moment that will make everything much easier, especially modifying existing metadata.
Until that’s ready, to modify the metadata of an existing Kotlin class, you can call KotlinClassMetadata.Class.accept with a visitor implementation that delegates to a writer (an instance of KotlinClassMetadata.Class.Writer), get the result from that writer and write it to the annotation via ASM. You can override visitFunction in your visitor implementation to check if the name is “getData” and call super.visitFunction with another name (“Ar” / “Br”). However, if you need the full signature to determine how to rename the function, you’ll basically need to provide a KmFunctionVisitor implementation that stores everything (type parameters, value parameters, types, contracts, …) in its fields and determines whether to pass this function to the delegate writer or not in its visitEnd. This is a lot of boilerplate, and I wouldn’t recommend trying that before KT-26602 is implemented.
To answer your other questions: Kotlin metadata is the source that is used by the compiler and kotlin-reflect to find out information about declarations in class files. Therefore, if your project for example is a final application which doesn’t use Kotlin reflection – it’s perfectly safe to remove Kotlin metadata via r8, since it wouldn’t affect anything anyway. The need for modifying metadata usually comes from use cases involving some bytecode processing tools which normally don’t differentiate Kotlin from Java classes (obfuscators, optimizers, etc) and either an application that uses kotlin-reflect, or a library (that needs to be read by the user’s compiler correctly). If you have such a use case, I imagine you indeed need to modify the metadata via kotlinx-metadata-jvm. Take a look at an example test case where metadata is produced from scratch and is verified by using kotlin-reflect on the generated class: kotlin/MetadataSmokeTest.kt at e542c9ea848c988f142364eeb5975495ecd64090 · JetBrains/kotlin · GitHub
@wumo You’re correct that kotlinx-metadata-jvm doesn’t provide any access to annotations stored in the class file. In fact, there’s no trace of those annotations in the metadata itself (@kotlin.Metadata annotation), so they will always need to be loaded by external means. For example, matching the ExecutableElement instance with the function metadata by the JVM signature (passed to JvmFunctionExtensionVisitor.visit). We might consider providing a bunch of small libraries on top of kotlinx-metadata-jvm to provide this matching for popular use cases (annotation processing, reflection, class file processing). Could you please report an issue?