Can't add annotation by compiler plugin

I’m trying to create kotlin compiler plugin that will parse all the doc comments and generate @Schema annotation for them. So what I’m doing now:

class OpenApiCommentsGenerationTransformer(
    private val pluginContext: IrPluginContext,
    private val msgs: MessageCollector,
) : IrElementTransformerVoidWithContext() {
    override fun visitClassNew(declaration: IrClass): IrStatement {
        val schema = IrSingleStatementBuilder(pluginContext, Scope(declaration.symbol), -1, -1).build {
            // schema generated here

        declaration.annotations += schema
        return super.visitClassNew(declaration)

But this works only when class already has some annotation. E.g. with this class

class TestObj

I get next bytecode generated: public final class TestObj

So both annotations are present. But if I have next declaration

class TestObj

I’m getting:

public final class TestObj

declaration.dump shows that Schema annotation is generated in both cases. But for some reason it’s not written to the result .class file. The same story for properties.