Validating generated Kotlin code

Support
1

I am writing a code generator to Kotlin. I am using KotlinPoet, so my concern lies not in the correctness of the code generation process; rather, I want to check that the resulting files have the correct package directive, correct imports, correct declarations and definitions (name, type, initial values for properties; if there are function calls present, I want to know if the correct functions are called with correct arguments). What are my options?

  1. Line-by-line comparison: I type up the expected results in a separate file, then compare the lines of the generated file against the lines of the test file. Well, what if KotlinPoet breaks a line over a certain amount of characters in two or puts the closing bracket on the same line as the last statement in the block or something? The code would still compile, so the generator didn’t do anything wrong, but my tests are likely to fail, unless I specifically tailor them to match the possible obscurities of whatever library I’m using for generation. Doesn’t sound very practical? But is rather simple to implement.

  2. Obtaining a PSI file for the generated file and validating it. Sounds a little bit better? I guess I could validate it against the PSI tree for the correct test file and possibly skip whitespaces/bracket placements/other formatting details (I assume, but I’m not sure…). But I’m not exactly sure how to accomplish that.

  3. Compiling the generated file, checking if it even compiles (if not, my code generation certainly doesn’t work!), then getting a class object and using reflection to check if it has all the expected properties and methods… I have no idea how to approach this. How do I even compile a Kotlin file from Kotlin runtime?

So, what would be the best approach? I’m not exactly familiar with writing, let alone testing, code generators.

2

I think the best approach is to use a combination of option 1 and 3 and also a 4th option.

  1. In my experience kotlin poets formatting is what you would expect a human to do. It depends on how complex the code you generate is but you should be able to check that your generated code is what you expect. At least if you are able to produce easy to test sample cases.

  2. You could do that, but at that point it’s probably easier to just compile the generated code and run it. Also I’m not sure that getting access to the PSI would be easier than just ensuring that option 1 properly reflects kotlin-poets formatting.

  3. I’d take a look at GitHub - tschuchortdev/kotlin-compile-testing: A library for testing Kotlin and Java annotation processors, compiler plugins and code generation. It’s a library designed specially for this task. It compiles the code and will also allow you to get references to any KClass in the generated code. That way you can check for the existence of functions/properties, etc.

  4. Have a test project using your code generator and test that. That way you should be able to run normal unit tests on the generated code.

1 Like