Ability to consume DSL scripts in runtime?

There is a problem:
Some big project(android + ios + web), after some discussions we decided to write a DSL for our project tests, I develop an android app and volunteered to create a dsl prototype and kotlin helps a lot. Because the project is multiplatform and I couldn’t convince other teams to switch to kotlin, this DSL will be implemented at least twice, where one requirement is sharing test scenarios using this dsl between platforms. So, I need to find a way to load test script which is written in our DSL in runtime.
I think of the solutions but all are the bed ones:

  1. manual parsing of script which almost fully discards kotlin dsl capabilities :frowning:
  2. embedding kotlin compiler inside app - I even scared to think about it
  3. assembing new apk for each script on demand by CI - slow, flaky, and somewhat cheaty
  4. reflection? I can’t find anyone uses reflection for these purposes and I hardly can imagine how to apply it for some string with code

I created this topic to know - if anybody created some kind of interpreter with kotlin and how to do it right?

Kotlin scripting would allow you to embed the Kotlin compiler in your app. There’s several nice examples but sadly there isn’t a hands-on to learn from. Here’s the link to the KEEP if you want an under the hood look at things: https://github.com/Kotlin/KEEP/blob/master/proposals/scripting-support.md

One thing to consider is where you will be writing the DSL and if you can get tooling support. Kotlin DSLs are great because they support tooling with no cost as it’s just part of the language.
When embedding the compiler you can get similar tooling when editing your custom scripts inside Intellij.

The result is a fully tooled custom file similar to build.gradle.kts. Instead of Gradle tooling included by default, your code is included. You can manage imports and handle compiling any way you want (e.i. You might provide an @dependsOn("myfile.custom.kts") annotation to enable script dependencies).

Another question to ask is: Is your DSL close to YML? Or in other words, is this something you can boil down to simple predefined actions so it looks like GitLabCI or GitHub Actions? A Kotlin DSL would definitely provide you with more control as you have full scripting access, type-safety, and autocompletion. However, it may still be better to start on how your use-case is modeled in a pure data format first.

It sounds like you want a portable DSL. If that DSL is simply data that doesn’t need to be scripted or have logic in it, see how it would look in a mark-up language. (This is the opposite of when you find you’re “coding in JSON/YML” as your mark-up starts to get control flow, operators, etc. That’s when you want to look for a more robust scripting solution).

If that’s the route you go, you can always add on a Kotlin DSL. For example, this Kubernetes DSL. This gives tooling and programmatic support to writing tests.

Finally, have you looked at existing DSLs for writing test scenarios? Cucumber is the first that comes to mind. You write “feature” files that describe your tests and implement that binds the tests to implementation in whatever language you want. This way would allow you to write BDD scenarios platform independently and implement the glue in the native language for the project. There’s tooling support for Cucumber in IntelliJ.

TL:DR
There are some really great benefits to Kotlin embedded scripting, but it may not fit your use case.
Is your DSL portable data? Or is it meant to be have programmatic control flow and more complex logic? (this might move things closer to a fully blown language)
Where are you writing the test DSL? IntelliJ, a text simple editor (VSCode, Atom, etc)? Is tooling important? (Or maybe it’s meant to be simple enough to not matter)
Have you looked at existing test solutions for describing test scenarios (Cucumber maybe)? I suspect Cucumber may provide what you’re looking for–independent test scenarios.

2 Likes

Here is another industry example from our lab member: https://github.com/pklimai/jsnapy-kotlin-dsl. The only major drawback of using kotlin scripting right now is the bundle size. If you are OK with 20 MB of additional binary size. it is ideal. I think it will be somehow optimized in future.

Main reason we are considering test dsl because of nature of data exchange with server - it’s not REST or http but socket based async data(real time game) - a lot of huge data structures with complex logic. Now, manual testers can’t reproduce some bugs because of that, junit tests don’t help. We need big scripts where we can craft a required network exchange( expected output/input), describe user input, validate model & ui. A prototype I created with kotlin is viable and we are planning to implement it. The only problem for now is how to have shared repository with tests written by an arbitrary tester and ability to execute them in our applicaiton. For now, I think the only way is to write a job in CI where test script will be copied into the codebase, compiled and run on emulators - it works but when script count will start to grow the project recompilation will be the most time/cpu consumer. Referenced KEEP is promising but I have some doubts if it achievable on android.