Using kotlin.script.experimental.api.* how to retrieve the value from the Script

Following my previous question, I’ve decided to use the kotlin.script.experimental.api.* instead of the JSR api.

Basically what I am trying to do is the following:

  • I’ve developed a library (DSL) that allows json transformations on a given input (basically it can be used to transform a JSON from one format to another);
  • I want to be able to run the DSL Transformation as scripts so I can dynamically change them, without the need of having the code compiled.

I’ve come-up with those solution at this point:

package net.andreinc.mapneat.scripting

import net.andreinc.mapneat.scripting.KotlinScriptRunner.evalScript
import org.apache.logging.log4j.kotlin.Logging
import kotlin.reflect.KClass
import kotlin.script.experimental.api.*
import kotlin.script.experimental.host.toScriptSource
import kotlin.script.experimental.jvm.dependenciesFromCurrentContext
import kotlin.script.experimental.jvm.jvm
import kotlin.script.experimental.jvmhost.BasicJvmScriptingHost

data class ProvidedProperty(val name: String, val type: KClass<>, val value: Any?) {
constructor(name: String, type: Class<
>, value: Any?) : this(name, type.kotlin, value)
}

object KotlinScriptRunner : Logging {

fun evalScript(scriptFile: SourceCode, props: List<ProvidedProperty>): ResultWithDiagnostics<EvaluationResult> {
    val compileConfig = ScriptCompilationConfiguration {
        jvm {
            dependenciesFromCurrentContext(wholeClasspath = true)
            defaultImports(
                "net.andreinc.mapneat.dsl.*",
                "net.andreinc.mapneat.operation.*"
            )
        }
        providedProperties(*(props.map { it.name to KotlinType(it.type) }.toTypedArray()))
    }
    val evaluationConfig = ScriptEvaluationConfiguration {
        providedProperties(*(props.map { it.name to it.value }.toTypedArray()))
    }

    return BasicJvmScriptingHost().eval(scriptFile, compileConfig, evaluationConfig)
}

}

fun main() {

val script1 = """
    println(a)
    println(b)
    a+b
"""

val props1 = listOf(
    ProvidedProperty("a", Int::class, 2),
    ProvidedProperty("b", Int::class, 3)
)

println(evalScript(script1.toScriptSource(), props1))

val script2 = """
    println(c)
    
    json("{}") {
        "name" /= "Andrei"
        "value" /= c
    }.getPrettyString()
    
"""

val props2 = listOf(
    ProvidedProperty("c", Int::class, 3)
)

println(evalScript(script2.toScriptSource(), props2))

}

If I run the previous code the output looks like this:

2
3
Success(value=EvaluationResult(returnValue=$$result: kotlin.Int = 5, configuration=kotlin.script.experimental.api.ScriptEvaluationConfiguration@6051e37e), reports=[DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt], DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt]])
3
15:55:14.160 [main] INFO  net.andreinc.mapneat.dsl.MapNeat - Transformation(id=a87079e1-da2e-4d52-af7b-e95b0cfa5253, parentId=null) INPUT = {}
15:55:14.170 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=a87079e1-da2e-4d52-af7b-e95b0cfa5253) "name" ASSIGN(/=) "Andrei"
Success(value=EvaluationResult(returnValue=java.lang.NoSuchMethodError: 'int Script.access$getC$p(Script)', configuration=kotlin.script.experimental.api.ScriptEvaluationConfiguration@3929b7f6), reports=[DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt], DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt]])

As you can for script1 from the evaluation result I can retrieve the last value (after computing a+b = 5).

But for the second script I am encountering the following problem: Success(value=EvaluationResult(returnValue=java.lang.NoSuchMethodError: 'int Script.access$getC$p(Script)...and so on'

My question is:

  • How can I retrieve the value from the second script ?
  • Why it’s not working just like for the first script ? I have done something very similar in both cases.

Hmm…
It might be helpful to move your script into a kts file and set a breakpoint within the script. I know you’re using a script string now so it will add a bit of overhead to switch it over to file evaluation to enable that.

Personally, I’d do a bunch of sanity checks (which you may have already done) such as; changing the name of c. Changing something to prove the failing part is the "value" /= c line (or not that line)–and follow up with trying to prove it is or is not something to do with the json method.

Of course if you go the route of enabling debugging, you can poke around more if the sanity checks are not enough.

1 Like

@arocnies I’ve modified the code like this:

  1. Moved the script in a separate file: “Script.kts” which now contains the following:
println(c)

json("{}") {
    "name" /= "Andrei"
    "value" /= c
}.getPrettyString()
  1. I’m also running the Transformation locally:
fun main() {

    // Running normally the script
    val c = 3
    println(
        json("{}") {
            "name" /= "Andrei"
            "value" /= c
        }.getPrettyString()
    )

    // Running the script using external properties
    val props2 = listOf(
        ProvidedProperty("c", Int::class, 3)
    )

    println(evalScript(File("Script.kts").toScriptSource(), props2))
}

The output is:

19:44:50.297 [main] INFO  net.andreinc.mapneat.dsl.MapNeat - Transformation(id=976b45f4-cd7e-456d-bf40-d2b08c4b7f3e, parentId=null) INPUT = {}
19:44:50.316 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=976b45f4-cd7e-456d-bf40-d2b08c4b7f3e) "name" ASSIGN(/=) "Andrei"
19:44:50.318 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=976b45f4-cd7e-456d-bf40-d2b08c4b7f3e) "value" ASSIGN(/=) 3
{
  "name" : "Andrei",
  "value" : 3
}
19:44:53.089 [main] INFO  net.andreinc.mapneat.dsl.MapNeat - Transformation(id=5c0f2e53-d1c8-403f-8c7a-219b58687f85, parentId=null) INPUT = {}
19:44:53.090 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=5c0f2e53-d1c8-403f-8c7a-219b58687f85) "name" ASSIGN(/=) "Andrei"
Success(value=EvaluationResult(returnValue=java.lang.NoSuchMethodError: 'int Script.access$getC$p(Script)', configuration=kotlin.script.experimental.api.ScriptEvaluationConfiguration@3d4b5da2), reports=[DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt], DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt]])

The script is indeed stoping at line "value" /= c, but there is no obvious error why is this happening. Also debugging the Script doesn’t work for me: OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended.

It’s weird, because the code representing the Script is running perfectly in “normal” Kotlin, but fails to get executed from the .kts file.

Also, I don’t know how to interpret this error: returnValue=java.lang.NoSuchMethodError: 'int Script.access$getC$p(Script)'.

Do you have any ideas:

  1. How to debug the Script
  2. What is the source of the previously mentioned error ?

(Thanks for looking into it!)

The debugging may only work on evaluating files instead of strings. Even though you made the file the same name, I wonder if the evaluation needs something special to tie it to that file that it normally gets from the SourceFile/ScriptFile (don’t remember the exact class)–not sure.

I suspect the error is due to not being able to call getC() within the json method. I’m not sure why it would have trouble though–your println shows it’s fine just before it enters the json method.

What is your script base class? Is it named Script (assuming this because of the error shows it’s calling the accessor on a Script class)?

If I keep c as the property the error is:
returnValue=java.lang.NoSuchMethodError: 'int Script.access$getC$p(Script)'

If I rename c to someVariable, the error changes to:
int Script.access$getSomeVariable$p(Script)

So I suspect it cannot retrieve the prop value (c or someVariable inside the DSL, but it works when I print it directly using “println” inside the Script.

It’s getting weirder.

Odd. What if you declare C in your script? Like this:

val c = 42
println(c)

json("{}") {
    "name" /= "Andrei"
    "value" /= c
}.getPrettyString()

@arocnies I was writing in the same time as you, see my above answer. You are right, it cannot retrieve the value of c.

It works:

20:04:48.271 [main] INFO  net.andreinc.mapneat.dsl.MapNeat - Transformation(id=29441874-dacf-45ee-b0c1-9623708f80cc, parentId=null) INPUT = {}
20:04:48.272 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=29441874-dacf-45ee-b0c1-9623708f80cc) "name" ASSIGN(/=) "Andrei"
20:04:48.272 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=29441874-dacf-45ee-b0c1-9623708f80cc) "value" ASSIGN(/=) 42
Success(value=EvaluationResult(returnValue=$$result: kotlin.String = {
  "name" : "Andrei",
  "value" : 42
}, configuration=kotlin.script.experimental.api.ScriptEvaluationConfiguration@37484a38), reports=[DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt], DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt]])

@arocnies

What is even weirder, is the following, If I change the Script like:

val c = someValue

json("{}") {
    "name" /= "Andrei"
    "value" /= c
}.getPrettyString()

Where someValue is supplied through props:

    val props2 = listOf(
        ProvidedProperty("someValue", Int::class, 3)
    )

Everything works fine:

Success(value=EvaluationResult(returnValue=$$result: kotlin.String = {
  "name" : "Andrei",
  "value" : 3
}, configuration=kotlin.script.experimental.api.ScriptEvaluationConfiguration@aea6716e), reports=[DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt], DEBUG Loading modules: [java.se, jdk.accessibility, jdk.attach, jdk.compiler, jdk.dynalink, jdk.httpserver, jdk.jartool, jdk.javadoc, jdk.jconsole, jdk.jdi, jdk.jfr, jdk.jshell, jdk.jsobject, jdk.management, jdk.management.jfr, jdk.net, jdk.scripting.nashorn, jdk.sctp, jdk.security.auth, jdk.security.jgss, jdk.unsupported, jdk.unsupported.desktop, jdk.xml.dom, java.base, java.compiler, java.datatransfer, java.desktop, java.xml, java.instrument, java.logging, java.management, java.management.rmi, java.rmi, java.naming, java.net.http, java.prefs, java.scripting, java.security.jgss, java.security.sasl, java.sql, java.transaction.xa, java.sql.rowset, java.xml.crypto, jdk.internal.jvmstat, jdk.management.agent, jdk.jdwp.agent, jdk.internal.ed, jdk.internal.le, jdk.internal.opt]])

But if I run the script like:

json("{}") {
    "name" /= "Andrei"
    "value" /= someValue
}.getPrettyString()

I get the error

1 Like

My guess is that something is going wrong with capturing external properties in lambdas. Maybe I found the wrong function but based on this implementation json is not an inline function. I’d would be interesting to see if this would work if you change it into one. My guess is that this would fix your bug.
In any case you should create an issue at: https://kotl.in/issue

2 Likes

@Wasabi375 I’ve changed the method to:

inline fun json(json: String, init: MapNeat.() -> Unit)  : MapNeat {
     return MapNeat(json)
             .apply(init)
} 

And now it works fine. Your assumption was correct.
But nevertheless, it looks like a bug. I will try to formulate a description and raise a bug.
Thanks for your help.

For the reference opened a bug here:
https://youtrack.jetbrains.com/issue/KT-43176

2 Likes

It seems that the bug was fixed, but the question remains.

Given this code:

object KotlinScriptRunner : Logging {

    fun eval(sourceCode: SourceCode, props: List<ProvidedProperty>): ResultWithDiagnostics<EvaluationResult> {

        val compileConfig = ScriptCompilationConfiguration {
            jvm {
                dependenciesFromCurrentContext(wholeClasspath = true)
                defaultImports("net.andreinc.mapneat.dsl.*")
            }
            providedProperties(*(props.map { it.name to KotlinType(it.type) }.toTypedArray()))
        }
        val evaluationConfig = ScriptEvaluationConfiguration {
            providedProperties(*(props.map { it.name to it.value }.toTypedArray()))
        }

        return BasicJvmScriptingHost().eval(sourceCode, compileConfig, evaluationConfig)
    }

    fun evalAsString(sourceCode: SourceCode, props: List<ProvidedProperty>) : Any? {
        return eval(sourceCode, props).valueOrThrow().returnValue
    }
}

If I run it like:

fun main() {

    val script = """
        "ABC" + aValue
    """

    val props1 = listOf(
        ProvidedProperty("aValue", Int::class, 3)
    )

    val props2 = listOf(
        ProvidedProperty("aValue", Int::class, 4)
    )

    repeat(1) {
        println(evalAsString(script.toScriptSource(), props2))
    }
}

I get:

$$result: kotlin.String = ABC4

How can i retrieve only the String ABC4?

ResultValue is a sealed class, which can be of type ResultValue.Value, ResultValue.Unit, or ResultValue.Error. Once you check the value is of ResultValue.Value, you can access its value and type properties.