You can already use TensorFlow from kotlin-jvm. See here for reference: Installer TensorFlow Java. Also you can use TensorFlow directly with kotlin-native: https://github.com/JetBrains/kotlin-native/tree/master/samples/tensorflow.
Pure kotlin probably won’t be ever as fast as optimized native version (the CPU time difference is not large, it is only about x1.5 - x2, but memory consumption is different). Currently, what we need is a good Kotlin API for all that. If you have some ideas about it, please join slack channels mentioned above and let us discuss it. Also you are welcome to contribute to kmath. I currently do not plan to use TensorFlow since I do not have experience with it, but integration with ND4J and deeplearning4j is planned. Probably we can use deeplearning4J for kotlin-jvm and TensorFlow for native.