Why kotlin build tooling is SO AWFUL?

1

I have been trying to migrate my multiplatform project to Kotlin 1.9.0 from Kotlin 1.5.21 for 6 hours with no luck… I just want to make a statement: build tooling around kotlin multiplatform is awful!

  1. No repetitive behaviour. The build fails at diffrerent places
  2. Build with gradle fails differently than build in Idea
  3. No clear messages. I cannot figure out what my be wrong after reading the message below provided this module even does not have tests!
FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':Designer:jsJar' (type 'Jar').
  - Gradle detected a problem with the following location: '...\out\production\designer'.
    
    Reason: Task ':Designer:jsJar' uses this output of task ':Designer:compileTestKotlinJs' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':Designer:compileTestKotlinJs' as an input of ':Designer:jsJar'.
      2. Declare an explicit dependency on ':Designer:compileTestKotlinJs' from ':Designer:jsJar' using Task#dependsOn.
      3. Declare an explicit dependency on ':Designer:compileTestKotlinJs' from ':Designer:jsJar' using Task#mustRunAfter.
1 Like
2

This same kind of failure happened to a colleague of mine in front of me, and he solved it without hesitation by reading the error message. I think this kind of issue is one that will go away as you become more familiar with Gradle.

Your module (or project) “Designer” has a task “jsJar” that is using the output of the “compileTestKotlinJs”. Are you declaring the “jsJar” task? Even if not, you can declare the dependency using this:

tasks.named("jsJar") {
    dependsOn("compileTestKotlinJs")
}

Hopefully that fixes the issue.

Just a note. Gradle is a heavier build tool expected to build complex projects, multiple languages, multi-module projects, targeting multiple platforms. On top of that it’s intended to be the primary control for the developer to run tests, produce artifacts, containers, and anything else the developer needs.
^ This is why Gradle has a large learning curve for complex projects.

If you’re coming from another build tool where all it does is dependency management, Gradle can do that pretty easily (basically a dependencies block and you’re done). However, if you’re doing the complex things Gradle allows you to tackle, then it requires a good amount of familiarization with Gradle and reading the docs. What’s worse is that most examples outside of the official docs show old ways of laying out Gradle projects.

For anyone working with Gradle beyond the basic dependency block, I highly recommend spending a few minutes skimming through the Gradle docs. Gradle is so much harder to learn by copy-paste StackOverflow IMHO.

JetBrains knows this is tough–especially since multiplatform is a complex build that doesn’t have to be complicated for Kotlin use-cases. Check out this recent blog post on their project Amper and notice how easy project configuration may become in the future.

3 Likes
3

First of all, thanks for advice, I will try it next time, when I have free 6 hours for hitting my head against walls trying to upgrade version of tools. For now, I have just rolled back to Kotlin 1.5.21.

Why should I put anything in my build scripts, which is not really required? Why task jsJar should depend on compileTestKotlinJs, if module does not have tests!?

In normal live, after such upgrade I just receive a bunch of compilation errors related with changes in language, which is expected and perfectly fine, but not with Gradle, no… I need to upgrade damned gradle wrapper, which I cannot do smoothly, because it starts dumping hundreds of errors, something about ‘kotlin plugin not compatible with build variants…’

Why for the hell sake, I need this wrapper?
Why its numerous versions dumped inside .gradle folder of my project dir?
Why upgrading it to newer version is related with what is inside my build scripts at all?!
Why I cannot exclude tests from execution by passing -x test? For some reason kotlin plugin still runs them…
And after all of this there is another problem, and then another…

Steep learning curve? I do not want to waste a minute of my time on learning this useless build tool, which cannot do the simpliest job of just shutting up and building a bunch of trivial kotlin modules…

To sum up and speaking generally, yet another statement from me: Gradle is a “piece of digestion product”, looking like it was designed and developped by drug addicts. Sadly, this ugly monster became popular and now it seems the only option to build Kotlin projects, hopefully JB will be able to fully complete their Amper.

2 Likes
4

Well, we don’t have to use Gradle, we can use kotlinc and other tools manually. That actually provides a good perspective on why we do need build tools and why they are more complicated than we expected them to be. We will see how much they do for us underneath.

Your specific case with this compileTestKotlinJs seems to me like a bug. Either somewhere in the Kotlin plugin or in the configuration of your project.

2 Likes
6

Yeah I think your problem here is with Gradle, which is not a Kotlin build tool specifically, but rather a JVM build tool. If you don’t like Gradle, use Maven.

I don’t disagree with you, FWIW; I also find Gradle to be an absolute pain in the rear end. When it works, it’s awesome, but when it fails, it fails in the most complicated, spectacular ways that can take you ages to debug.

1 Like
7

@vk I’m not an authority on Gradle by any means, but allow me to answer your seemingly rhetorical questions to give you some perspective.

You don’t. Feel free to install Gradle globally on your machine instead. It’s just much better to use the wrapper so that everyone running your project actually uses the same Gradle version as you.

As written in the first Google result: “The Wrapper is a script that invokes a declared version of Gradle, downloading it beforehand if necessary”. So basically when you set it up, people can simply clone your project and run ./gradlew build. They don’t need to know which version of the tooling to install.

This is so that only one person has to set it up, instead of every contributor. So again, you don’t need the wrapper, but if you don’t have it then you will need to install Gradle itself in the correct version on your machine, and you might not even need the same version in every project, so that’s additional overhead that you would need to deal with.

They are not. Also, you seem to be confusing the Gradle wrapper and Gradle itself. The wrapper is just a little script (+ a jar) that ensures you’re running the version of Gradle that you want. These versions of Gradle are downloaded into the ~/.gradle/wrapper/dists folder on your machine, which is common to all projects.

The project’s .gradle directory mostly holds project-specific Gradle caches. What’s inside shouldn’t really matter to you. But if we want to dig, I believe the versioned folders that you see inside it simply contain pieces of the build cache or config cache produced by the corresponding Gradle version. They are probably separated for compatibility reasons (so an old version of Gradle doesn’t try to use the build cache produced by a more recent version and vice-versa).

When you change the version in gradle-wrapper.properties, you don’t update the version of the wrapper itself, you actually tell the wrapper which version of Gradle to use (or download if not present).

Upgrading the Gradle version is definitely related to the contents of your build files: Gradle is the build tool that reads those files. Like every tool, it evolves and so does the build script’s APIs and DSLs. In a very old version of Gradle, some new concepts didn’t exist; in more recent versions, some old concepts may disappear. How would you expect the configuration format to become better over time without this?

Upgrading the wrapper itself is done by running ./gradlew wrapper (with optional parameters). That will update the scripts and the jar of the wrapper so they match either the version of Gradle that you’re running, or the version you specified as an argument. This is what actually updates the wrapper itself, and it shouldn’t impact your build in any significant way.

That might be a bug, but also using -x test is usually a bad idea. Applying plugins to your build configures a lot of things for you, and in particular a task graph. Compiling the files is necessary in order to execute the tests for instance, so the task that runs tests depends on the task that compiles. Excluding arbitrary tasks using -x is a bad idea because it can break the build by removing a required dependency. Instead, use the task that does what you want to do. If you just want to build your artifacts, use assemble instead of build (because the latter will also run the tests).

4 Likes
8

Just as a side note, you’re updating a project from a version that is almost 2.5 years old. During such amount of time, many things may have changed around the tooling including the build tool and the IDE, which is why it may not be as smooth as it usually is.

Typically the Kotlin Gradle plugin evolves together with Gradle, which is why you might have to upgrade both of them. In turn, Gradle is trying to improve the developer experience by removing footguns here and there, so if you upgrade from a very old version of Gradle, it is possible that some of your old constructs now produce warnings or errors. It’s a pain but it’s also for the best, that’s what it means to upgrade: it allows you to bring your projects to current standards as well.

Similarly, the Kotlin IDEA plugin evolves together with Kotlin AND IntelliJ IDEA.

All these pieces are not infinitely compatible backwards and forwards, so I guess it’s reasonable to expect that they should be updated together, or at least to some versions that are not too far apart.

Note that you might have a smoother process if you try to upgrade slightly more incrementally. For instance you could first fix all the warnings you had in your current version of Kotlin and Gradle, then upgrade to 1.6.21, then see what you get, then move on to 1.7.21, 1.8.22, and finally 1.9.20. Or you could go slightly faster by skipping every other feature version and jump 2 at a time.

1 Like
9

I would actually suggest the opposite: do not update “gently”, by going one version at a time. Instead, nuke everything and jump straight to the newest versions of everything by generating entirely new project config. Then add missing bits.

My experience with various Gradle projects is that people often add a lot of mess to their project configs. Gradle was designed with the concept of convention over configuration. That means less is more, we should add as little config as only possible to achieve our goals. This way Gradle works the the best, it is the most reliable and we have the least problems with maintenance in general. People tend to do the opposite: they copy random fragments of configs from the internet to their projects, often incompatible with each other. Then they glue these random fragments in random patterns until the project accidentally starts working. Resulting config monstrosity blows up whenever we need to change anything.

Of course, this is partially a fault of Gradle. It is complicated and over years they changed many basic concepts. As a result, there are multiple ways of doing same things, depending on the version of Gradle, and sources in the internet aren’t always relevant to our project. I think this is a natural consequence of improving the tool, but I understand the frustration of people.

Secondly, this is a fault of people. I don’t know why, but many people assume this is normal they have to learn Java, Kotlin, Android, Ktor, Spring, Compose, coroutines, Room, etc., but Gradle? No, no, let’s not learn and instead copy random configs from the internet.

Going back to my first thought. If the project config is clean, and by “clean” I mean it is relatively simple, almost empty, then either we use incompatible versions of Koltin plugin vs Gradle vs something, so I suggest checking compatibility matrix, or maybe the version we chose has some kind of a bug (?). If the config isn’t clean then I suggest starting from scratch as this will be much easier than trying to migrate a faulty config.

5 Likes
10

@broot While I understand where you’re coming from and the rest of your post (which I almost 100% agree with), I don’t think “nuke everything and re-write the build from scratch” is viable for a lot of projects. Most non-toy projects out there simply have inherent complexity that needs some configuration, thus a sizable amount of configuration is actually necessary (even when done right, and even in the latest version).

Of course, upgrading (to any version) takes discipline. You need to actively seek out outdated pieces of config, and for this you need to make the effort to understand which piece is here for which reason. This is an effort a lot of people are not ready to make, and like you I don’t quite understand why people consider that it’s not part of their job description. I get that they want to get things done, but knowing how your stuff is built is important, and even necessary at times.

Fortunately, the tooling should warn you about things that shouldn’t really be used anymore, so upgrading incrementally will not lead you to a broken build down the road. That said, it doesn’t solve the problem of extra pieces of configuration that were written by copy-paste a long time ago, and that don’t make sense anymore – this still requires conscious effort to make things clean.

I would even add that the nuke-and-rewrite approach doesn’t work any better for someone who’s not willing to look for the idiomatic ways to do things. Copy-pasting the same outdated pieces of code in recent versions will not help rewriting a correct config after nuking.

4 Likes
11

Yes, I agree. I just assume the most typical case is an Adnroid/web application without or almost without any special needs for building. From my experience, in most cases the custom code in the build config isn’t really related to the specifics of the project, but is a result of a past workarounds of some sort.

Of course, there are much more complicated cases. Once, I worked on a tool for testing the security posture of Android devices. Project consisted of a desktop UI app, Android app, web app and it had plugins stored in separate modules. Desktop app automatically installed the Android app over the USB, so technically, desktop app contained the Android app. Everything was kept in a single Gradle project, separated in modules and it was built and run with a single magic ./gradlew run with minimal setup on the machine (I believe only JDK). Yeah, power of Gradle. You can imagine there was quite a lot of build config there. But it was for a reason! Almost every line in the build config was intentional and meaningful. I can’t say the same about many projects I looked into.

Also, sometimes build configs are complicated not due to the project needs, but company needs. Internal artifact repos, artifact signing, containerization and other similar requirements.

2 Likes
12

Dude 100% this, at my work we have a lot of older Groovy+Gradle microservices, and the build.gradle files are almost always full of the same copy-pasted code, with little to no regard for whether it’s actually needed for the specific project.

13

I had similar problems upgrading from Kotlin 1.5. The problem was not so much Kotlin or gradle but more to do with dependencies. It is what is typically called dependency hell. The jump from Kotlin 1.5 to 1.6 had major changes that required changes in your dependencies. It makes it very hard to upgrade 1 piece at a time because upgrading that one piece will upgrade other transitive dependencies. In my case it was further complicated by the fact that a major source of dependencies went away at the time so I couldn’t get to some versions

You may have better luck upgrading everything at once as others have suggested.

14

I would argue that it’s pretty bad advice. First of all it doesn’t work for MPP, but even JVM is way behind Gradle in terms supported features and speed with Kotlin (and even without Kotlin it may be not the best solution to go with Maven for larger project)

16

I believe you didn’t attach the demo project. Do you have a dependency for the test library?