I wrote a blog post a while ago complaining about Gradle, mostly from a position of ignorance: I had never really taken the time to properly learn the tool and I was frustrated while trying to stumble my way through working with it. But luckily, my company encourages learning and experimentation so I was given an entire week–40 whole working hours–to do nothing but learn Gradle. So I did. And I still hate it, but for different reasons.

How to learn Gradle

During my period of intense frustration with Gradle I found this condescending blog post claiming that anyone who dislikes it is simply using it wrong. So when I finally got the opportunity to learn it, I was adamant that I would learn it the “right way”. No outdated tutorials, no hacks, no workarounds. I wanted to learn only the modern, idiomatic, as-God-intended method of Gradle usage.

To be fair, this is always the best way to learn any tool. Always be wary of learning something through only practical examples, how-to guides, or copy-pasted code. Because us programmers are universally terrible at everything we do, and if you learn that way you will invariably end up learning someone else’s bad practices. It is always better to go back to first principles; find resources that zoom out and tell you what the tool is for, how it is generally intended to be used, and the overall design patterns used with it. Ideally only trust sources very closely associated with the development of the tool itself since second- and third-hand sources can accidentally introduce their own baggage.

Having learned idiomatic Gradle, I can say that almost none of the resources out in the wild are good. The good list is extremely short:

  • Understanding Gradle is a great video series walking through most of the basic pieces of gradle and what they are for. It doesn’t have enough detail to teach you how to get a fully idiomatic build, but at least it gives you a sense of what kind of stuff you should be doing.
  • The official docs on structuring large projects presents similar information in a text form. I object to the word “large” in the title since it makes it sound exceptional in some way, as if most gradle projects won’t benefit from this approach. I think a more accurate title would be “structuring nontrivial projects”. If you’re doing some toy project in gradle, sure you don’t need this. But likely any serious project should be aware of these practices.
  • The idiomatic Gradle repo is a complex project applying all of the best practices. Not super useful when you’re first learning because it will look like alien hieroglyphs, but once you start to grok Gradle, it provides lots of interesting examples to consider.

Probably the most surprising discovery for me was just how useless most of the official Gradle docs are. Many of them are outdated, incomplete, or so specialized that they’re useless for general learning.

What is Gradle?

I won’t try to fully explain Gradle, that’s what the links above are for. But I can tell you the one reason you are probably failing to understand Gradle. First off, pretty much all of the common reasons given online are wrong. Well, not exactly wrong, but rather they are misleadingly simple.

Here is why you actually don’t get it: Gradle is not a build tool, rather it is a platform for building build tools. As soon as you stop seeing it as a simple means of hooking together tasks and dependencies you’ll start seeing why it is so complex.

Why does it suck?

I love this quote by Peter Bhat Harkins:

One of the most irritating things programmers do regularly is feel so good about learning a hard thing that they don’t look for ways to make it easy, or even oppose things that would do so.

I almost fell for this trap. Once I started actually solving problems with Gradle I noticed that I started talking about its complexities in a more positive light. But that’s stupid. When I took a step back I realized that all of the problems I had with Gradle before still existed, I just understood how to use them to get work done. But the fundamental problem with Gradle is that it simply does not justify these complexities.

A case study

To learn Gradle, I set myself a task of incorporating some custom code generation into our build process. This is the kind of thing I can easily do with shell scripting, but I wanted it to be a proper part of Gradle since it perfectly fits the traditional build-tool model of monitoring when an input file changes to determine when to run a task that generates an output file.

At a high level, the process is this:

  1. Run some command line stuff to prepare the code generation tools
  2. Run a script on some input files to produce intermediate metadata needed for code generation
  3. Run a script on the intermediate metadata to produce the code
  4. Compile the code normally

The goal is that I can run a single build task and it checks if either the original input files or metadata have changed and reruns whatever parts of the process are needed before building my code.

Could I have accomplished this just by opening up build.gradle and writing a bunch of custom tasks? Absolutely. But doing it that way would be like writing your entire program logic inside of main: technically feasible, but not idiomatic.

So here is what I had to do in order to do it the right way:

  1. Since the part of the project relating to input files and metadata is logically separate from the rest of our app, split it into a Gradle subproject to follow best practices
  2. Now that we have separate projects, best practice dictates that each has its own “convention plugin” to distinguish the type of build it is performing, so define a sub-build for storing custom plugins
  3. Develop a custom plugin for the metadata generation. This involved going down a huge rabbit hole of trying to use an open source community plugin since technically some of the code generation stuff I was using could be run on the JVM and thus could be built into gradle
  4. Find out that the community plugin had a critical bug in one of my required use-cases so I had to throw it out and start over. Develop a complete custom Gradle plugin for running the command line steps I had been doing manually before
  5. Best practice is to define custom task types in your convention plugin rather than customizing built-in tasks, so subclass gradle’s Exec task to specify the kind of inputs and outputs needed for each command
  6. Work around the fact that Gradle’s built-in Exec task doesn’t work on Windows
  7. Best practice is to not tightly couple your build script to your plugin’s tasks, so develop a mini DSL extension so that the tasks can change independently of the build script
  8. Now that the convention plugin is so complex, write end-to-end tests that create mini gradle builds that apply the plugin and verify that the task output is correct
  9. Write another convention plugin for turning the metadata into source code
  10. Break your brain trying to figure out how to apply Kotlin plugins to your convention plugin while applying different versions of the same plugins to your actual app build since Gradle bundles its own internal version of Kotlin which is older than the one for your app
  11. Create the code generation task. This is actually the easiest part since you can pretty much write normal code with normal tests, though you still need to figure out the right input/output types for the task since the best practice is that all such types in Gradle are redirected through “provider” types instead of being used directly
  12. Write another mini DSL extension for customizing the code generation task

  13. Figure out how to add generated code to a source set via the Kotlin Gradle plugin. Duh, it’s obviously

    kotlin {
  sourceSets {
    val commonMain by getting {
      kotlin {
        srcDir(...)
      }
    }
  }
}
  14. Finally start setting up the actual Gradle builds. Apply the first convention plugin and configure the custom metadata-producing tasks. Easy. Wait. Shit. How do you get the output of the subproject task to be visible to the other project so that Gradle understands the task dependency between the two projects?
  15. Define a “consumable configuration” in the subproject and add the task output to it by declaring an “artifact”. Oops now your build script is tightly coupled to the plugin task. This seems unidiomatic and unavoidable.
  16. Remember to use flatMap on the task to get its output because task creation is lazy!
  17. Apply the other convention plugin where code generation is needed. Configure the code generation task to get… wait, how do I get the artifact that I declared in the subproject?
  18. Define a “resolvable configuration” here. Declare a dependency which points the resolvable configuration at the consumable configuration from the subproject

  19. Wonder what the hell a configuration is and why I’m declaring them. Check the docs:

    A configuration is a named set of dependencies grouped together for a specific goal.

    Yep that clarifies nothing

  20. Hook up the resolvable configuration to the code generation task. Wait, it needs a file, not a configuration. Arbitrarily grab the first file from the configuration (since we know the subproject only added one file). That feels shitty
  21. Remember to use map on the configuration since it’s lazy!

And, believe it or not, it’s as simple as that.

tl;dr

What sucks about the above process is not that it’s long. It’s the sheer number of concepts that must be learned and interacted with: subbuilds, subprojects, plugins, tasks, providers, extensions, source sets, configurations, artifacts, dependencies, and all of the quirks and workarounds involved with each of those things.

All of this complexity makes working with Gradle a slog. You can fully, deeply understand every single aspect of your build until you want to do one thing slightly differently and suddenly it doesn’t work because you were supposed to be using some totally different part of Gradle you never heard about before.

The whole idea of using community-developed plugins becomes a minefield because the chance of them applying all of the concepts completely correctly is essentially 0%, meaning you might be okay as long as you’re only doing basic stuff, but as soon as you stray off the beaten path stuff starts breaking and you have almost no chance to fix it properly yourself.

The complexity does come with some small benefits. It’s cool that our custom Gradle plugins have their own tests so we can independently test our code generation process without touching any of our actual app code. Once you understand how Gradle likes to structure things, it does make your build scripts less cluttered and each bit of build logic ends up compartmentalized in a sensible place.

But goddam it feels like there must be an easier way to accomplish this stuff. I’ve learned dozens of different tools over my software development career and compared to them the amount of comfort I feel with Gradle after 40 hours of dedicated study is atrociously low. I have since gone on to further enhance our Gradle build process and it honestly hasn’t felt any easier. The only difference from before is that when working on Gradle stuff I used to spend way more time than I expected, get incredibly frustrated, and give up; now I spend way more time than I expected, get incredibly frustrated, and eventually (somehow) find a fix.