Notes on Gradle Microservices Monorepo setup

The product I am building/architecting at work these days uses Monorepo[1] for all our Microservices. Our Microservices are primarily built using Java 17 and Spring Boot 2.6.x. For frontend and platform code(Terraform, Helm charts, configuration files, etc) we have different Git repositories. We use Gradle 7.3 as our build tool. We also make use of shared libraries for code reuse. I know people suggest you should avoid using shared libraries in Microservices but as I discussed in an earlier post[2] I think there are valid reasons to use shared libraries in Microservices.

I prefer Monorepo for three main reasons:

• Better visibility and control. 
• Atomic code refactoring across Microservices. This is common in the initial phase of development. 
• Easy Code sharing between Microservices

Below is a representative structure of our Microservices monorepo. All our Microservices sit inside the services directory. And, shared libraries sit inside the shared libraries.

I don’t think the setup I mentioned above is unique. There are many organizations using similar setup. There were a couple of architecturally significant decisions[3] we took with respect to Gradle usage in our Microservices monorepo. In this post I will cover those decisions in some detail.

Decision #1: Use of Gradle composite builds for autonomous Microservices

There are two ways you can use Gradle with your Microservices monorepo.

• Multi-project setup: In this setup we have a root project and then we have one or more sub projects. You will have a single settings.gradle in the project root directory. Also, if you use Gradle wrapper then it will also be in the project root directory. In your root build.gradle you can use dependency management plugin to define base versions of libraries and plugins for services and dependencies. This is a clean setup that you can use to avoid build script duplication and tighter control of versions. But, the downside is that you have tight coupling. For Microservices independent deployability matters so this setup might constraint you. I used this setup 4 years back when I built a real-time pricing engine. The system had 8 Microservices and the development team was composed of only 6 developers. It worked fine for that setup. I was able to build all the Microservices by executing `./gradlew clean build` at the root of the project. Also, IntelliJ support for Gradle multi-project works like a charm so the development team was very productive. Most Gradle developers know about multi-project builds.

• Composite build setup: Late last year I started work on another product development initiative and this time we went with the composite build setup. 

A composite build is simply a build that includes other builds. In many ways a composite build is similar to a Gradle multi-project build, except that instead of including single projects, complete builds are included.

The main reasons we went with composite build are:

• Not every developer wants to import all the Microservices in their IDE for obvious reasons. Using IntelliJ composite build support you can either import all Microservices or only a few. This time our development team size is much larger (close to 50 devs). I prefer to import the complete project in IntelliJ. IntelliJ has great support for composite builds as well. In my root settings.gradle I have the following code to import all services and shared libraries.

rootProject.name='app'

def projectDirs = [
        'shared',
        'services',
]

projectDirs.each {
    file(it).eachDir {dir ->
        includeBuild dir
    }
}

• Different CI and CD for each Microservice. We use Azure DevOps and it gives us the flexibility to create different CI/CD pipelines for each Microservice by making use of the trigger.paths.include [4] option of Azure CI yaml.
• Composite build allows you to include shared libraries using the includeBuild option. For example, if service1 needs to use shared libraries lib1 and lib2 so you will first add them to settings.gradle as shown below.

rootProject.name = ‘service1’
includeBuild ‘../../shared/lib1’
includeBuild ‘../../shared/lib2’

Next, you add the dependency in the Microservice build.gradle

dependencies {
   implementation “com.example.app:lib1”
   implementation “com.example.app:lib2”
}

You can now change both the libraries and services together without the need for any dependency release process. One thing you should know is when you build service1 then lib1 and lib2 will be built but tests will not run for lib1 and lib2.

• Composite build allows you to have independent Gradle projects that can have their own dependencies, plugins, lifecycle. There is no root project here. All services and libraries are at the same level. This means if we want we can move libraries to their own Git repositories. We just need to ensure we include the correct path in the includeBuild directive.

Decision #2: Use of Gradle version catalog for centeralizing dependency versions

We have been actively doing the development for the last four months. We started with Spring Boot 2.4.x and the latest Spring Boot version is 2.6.3. Spring Boot is just one example. We also have few other dependencies that are not mentioned in Spring Boot parent BOM so we pick their latest versions from Maven central. 

Since last one month I started seeing different versions of the same dependencies. A couple of our libraries required Jackson so the developer added it to build.gradle but they used an old version. There were also different Spring Boot versions. It will be a few more months before our application will be made available to end customers. I want to ensure we use consistent library versions so that it is easy to migrate to the latest versions and we don’t get any classpath issues. Also, I don’t want the new tech stack to become old before it is released. 

So, I gave this task to one of our developers to figure out a way to lock version numbers but still give Microservices the independence to override them if required. She (the developer who was assigned the task) did research and found out about the Gradle version catalog, a feature introduced in the latest Gradle version. I have earlier done something similar using gradle.properties. I sometimes refrain from telling how I did earlier since I don’t want to deprive them of a learning opportunity. Also, we might miss an opportunity to find a better answer. It does not mean that I don’t help, I try to nudge towards the answer . I also learn in the process. Gradle version catalog is a much better solution than what I knew. 

The way it works is you start by creating a libs.version.toml file in the root gradle directory.

The TOML file has four major sections.

• the [versions] section is used to declare versions which can be referenced by dependencies
• the [libraries] section is used to declare the aliases to coordinates
• the [bundles] section is used to declare dependency bundles
• the [plugins] section is used to declare plugins

Below is an example toml file with dependency versions clearly defined. 

[versions]
springBootVersion2_6_3 = “2.6.3”
okhttpVersion4_9_3 = “4.9.3”

[libraries]
springBootLib2_6_3 = { module = “org.springframework.boot:spring-boot”, version.ref = “springBootVersion2_6_3”}
okHttpLib4_9_3 = { module = “com.squareup.okhttp3:okhttp”, version.ref = ”okhttpVersion4_9_3” }

[plugins]
springBootPlugin2_6_3 = { id = “org.springframework.boot”, version.ref = “springBootVersion2_6_3”}

[bundles]
testContainerDeps = [“postgresTestContainer”, “junitTestContainer”]

Then, each project (Microservice and shared library) import this version catalog in their settings.gradle

dependencyResolutionManagement {
    versionCatalogs {
        libs {
            from(files("../../gradle/libs.versions.toml"))
        }
    }
}

Then, in build.gradle we use the aliases instead of their name.

plugins {
  alias(libs.plugins.springBootPlugin2_6_3
}

dependencies {
implementation libs.okHttpLib4_9_3

You can read more about the version catalog in the Gradle documentation[5].

Conclusion

I believe Gradle has the right primitives to support Monorepos for Microservices. As I discussed in this post you can get the best of both worlds using composite builds and version catalog.

References

1. My thoughts on Monorepo – Link
2. When to use shared libraries in Microservices architecture – Link
3. Architecturally Significant Decisions – Link
4. Azure CI yaml schema reference – Link
5. Gradle version catalog – Link