My Thoughts On Monorepo

Before we start let me give some context on my background so that you can better understand my thoughts on Monorepo. 

I head technology at an IT services organization. Most of the products that I build are using Microservices architecture, have multiple frontends(web and mobile). The biggest product that I recently built had close to 30 microservices, 1 web client written in React,  and native mobile app built using React Native. These numbers are nowhere near the numbers big product companies have shared. 

I prefer Macroservices over Microservices. I think most products don’t need more than 10 microservices. 

The reason I am clearly specifying I belong to the IT services world is because most of the stuff we consume on software development is written by engineers and senior tech people at the product companies. The stuff they write and share is based on the real problems and challenges they face at work. There are times when those problems resonate with problems other software engineers face at their work but there are times they are solutions to the problems we don’t have. So, we have to look at these solutions from the lens of our problems.

The post is based on my experience building software, leading and managing software delivery teams, and learning from the great articles written by engineers using Monorepos. Please refer to the references section for good resources on monorepos.

Let’s get back to the topic at hand.

So, what is a Monorepo?

A monorepo is a software development strategy where a single version control repository has source code for multiple projects, libraries, and applications irrespective of their programming language. Also, the organizations using Monorepo strategy often use a common build tool (like Bazel, Pants, Buck) to manage all the source code. Some of the popular examples of organizations that employ monorepo strategy are Google, Facebook, Twitter, Microsoft, and Uber. 

The alternative to monorepo is polyrepo/multirepo. In multirepo, you have a separate version control repository for each component. This is the common strategy used by most organizations to structure their code. This in my view has been largely driven by Microservices architecture style and small modules movement.

As mentioned in the paper[1] (Advantages and Disadvantages of a Monolithic Repository – A case study at Google), Monorepos have following properties:

• Centralization: The codebase is contained in a single repo encompassing multiple projects.
• Visibility: Code is viewable and searchable by all engineers in the organization.
• Synchronization: The development process is trunk-based; engineers commit to the head of the repo.
• Completeness: Any project in the repo can be built only from dependencies also checked into the repo. Dependencies are unversioned; projects must use whatever version of their dependency is at the repo head.
• Standardization: A shared set of tooling governs how engineers interact with the code, including building, testing, browsing, and reviewing code.

My understanding is that to successfully use monorepo you will have to satisfy all the properties. Otherwise, you will not get benefits intended from monorepo. 

Advantages of Monorepos

There are valid reasons why many big product organizations prefer Monorepo. Following are the main reasons:

Reason 1: Simplified dependency management

Monorepos make dependency management simple by:

1. You can easily depend on other projects/modules in a monorepo without the need for artifact management tools like Nexus, Artifactory etc.
2. You avoid diamond dependency problem. Diamond dependences occur when a project has two dependencies which depend on the same underlying library. When a developer upgrades a dependency, they run the risk of breaking a diamond in the dependency graph.
3. It is easier to keep all dependencies on the same version by using a centralized way to manage version numbers.

This is simplified further by using a single build tool. I have not used Bazel, Bucks, or Pants. I was watching a talk on Twitter monorepo journey where they talked about Gradle being too slow for their use case. For the size of applications I have built Gradle has worked just fine.

Reason 2: Code sharing and reusability

The second big benefit of Monorepo is that developers can share code across projects. It is easier to enforce best practices across the code base by using monorepo. Another related point is that with monorepo we don’t end up creating silos. This is important in an enterprise setup because it leads to passing the buck and bugs falling through the cracks of the boundaries. In my experience with multirepo setup people only care about their Microservice running fine. They miss the point that value is achieved by integrating the software and collaboration. In IT service organizations where there is more bureaucracy and uneven distribution of skilled developers the problem scales very quickly with multirepo setup. Yes, I know it is a culture problem but most IT service organizations can’t burn investor dollars to build the culture.

Reason 3: Atomic changes

This I didn’t realize before I read literature on Monorepo. There is a lot of benefit in seeing related changes in a single commit. If you are working on a story that requires changes in multiple components then in a multirepo scenario you will have to see changes in multiple repositories and merge the PRs in some sequence so that you are in a healthy state. WIth monorepo you save the pain of trying to coordinate commits across multiple repositories. Also, this leads to better code reviews as all the changes are in one place.

Reason 4: Large-scale code refactoring

This is related to reason 3. With a monorepo, you can refactor the API and all of its callers in one commit. You see all the usages of an API at a single place and it is much easier to do than with multirepo where you might not even have all the code checked out. In my experience with multirepo setup most developers don’t keep all the repos updated with the upstream changes. Monorepos enables continuous improvement on global level that multirepo you do at local level.

Reason 5: Less bureaucracy

With some organizations I have worked at, you have to create ServiceNow tickets to create a repository. It can take a couple of days before you get your empty repository. With monorepo you don’t have to go through this pain. 

Disadvantages of Monorepo

Nothing comes for free. There are always trade offs involved. Your job as a software engineer is to figure out if advantages weigh more than trade offs or not. 

In my view following are the downsides of monorepo:

1. Monorepos could slow down developers because of slow build times, poor tooling, and merge conflicts. 
   1. Most developers still in 2020 struggle to cleanly merge code. 
   2. Git is slow for projects with large numbers of files and history.
2. There is cognitive overhead involved as developers have to get comfortable with a much larger code base than they would have with multirepo setup.
3. To do monorepo well require investment in tooling that most organization non-tech leadership will fail to understand

So, what’s my view on monorepos?

Before I talk about my views on Monorepo let’s understand three main constraints of IT services organization. 

1. We work with multiple customers so we can’t keep code of all customers in the same repository even when we host their code in our version control for obvious reasons. Also, we can’t give access to all our repositories to all our developers because of security and IP related issues. So, we will keep our discussion focused on how to manage repos for a single customer.
2. IT services organizations have a high ratio of junior(< 5 years) to senior engineers(> 10 years) somewhere in the range of 10:1 to 100:1 or may be higher in bigger IT service organizations. The reason I am bringing this point is that monorepos requires discipline and it is tough to achieve without senior engineers driving it using a well-defined process.
3. People come and go at a faster rate.

Given the above two constraints and the disadvantages of monorepos it might seem that monorepos will not work for us. But, I see real problems faced by software delivery teams that can be solved by monorepos.

We build products for different customers. These products usually follow Microservice architecture, have multiple frontends – web and mobile, functional tests, scripts for deployment automation. In the multirepo strategy, you will create at minimum 5 repositories – 1 for backend with all microservices, 1 for SPA frontend, 1 or 2 repo for mobile depending on whether you are building pure native or using some native framework like React Native or Flutter, 1 for functional tests, 1 for deployment automation scripts.  More often than not your team will use one repo per Microservice then only god knows how many repos you end up creating.

Let me tell you a real story. I was once working with a client that had more than 1000 repositories in their version control system. They were using the Gitlab version control platform. They had 5 products and each product was made up of multiple Microservices. The first question I asked them was to help us understand which all services and their respective code repositories were part of product A. Their chief architect had to spent a day figuring out all the repositories that made the product A. After spend a day still she was not sure if she has covered all the services.

Let’s discuss problems that I face with software delivery teams using the multirepo strategy. Just to reiterate these problems are in the context of a single customer.

1. Lack of accountability: Humans are good at creating boundaries and silos. They don’t care what happens outside those boundaries. They don’t care about the bigger picture.
2. Version drift. 10 different versions of Spring Boot, three different JDK versions, multiple versions of React, and god knows how many different versions of libraries.
3. Repository sprawl. Everything ends up becoming a repository.
4. Health of projects. Using the same tools for all projects. Code consistency. Single source of truth
   1. Architecture patterns
   2. Coding style
   3. Testing practices
5. When you use multirepo everything ends up become a separate repository. And soon you lose the sight on how many repos you have. 

Can monorepo work for us?

Yes. For a single customer we don’t have to scale to millions of lines of code and 1000’s of developers. For a single customer we are under a million lines of code and our delivery teams are well under 100. We don’t have large version control histories and our developers commit less than 1000 commits in a week. So, we are well below the numbers shared by Google and Facebook. 

This means existing tools like Git and Maven/Gradle work fine for us. 

We are already doing that for one of our big customers. Below is the mono starter repo we use for new products.

References

1. Advantages and Disadvantages of a Monolithic Repository – Link
2. Monorepos: Please don’t! – Link
3. Why Google Stores Billions of Lines of Code in a Single Repository – Link
4. Advantages of Monorepo – Link