How Using a Monorepo is Helping Us Work Better

Similar to many other mature startups moving towards their growth stage, Minute Media had to rebuild some of its infrastructure.

Our initial infrastructure was a Monolithic application in Ruby on Rails, but we realized we needed to explore other options as our company and team grew. As we set out to explore more options, our goal was to create tools and company-wide standards that would help facilitate open conversation and company growth…

In our company structure, our teams work in cross-functional organization with clear and sometimes separate business goals. During our exploration, each team ventured out on its own, testing technologies like Node and Go. A few weeks in, we realized we had some communication problems. When we had the opportunity to share our work, we noticed similar technical and design problems arose across teams.

To avoid duplicating efforts across multiple teams we tried our new development efforts in a monorepo.

What is a monorepo?

A monorepo is, simply put, a monolithic repository. The idea is to keep all, or a majority of, the projects in the company, in one single repository. This concept is not new and has been in practice for 10 years, most notably, it’s being used by large companies and corporations such as Google, Facebook, Twitter, and Uber.

After using monorepo for over a year, I would like to share some of the benefits and lessons we have learned:

Introducing Transparency

Our codebase is available to all team members, but when projects each had their own repos, we rarely found ourselves reviewing the other team’s code.

Now, we can enable all team members to review the entire code base. Most of the code is still owned by separate teams, but if someone is interested in seeing what other teams are doing, all they have to do is take a look at the pull request.

The barrier to doing so is extremely low, since it is just another PR in the repository . In fact, team members regularly find themselves browsing other teams’ PRs.

This process has led us to subconsciously create better and more readable code, since all of the code is in the open to all teams, similar to the open source effect.

Building Alignment

When everything is visible and accessible in a single repo, it is much easier to agree and re-enforce standards. Previously, every team had its own specific standards, but as we better aligned ourselves , . we came up with a process for company-wide standards. In this process, all standards are tested and implemented in each respective team, and once proven useful, all teams move forward and the new standard is implemented across the entire monorepo. Standards we have recently implemented are:

• Directory structure for restful services in Go
• Switching from Javascript to Typescript
• Linting rules for Javascript projects
• Documentation formatting
• Authentication
• Deployment and monitoring

Sharing Ideas

Since switching to monmorepo, we are seeing better cross system architectural designs. This was very hard to spot before, especially for services which are in separate repositories, but now similar functionality is easy to spot and extract. Building an in-house framework to support and increase delivery velocity, we use the repo in one service and then apply it. Using one codebase allows participants to spot new or similar ideas. For example, you can use a service that categorizes videos according to the transcript, or one that categorizes news articles as well.

Integrating Early

Since the whole system is in one repo, writing integration tests and launching the entire system in Docker is a breeze. Today, Minute Media is able to detect integration problems extremely early.

Using Docker Compose, you can spin up the entire end-to-end system on your laptop. This enables more elaborate features, like spinning ad hoc QA environments or demos for clients.

Allowing for Simpler Maintenance

In addition, language upgrades and security patches are applied across all of the code at once. Maintaining and updating external dependencies is much simpler, and more accessible than before.

For example, upgrading from Go 1.10 to 1.11 was done by a single developer over a period of two to three days, on 20 different services.

Smoothing Corners

Like everything else in software development, a monorepo requires some discipline. Just like sharing ideas is easy, so is breaking service encapsulation.

Not everybody feels comfortable sharing all of the code all the time and sometimes sharing creates more complicated discussions, where there should just be one simple decision.

Furthermore, once the code grows it becomes slower to push/pull requests - which is why we are working towards better integration. We are still at the beginning of this experiment, and it’s obvious that there are many things we still need to add to our tool box, such as:

• List of rules for monorepo
• Automated project generation
• Template deploy scripts
• Pull request templates
• Usage of  directed graph build systems, like Bazel or Buck

Summary

There are a lot of technical benefits which can be achieved by using a monorepo. With some work, I believe the same technical benefits can be obtained as when using a multi repo.

Most importantly though, introducing a tool that generates transparency and alignment is far more important than any other technical advantage.