Terraform - Monorepo Vs. Multirepo
In this short post I'm going to discuss the monorepo vs multirepo strategy in Terraform
What is Terraform
I'm not going to to into too much depth, I will assume that since you are here you already know but incase you don't; Terraform is an open source project by Hashicorp that provides Infrastructure as Code, the real benefit of this is that Terraform has multiple prodivers, so from one tool you can create, manage and destroy different clous tools (Azure, AWS, Datadog etc.)
I've written other posts about Terraform on this site, so please take a look.
As stated above Terraform a tool for creating "Infrastructure as Code" the files are written in the Hashicorp Configuration Language and given the extension .tf, the CLI tool is then used to perform various steps, from the initialisation to the plan, apply and destroy steps.
As with all code it's common practice that you store this in source control e.g GitHub or something similar.
Now like most code you can either create a monolith application with all the code living in the one repository and have multiple devs working on it at once, or you can opt to separate out concerns and form individual teams that manage and maintain their own services.
A brief journey in time:
When I first started at FA the team I joined were the only team using Terraform to manage infrastructure in Azure, in those early days we had all the code living in one repo, which was fine when we had 4-5 services to manage, we could create and update the files and then get the expected infrastructure in Azure once, the difficulty came as we started to create more services with varying complexity.
In order to improve our situation we took advantage of the
modules concept in Terraform, this allowed us to keep the mono repo but deploy each "module" as required, this quickly freed up the devs to deploy their changes individually but didn't necessarily mean conflicts in the terraform process would be resolved, a common one in the current approach is if a one team create an experimental piece of code that gets merged into the PR by another team then it can result in conflicts, sure we could probably only include the files we need in a PR, but this then becomes a human managed process and a bit tedious.
This is currently where the team sits, and whilst we managed this rather well, we do occasionally hit the odd snag.
For the most part it's okay, we are one team looking after several services, so in some ways it makes sense to have a mono repo approach, we can assert a level of control and quality over the files and make that part of our "standard" approach.
The difficulty comes when other teams looking to use terraform to manage their Azure infrastructure want to participate, it can result in merge conflicts, or in definition conflicts when the stages of terraform are ran in a pipeline (see above point). It's good for driving the conversation but can slow down or impede a teams ability to deliver.
Opportunity arose when I was approached by another product team about getting set up with Terraform. My first thought was "use ours" but then a little nagging feeling in my mind told me here was the chance to separate out the work of this team from the large cumbersome monorepo and hand back the control to them, they'd benefit from our learning but be able to run and edit thier own work freely.
I spent a few days talking and working through the strategy and getting things setup with the other team, one thing that we did was to use a purely YAML based approach within Azure DevOps as opposed to GUI defined release pipelines. Now I don't wish to take any credit for this, my input was simply to discuss a couple of things, the multi or independent repo and then a little on the handling of how we would store the state file in blob storage. The real genius has to go to my colleagues in this team, who took the idea and implemented some very thorough and well thought out templates in the yaml pipelines.
The result, I can honestly say was glorious, the templates were simple but solved a couple of key issues with the mono repo:
- A single source of truth with a mainline "build" and a promotion of build artifacts into environments
- Parameterisation of environments in the YAML templates to allow for rapid deployment into different environments
- Generic enough to enforce a standard approach, but give other teams the flexibility to achieve their own goals
The fun didn't stop their though, with these templates to hand we could start to enable other teams to make use of them and this is exactly what's been happening.
I've had the pleasure of spreading the good news, so to speak and point others seeking to utilise terraform in the direction of the scripts so that they too can start to manage and maintain their own infrastructure.
This level of abstraction has in my eyes been quiet liberating for these teams as I can see that they are releasing faster than ever and not getting stuck waiting on the approval of others, at this early adoption stage it's working, however my reservations may come; one thing that does concern me is that some teams may structure or approach it differently and we'd end up with mis configured infrastructure or infrastructure that wasn't meeting the expecting compliance levels etc. Time will tell, but I will also help steer the ship in the right direction, so that the "best practice" is being met across teams.
So which one?
Well I think the writing speaks for itself, I'm a fan of the multi repo approach, from what I've seen it rapidly enables product teams to deploy the required infrastructure, and promoting changes into another environment are as simple as supplying the variables. It's fast and that means less time spent on writing IaC and more time spent getting the idea out into the hands of users so it can be tried and tested.
However I would say that this also depends on the business and their approach to the different services that IT might be providing, Footasylum is very muich a product focused place, with individual teams working on and delivering thier product, be it the loyalty scheme, the commerce tools etc, for us it makes sense to use the multi repo approach as the benefits to the teams is very apparent.
Hope you found this informative, please reach out and let me know!