Terragrunt for DRY infrastructure

We have seen in the last article benefits of using terraform to create an infrastructure on AWS cloud and deploy an example of a full stack application based on strapi and NextJs, check previous articles for more details about the context. Requirement for this tutorial:

AWS account AWS CLI Installed Docker Installed Terraform installed Terragrunt installed Basic knowledge of AWS components commands and Docker Terraform modules from the previous post of this series terraform deployment

Why Terragrunt?

Using only terraform for small projects may be enough, but when we deal with large and complex projects, things become tedious and repetitive, which increases errors. Consider the following file structure, which defines three environments (prod, qa, stage) with the same infrastructure in each one (an app, a MySQL database, and a VPC):

The contents of each environment will be more or less identical, except perhaps for a few settings (e.g. the prod environment may run bigger or more servers). As the size of the infrastructure grows, having to maintain all of this duplicated code between environments becomes more error prone. You can reduce the amount of copy paste using Terraform modules, but even the code to instantiate a module and set up input variables, output variables, providers, and remote state can still create a lot of maintenance overhead. How can you keep your Terraform code DRY so that you only have to define it once, no matter how many environments you have?

Based on the official documentation site web is a thin wrapper that provides extra tools for keeping your configurations DRY (don’t repeat yourself)

why should use it / benefice:

• keep terraform code DRY
• keep remote state configuration DRY
• keep command line interface flags DRY
• simplify infrastructure architecture
• execute terraform commands on multiple modules at once

Terragrunt has the ability to download remote Terraform configurations. The idea is that you define the Terraform code for your infrastructure just once, in a single repo, called, for example, modules:

This repo contains typical Terraform code, with one difference: anything in your code that should be different between environments should be exposed as an input variable.

In a separate repo, called, for example, live, you define the code for all of your environments, which now consists of just one terragrunt.hcl file per component (e.g. app/terragrunt.hcl, mysql/terragrunt.hcl, etc). This gives you the following file layout:

Notice how there are no Terraform configurations (.tf files) in any of the folders. Instead, each terragrunt.hcl file specifies a terraform { … } block that specifies from where to download the Terraform code, as well as the environment-specific values for the input variables in that Terraform code.

How to use it in practice?

keep terraform code DRY

In the previous guide, we build a bunch of terraform modules, aiming to create and define multiple resources that will need to create our infrastructure.

The illustration above is the folder structure, the name “terraform” is not required you can name your folder as you like. In case your project is bigger enough to have multi-applications, you can add another level in the folder structure to join modules separately for each sub-application.

create a folder and name it as you like (backend in our case). In this folder, we defined the code of our environment which contains terragrunt.hcl file for each environment.

We defined two environments; you can add as much as you need -> Flexibility. The content of terragrunt.hcl:

Let’s explain each part :

Terraform {…} block -> indicates where terraform code is located. (Note: the double slash // is intentional and required) You can use a remote file, for example, a GitHub repository with ssh:

Locals {…} block -> used to define aliases for Terragrunt expressions that can be referenced within the configuration. All the arguments passed into locals are available under the reference local.ARG_NAME throughout the Terragrunt configuration file.

Include {…} block -> used to specify inheritance of Terragrunt configuration files. The included config (also called the parent) will be merged with the current configuration (also called the child) before processing.

Input -> variables and properties for this environment (you can refer to these inputs as terraform outputs or declared environment variables). Assuming that terraform code use variables as we saw in the previous post.

The illustration shows the difference between directory structure with and without terragrunt:

As we said before, the impact of terragrunt might not be seen in a small project, but imagine you have 4 or 5 environments, each consisting of many sub-applications.

keep remote state configuration DRY

Terraform supports remote state storage via backends as we did in the last post, but the problem with those ones is they lack support of variables or functions, which forces us to copy/paste the same Backend configuration into each terraform sub-application folder.

To fix this with terragrunt, we create a terragrunt.hcl file in the root folder of the project file source and another one (also terragrunt.hcl named Child in documentations) in terraform modules folder to inherit the root’s configuration.

Child file: In our case, Strapi is the folder that contains the child file. The AWS folder structure should look like this:

The file content should be like this:

Root file:

The root file structure should be like this

The file contains the configuration of the backend.tf :

The file holds the remote state configuration for the terraform to be stored in S3.

The terragrunt built-in function path_relative_to_incluse() returns the relative path between the current terragrunt file and the other child one.

Here, we created a backend configuration based on remote_state block, but there is an alternative way, use generate a backend terraform block as shown in the illustration above:

What is the difference:

The generate block introduces a problem: in case we used the s3 backend, terraform expects the S3 bucket to already exist for it to upload the state objects. Ideally you can manage the S3 bucket using Terraform, but what about the state object for the module managing the S3 bucket? How do you create the S3 bucket, before you run terraform, if you need to run terraform to create the bucket?

keep command line interface flags DRY:

Step 1:

Let initialize the backend:

Terragrunt init

The message of success shows if all run correctly, which means that connected successfully with the S3 backend already created in the last post else check your aws credentials or profile

Step 2:

Apply changes:

Terragrunt apply

Conclusion

Great job this was the last post in this series of tutorials, you have now seen how to build a full-stack web application with Strapi and Nextjs and deploy it using docker-compose in the first post, the second one it was about how to deploy in AWS with terraform, last and this one you have seen how to manage your infrastructure and keep your terraform DRY with Terragrunt

Sources :

https://terragrunt.gruntwork.io/