Managing Multiple Environments in Terraform

When managing infrastructure, it's common to configure multiple environments for deploying code. For example, dev, staging and production. In these cases, it's important for each environment to be as close as possible to the others. This ensures that minute differences in architecture—as minor as a PostgreSQL patch version or as major as using RabbitMQ in staging and IBM MQ in production -don't introduce problems when applications are promoted to production. Using Terraform to manage environments can help. Terraform is declarative, and the same declaration can be used to manage multiple environments. To do so, we'll explore two common solutions: Terraform Workspaces, and Terraform Modules.

Terraform Workspaces

This is a great way to isolate different versions of your state. The declarative infrastructure is still read by the same .tf files within your directory, while the state used to reconcile against your currently deployed infrastructure is isolated from other workspaces (Similar to the way version managers allow multiple language versions without altering the project's code).

When using workspaces, your directory structure might look something like this:

├── README.md
├── main.tf
├── variables.tf
├── outputs.tf
├── eks.tf
├── vpc.tf
├── iam.tf

With this structure, you can create a workspace for each environment:

$ terraform workspace new stage
$ terraform workspace new prod
$ terraform workspace select stage

These commands will create a separate subdirectory for each workspace within your terraform back end, and use this directory to store the current state. This means when you run terraform plan in an environment that is currently up to date, you will have no changes to apply because the state within this workspace's directory is in sync with your infrastructure. By contrast, the same command when using a different workspace could show pending changes.

Workspace Pros

• Your infrastructure definitions are identical for each environment.
• Directory structure is easy to navigate and manage.
• It's trivial to spin up additional environments with terraform workspace new [environment]

Workspace Cons

• While each environment is identical, that's not always what you want.
• You may have conditional operators in your code, which can get messy. IE: instance_count = terraform.workspace == "prod" ? 10 : 2
• It's actually not recommended by HashiCorp.

Terraform Modules

As an alternative, Terraform also offers modules. This approach encourages you to abstract the complex configuration of your environment into a module, and then reference that module in separate terraform "environment" folders within your project. In this case, your directory structure might look something like this:

├── README.md
├── modules/
│   ├── eks/
│   │   ├── eks.tf
│   │   ├── iam.tf
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   ├── variables.tf
│   │   ├── vpc.tf
├── stage/
│   ├── eks.tf
│   ├── main.tf
│   ├── variables.tf
├── prod/
│   ├── eks.tf
│   ├── main.tf
│   ├── variables.tf

In the above example, the main.tf file within both stage and prod would have its own terraform configuration, including its own backend. This allows you to store your tfstate in a completely isolated location for each environment. Additionally, the eks.tf within both stage and prod would simply call the eks module (where most of the infrastructure is defined) and pass in its own unique set of variables specific to that environment.

Modules Pros

• Complex infrastructure constructs are abstracted.
• You may want some resources in prod but not stage.
• Increased upfront investment.

Modules Cons

• More complex code and directory structure.
• There aren't good conventions for how to structure your module.
• It can sometimes be confusing where to make changes to your infrastructure.

So why might you use one method over the other? Use whichever method makes the most sense for your use case. A project with 1-3 engineers and relatively few cloud needs may benefit from Terraform Workspaces, while a much larger engineering team could benefit from the rigidity of modules, and know that the upfront investment will pay dividends as their team and needs grow. In short: be pragmatic! I've found that pragmatism and considering my current needs is always the best approach.