Multiple Terraform CLI Workspaces Using Only One TFVars File

Terraform

Terraform Cloud offers an excellent feature that allows you to set variables and override them with environment-specific variables. Unfortunately, replicating this behavior with the Terraform CLI requires some creativity.

Authored by Corey Regan

Published on June 10, 2024

6 min read

Mimic The Gold Standard: Terraform Cloud Workspaces

Terraform Cloud Workspaces are wonderful. You can create variables in your Terraform code, and set default values for each environment.

var.server_name in the dev Terraform Cloud Workspace can be set to dev.my-server.my-company.com, and to prod.my-server.my-company.com in the prod Terraform Cloud Workspace.

It makes sense, right? Each environment overrides var.server_name as you would expect.

Terraform CLI Workspaces

Confusingly, Terraform CLI Workspaces behave nothing like Terraform Cloud Workspaces. The only thing they do is change a terraform.workspace variable to reflect the current workspace, and output to a different state file.

That's it.

Ugly Setups Of "One Workspace Per Environment"

Now Terraform will tell you to create separate *.tfvars files named by environment, so you can add the -var-file= flag to every command issued, yuck!

bash:Multiple *.tfvars files suck

.
├──
│   main.tf
│   outputs.tf
│   providers.tf
│   terraform.dev.tfvars
│   terraform.prod.tfvars
│   terraform.stg.tfvars
│   variables.tf
 
$ terraform apply -var-file=terraform.prod.tfvars

Or even worse, you could copy your *.tf files into a separate folder for each environment where you have a duplicate or triplicate of everything, meaning one change has to be replicated to the other environments, good luck keeping that perfectly synchronized. Each providers.tf would have a different remote Terraform state file configured.

bash:Multiple environment folders also sucks

.
├── dev
│   ├── main.tf
│   ├── outputs.tf
│   ├── providers.tf
│   ├── terraform.tfvars
│   ├── variables.tf
├── prod
│   ├── main.tf
│   ├── outputs.tf
│   ├── providers.tf
│   ├── terraform.tfvars
│   ├── variables.tf
├── stg
│   ├── main.tf
│   ├── outputs.tf
│   ├── providers.tf
│   ├── terraform.tfvars
│   ├── variables.tf
 
$ cd prod
$ terraform apply prod

There has to be a simple solution, right?

Terraform CLI, A Workspace Per Environment, One TFVars File

Ideally, you should have one copy of your code with each environment's variables overriding defaults. With this scheme, the remote Terraform state file will have the environment appended to it.

bash:This is desired outcome

.
├──
│   main.tf
│   outputs.tf
│   providers.tf
│   terraform.tfvars
│   variables.tf
 
$ terraform workspace select prod
$ terraform apply

At first-glance you might be thinking

well hang on, issuing just one terraform apply -var-file=terraform.prod.tfvars command is quicker than issuing two commands, terraform workspace select prod and terraform apply

And, well, you're kind of right, if, and only if you plan to apply once. Let's contrast the two:

bash:Using -var-file=

$ terraform plan  -var-file=terraform.prod.tfvars
$ terraform apply -var-file=terraform.prod.tfvars
$ terraform plan  -var-file=terraform.prod.tfvars
$ terraform apply -var-file=terraform.prod.tfvars
$ terraform apply -var-file=terraform.prod.tfvars

bash:Using workspace select

$ terraform workspace select dev
$ terraform plan
$ terraform apply
$ terraform plan
$ terraform apply
$ terraform apply

As you can see, in a practical example where you make changes, test their plan, apply approved changes, realize you need to tweak something, plan again, apply again, it fails to apply, and a second apply passes. So yeah, in the real world it's faster to type a simple terraform plan or terraform apply command than it is to use -var-file=.

Multiple Environments In Only terraform.tfvars

Normally, you would define a simple variable like so:

terraform:variables.prod.tf

variable "database_name" {
  type        = string
  description = "The name of the database"
  default     = "my_production_db_name"
}

To support multiple environments, we consolidate all environment variables into one terraform.tfvars:

terraform:variables.tf

variable "database_name" {
  type        = map(string)
  description = "The name of the database"
  default = {
    prod = "my_production_db_name"
    stg = "my_staging_db_name"
    dev = "my_development_db_name"
  }
}

Accessing the correct variable is easy, simply pass in the current Terraform Workspace name as the map's key:

terraform:main.tf

resource "azurerm_postgresql_flexible_server" "test_db" {
  name = var.database_name[terraform.workspace]
}

You can of course target a specific value if needed, like calling var.database_name["prod"] or var.database_name.prod

Limiting resource creation to specific environments

Let's say you only want to create a resource in production, like a Container Registry, where other environments will share that same resource. You can take advantage of count and testing the value of terraform.workspace:

terraform:main.tf

resource "azurerm_postgresql_flexible_server" "test_database" {
  count = terraform.workspace == "prod" ? 1 : 0
  name  = var.database_name[terraform.workspace]
}

Now, you'll need to access the correct index of azurerm_postgresql_flexible_server.test_db, which will always be [0] with this method

terraform:outputs.tf

output "database_id" {
  value = azurerm_postgresql_flexible_server.test_database[0].id
}

Another advantage is if you have a variable that requires iterating through with a for_each loop, you can simply leave some environments blank.

terraform:Using for_each loops to iterate environments

variable "database_names" {
  type = map(list(object({
    name     = string
    location = string
    sku      = string
  })))
  description = "The name of the database"
  default = {
    prod = [
      {
        name     = "prod-primary_db-psql"
        location = "eastus"
        sku      = "Standard"
      },
      {
        name     = "prod-secondary_db-psql"
        location = "westus"
        sku      = "Standard"
      }
    ]
    stg = [] # No DB needed in Staging
    dev = [
      {
        name     = "dev-primary_db-psql"
        location = "westeurope"
        sku      = "Standard"
      },
    ]
  }
}
 
resource "azurerm_postgresql_flexible_server" "test_database" {
  for_each = toset(var.database_name[terraform.workspace])
  name     = each.value.name
 
  location            = each.value.location
  resource_group_name = "${terraform.workspace}-rg" # prod-rg, stg-rg, dev-rg
}

Instead of using a list of object, we can use a map to give each database an easy-to-reuse key. This makes it easier to precisely extract a specific piece of data from a potentially complicated dataset. Here is the same example, slightly tweaked:

terraform:Using for_each loops to iterate environments

variable "database_names" {
  type = map(map(map({
    name     = string
    location = string
    sku      = string
  })))
  description = "The name of the database"
  default = {
    prod = { # Use curly braces
      primary = {
        name     = "prod-primary_db-psql"
        location = "eastus"
        sku      = "Standard"
      } # Commas not needed
      secondary_db = {
        name     = "prod-secondary_db-psql"
        location = "westus"
        sku      = "Standard"
      }
    }
    stg = {} # No DB needed in Staging
    dev = {
      primary = {
        name     = "dev-primary_db-psql"
        location = "westeurope"
        sku      = "Standard"
      }
    }
  }
}
 
resource "azurerm_postgresql_flexible_server" "test_database" {
  for_each = { for k, v in var.database_name[terraform.workspace] : k => v }
  name     = each.value.name
 
  location            = each.value.location
  resource_group_name = "${terraform.workspace}-rg" # prod-rg, stg-rg, dev-rg
}
 
output "primary_db_id" {
  value = azurerm_postgresql_flexible_server.test_database["primary"].id
}
output "prod_db_name" {
  value = var.database_names["prod"]["primary"].name
}