Infrastructure as Code emerged as a response to the problem of undocumented, inconsistent, manually-managed infrastructure. Click-ops servers where nobody knows what’s actually configured. Snowflake environments that differ from production in undocumented ways. Infrastructure that can’t be reproduced.
IaC solved these problems by treating infrastructure as code. The irony is that many teams write infrastructure code with none of the engineering rigor they apply to application code. No reviews, no tests, no code quality standards, no documentation. The result: IaC that has the same problems as the click-ops it replaced, plus the added complexity of Terraform state files.
The Software Engineering Practices That Apply
Version Control and Code Review
IaC must be in version control. This is baseline. Every infrastructure change is a commit, with a history that shows what changed, why, and who made the change.
Beyond version control: all infrastructure changes should go through code review, with the same scrutiny as application code.
What to look for in IaC review:
Security posture: security groups with 0.0.0.0/0, IAM policies with *, S3 buckets without encryption or public access blocks.
# Security issue: overly permissive security group
resource "aws_security_group_rule" "allow_all" {
type = "ingress"
from_port = 0
to_port = 65535
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"] # FLAG: should this be restricted?
}
Tagging: without tags, cost allocation and resource ownership become opaque at scale.
# Every resource should have consistent tags
resource "aws_instance" "app" {
ami = var.ami_id
instance_type = var.instance_type
tags = merge(var.common_tags, {
Name = "${var.environment}-app-server"
Component = "application"
Owner = "platform-team"
Environment = var.environment
})
}
Resource naming: inconsistent naming makes environments hard to navigate. Define naming conventions and enforce them.
Dependency declarations: explicit depends_on where implicit dependencies aren’t enough, data source references that might be incorrect.
Testing Infrastructure Code
Application code has unit tests, integration tests, end-to-end tests. Infrastructure code can and should have equivalent validation.
Format and syntax validation (the equivalent of compilation):
terraform fmt -check -recursive
terraform validate
These catch formatting inconsistencies and basic configuration errors before a plan is even attempted.
Static analysis with tools like tfsec or Checkov:
# tfsec — security scanning for Terraform
tfsec . --format json > security-report.json
# Checkov — policy-as-code scanner
checkov -d . --output cli
These catch security misconfigurations: unencrypted storage, overly permissive access controls, missing logging configuration.
Infrastructure testing with Terratest (Go testing framework):
func TestVPCModule(t *testing.T) {
terraformOptions := &terraform.Options{
TerraformDir: "../modules/vpc",
Vars: map[string]interface{}{
"cidr_block": "10.0.0.0/16",
"environment": "test",
},
}
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
// Verify the VPC was created with expected properties
vpcId := terraform.Output(t, terraformOptions, "vpc_id")
aws_helper.AssertVpcExists(t, vpcId, "eu-west-1")
// Verify no public access on private subnets
subnetIds := terraform.OutputList(t, terraformOptions, "private_subnet_ids")
for _, subnetId := range subnetIds {
aws_helper.AssertSubnetHasNoAutoAssignPublicIp(t, subnetId, "eu-west-1")
}
}
Terratest actually applies infrastructure, validates it, and destroys it. Slow but thorough — suitable for CI on module changes.
Module Design and Reuse
Infrastructure modules are the IaC equivalent of library functions. They should be:
Composable: modules accept inputs and produce outputs. Don’t hardcode values that differ between environments or use cases.
Documented: what does this module create? What are the required inputs? What does it output?
# modules/rds-postgres/variables.tf
variable "identifier" {
type = string
description = "RDS instance identifier. Must be unique within the region."
}
variable "instance_class" {
type = string
description = "RDS instance class (e.g., db.t3.micro, db.r6g.large)"
}
variable "deletion_protection" {
type = bool
description = "Prevent accidental deletion. Set to false only in non-production environments."
default = true
}
Versioned: modules shared between teams should be versioned. Consumers pin to a version and upgrade deliberately.
Minimal: modules should do one thing well. A “production environment” module that creates VPC, ECS cluster, RDS, ElastiCache, and ALB in one call is hard to understand, hard to test, and impossible to use for partial deployments.
Environment Parity
One of the primary values of IaC is the ability to reproduce environments consistently. This only works if you use the same code for all environments.
# environments/staging/main.tf
module "api" {
source = "../../modules/ecs-service"
environment = "staging"
desired_count = 1 # Smaller in staging
cpu = 256
memory = 512
}
# environments/production/main.tf
module "api" {
source = "../../modules/ecs-service"
environment = "production"
desired_count = 3 # Larger in production
cpu = 1024
memory = 2048
}
Same module, different configuration. What you should not have: a staging module that’s a simplified, manually-adjusted version of the production module. When staging drifts from production, staging stops being useful for testing production behavior.
The State Problem at Scale
State files grow with infrastructure. Managing state at scale requires explicit structure:
Separate state per environment: staging and production should never share state. A destructive operation in staging state shouldn’t touch production resources.
Separate state per component: a terraform apply in the networking configuration shouldn’t be able to accidentally modify the application tier state. Blast radius isolation.
State backup and recovery: if state is corrupted or lost, can you recover? Remote backends (S3, GCS) with versioning enabled provide point-in-time recovery.
State migration documentation: when reorganizing state (splitting, merging, moving resources), document the procedure. State operations are irreversible when done wrong.
The Automation Pipeline
Infrastructure changes should go through the same CI/CD discipline as application changes:
# .gitlab-ci.yml
stages: [validate, plan, apply]
validate:
script:
- terraform fmt -check
- terraform validate
- tfsec .
- checkov -d .
plan:
script:
- terraform init
- terraform plan -out=tfplan -detailed-exitcode
artifacts:
paths: [tfplan]
when: always
apply-staging:
script:
- terraform apply tfplan
environment: staging
when: manual
dependencies: [plan]
apply-production:
script:
- terraform apply tfplan
environment: production
when: manual
needs: [apply-staging]
The plan stage is always run and its output is saved. apply stages require manual approval and use the saved plan — so you know exactly what will be applied.
Documentation as Code
The gap between what IaC does and what engineers think it does causes incidents.
Module README files are minimal viable documentation:
- What does this module create?
- What are the security implications?
- What monitoring is included?
- What does the operator need to know for incidents?
# modules/rds-postgres
Creates a PostgreSQL RDS instance with:
- Multi-AZ for production (single-AZ for non-production environments)
- Automated backups with 7-day retention
- Encryption at rest with AWS KMS
- Performance Insights enabled
## Operational Notes
- Connection string available via Secrets Manager at `/{environment}/postgres/{identifier}/connection`
- Monitoring dashboard: [CloudWatch Dashboard](link)
- Backup restores: documented in runbook/rds-restore.md
## Known Limitations
- storage_type=gp3 is not supported in eu-south-2 (use gp2)
Infrastructure knowledge that lives only in people’s heads is a bus-factor problem.
The Discipline
The discipline of IaC as software comes down to one mental shift: infrastructure code has the same properties as application code — it can be wrong, it can drift, it can accumulate debt, and it requires engineering rigor to maintain.
The teams that treat IaC as a configuration file to be updated manually in emergencies will eventually have an infrastructure incident that their codebase can’t explain or recover from quickly. The teams that treat it as software will have it documented, reviewed, tested, and deployable with confidence.