---
title: How to Build a Safe Terraform Apply Workflow on AWS: Approval Gates, Plan Review, and Rollback
description: One bad `terraform apply` still deletes databases. July 2026: plan, review, apply, no -auto-approve, OIDC CI, Atlantis/HCP Terraform gates.
url: https://www.joinhorizon.co/blog/safe-terraform-apply-workflows-approval-gates-aws/
datePublished: 2026-04-04T00:00:00.000Z
dateModified: 2026-10-28T00:00:00.000Z
author: manojkumar-varatharajan
category: DevOps & CI/CD
tags: terraform, aws, devops, infrastructure-automation, cicd-safety
---

# How to Build a Safe Terraform Apply Workflow on AWS: Approval Gates, Plan Review, and Rollback

> One bad `terraform apply` still deletes databases. July 2026: plan, review, apply, no -auto-approve, OIDC CI, Atlantis/HCP Terraform gates.

Somewhere, right now, someone is tempted to run `terraform apply -auto-approve` in production. As of **July 2026**, the safe pattern is unchanged: **plan, human review, then apply the saved plan artifact**, with CI using **OIDC to AWS** (no long-lived keys) and a gate (Atlantis, **HCP Terraform**, Spacelift, env0, or equivalent).

**First-party benchmark (illustrative failure modes):** a rename without `moved` replaces RDS, a wrong account profile destroys resources in prod, a typo opens `0.0.0.0/0`. Prevention is minutes of review; recovery is hours.

**Reproduce this:** [Plan-review-apply checklist](https://www.joinhorizon.co/examples/architecture-blog-2026/terraform-apply-gates/plan-review-apply-checklist.md)

## The cost of a bad apply

**Illustrative failure mode A:** Refactor a resource name without `moved` / state mv, and Terraform plans a destroy and create on RDS. Restore from backup can take hours.

**Illustrative failure mode B:** Local apply with the wrong AWS profile/account, and resources change in prod when you intended staging.

**Illustrative failure mode C:** Variable typo opens a security group to the world, found at the next audit rather than at apply time.

Prevention (review + block `-auto-approve` in prod) is minutes. Failure is hours and real money.

## The three-gate model: plan, review, apply

![Architecture diagram of a Terraform plan waiting for review before apply](../../assets/images/blog/safe-terraform-apply-workflows-approval-gates-aws-gates.webp)

_Figure: the three gates in the article. The plan shows the diff, a person reviews it, and only then does apply run. Destroy, auto-approve, and very high parallelism stay blocked in production. This is Terraform, not a CloudFormation change set. [Open the draw.io source](https://www.joinhorizon.co/examples/architecture-blog-2026/safe-terraform-apply-workflows-approval-gates-aws/gates.drawio)._

A safe workflow has three gates:

### Gate 1: Plan

```bash
terraform plan -out=tfplan
```

Output the plan to a file. Never rely on console-only output (which scrolls away and is hard to review).

The plan shows:

```
  # aws_db_instance.main will be destroyed
  - resource "aws_db_instance" "main" {

  # aws_security_group.app will be updated in-place
  ~ resource "aws_security_group" "app" {
        ~ ingress {
              + cidr_blocks = ["0.0.0.0/0"]
              from_port   = 443
              to_port     = 443
            }
        }
```

A reviewer should read this and say "yes, this is what I expected" or "wait, why is the database being destroyed?"

Plan safety tips:

- Always output to a file (a saved plan applies exactly what was reviewed; console output can't be applied)
- Commit the plan to CI/CD so there's an audit trail
- If the plan is larger than 100 lines, display it in a tool that's designed for reading (not a text scroll)

### Gate 2: Review

A human reads the plan. Not the person who wrote the code, but someone else. Ideally someone senior.

A reviewer should ask:

- "Are any critical resources being destroyed?" (databases, load balancers, security groups)
- "Are any IAM permissions being changed?" (could break applications)
- "Are any resource replacements happening?" (which means downtime)
- "Does this match the ticket/PR description?"

The review happens before apply. The review blocks apply if something looks wrong.

### Gate 3: Apply

Apply runs only after review approval, and it should run:

- **In CI/CD**, not on a developer's laptop
- **With audit logging** (who applied it, when, what changed)
- **With the exact plan that was reviewed** (not a fresh plan that could be different)

Terraform supports this with `terraform apply tfplan`. Terraform refuses to apply a saved plan if the state has changed since the plan was created, so apply runs exactly what was reviewed or fails as stale.

## What to audit in a Terraform plan

Not everything in a plan is dangerous, but some things are red flags.

### Red flag 1: Resource destruction

```
  # aws_rds_db_instance.main will be DESTROYED
```

Databases should never be destroyed by accident. If you see a database destruction, pause and understand why:

- Is it a resource rename? (In which case, use `terraform state mv`)
- Is it a legitimate decommissioning? (In which case, require extra approvals)
- Is it a mistake in the code change? (Fix and re-plan)

### Red flag 2: Resource replacement

```
  # aws_db_instance.main will be destroyed and recreated
  - will be destroyed
  + will be created
```

This is dangerous because it means downtime (the resource is gone during the recreation). For databases, it means data loss (usually).

### Red flag 3: Large security group changes

```
  ~ resource "aws_security_group" "app" {
        ~ ingress {
              + cidr_blocks = ["0.0.0.0/0"]
            }
        }
```

Opening access to 0.0.0.0/0 (the entire internet) should be questioned. Is this intentional?

### Red flag 4: IAM policy changes

```
  ~ resource "aws_iam_role_policy" "app_role" {
        + "s3:*"
        - "s3:GetObject"
        - "s3:PutObject"
    }
```

Adding broad permissions (like `s3:*` instead of specific actions) is a security issue.

### Red flag 5: Encryption or backup settings disabled

```
  ~ resource "aws_rds_db_instance" "main" {
        ~ storage_encrypted = true -> false
        ~ backup_retention_period = 30 -> 0
    }
```

Disabling encryption or backups is almost never intentional. Question this.

### Green flag: additive changes only

```
  + resource "aws_s3_bucket" "backup" { ... }
  + resource "aws_iam_role" "service" { ... }
```

Creating new resources with no changes to existing ones is low risk. These plans can be approved quickly.

## Blocking dangerous commands in CI/CD

Some commands should never run in production. Set up guards:

### Block `-auto-approve` in Production

The `-auto-approve` flag skips the approval step entirely. It should only exist in dev.

**In your CI/CD pipeline:**

```bash
if [[ "$ENVIRONMENT" == "production" ]] && [[ "$TERRAFORM_ARGS" == *"-auto-approve"* ]]; then
  echo "❌ -auto-approve is forbidden in production"
  exit 1
fi
```

### Block `terraform destroy` in Production

```bash
if [[ "$ENVIRONMENT" == "production" ]] && [[ "$COMMAND" == "destroy" ]]; then
  echo "❌ terraform destroy is forbidden in production. Use drift detection instead."
  exit 1
fi
```

If you need to destroy resources in production, require a separate approval process or don't allow it through normal CI/CD.

### Block `-parallelism=1000` in Production

Terraform's `-parallelism` flag controls how many resources change simultaneously. High parallelism can cause issues:

```bash
if [[ "$ENVIRONMENT" == "production" ]]; then
  terraform apply -parallelism=5 tfplan
else
  terraform apply -parallelism=10 tfplan
fi
```

Limiting parallelism means changes happen more slowly, giving you time to notice problems.

## Per-environment policies: auto-approve for dev, manual gate for prod

![Architecture diagram of one Terraform plan auto-applying in dev and waiting for a person in production](../../assets/images/blog/safe-terraform-apply-workflows-approval-gates-aws-envs.webp)

_Figure: per-environment policy in the article. Development can apply the plan on its own. Production stops until a person approves. The two environments are not two steps of one apply. [Open the draw.io source](https://www.joinhorizon.co/examples/architecture-blog-2026/safe-terraform-apply-workflows-approval-gates-aws/envs.drawio)._

Different environments have different risk profiles.

| Environment | Approval Required | Auto-Approve OK | Parallelism | Policy                                                |
| ----------- | ----------------- | --------------- | ----------- | ----------------------------------------------------- |
| Dev         | No                | Yes             | 10+         | Speed matters; we accept risk                         |
| Staging     | Maybe             | No              | 5           | Simulate production, but still safe to experiment     |
| Production  | Always            | No              | 3-5         | Every change is reviewed; destructive ops are blocked |

**Example CI/CD configuration:**

```yaml
# .github/workflows/terraform.yml

on: [push, pull_request]

env:
  TF_VAR_environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: hashicorp/setup-terraform@v2

      - name: Terraform Plan
        run: |
          terraform init
          terraform plan -out=tfplan

      - name: Require Approval (Production Only)
        if: env.TF_VAR_environment == 'production'
        uses: actions/github-script@v6
        with:
          script: |
            github.rest.pulls.requestReviewers({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number,
              reviewers: ['senior-infra-engineer']
            })

      - name: Wait for Approval (Production Only)
        if: env.TF_VAR_environment == 'production'
        run: |
          # Block until PR is approved
          # (Implementation depends on your approval strategy)

      - name: Terraform Apply (Auto for Dev, Conditional for Prod)
        run: |
          if [[ "$ENVIRONMENT" == "production" ]]; then
            terraform apply tfplan  # Requires prior approval
          else
            terraform apply -auto-approve tfplan
          fi
        env:
          ENVIRONMENT: ${{ env.TF_VAR_environment }}
```

## AWS-specific risks and mitigations

Some Terraform operations are particularly risky on AWS.

### Risk 1: RDS resource replacement

RDS instances can't be replaced (updated in place) for certain changes:

```hcl
resource "aws_db_instance" "main" {
  allocated_storage = 100  # Changed from 50
  skip_final_snapshot = false  # Safe
  apply_immediately = true  # Dangerous! Causes immediate downtime
}
```

If `apply_immediately = true`, the change happens now, not during your maintenance window. Your database is unavailable.

**Mitigation:** Review RDS changes extra carefully. Use `apply_immediately = false` in production.

### Risk 2: ElastiCache node replacement

Changing node types in ElastiCache causes the cache to be recreated, flushing all cached data.

```hcl
resource "aws_elasticache_cluster" "main" {
  node_type = "cache.t3.micro"  # Changed from cache.t3.small
}
```

This is a cache replacement. Plan for cache misses and increased load on your database.

### Risk 3: Security group rule changes during active traffic

Removing a security group rule during active traffic can drop connections mid-stream.

```hcl
resource "aws_security_group_rule" "app_ingress" {
  type              = "ingress"
  from_port         = 443
  to_port           = 443
  protocol          = "tcp"
  cidr_blocks       = ["10.0.0.0/8"]  # Removing this rule breaks connections
}
```

**Mitigation:** Make security group changes during maintenance windows, or apply them gradually (update code, apply change, verify, then roll forward).

## Rollback options when apply goes wrong

If `terraform apply` causes problems, you have options.

### Option 1: Terraform state rollback

If the plan that was applied was bad, you can use `terraform state push` to revert to the previous state:

```bash
# Save current state
terraform state pull > current-state.json

# Restore previous state (from backup)
terraform state push previous-state.json

# Re-plan (should show how to recreate the destroyed resources)
terraform plan
```

This is a last resort and it isn't clean, but it works when you need to undo a disaster quickly.

### Option 2: Destroy and rebuild

For some resources, it's faster to destroy and recreate:

```bash
terraform destroy -target=aws_instance.web
terraform apply -target=aws_instance.web
```

This removes the corrupted resource and rebuilds it cleanly.

### Option 3: Manual AWS console changes

If Terraform is causing problems, make changes directly in the AWS console to stabilize, then fix Terraform code and re-apply:

1. Manually fix the problem in AWS console
2. Update Terraform code to match
3. Run `terraform import` if necessary to bring it under Terraform management
4. Run `terraform plan` to verify zero changes

## Tools for safe workflow automation

Several tools specialize in safe Terraform workflows.

### Atlantis

Atlantis is a self-hosted tool that runs `terraform plan` on pull requests and manages `terraform apply` approvals.

**Workflow:**

1. Developer opens PR with infrastructure changes
2. Atlantis runs `terraform plan` and posts the plan in the PR
3. Reviewers comment `atlantis apply` to approve
4. Atlantis runs `terraform apply` with full audit logging

Benefits:

- Plan output is visible in the PR
- No developer access needed to run apply
- Full audit trail of who approved what

### Spacelift

Spacelift (or **HCP Terraform**, formerly Terraform Cloud) adds approval workflows, policy enforcement, and drift detection.

**Features:**

- Require approval before apply
- Block dangerous operations (destroy, auto-approve)
- Policy as Code (enforce naming conventions, required tags, etc.)
- Drift detection and remediation

### GitHub Actions with required approvals

If you're using GitHub, you can use GitHub's built-in approval mechanisms:

```yaml
- name: Create Approval Issue
  if: github.event_name == 'pull_request'
  uses: actions/github-script@v6
  with:
    script: |
      github.rest.issues.create({
        owner: context.repo.owner,
        repo: context.repo.repo,
        title: 'Approval Required: Infrastructure Changes',
        body: 'This PR modifies production infrastructure. Requires approval from @senior-infra-engineer'
      })
```

## Testing your safe workflow

Before deploying to production, test your approval workflow in staging:

1. Create a change in staging that would be dangerous (like increasing instance size)
2. Verify the plan is created correctly
3. Verify the approval requirement blocks apply
4. Verify approval enables apply
5. Verify the change applies correctly

If this process works in staging, you can trust it in production.

## What this post doesn't cover

OpenTofu-specific CI plugins and full Sentinel/OPA policy libraries. The same gates apply; tool names differ.

## What to do Monday morning

1. Ban `-auto-approve` on prod; require `terraform apply tfplan` after review.
2. Move CI credentials to **OIDC** with short-lived AWS roles.
3. Put Atlantis or HCP Terraform/Spacelift in front of prod applies.
4. Walk the [apply-gates checklist](https://www.joinhorizon.co/examples/architecture-blog-2026/terraform-apply-gates/plan-review-apply-checklist.md).
5. Next: [state import/move](https://www.joinhorizon.co/blog/terraform-state-management-aws-import-move-repair/), [drift detection](https://www.joinhorizon.co/blog/aws-infrastructure-drift-detection-terraform/), [Control Tower](https://www.joinhorizon.co/blog/how-to-set-up-aws-control-tower-multi-account-governance/).
6. [Contact](https://www.joinhorizon.co/contact-us/) for a Terraform governance pass.

---

*Source: https://www.joinhorizon.co/blog/safe-terraform-apply-workflows-approval-gates-aws/*