Run Terraform in Automation with Atlantis

Updated on February 3, 2021
Run Terraform in Automation with Atlantis header image

Introduction

Automation of Terraform is a way to ensure consistency and implement features such as integration with version control hooks, and it's useful for development and production servers. Following this guide, you'll set up an example CI/CD pipeline with Atlantis, GitHub, and Terraform.

Prerequisites

This guide uses a Vultr Ubuntu 20.04 VPS with 1 vCPU and 1 GB of RAM. This is an advance guide, requiring a basic understanding of Terraform and CI/CD concepts.

This guide uses variables with uppercase values in angle brackets. Replace these with your values.

1. Install Terraform CLI

Atlantis uses Terraform to plan, create, and destroy cloud resources.

  1. Download the 64-bit Linux version of Terraform, found on the Terraform download page. The example below is for version 0.14.5.

     # wget https://releases.hashicorp.com/terraform/0.14.5/terraform_0.14.5_linux_amd64.zip
  2. Extract the Terraform .zip file.

     # apt install -y unzip
     # unzip terraform_*_linux_amd64.zip && rm terraform_*_linux_amd64.zip
  3. Move terraform to /usr/bin.

     # mv terraform /usr/bin/terraform
  4. Check the terraform installation.

     # terraform --version
     # which terraform

2. Install Atlantis

  1. Download the 64-bit Linux version of Atlantis, found on the Atlantis release page. The example below is for version 0.16.0.

     # wget https://github.com/runatlantis/atlantis/releases/download/v0.16.0/atlantis_linux_amd64.zip
  2. Extract) the Atlantis .zip file.

     # unzip atlantis_linux_amd64.zip && rm atlantis_linux_amd64.zip
  3. Create an Atlantis user with a home directory.

     # adduser -m atlantis
  4. Move atlantis to its home directory /home/atlantis

     # mv atlantis /home/atlantis/atlantis
  5. Change the ownership of the file to Atlantis user.

     # chown atlantis:atlantis /home/atlantis/atlantis
  6. Check the Atlantis installation.

     # /home/atlantis/atlantis --help

3. Setup a GitHub Repository

Create a private GitHub repository. The recommended organization for the Github repository is shown below. Replace vultr-vc and instance1 (flagged with double asterisks below) with your corresponding module and service name.

    ├── modules
    │   └── vultr-vc **
    │       ├── provider.tf
    │       ├── variable.tf
    │       ├── output.tf
    │       └── main.tf
    │
    ├── services
    │   └── instance1 **
    │       ├── terraform.tfvars.example
    │       ├── variable.tf
    │       ├── output.tf
    │       └── main.tf
    ├── .gitignore
    ├── README.md
    └── atlantis.yaml

On GitHub:

  1. Go to Settings then Webhook.
  2. Click Add webhook.
  3. Set Payload URL to http://<INSTANCE_IP_OR_URL>/events
  4. Set Content type to application/json
  5. Enter a random string in Secret. More than eight characters is recommended.
  6. Pick the events to send.
    • Pushes
    • Pull request review comments
    • Pull requests
    • Issue comments
  7. Leave Active ticked.
  8. Click Add webhook.

Create a Github token:

  1. Go to Profile > Developer Settings > Personal access token
  2. Click Generate new token
  3. Select scopes.
    • repo
  4. Give the token a name, For example: Atlantis Token
  5. Click Generate Token and copy the token.

Create an atlantis.yaml file in the GitHub repository, which tells Atlantis which directories and workflows to apply. It also creates a tfvars file on /home/atlantis/tfvars/service.tfvars to execute terraform apply -var-file <FILENAME>.

version: 3
projects:
- name: instance1
    dir: services/instance1
    workflow: service
    autoplan:
    when_modified: ["modules/**/*.tf","*.tf"]
    enabled: true

workflows:
service:
    plan:
    steps:
    - init
    - plan:
        extra_args: ["-var-file", "/home/atlantis/tfvars/service.tfvars"]

4. Create a Terraform Module

A Terraform module is a universal template that speeds up the creation of similar services.

  1. Create a Vultr provider.tf file.

     terraform {
         required_providers {
             vultr = {
             source = "vultr/vultr"
             version = "2.1.2"
             }
         }
     }
    
     provider "vultr" {
         api_key = var.vultr_token
         rate_limit = 700
         retry_limit = 3
     }
  2. Create a main.tf file.

    This example creates a Vultr Cloud Compute instance.

     resource "vultr_instance" "instance" {
         plan                = var.server_type
         region              = var.region
         os_id               = var.server_os
         label               = var.server_label
         tag                 = "terraform"
     }
  3. Create a variable.tf file, which defines the variables passed to the Terraform.

     variable "vultr_token" {
         description   = "Vultr api token"
         type          = string
     }
    
     variable "server_os" {
         description   = "Vultr OS"
         type          = string
     }
    
     variable "server_type" {
         description   = "Vultr Server Type Ex. vc2-1c-1gb"
         type          = string
     }
    
     variable "server_label" {
         description   = "Vultr Server Name Labeling"
         type          = string
     }
    
     variable "region" {
         description   = "Vultr Region"
         type          = string
     }

5. Create a Terraform Service

  1. Create a main.tf file.

    Terraform stores state about your infrastructure and configuration in a .tfstate file, which is declared in the terraform backend block. The module block sets the variables for the terraform module.

     terraform {
         backend "local" {
             path = "/home/atlantis/tfstate/service-instance1.tfstate"
         }
     }
    
     module "server1" {
         source          = "../../modules/vultr-vc"
         vultr_token     = var.vultr_token
         server_type     = "vc2-1c-1gb"
         server_os       = "387"
         server_label    = "Server1-Terraform"
         region          = "SGP"
     }

    Use the Vultr API to find valid server_os values.

     $ curl https://api.vultr.com/v2/os

    For valid region values:

     $ curl https://api.vultr.com/v2/regions
  2. Create a variable.tf file to pass the API token as a secret variable to Terraform.

     variable "vultr_token" {
         description   = "Vultr api token"
         type          = string
     }

6. Get an API Key

  1. Navigate to the Vultr Customer Portal.
  2. Go to Account > API.
  3. Click Enable API.
  4. Copy the Vultr API key.
  5. Go to Access Control.
  6. Add the Server IP to access control.

7. Set up the Atlantis Service

  1. Set up the tfvars file.

     # mkdir -p /home/atlantis/tfvars
     # nano /home/atlantis/tfvars/service.tfvars
  2. Add your Vultr API key as shown.

     vultr_token = "<VULTR_API_KEY>"
  3. Grant ownership of the tfvars folder to the atlantis user.

     # chown -R atlantis:atlantis /home/atlantis/tfvars
  4. Set up the tfstate local backend directory.

     # mkdir -p /home/atlantis/tfstate
     # chown -R atlantis:atlantis /home/atlantis/tfstate
  5. Set up the repos.yaml file in the atlantis user home directory.

     # nano /home/atlantis/repos.yaml

    The repos.yaml file sets the atlantis.yaml file's abilities in the GitHub repo. By setting it to mergeable, the action can only occur if the Pull Request is mergeable.

     repos:
     - id: /.*/
     apply_requirements: [mergeable]
     allowed_overrides: [workflow]
     allow_custom_workflows: true
  6. Grant ownership of repos.yaml to the atlantis user.

     # chown atlantis:atlantis /home/atlantis/repos.yaml
  7. Create a new systemd service.

     # cd /etc/systemd/system
     # nano atlantis.service or vi atlantis.service

    Paste the following:

     [Unit]
     Description=Atlantis
    
     [Service]
     User=atlantis
     WorkingDirectory=/home/atlantis
     ExecStart=/home/atlantis/atlantis server --allow-fork-prs --gh-user=<Github Username> --gh-token=<Github Token> --repo-whitelist=github.com/<GITHUB_USERNAME>/<REPOSITORY_NAME> --atlantis-url=http://<INSTANCE_IP_OR_URL> --gh-webhook-secret=GITHUB_SECRET> --repo-config=/home/atlantis/repos.yaml --log-level debug --port 80
     Restart=always
    
     [Install]
     WantedBy=multi-user.target
    
    
     # systemctl daemon-reload
     # systemctl start atlantis.service
  8. Check the installation.

     # systemctl status atlantis.service
     # journalctl -xe

8. Create a New Server / Service

On your local computer or GitHub:

  1. Checkout to a new branch.

     $ git checkout -b SERVICE-2
  2. Create a new service folder. It's recommended to use the service name.

  3. Add an atlantis.yaml file in that folder. Paste the following:

     projects:
     - name: instance1
         dir: services/instance1
         workflow: service
         autoplan:
         when_modified: ["modules/**/*.tf","*.tf"]
         enabled: true
     - name: instance2
         dir: services/instance2
         workflow: service
         autoplan:
         when_modified: ["modules/**/*.tf","*.tf"]
         enabled: true
  4. Create a pull request on GitHub to the main branch.

  5. Check the automated Terraform Plan. For example, you may see:

     An execution plan has been generated and is shown below.
     Resource actions are indicated with the following symbols:
     + create
    
     Terraform will perform the following actions:
    
     # module.server1.vultr_instance.instance will be created
     + resource "vultr_instance" "instance" {
         + activation_email       = true
         + allowed_bandwidth      = (known after apply)
         + app_id                 = (known after apply)
         + backups                = "disabled"
         + date_created           = (known after apply)
         + ddos_protection        = false
         + default_password       = (sensitive value)
         + disk                   = (known after apply)
         + enable_ipv6            = false
         + enable_private_network = false
         + features               = (known after apply)
         + firewall_group_id      = (known after apply)
         + gateway_v4             = (known after apply)
         + hostname               = (known after apply)
         + id                     = (known after apply)
         + internal_ip            = (known after apply)
         + kvm                    = (known after apply)
         + label                  = "Server1-Terraform"
         + main_ip                = (known after apply)
         + netmask_v4             = (known after apply)
         + os                     = (known after apply)
         + os_id                  = 387
         + plan                   = "vc2-1c-1gb"
         + power_status           = (known after apply)
         + ram                    = (known after apply)
         + region                 = "SGP"
         + reserved_ip_id         = (known after apply)
         + script_id              = (known after apply)
         + server_status          = (known after apply)
         + snapshot_id            = (known after apply)
         + status                 = (known after apply)
         + tag                    = "terraform"
         + user_data              = (known after apply)
         + v6_main_ip             = (known after apply)
         + v6_network             = (known after apply)
         + v6_network_size        = (known after apply)
         + vcpu_count             = (known after apply)
         }
    
     Plan: 1 to add, 0 to change, 0 to destroy.

    To re-plan, you could use atlantis plan.

  6. Execute the terraform plan using atlantis apply

    The output of apply should look like this:

     module.server1.vultr_instance.instance: Creating...
     module.server1.vultr_instance.instance: Still creating... [10s elapsed]
     module.server1.vultr_instance.instance: Still creating... [20s elapsed]
     module.server1.vultr_instance.instance: Still creating... [30s elapsed]
     module.server1.vultr_instance.instance: Still creating... [40s elapsed]
     module.server1.vultr_instance.instance: Still creating... [50s elapsed]
     module.server1.vultr_instance.instance: Still creating... [1m0s elapsed]
     module.server1.vultr_instance.instance: Still creating... [1m10s elapsed]
     module.server1.vultr_instance.instance: Still creating... [1m20s elapsed]
     module.server1.vultr_instance.instance: Still creating... [1m30s elapsed]
     module.server1.vultr_instance.instance: Still creating... [1m40s elapsed]
     module.server1.vultr_instance.instance: Still creating... [1m50s elapsed]
     module.server1.vultr_instance.instance: Creation complete after 1m54s [id=1ef9bc13-cb27-48b2-b776-30559caffc3f]
    
     Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
    
     The state of your infrastructure has been saved to the path
     below. This state is required to modify and destroy your
     infrastructure, so keep it safe. To inspect the complete state
     use the `terraform show` command.
    
     State path: /home/atlantis/tfstate/service-instance1.tfstate

More Resources