Navigator’s Guide: Initial Environment Setup

Posted on by dawid.slusarczyk

Note: This is an early release version of the contents of the Navigator’s Guide book, an offering from the DigitalOcean Solutions Engineers. The goal of the book is to help business customers plan their infrastructure needs, provide working examples along the way, and include technical nuance and the “why” that makes some decisions better than others.

The book and accompanying code will be publicly available in a GitHub repository. Because this is an early release, the book is not yet complete and the repository is not yet public, but stay tuned!

This is the first hands-on portion of the book. First, we’ll go over the tools we’ll be using, how they fit together, and how they can be beneficial to you as you begin to create and manage your infrastructure on DigitalOcean.

Then, we’ll set up a single Droplet which we’ll use as as a controller to run and use the rest of our tool belt.

Our Tool Belt

We’ll primarily be using Terraform, Ansible, terraform-inventory, and Git.

Terraform

Terraform is a open-source tool that allows you to easily describe your infrastructure as code. This means you can version control your resources in the same way you would if you were writing a program, which allows you to roll back to a working state if you hit an error.

Terraform uses a declarative syntax (HCL) that is designed to be easy for humans and computers alike to understand. HCL lets you plan your changes for review and automatically handles infrastructure dependencies for you.

We’ll be using Terraform to create our infrastructure — that is, creating Droplets, Floating IPs, Firewalls, Block Storage Volumes, and DigitalOcean Load Balancers — but we won’t be using it to configure those resources. That’s where Ansible comes in.

Ansible

Ansible is a configuration management tool which allows you to systematically handle changes to a system in a way that maintains its integrity over time. Ansible’s standard library of modules is extensive, and its architecture allows you to create your own plugins as well.

Playbooks are YAML files which define the automation you want to manage with Ansible. Like Terraform, you can version control your playbooks. Unlike Terraform, a change in the configuration of a resource does not require the destruction and recreation of that resource.

Ansible was created to push configuration changes outward which differs from other configuraiton management tools like Puppet and Chef. It also doesn’t require that an agent be installed on the target nodes beforehand since Ansible leverages simple ssh connections to configure your infrastructure. Ansible does however require knowledge of what endpoints it needs to reach out to. That’s normally taken care of with a simple inventory file. Because we’re using Terraform to deploy, and it maintains your infrastructure state in a file, we’ll be using terraform-inventory to dynamically feed Ansible its list of target machines.

terraform-inventory

terraform-inventory is a dynamic inventory script that pulls resource information from Terraform’s state file and outputs it in a way that Ansible can use to target specific hosts when executing playbooks. It gets a little more complicated than that, but the key point is that terraform-inventory makes it easier for you to use Terraform and Ansible together.

Git

We’ll use Git as our version control system. You don’t need in-depth knowledge of Git in particular, but understanding committing changes, tracking, and cloning. Because we can version control our Terraform and Ansible files, we can run tests on different versions of our infrastructure by specifying a version of a Terraform module or Ansible role.

The repository for this book is hosted on GitHub. When writing your own modules and roles, you can use other Git services like GitLab or Bitbucket, but the way you specify your module and role sources can vary depending on the service.

Optional Tools

The DigitalOcean CLI utility, doctl, is often helpful in quickly accessing your account through the API to create or retrieve resource information. You can find instructions to set up doctl in the project README and full usage information in its official documentation. It makes grabbing image and SSH key IDs much easier and faster than typing out and running a curl command.

Setting Up the Controller Droplet

We’re going to use a Ubuntu 16.04 x64 (Xenial Xerus) Droplet as our controller machine. This is the server from which we’ll run our tools.

To start, you’ll need:

Now it’s time to create the Droplet. You can use How To Create Your First DigitalOcean Droplet for a detailed walkthrough. We’ll be using the following options:

  • Image: Ubuntu 16.04 x64.
  • Size: 1GB Standard Droplet.
  • Datacenter region: Your choice.
  • Additional options: private networking, backups, user data, and monitoring.
  • SSH keys: Select yours.

When you select the user data option, a text field will open up. User data is arbitrary data that a user can supply to a Droplet at creation time. User data is consumed by CloudInit, typically during the first boot of a cloud server, to perform tasks or run scripts as the root user.

Copy and paste the following script into the user data text field. It installs Python 2.7, pip (a Python package manager), Git, zip, Terraform, terraform-inventory, and Ansible. The only modification you need to make to this file is setting your desired username and your public SSH key.

#cloud-config
# Source:  https://github.com/{{ config.nav-repo }}/example-code/01-intro/ch03/cloud-config.yaml

users:
  - name: your_desired_username_here # <-- Specify your username here.
    groups: sudo
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - your_public_key_here # <-- Specify your public SSH key here.

package_upgrade: true

packages:
  - python
  - python-pip
  - git
  - zip

runcmd:
  - [curl, -o, /tmp/terraform.zip, "https://releases.hashicorp.com/terraform/0.11.3/terraform_0.11.3_linux_amd64.zip"]
  - [unzip, -d, /usr/local/bin/, /tmp/terraform.zip]
  - [curl, -L, -o, /tmp/terraform-inventory.zip, "https://github.com/adammck/terraform-inventory/releases/download/v0.7-pre/terraform-inventory_v0.7-pre_linux_amd64.zip"]
  - [unzip, -d, /usr/local/bin/, /tmp/terraform-inventory.zip]
  - [pip, install, -U, pip, ansible]
  - [git clone https://github.com/digitalocean/navigators-guide.git]

From here, click Create. The Droplet itself will be up and running quickly, but the commands in its user data will take a little time to finish running. You can log into the Droplet and look at /var/log/cloud-init-output.log to check its status.

Note: If you’re more comfortable with another operating system, like a Linux/BSD distribution or macOS on your local computer, you can use that instead as long as it meets Ansible’s system requirements.

You can also install this software manually if you prefer. Terraform and terraform-inventory are Go binaries that need to be placed within your $PATH. We recommend installing Ansible with Pip instead of a system package manager like APT because it stays up to date and allows you to install it within a virtualenv.

The last step is to create an SSH key for the controller Droplet. We’ll later place this on each of the nodes in our infrastructure. You can run this one-liner when logged into your server to create a key and comment it with your Droplet’s hostname:

ssh-keygen -t rsa -C $(hostname -f)

You’ll be able to see the public and private key pair in /home/your_username/.ssh/. Later on, we’ll use these to configure our Terraform variables.

Now we can start using the tools and controller Droplet we just set up to start creating some usable infrastructure. By the end of the next chapter, you’ll start seeing the differences between Ansible and Terraform and have a better idea of how they’ll both fit into your deployments.

Source: DigitalOcean News Navigator’s Guide: Initial Environment Setup