Test Driven Devops with Ansible

Or using Ansible as an acceptance test tool.

Problem

My blog has received major overhaul couple days ago:

Preserving site permalinks was my major migration concern and it’s easy to break them since there are many changes involved.

Idea

If the migration is an implementation refactor then urls preservation could be verified using a regression test.

The idea is to have a list of relative url paths and run requests to both old and new sites. Having “200 Ok” responses would be considered as pass and fail otherwise.

It’s worth noting that “test driven” is core concept of Ansible as it acts against specs playbooks provide. But, in this case, Ansible is used only for its testing capabilities without any required actions: purely as an acceptance testing tool.

Plan

  1. test fails with the new address
  2. make changes: provisiton, setup, etc…
  3. test passes with the new address
  4. repeat 2-3 until 3 succeeds

The only prerequisite is that test must pass against the old address to be considered valid.

The Test

Getting a list of urls is trivial either from sitemap.xml or from an index.html.

Here’s an excerpt from my list:

/blog/2015/02/25/gorack-go-webserver-rack
/blog/2014/04/04/share-by-communicating
/blog/2014/02/04/why-i-stopped-contributing-to-vundle
/blog/2013/08/19/three-ways-to-get-your-ios-app-data
/blog/2013/08/19/MHAboutView-iOS

After obtaining a list of urls next question was which tool to use for testing.

Fortunately there’s no need to use any special tools, since Ansible allows doing this:

 ---

- name: tests different aspecs of gmarik.info changes
  hosts: all
  gather_facts: false
  connection: local

  tasks:
    - debug: msg="Running against {{domain_name}}"

    - name: test redirects reach destination
      uri: method=GET url={{domain_name}}{{item}} follow_redirects=yes status=200
      with_lines: cat gmarik.info-permalinks.csv

is a content of a playbook saved at tests/gmarik.info_test.yml

where:

see uri module documentation for more details about how it works.

Test validation

To make sure the test is valid it’s run against the old address:

$ ansible-playbook -i 'localhost,' tests/gmarik.info.yml -e 'domain_name=http://gmarik-info.herokuapp.com'

PLAY [tests different aspecs of gmarik.info changes] ************************** 

TASK: [debug msg="Running against {{domain_name}}"] ****************** 
ok: [localhost] => {
    "msg": "Running against http://gmarik-info.herokuapp.com"
}

TASK: [test path reachable] *************************************************** 
ok: [localhost] => (item=/about)
ok: [localhost] => (item=/blog/2015/02/25/gorack-go-webserver-rack)
ok: [localhost] => (item=/blog/2014/04/04/share-by-communicating)
ok: [localhost] => (item=/blog/2014/02/04/why-i-stopped-contributing-to-vundle)
ok: [localhost] => (item=/blog/2014/01/14/tls-auth-in-soa)
ok: [localhost] => (item=/blog/2013/08/19/three-ways-to-get-your-ios-app-data)
ok: [localhost] => (item=/blog/2013/08/19/MHAboutView-iOS)
...trimmed

where -e 'domain_name=http://gmarik-info.herokuapp.com' sets the domain to request to. In this case it’s the old address.

Test passes according to the output: the test is valid.

Changes

Details of all the changes go out of the scope of this post, but briefly it’s:

Testing the new address

Running the test against the new address resulted in bunch of 404 errors, test failed:

failed: [localhost] => (item=/blog/2013/08/19/MHAboutView-iOS) => {"redirected": false, "server": "nginx", "status": 404, ... trimmed}

msg: Status code was not [200]

Short Google-ing revealed it was because of Hugo’s default to lowercase slugs

# Do not make the url/path to lowercase
disablePathToLower:         false

(which is also a double negative and confusing)

Pass

After couple back and forth, “enabling disabling”, regenerating the site, everything worked and the test passed with the new address: all the url paths were preserved.

Great success!

Summary

Comments