Test Driven Devops with Ansible

Or using Ansible as an acceptance test tool.


My blog has received major overhaul couple days ago:

Preserving site permalinks was my major migration concern and it’s easy to break them since there are many changes involved.


If the migration is an implementation refactor then urls preservation could be verified using a regression test.

The idea is to have a list of relative url paths and run requests to both old and new sites. Having “200 Ok” responses would be considered as pass and fail otherwise.

It’s worth noting that “test driven” is core concept of Ansible as it acts against specs playbooks provide. But, in this case, Ansible is used only for its testing capabilities without any required actions: purely as an acceptance testing tool.


  1. test fails with the new address
  2. make changes: provisiton, setup, etc…
  3. test passes with the new address
  4. repeat 2-3 until 3 succeeds

The only prerequisite is that test must pass against the old address to be considered valid.

The Test

Getting a list of urls is trivial either from sitemap.xml or from an index.html.

Here’s an excerpt from my list:


After obtaining a list of urls next question was which tool to use for testing.

Fortunately there’s no need to use any special tools, since Ansible allows doing this:


- name: tests different aspecs of gmarik.info changes
  hosts: all
  gather_facts: false
  connection: local

    - debug: msg="Running against {{domain_name}}"

    - name: test redirects reach destination
      uri: method=GET url={{domain_name}}{{item}} follow_redirects=yes status=200
      with_lines: cat gmarik.info-permalinks.csv

is a content of a playbook saved at tests/gmarik.info_test.yml


see uri module documentation for more details about how it works.

Test validation

To make sure the test is valid it’s run against the old address:

$ ansible-playbook -i 'localhost,' tests/gmarik.info.yml -e 'domain_name=http://gmarik-info.herokuapp.com'

PLAY [tests different aspecs of gmarik.info changes] ************************** 

TASK: [debug msg="Running against {{domain_name}}"] ****************** 
ok: [localhost] => {
    "msg": "Running against http://gmarik-info.herokuapp.com"

TASK: [test path reachable] *************************************************** 
ok: [localhost] => (item=/about)
ok: [localhost] => (item=/blog/2015/02/25/gorack-go-webserver-rack)
ok: [localhost] => (item=/blog/2014/04/04/share-by-communicating)
ok: [localhost] => (item=/blog/2014/02/04/why-i-stopped-contributing-to-vundle)
ok: [localhost] => (item=/blog/2014/01/14/tls-auth-in-soa)
ok: [localhost] => (item=/blog/2013/08/19/three-ways-to-get-your-ios-app-data)
ok: [localhost] => (item=/blog/2013/08/19/MHAboutView-iOS)

where -e 'domain_name=http://gmarik-info.herokuapp.com' sets the domain to request to. In this case it’s the old address.

Test passes according to the output: the test is valid.


Details of all the changes go out of the scope of this post, but briefly it’s:

Testing the new address

Running the test against the new address resulted in bunch of 404 errors, test failed:

failed: [localhost] => (item=/blog/2013/08/19/MHAboutView-iOS) => {"redirected": false, "server": "nginx", "status": 404, ... trimmed}

msg: Status code was not [200]

Short Google-ing revealed it was because of Hugo’s default to lowercase slugs

# Do not make the url/path to lowercase
disablePathToLower:         false

(which is also a double negative and confusing)


After couple back and forth, “enabling disabling”, regenerating the site, everything worked and the test passed with the new address: all the url paths were preserved.

Great success!