Have you ever tested whether all pages of a website return with a 200 status code before you deployed to production? No? Then let me introduce sireg: a tool for website regression testing.

Over a year ago, I wrote an article on how I regression test the sitemap of my website before I deploy an update. Back then, I used a simple unit test setup to download the sitemap, rewrite all links to target the staging server and fire some HTTP requests. While being involved in the relaunch of the website of the company I work for (check it out: www.mimacom.com) I felt I just had to turn the idea of website regression testing into a standalone project. Say hello to sireg! :-)

Getting Started with sireg

First, you will need to download the sireg binary from GitHub releases (it’s cross-platform thanks to zeit/pkg). As a first step, let’s see whether all URLs listed in a sitemap really do exist:

$ sireg exec --loader-sitemap-sitemap "https://<DOMAIN>/sitemap.xml"

With the above command sireg downloads the sitemap and fires a HTTP request to all sites listed in the sitemap. All pages that do not end up at a 200 status code are reported broken. Of course, sireg follows all redirects along the way.

Using sireg to Test a Staging Environment

Suppose we have setup a staging environment of a new version of the website to be deployed. Before we deploy to production we want to make sure that all pages available of the deployed website are working fine on our staging server. We can use sireg to rewrite the URLs retrieved from sitemap and target the staging server:

$ sireg exec \
    --loader-sitemap-sitemap "https://<DOMAIN>/sitemap.xml" \
    --replacer-static-replace "https://<DOMAIN>/" \
    --replacer-static-with "http://localhost:8080/"

All URLs loaded from the sitemap are matched against the replace argument and if there is a match, the match is replaced by the with argument. Starting the staging server on a build agent allows to replace the domain found in the URLs of the sitemap to point to localhost leaving all the important paths after the FQDN intact.

The sireg Workflow

sireg offers a customizable workflow to support various use cases. The basic workflow of the tool is:

Load: sireg first loads URLs to be tested from any source. URLs can currently be loaded from a sitemap or from a simple text file.

Filter: If the number of loaded URLs is too large, sireg supports the ability to add filters to reduce the test size. Filters can also be useful to create quick smoke tests by choosing a random sample from all URLs.

Replace: sireg use cases typically include targeting a different domain (e.g. localhost) to test against. Replacers therefore allow to rewrite a URL before the actual HTTP request is executed.

Request: sireg will then concurrently request all URLs and follow any redirects along the way. The redirect chain and HTTP headers are stored internally for evaluation.

Report: The results of the HTTP requests are matched against the expected values (e.g. a status code of 200) and any deviations are reported as violations. Reporters then output the results of the evaluation to different formats (e.g. file or console output).

sireg Test Suites

sireg can be invoked with command line options as shown in the examples above, but also via a JSON test suite file. For example, I use the following test suite configuration to test this website in the continuous delivery pipeline:

{
  "testSuite": "Regression test fabian-keller.de",
  "loaders": [
    {
      "loader": "sitemap",
      "options": {
        "sitemap": "https://www.fabian-keller.de/sitemap"
      }
    }
  ],
  "replacers": [
    {
      "replacer": "static",
      "options": {
        "replace": "https://www.fabian-keller.de",
        "with": "http://localhost:9000"
      }
    }
  ]
}

See the docs for a full list of available loaders, filters, replacers, and reporters and their options.

Add sireg to Your Project

Give sireg a try and add it to your project. The CLI options make it easy to give it a quick spin and to see the benefits of regression testing a website. If you are using sireg I’d be happy to hear about your use case! So feel free to leave a comment here :-)