In the previous post, we automated the site deployment to Production. In this part of the series, we’ll delve into automating the site deployment to different environments along with running the tests automatically.

Let’s recap the workflow from the previous posts:

  1. Build the site with the jekyll build command ran on top of a Jekyll container (described in part 1 and in part 2)
  2. Deploy the site to a specified environment (described in the previous post)
  3. Unless it’s not Production, run the tests in that environment (this part)

Of course, the next step wouldn’t be executed if the previous step failed. In this post, we won’t automate the whole flow, but only the last two parts of it.

We’ll run the tests using Python’s unittest built-in library. As you may have guessed, we’ll test our site somewhere safe before we show it to our readers. Hence, we’ll implement environment separation.

In order to do so, we’ll implement environment separation at three levels:

  1. Dev (or development) environment that we’ll run locally to test our site on our computer
  2. Production-like (or “staging”) environment to run all the tests in a Production-like setup after we completed building the site
  3. The Production environment (the money time)

On top of the environment separation, we’ll automate the deployment and testing steps by writing a docker-entrypoint.sh script, which will be run on top of the deployment container.

Environments Set Up

If we revisit the environments, we’d discover that we have all the moving parts for the local (Development) environment and the Production one. In the case of the Dev environment, we’ve already created settings to serve our site locally with NGINX.

In regards to the Production environent, we already have the DNS entries in Cloudflare and the S3 bucket set up.

As we stipulated earlier, we’d need to set up a Production-like environment to test our site against right before exposing it to our fellow visitors. We’ll mimic the Production environment by performing the following actions:

  • Create an S3 bucket named staging.yourdomain.com. You should copy all the settings from the www.yourdomain.com bucket you’d created in the previous post.
  • Add the staging DNS entry in Cloudflare that would point to the newly created bucket.

After we’re done configuring the Staging environment, we’re good to go.

Configurations Files

Our site should behave the same way in all the environments. In general, it’s a good pratice our business logic behaves the same way independently of the environments it runs on.

In this series, we’ll use .env files to store environment-related configs.

As their names suggest, the .env files contain additional environment variables that can be used per each environment.

We’re using .env files to store configs, since docker-compose supports passing them as environment configurations during the runtime.

The files would be created under the site’s main directory.

We’ll create a .env file for each environment:

  1. .env.prod
  2. env.staging
  3. env.dev

Their contents will be as follows:

.env.prod

S3_BUCKET_NAME=www.yourdomain.com
BASE_URL=https://www.yourdomain.com
SERVER_HEADER=server
EXPECTED_SERVER=cloudflare
ENVIRONMENT=production

.env.staging

S3_BUCKET_NAME=staging.yourdomain.com
BASE_URL=https://staging.yourdomain.com
SERVER_HEADER=server
EXPECTED_SERVER=cloudflare
ENVIRONMENT=staging

.env.dev

BASE_URL=http://site
SERVER_HEADER=Server
EXPECTED_SERVER=nginx
ENVIRONMENT=dev

Note that the Dev environment differs from the Staging environment and Staging environment’s configs look identical to the Production ones. This is to show that Production-like (e.g. the Staging) environment should contain identical or almost identical configurations to the Production environment.

This way, we can simulate better how our code behaves in the Production environment.

Preparing the Tests

Prior to devising the tests, we’d need to update the requirements.txt file in the cicd directory to include all the needed dependencies needed to perform the tests.

Update the Dependencies

Note the new version of the requirements.txt file:

boto3==1.20.15
requests==2.26.0
beautifulsoup4==4.10.0

Two dependencies were added in order for us to be able to perform the required tests, i.e.:

  • beautifulsoup4 - parses the HTML page into an object, which can be navigated by Python
  • requests - allows to issue HTTP requests in a simple format

The Code

We’ll produce a single file called tests.py under the cicd directory. This file will contain all the tests that we’d run.

Here are its contents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

#!/usr/local/bin/python
import unittest
import requests
import os
import sys
from bs4 import BeautifulSoup
from pathlib import Path

BASE_URL_KEY = 'SITE_BASE_URL'
SERVER_HDR_KEY = 'SERVER_HEADER'
HDR_VALUE_KEY = 'EXPECTED_HEADER_VALUE'
CONTENT_TYPE = 'Content-Type'

base_url = os.environ.get(BASE_URL_KEY)
server_header = os.environ.get(SERVER_HDR_KEY)
header_value = os.environ.get(HDR_VALUE_KEY)

class TestSiteLinks(unittest.TestCase):

    def setUp(self):
        self.page_response = requests.get(base_url)

    def test_ok_response(self):
        self.assertEqual(self.page_response.status_code, 200)
    
    def test_server_header(self):
        self.assertIn(header_value, self.page_response.headers[server_header])

    def test_page_links(self):
        soup = BeautifulSoup(self.page_response.text, 'html.parser')
        links = [link['href'] for link in soup.find_all('a')]
        for link in links:
            if 'http' not in link:
                req_url = f'{base_url}{link}'
            if req_url == base_url+'/': # no need to test the base url twice
                continue
            with self.subTest(link=req_url):
                response = requests.get(req_url)
                self.assertEqual(response.status_code, 200)
    
    def test_content_type_header(self):
        self.assertIn('html', self.page_response.headers[CONTENT_TYPE])

if __name__ == '__main__':
   test_result = unittest.main()

Let’s dissect the script into the following parts:

The Setup

In order to run the tests, we’d first need some data, such as to where send our requests to and what headers to expect. In our case, we’re consuming these configurations from the environment variables provided to the container.

In lines 9-16 of the script, we get all the relevant configurations to use them later in the code.

Afterwards, we set up an input against which we’re going to run the tests.

In line 21, we execute a single request to the site’s base URL, so all of our tests can be run against the response’s contents.

The Tests

The table below lists all the tests in the script:

Method name What it does
test_ok_response Tests if the response code is 200 (OK)
test_server_header Tests that the content is served with the correct server: either NGINX or Cloudflare, depending on the environment
test_page_links Gets all the links from the site by traversing the response’s body with the BeautifulSoup library and tests whether there are dead links
test_content_type_header Tests the main response’s Content-Type header to see whether this is correct and suits the expected MIME type. As you recall, specifying an incorrect Content-Type header will result in unexpected errors in our browser while it tries to render the page(s) and their related content

The Entrypoint Script

In order to automate the last two parts of the flow (.e.g deploying and testing), we’ll create an entrypoint script named docker-entrypiint.sh script. This script will be added under the cicd directory:

#!/bin/sh
echo "#### Starting the site's CI/CD process ####"
if [ -z "$S3_BUCKET_NAME" ]; then # if there's no bucket name specified, skip the deployment
    echo "#### Dev environment. Skipping the site deployment ####"
else 
    echo "#### Deploying the site ####"
    ./deployment.py 2> deployment_errors.txt #redirect all the error messages from the deployment to the txt file
    deployment_result=$(grep -i "exception" deployment_errors.txt) 
    if [ -n "$deployment_result" ]; then # if the file's contents aren't empty, there were errors
        echo "Deployment failed with the following message(s):"
        cat deployment_errors.txt
        echo "Stopping the next tests"
        exit 1
    fi
fi
if [ "$ENVIRONMENT" == "production" ]; then
    echo "#### Production environment. Skipping the tests ####"
    exit 0
fi
echo "#### Running the tests ####"
./tests.py 2> errors.txt #redirect all the stderr to the errors.txt file
test_result=$(grep -i "failure" errors.txt) #search in the file if the tests failed
if [ -n "$test_result" ]; then
    echo "Tests failed with the following message(s):"
    cat errors.txt
    exit 1
fi
echo "#### Tests passed ####"

As stated in if [ -z "$S3_BUCKET_NAME" ], if the bucket name is empty we’re in the Dev environment, so we don’t need to deploy the site and then test it. Otherwise, the script proceeds with the deployment of the site to a specified S3 bucket.

The following script exceprt determines whether the tests should be run depending on the environment:

if [ "$ENVIRONMENT" == "production" ]; then
    echo "#### Production environment. Skipping the tests ####"
    exit 0
fi

If the script is still running, it proceeds to executing the tests and prints whether the tests were successfull or not.

Note that in ./deployment.py 2> deployment_errors.txt and in ./tests.py 2> errors.txt the error messages printed by both scripts are collected to a text file.

A note about the streams in Linux: there are three streams that every program in Linux can get:

  • stdin (marked as 0), from where a program can get its input
  • stdout (marked as 1). This where the standard output is written, e.g. the non-error messages that are printed.
  • stderr (marked is 2). This where all the error messages are printed.

Refer here and here for more info about the standard streams.

The above means that we redirect the messages from the stderr to (>) a file.

In the next stage, the deployment script evaluates the contents of the files and determines whether there were errors in the deployment and testing stages respectively. This is done by greping the files’ contents for expressions, such as “exception” or “failure” which are printed by the Python’s error stacktrace and by the unittest library respectively.

Finally, the script prints whether the tests succeeded or failed. If the tests failed, it prints the entire contents of the error message(s). This way, you’re able to receive an indication which issues you site has, so you can fix them as quickly as possible.

The Deployment Docker Image

FROM python:3.9.9-alpine3.14
ENV SITE_CONTENTS_PATH=/site
ENV AWS_SHARED_CREDENTIALS_FILE=/deploy/credentials
RUN apk update && apk upgrade
WORKDIR /deploy
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY deployment.py tests.py docker-entrypoint.sh ./
RUN chmod 755 *.py *.sh
ENTRYPOINT ["./docker-entrypoint.sh"]

Here are some differences between the original Dockerfile.deploy and this version of the file:

  • We’ve removed the S3_BUCKET_NAME environment variable, since it’s going to be specified in one of the .env files that we’d pass to the deployment container when running the docker-compose command.
  • We’ve added the tests.py file that contains all of our tests.
  • We’ve added the docker-entrypoint.sh script and made the container’s entrypoint so it gets executed upon the container initialization, as opposed to the deployment.py script.

Now we have the entrypoint script that orhestrates the deploying and testing steps in our flow. In the next part, we’ll discuss how exactly we perform the deployment and testing steps for different environments.

Let’s build the container while we’re in the cicd directory:

docker build -t sitedeployment -f Dockerfile.deploy .

The Docker Compose File

Let’s review our docker-compose file under the site’s main directory:

version: "3"

services:
    site:
        image: nginx:1.21.4-alpine
        volumes:
          - $PWD/_site:/usr/share/nginx/html
          - ./nginx.conf:/etc/nginx/nginx.conf:ro
        ports:
          - 80:80
        container_name: site
        networks:
            - internal
    
    sitebuilder:
        image: sitebuilder
        volumes:
            - $PWD:/site
        container_name: sitebuilder
        entrypoint: ["jekyll", "build"]

    deployment:
        image: sitedeployment
        volumes:
            - $PWD/_site:/site
            - ~/.aws/credentials:/deploy/credentials:ro
        networks:
            - internal
        depends_on: 
            - site
        environment:
            - S3_BUCKET_NAME=${S3_BUCKET_NAME}
            - SITE_BASE_URL=${BASE_URL}
            - SERVER_HEADER=${SERVER_HEADER}
            - EXPECTED_HEADER_VALUE=${EXPECTED_SERVER}
            - ENVIRONMENT=${ENVIRONMENT}

networks:
    internal:

As you can notice, the composition of the services and relations between them has not been changed. However some minor changes were made to the file.

Let’s look at them more closely:

  • Networks. A new internal network was added. Custom bridge networks in Docker support internal DNS resolution. Hence, this why the internal network was added. This way, our site container can be located by the test script that runs in the local environment.
  • Four environment variables were added under the deployment service. docker-compose supports environment variables templates. These environment variables’ values will be taken from the .env files we pass as an argument to the docker-compose command.
  • More info about passing the .env files to the docker-compose command is available here

Semi-Automating the Flow

Finally, we can run almost the whole CI/CD flow semi-automatically by issuing a series of commands to the shell.

If we need to build the site, we can power up the sitebuilder container as following:

docker-compose up sitebuilder --abort-on-container-exit --exit-code-from sitebuilder

Note the --abort-on-container-exit and the --exit-code-from flags. These flags allow us to pass a container’s exit code to the shell. You can read more about these flags here. This piece of information would be useful in our subsequent post (stay tuned).

You can ignore the error messages about the missing environment variables, since the sitebuilder image doesn’t rely upon any environment variables. The same applies to the site image mentioned further.

If you want to run and test your site locally:

docker-compose up -d site

Afterwards, this command can be run:

docker-compose --env-file .env.dev up deployment \
    --abort-on-container-exit --exit-code-from deployment

After the deployment container has completed running all the tests, you’ll get an indication whether the tests succeeded or failed. To terminate all the containers and to remove the temporarily created network, run the docker-compose down command.

In order to deploy and test the site in the Staging environment, run the following command:

docker-compose --env-file .env.staging up deployment \
    --abort-on-container-exit --exit-code-from deployment

After the tests succeeded you’d probably want to deploy your site to the Production environment. To do so, just change the env.staging file to env.prod file under the --env-file option.

In order to remove all the containers and associated temporary elements, run the docker-compose command.

The source code is available here.

Stay tuned for more posts.