Deployment Automation using Docker Compose

Goal - we will explore a use case where we have a two tier application - web tier and db tier. We will use docker to build deploy both together.



Directory tree

$ tree ~/docker-app/
/home/training/docker-app/
├── cassandra
│   ├── Dockerfile
│   └── init.cql
├── docker-compose.yml
└── flask-app
    ├── app.py
    ├── Dockerfile
    └── requirements.txt


Create docker-compose file. It defines two services - web-service and db-service. DB-service depends on web-service. DB_service hosts a cassandra database. Data of cassandra is stored on /var/lib/cassandra/data. This data directory is mapped to a volume on the docker host machine - with a purpose that if the container is recreated, we want to persist the data outside the container environment.

$ cat ~/docker-app/docker-compose.yml
version: "3"
services:
  web-service:
    build: ./web-service
    ports:
      - "5000:5000"
    volumes:
      - "./web-service:/app"
    depends_on:
      - db-service

  db-service:
    build: ./db-service
    volumes:
      - "/data:/var/lib/cassandra/data"


Define a dockerfile for DB-service.

$ cat ~/docker-app/db-service/Dockerfile
FROM cassandra:3.11.3
VOLUME /var/lib/cassandra/data
USER cassandra
RUN cassandra


Define a docker file web service. We are using a customized ubuntu instance that consists of python and python libraries - flask web framework and cassandra-driver - as the name indicates it is a python driver for Cassandra database.

$ cat ~/docker-app/web-service/Dockerfile
FROM  abasar/web_tier:1.0
COPY . /app
WORKDIR /app
CMD ["python", "app.py"]


Below is code of a web application - it exposes three path

  • / - show hello world
  • / init - initializes the database with requires schema
  • /counter - increment view count and show the current count
$ cat ~/docker-app/web-service/app.py
from cassandra.cluster import Cluster, NoHostAvailable
import socket

from flask import Flask
app = Flask(__name__)

session_ = None
db_host_ = "db-service"


def get_session():
  global session_
 if session_:
  return session_
 try:
  socket.gethostbyname(db_host_)
  cluster = Cluster([db_host_])
  session_ = cluster.connect()
 except socket.error:
  print("DB host is not found")
 except NoHostAvailable:
  print("DB host is up but the DB service is not available")
 return session_
  
@app.route("/", methods = ["get", "post"])
def hello():
    return "Hello world"

@app.route("/init")
def init():
    session = get_session()
    if session:
        session.execute("CREATE KEYSPACE IF NOT EXISTS demo WITH replication = {'class': 'SimpleStrategy','replication_factor': 1}")
        session.execute("CREATE TABLE IF NOT EXISTS demo.counters(name text primary key, count counter)")
        return "DB has been initialized"
    return "DB Session is not active"

@app.route("/counter")
def counter():
    session = get_session()
    session.execute("update demo.counters  set count += 1 where name = 'home'")
    count = session.execute("select count from demo.counters where name = 'home'").one()[0]
    return "Page view count: " + str(count)

if __name__ == "__main__":
    app.run(debug = True, host = "0.0.0.0", port = 5000)


Validate the docker-compose.yml file

$ sudo docker-compose config
services:
  db-service:
    build:
      context: /home/training/docker-app/db-service
    volumes:
    - /data:/var/lib/cassandra/data:rw
  web-service:
    build:
      context: /home/training/docker-app/web-service
    depends_on:
    - db-service
    ports:
    - 5000:5000/tcp
    volumes:
    - /home/training/docker-app/web-service:/app:rw
version: '3.0'



$ sudo docker-compose build
Building db-service
Step 1/4 : FROM cassandra:3.11.3
 ---> d05bb5bbfe7d
Step 2/4 : VOLUME /var/lib/cassandra/data
 ---> Using cache
 ---> c80e5ea58ce9
Step 3/4 : USER cassandra
 ---> Using cache
 ---> 29e1c9c5b13b
Step 4/4 : CMD ["cassandra"]
 ---> Using cache
 ---> de307c093a33
Successfully built de307c093a33
Successfully tagged docker-app_db-service:latest
Building web-service
Step 1/4 : FROM  abasar/web_tier:1.0
 ---> d6a39aa114c8
Step 2/4 : COPY . /app
 ---> Using cache
 ---> f70fccac3702
Step 3/4 : WORKDIR /app
 ---> Using cache
 ---> 117171fb6177
Step 4/4 : CMD ["python", "app.py"]
 ---> Using cache
 ---> 1ea7ab33e4bf
Successfully built 1ea7ab33e4bf
Successfully tagged docker-app_web-service:latest


You can find the files in the github link below.

https://github.com/abulbasar/docker-examples/tree/master/docker-app


Since we are using volume mapping from the DB service to /data, create the directory and give necessary permission - in this case we are giving full permission.

$ sudo mkdir /data
$ sudo chmod 777 /data


Start the services

$ sudo docker-compose up
Starting docker-app_db-service_1 ... done
docker-app_web-service_1 is up-to-date
Attaching to docker-app_db-service_1, docker-app_web-service_1
web-service_1  |  * Serving Flask app "app" (lazy loading)
web-service_1  |  * Environment: production
web-service_1  |    WARNING: Do not use the development server in a production environment.
web-service_1  |    Use a production WSGI server instead.
web-service_1  |  * Debug mode: on
web-service_1  |  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
web-service_1  |  * Restarting with stat
web-service_1  |  * Debugger is active!
...


Initialize the data base

$ curl localhost:5000/init
DB has been initialized
$ curl localhost:5000/counter
Page view count: 1


Simple stress testing

On host machine, install apache-utils

$ sudo apt install apache-utils

Run the stress test

$ ab -n 1000 -c 10 -k http://localhost:5000/counter
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Werkzeug/0.14.1
Server Hostname:        localhost
Server Port:            5000

Document Path:          /counter
Document Length:        19 bytes

Concurrency Level:      10
Time taken for tests:   8.359 seconds
Complete requests:      1000
Failed requests:        984
   (Connect: 0, Receive: 0, Length: 984, Exceptions: 0)
Keep-Alive requests:    0
Total transferred:      178067 bytes
HTML transferred:       20067 bytes
Requests per second:    119.64 [#/sec] (mean)
Time per request:       83.586 [ms] (mean)
Time per request:       8.359 [ms] (mean, across all concurrent requests)
Transfer rate:          20.80 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       6
Processing:    35   83 100.4     69    1174
Waiting:       35   82 100.5     68    1173
Total:         35   83 100.9     69    1179

Percentage of the requests served within a certain time (ms)
  50%     69
  66%     75
  75%     81
  80%     85
  90%    101
  95%    115
  98%    151
  99%    861
 100%   1179 (longest request)