Dockerizing your CI/CD Pipeline

Paul Dragoonis

  • From Glasgow (a weegie :-)
  • Full-Stack Software Consultant
  • Quality Assurance Engineer
  • @dr4goonis /

Analyse and influence on quality of

  • code quality
  • environmental quality (local, staging, prod)
  • Behavioural-driven-development
  • continuous integration pipelines
  • security / scaling
  • continuous deployment / rollback quality


ps aux | grep java | grep -v grep | awk '{print $2}' | xargs kill

while read in; do host "$in"; done < sites.txt | grep -iv "GOOGLE" | grep -E '1\.2\.3\.4|5\.6\.7\.8' | sed -e 's/has\ address\' | sed -e 's/has\ address\' | while read sites; do curl -sL -w "%{http_code} %{url_effective}\\n" "$sites" -o /dev/null; done | grep -ivE '4.*|5.*' | sed -e 's/200//' | sed -e 's/HTTP/http/'

Automation > google-referrers.txt

cron entry for: ;-) <3 selenium

Continuous Integration

repeatable, continual way to automate tasks

gain feedback on those tasks like: metrics, measurements, reports

Continuous Deployment
Continuous Delivery

Continuous Deployment

your pipline builds continuously

but, you decide when you want to push the big red button

Continuous Delivery

your pipline builds continuously

once you have a green build, it promotes and goes to production with no manual intervention

What is a pipeline?

results moving through a series of steps from one side to the other.

What does a pipeline consist of?

  • jobs, lots of jobs!
  • that have their own single responsibility

What to put in your pipeline

  • your code - More than one repo?
    Jenkins MultipleSCM Plugin
  • Quick initial checking
  • lint checking
  • unit tests

Perform thorough testing

  • integration tests
  • acceptance
  • end-to-end testing
  • security testing
  • stress testing

Linking your pipeline together


(sequencing of the pipeline)

  • downstream triggers - what to trigger next
  • upstream triggers - what to trigger before running current one

Kicking off your pipeline

  • Commit triggered pipelines
  • Manually triggered(parameterised) pipelines

Managing the Pipline

Build Pipeline Plugin

Build Pipeline Plugin

Delivery Pipeline Plugin

Delivery Pipeline Plugin

Maintaining your pipeline/jobs

CloudBees folders plugin
group your jobs

JobDSL Plugin

job('PROJ-unit-tests') {
    scm {
    triggers {
        scm('*/15 * * * *')
    steps {
        maven('-e clean test')

JobDSL Plugin

job('unit-tests') {
    scm {
    steps {

JobDSL Plugin





Problems with standlone Jenkins

Scaling is hard, one slave per machine

One environment for your all your jobs

installed directly onto the jenkins host

CI env inconsistent from staging/prod

Docker to the rescue

What is docker?

Docker is an open-source project, which allows you to create lightweight, portable, distributable, self-sufficient containers, of an application with all of its dependencies

Containers running on a single machine all share the same OS/kernel/filesystem so they start instantly and make more efficient use of RAM.

What is docker?

Docker on different OS

Docker Hub (official registry)

Running an image from docker hub

    docker pull image_name:tag
    docker pull nginx:latest
    docker pull node:5.6
    docker pull mysql-server:5.7

Making your own image

FROM ubuntu:14.04

RUN apt-get update && apt-get install -y python python-dev postgresql

USER postgres

RUN /etc/init.d/postgresql start && createdb -T template0 -E UTF8 my_database

USER root


Making your own image

FROM php7.0-cli:latest

RUN apt-get update && apt-get install -y nginx

USER www-data
ENTRYPOINT nginx -g 'daemon off;'

Building your image

docker build -t ed_techmeetup:1

Docker compose (container orchestration)


  build: .
  dockerfile: path/to/my/apps/Dockerfile
      - "8080:80"

  image: ed_techmeetup:1

  image: redis:latest
    - "6379:6379"

Running your containers

                docker-compose up  <--- you will see all logs of all containers

                docker-compose up -d  <--- daemon mode, runs in background

                docker-compose logs <--

Back to Jenkins land

Docker Jenkins Plugin

Docker Jenkins Plugin

connects to any host running docker

creates an isolated environment for your jobs

runs your jobs inside it

spins down isolated environment

Build app image out of green test run

docker build -t${BUILD_NUMBER} <- jenkins var
docker build -t

CI Workflow with Docker

Integration testing with your containers

    cp docker-compose.yml

    docker-compose -d up

Pushing to private docker registry

docker push${BUILD_NUMBER}

Overview of docker pipeline

Lessons learned about dockerizing pipelines

Have lots of cleanup tasks, disk space fills up quickly

You can build and run production images on your localhost very easily

Thank you Edinburgh!