Dockerizing your CI/CD Pipeline

Paul Dragoonis

  • From Glasgow (a weegie :-)
  • Full-Stack Software Consultant
  • Quality Assurance Engineer
  • @dr4goonis /

Analyse and influence on quality of

  • code, environments, tech stack
  • continuous integration pipelines
  • deployment / rollback quality
  • security / scaling


  • Continuous Deployment vs Delivery
  • Building a strong CI pipeline
  • Tips and Tricks with Jenkins
  • Dockerizing your CI pipeline
  • Outline benefits of dockerizing your dev env / pipeline

Continuous Deployment vs Delivery

  • Depends on your customer / product
  • Depends on your rate of change
  • Depends on your team
  • Depends on your infrastructure
  • Dockerizing your CI pipeline

Building a strong CI pipeline

What is a pipeline?

changes / builds / results moving through a series of broken down steps from one side to the other.

Triggering Builds

  • SCM triggered builds
  • parameterised builds - passing down branch name

What to put in your pipeline

Setup system - pull down code

  • Pull down dependencies (More than one repo? Jenkins MultipleSCM Plugin)
  • or
  • parameterised builds - passing down branch name

Quick initial feedback

  • lint checking
  • unit tests

Perform thorough testing

  • integration tests
  • acceptance
  • end-to-end testing

Linking your pipeline together


(sequencing of the pipeline)

  • downstream triggers - what to trigger next
  • upstream triggers - what to trigger before running current one

Managing the Pipline

Build Pipeline Plugin

Build Pipeline Plugin

Delivery Pipeline Plugin

Delivery Pipeline Plugin

CloudBees Folders Plugin

Jobs Maintenance

CloudBees Folders Plugin

JobDSL Plugin

job('PROJ-unit-tests') {
    scm {
    triggers {
        scm('*/15 * * * *')
    steps {
        maven('-e clean test')

JobDSL Plugin

job('PROJ-release') {
    scm {
    // no trigger
    authorization {
        // limit builds to just Jack and Jill
        permission('hudson.model.Item.Build', 'jill')
        permission('hudson.model.Item.Build', 'jack')
    steps {
        maven('-B release:prepare release:perform')

JobDSL Plugin

freeStyleJob(String name, Closure closure = null)

buildFlowJob(String name, Closure closure = null)

ivyJob(String name, Closure closure = null)

matrixJob(String name, Closure closure = null)

mavenJob(String name, Closure closure = null)

multiJob(String name, Closure closure = null)

workflowJob(String name, Closure closure = null)

multibranchWorkflowJob(String name, Closure closure = null)

JobDSL Plugin





Problems with standlone Jenkins

Scaling is hard, one slave per machine

One integration environment for your platform, installed on jenkins host

Jenkins env inconsistent from staging/prod

Docker to the rescue

What is docker?

What is docker?

Docker on different OS

Docker Hub (official registry)

Running an image from docker hub

    docker pull image_name:tag
    docker pull nginx:latest
    docker pull node:5.6
    docker pull mysql-server:5.7

Making your own image

FROM ubuntu:14.04

RUN apt-get update && apt-get install -y python python-dev postgresql

USER postgres

RUN /etc/init.d/postgresql start && createdb -T template0 -E UTF8 my_database

USER root


Building your image

docker build -t falkirk_db_image

Docker compose (container orchestration)

  image: falkirk_db_image

  image: redis:latest
    - "6379:6379"

  build: .
  dockerfile: path/to/my/apps/Dockerfile
      - "8080:80"
      - database
      - redis


Running your containers

                docker-compose up
                docker-compose up -d  <--- daemon mode

Back to Jenkins land

Docker Jenkins Plugin

Docker Jenkins Plugin

connects to any host running docker

creates a jenkins agent, runs job(s)

spins down jenkins agent

CI Workflow with Docker

CI Workflow with Docker

Spinning up containers for testing

    docker-compose -f up -d

Build app image out of green test run

    docker build -t falkirk_lug_app:${BUILD_NUMBER}    <-- jenkins var

CI Workflow with Docker

CI Workflow with Docker

Building with dynamic tag

    docker build -t${BUILD_NUMBER}
    docker build -t
    docker build -t

    docker push

Pushing to private docker registry

    docker push

Registries overview

CI Workflow with Docker

Build promotion

Overview of docker pipeline

Thank you Falkirk!