Linked-in profile Github profile Twitter profile Curriculum vitae
Categories: Automation Drupal
30th May 2019
11th July 2020

Continuous Delivery, with Jenkins declarative pipelines, Composer, and Drupal 8+

WordPress, and Drupal 7 support coming at some point…

Nobody wants to deploy manually anymore, besides, why would you want to with the plethora of tools available to us?

Recently I was given the opportunity to move several Drupal projects into automation. Each environment is a little different, be it infrastructure, or operating system. But at least the CMS is the same, along with my options to interface with the sites, a command line.

I also need to be able to track project specific things to setup and do, like tool chains for style compilation, dependencies, and tests.

Anyway, long story short, I settled with Jenkins and it’s declarative pipelines system to carry out the heavy-lifting. With Jenkins, I can keep job pipeline steps in VCS. Now I just need something to orchestrate this process. Alas, I couldn’t find what I was looking for! So I decided to just throw together the exact same manual commands I use every day, and fine tune them for speed and fail-over.


I wanted to keep the deployments as short as possible, and reduce down-times to an absolute minimum. Not only was I able to reduce the down time, I found a way of avoiding maintenance mode altogether, and instead use the read-only module, along with a Blue-Green software-based deployment process, where the new build was setup along side the active build, and simplying updating symlinks.


There is no requirement for passwords to be stored within the JWSA codebase for your projects. Only the path to a GPG key for use with SSH authentication, and the reliance on a deployment user environment that provides credentials for the scripts to connect to databases, such as a .my.cnf file for the database, in the user’s workspace – remember not to dial those permissions over 600 😉.


With all this on my mind, I couldn’t be bothered with a fancy name, so Jenkins Web Scripts by Alex (JWSA) it is!

The scripts are available on GitHub for you to use, these are being actively developed, so keeps eyes open for more features. Or feel free to contribute

Note: The README on the repository may be more upto date than this post!

The build script

The build script intends to setup and bootstrap the entire Drupal site workspace, on the Jenkins server. With an exisiting database, either copied from the destination server, or default to using whatever is available, as defined in the environment file.

# These environment variables are the format used in Jenkins Groovy pipelines ${env.JOB_NAME} ${env.WORKSPACE} /path/to/${env.JOB_NAME}/.env



The deploy script

The deploy script will copy the entire Jenkins workspace to the specified paths, on the destination server. Deployments are over SCP or RSYNC, so there is no need to install a VCS, Node.js, Composer etc on the destination server, just a LAMP stack, PHP extensions required by Drupal 8, and credentials for the deployment user, see Project files.

# These environment variables are the format used in Jenkins Groovy pipelines ${env.JOB_NAME} ${env.WORKSPACE} ${env.BUILD_ID} /path/to/${env.JOB_NAME}/.env



The JWSA .env

# Get last successful Jenkins build number for the project
LAST_BUILD_ID=`curl http://jenkins.test:8080/job/$PROJECT_NAME/lastSuccessfulBuild/buildNumber`

Project files

An example of a JWSA project, can be found at ~/project/example-test

A project file

Here are all possible variables you can define for a JWSA project environment.

# CMS type (drupal8 only. Coming soon; drupal7, wordpress)
# Webserver environment (apache, nginx)
# Destination host
# REMEMBER: SSH keys need to be manually installed on the Jenkins deployment server.
# Base deployment path
# Product database dump location for reversion
# Project site assets (such as Drupal public files)
# Location of the deployed builds
# Location of this build
# Drupal's Twig storage path
# Location of this build
# The SSH user to connect to the deploy environment
# The web server user for project ownership & permissions
# Local database
# Deployment database (this is suffixed in 'prod' environments. with incremental job numbers "__n")
# Webroot symlink, that will link to the active build
# Drupal public files path in the build, this is linked to the project's main files directory
# Drupal settings location
# CLI tool (Drush, WP-CLI)
# Services to restart (used at various steps in the deployment process)
declare -a DEST_SERVICES_RESTART=("php-fpm")
# Services to reload (used at various steps in the deployment process)
declare -a DEST_SERVICES_RELOAD=("nginx")
# Database table data to ignore, on data dumps.
# Commands to execute before the platform step (The Drupal specific parts) of the build process.
"echo \"hi\""
# Rsync flags and parameters
# REMEMBER: Exclude .env, assets, cache, test, and tool directories to speed up transfers
# MUST: Suffix with the with "-e" flag, to allow succeeding text to be executed remotely.
RSYNC_FLAGS="-al --stats --delete-before --exclude=.env --exclude=.git --exclude=.sass-cache --exclude=node_modules --exclude=simpletest --exclude=tests --exclude=/storage --exclude=/private --exclude=/web/sites/default/files -e"
# SSH connection string

A project .env file

Variables injected into the Drupal environment, specifically the settings.php file

# The type of environment we're working with (local, dev, test, prod. etc..)

# Database credentials

# Drupal bits

A Drupal 8 project settings.php


// We have a JWSA environment.
if (($env = getenv('JWSA_APP_ENV')) !== FALSE) {

	// Drupal hash salt.
	if (($val = getenv('JWSA_HASH_SALT')) !== FALSE) {
		$settings['hash_salt'] = $val;

	// Drupal YAML config sync path.
	if (($val = getenv('JWSA_CONFIG_SYNC_PATH')) !== FALSE) {
		$config_directories['sync'] = $val;

		// Or maybe something like this... (if you're looking for an alternative to 'configuration split')
//		$config_directories['sync'] = '../config/' . $env . '/sync';

		// Or stick to using ../config/sync with production configs. Overwriting config variables using environment settings files below.

	// Drupal private assets path.
	if (($val = getenv('JWSA_PRIVATE_PATH')) !== FALSE) {
		$settings['file_private_path'] = $val;

	// Drupal TWIG template storage (good for AWS S3 bucket use-cases for public and/or private assets)
	if (($val = getenv('JWSA_TWIG_PHP_STORAGE_PATH')) !== FALSE) {
		$settings['php_storage']['twig']['directory'] = $val;

	// Drupal default database.
	$databases['default']['default'] = [
		'database' => getenv('JWSA_SQL_DATABASE'),
		'driver' => 'mysql',
		'host' => getenv('JWSA_SQL_HOSTNAME'),
		'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
		'password' => getenv('JWSA_SQL_PASSWORD'),
		'port' => getenv('JWSA_SQL_PORT'),
		'prefix' => '',
		'username' => getenv('JWSA_SQL_USER'),

	// Get any environment specific settings.
	$file = $app_root . '/' . $site_path . '/settings.' . $env . '.php';
	if (file_exists($file)) {
		include $file;

A project Jenkinsfile

Here is a really basic pipeline for a project. for the build and deploy scripts

#!/usr/bin/env groovy

pipeline {
	agent any

	stages {
		stage('Composer') {
			steps {
				sh 'composer install --no-interaction'

		stage('NPM') {
			steps {
			    sh 'npm install'

		stage('Grunt') {
			steps {
			    sh 'grunt foobar'

		stage('Build') {
			steps {
				sh "/path/to/jwsa/ ${env.JOB_NAME} ${env.WORKSPACE} /path/to/env/${env.JOB_NAME}/.env"

		stage('Test') {
			steps {
				sh 'vendor/bin/phpunit foobar'

		stage("Deploy") {
			steps {
				sh "/path/to/jwsa/ ${env.JOB_NAME} ${env.WORKSPACE} ${env.BUILD_ID} /path/to/envs/${env.JOB_NAME}/.env"

Leave a Reply

Your email address will not be published. Required fields are marked *