Improve your Local Env with Docker+Jenkins+Selenium to achieve Continuous Delirery

The path to Continuous Delivery (CD) happens to have stable execution results of automated test, so will be ideal if the developers could pass the automated test on their local machine before they integrate their code with the base code. But  if we want to execute the automated test on the developer local machine we have to be aware of:

1. The executions of the tests must be not intrusive, allowing the developers to focus on other tasks while the tests are executing.
2. The solution must be easily adaptable to changes, for example, if new tests suites are available or disabled,  this doesn't have to be an overhead work for the developers to synchronize with their local environment.

Sometimes these points are difficult to accomplish, i.e. if we have more than 500 automated functional test to be tested with a real browser (e.g. Firefox).  

Knowing the global solution explained in my previous post, it's easy with Docker and Jenkins to have a template that could build a "private platform" where execute 'locally'  all the  tests (unit, integrated, functional...)

You can see the simplified solution in the next diagram:

As you can see on the diagram, the solution is based on Docker compose which builds Jenkins in a Docker container: a Jenkins Master and n-slaves Jenkins  (depending on configuration parameters). It will execute the test with Selenium/Firefox, where the test live in the local filesystem of the host. In this case, for convenience and not to overload the developers' machines, we won't test on IE  (but we could use a Vagrant virtual-machine with Windows and IE)...

The key points of this kind of configuration are:

• Run Automated Test in background.
• Light GUI containers to run Firefox with Selenium.
• Parallel execution of the tests. 
• Easy installation and updates.
• We can see the visual errors of the navigation in the pdfs generated by the framework of test (see previous post)

Some Technical details of the solution

Docker-Compose

With Docker-Compose we have a simplified configuration for the developer:

# Jenkins Master
jenkins:
  image: s2obcn/jenkins
  container_name: jenkins
  ports:
   - "8090:8080"
   - "50000"
  volumes:
        - /home/s2o/vDocker/jenkins:/jenkins
  env_file:
    - jenkins-master.env
# Jenkins Slave
slave:
  image: s2obcn/jenkins-swarm
  links:
    - jenkins:jenkins
  volumes:
      - /home/s2o/vDocker/jenkins_shared/workspace:/opt/jenkins/workspace
  env_file:
    - jenkins-slave.env

Jenkins docker image with swarm plugin

Speed up the execution of the tests by running them in parallel, grouping the tests by suites and executing any suite by a different job. A second level of parallelisms is splitting the execution of the jobs by different Jenkins slaves.

• In this case the Jenkins slaves will connect to the Jenkins master by the Swarm plugin of Jenkins:

RUN wget --no-check-certificate --directory-prefix=${SWARM_HOME} \
      http://maven.jenkins-ci.org/content/repositories/releases/org/jenkins-ci/plugins/swarm-client/${SWARM_VERSION}/swarm-client-${SWARM_VERSION}-jar-with-dependencies.jar  && \
    mv ${SWARM_HOME}/swarm-client-${SWARM_VERSION}-jar-with-dependencies.jar ${SWARM_HOME}/swarm-client-jar-with-dependencies.jar && \
    mkdir -p ${SWARM_WORKDIR} && \
    mkdir -p ${SWARM_WORKDIR}/workspace/{TEST_PROJECT} && \
    chown -R jenkins:jenkins ${SWARM_HOME} && \
    chown -R jenkins:jenkins ${SWARM_WORKDIR} && \
    chmod +x ${SWARM_HOME}/swarm-client-jar-with-dependencies.jar

Docker with the correct Firefox configuration

To get a custom profile with Firefox the best way is execute Firefox, change the configuration as needed and save the current profile for the next executions.

• Now I get Firefox running within a docker container and rendered by the local X Server, so in this way it can be easy configured:

docker run -it --privileged -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/s2o/tmp/a:/opt/jenkins s2obcn/jenkins-swarm bash

Sharing local IDE workspace with Jenkins Job workspace

• The test, to be executed, will be loaded from an SCM or shared from a local FS. The trick here is to share the workspace between all the jobs/Jenkins.

Naming conventions (Jenkins with Job DSL Plugin)

• The jobs to execute will have the same name of the suites to execute. 
• We can build/update the jobs in the local Jenkins with the plugin Job DSL.