Improve your Local Env with Docker+Jenkins+Selenium to achieve Continuous Delirery

The path to Continuous Delivery (CD) happens to have stable execution results of automated test, so will be ideal if the developers could pass the automated test on their local machine before they integrate their code with the base code. But  if we want to execute the automated test on the developer local machine we have to be aware of:

1. The executions of the tests must be not intrusive, allowing the developers to focus on other tasks while the tests are executing.
2. The solution must be easily adaptable to changes, for example, if new tests suites are available or disabled,  this doesn't have to be an overhead work for the developers to synchronize with their local environment.

Sometimes these points are difficult to accomplish, i.e. if we have more than 500 automated functional test to be tested with a real browser (e.g. Firefox).  

Knowing the global solution explained in my previous post, it's easy with Docker and Jenkins to have a template that could build a "private platform" where execute 'locally'  all the  tests (unit, integrated, functional...)

You can see the simplified solution in the next diagram:

As you can see on the diagram, the solution is based on Docker compose which builds Jenkins in a Docker container: a Jenkins Master and n-slaves Jenkins  (depending on configuration parameters). It will execute the test with Selenium/Firefox, where the test live in the local filesystem of the host. In this case, for convenience and not to overload the developers' machines, we won't test on IE  (but we could use a Vagrant virtual-machine with Windows and IE)...

The key points of this kind of configuration are:

• Run Automated Test in background.
• Light GUI containers to run Firefox with Selenium.
• Parallel execution of the tests. 
• Easy installation and updates.
• We can see the visual errors of the navigation in the pdfs generated by the framework of test (see previous post)

Some Technical details of the solution

Docker-Compose

With Docker-Compose we have a simplified configuration for the developer:

# Jenkins Master
jenkins:
  image: s2obcn/jenkins
  container_name: jenkins
  ports:
   - "8090:8080"
   - "50000"
  volumes:
        - /home/s2o/vDocker/jenkins:/jenkins
  env_file:
    - jenkins-master.env
# Jenkins Slave
slave:
  image: s2obcn/jenkins-swarm
  links:
    - jenkins:jenkins
  volumes:
      - /home/s2o/vDocker/jenkins_shared/workspace:/opt/jenkins/workspace
  env_file:
    - jenkins-slave.env

Jenkins docker image with swarm plugin

Speed up the execution of the tests by running them in parallel, grouping the tests by suites and executing any suite by a different job. A second level of parallelisms is splitting the execution of the jobs by different Jenkins slaves.

• In this case the Jenkins slaves will connect to the Jenkins master by the Swarm plugin of Jenkins:

RUN wget --no-check-certificate --directory-prefix=${SWARM_HOME} \
      http://maven.jenkins-ci.org/content/repositories/releases/org/jenkins-ci/plugins/swarm-client/${SWARM_VERSION}/swarm-client-${SWARM_VERSION}-jar-with-dependencies.jar  && \
    mv ${SWARM_HOME}/swarm-client-${SWARM_VERSION}-jar-with-dependencies.jar ${SWARM_HOME}/swarm-client-jar-with-dependencies.jar && \
    mkdir -p ${SWARM_WORKDIR} && \
    mkdir -p ${SWARM_WORKDIR}/workspace/{TEST_PROJECT} && \
    chown -R jenkins:jenkins ${SWARM_HOME} && \
    chown -R jenkins:jenkins ${SWARM_WORKDIR} && \
    chmod +x ${SWARM_HOME}/swarm-client-jar-with-dependencies.jar

Docker with the correct Firefox configuration

To get a custom profile with Firefox the best way is execute Firefox, change the configuration as needed and save the current profile for the next executions.

• Now I get Firefox running within a docker container and rendered by the local X Server, so in this way it can be easy configured:

docker run -it --privileged -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/s2o/tmp/a:/opt/jenkins s2obcn/jenkins-swarm bash

Sharing local IDE workspace with Jenkins Job workspace

• The test, to be executed, will be loaded from an SCM or shared from a local FS. The trick here is to share the workspace between all the jobs/Jenkins.

Naming conventions (Jenkins with Job DSL Plugin)

• The jobs to execute will have the same name of the suites to execute. 
• We can build/update the jobs in the local Jenkins with the plugin Job DSL. 

Continuous integration: Functional Testing - Distributed Testing with Selenium and Jenkins

A way to distribute Selenium tests can be use Selenium Grid alone or Jenkins and the Selenium Jenkins plugin. However if more control is required, there's another way with Jenkins. You can configure a Jenkins Master with n-Jenkins Slaves that will execute the Selenium tests like local execution. In other words, Jenkins Slave will execute exactly as developers do in their local environment without any overhead configuration.

So, which are the pros to use this technique instead of Selenium Grid:

• Easy integration: it can be integrated seamlessly into the continuos integration (CI) of our product. 
• Distribution: the same infrastructure can be used to distribute the execution of ANY functional test (jMeter, SoupUI, Selenium...). 
• Dynamic: assignment different browsers capabilities test are more dynamic than with Selenium Grid. With Jenkins Master/Slave, the slave has the labels to conform different tests types that can be executed by any node. For example, the number of executors (instances of browsers) or node labels can be changed directly in Jenkins without restarting the node.
• Reporting: the execution and post execution (reports..) are distributed overall Jenkins nodes. 
• Centralized control: the distribution control is centralized in Jenkins.

Knowing that solution, we can implement this solution with:

• TestNg as core test framework:
• Group functional test in suites (that will be executed directly by the Jenkins jobs).
• Filter test execution by parameters ("@Test(groups='demoUser','pro'"... where we can have different kind of groups, such as different type of users, different environments... ) 
• Definition of executions  "timeouts" (at different levels).
• An easy way to execute the test (from maven, directly with java...)
• Data Driven Test
• Reports
• ...
• Test will be built following the Page Object Pattern. 
• A custom core framework that:
• Centralize configuration management of the test execution (main homepage, main browser type, user, password....).
• Centralize the startup of the browser (select browser by configuration properties)
• Automate the login of the test: every test will only take care of the page to test, navigation through the pages will managed by the core framework
• Execution retry policy (you know that test over IE may fail without reason)
• Manage errors 
• Data Driven Test: data stored in excels organized by 'tables'
• Custom detailed reports
• ...
• Testlink as Test Management.

This is a simplified picture of the platform to automate the execution of Selenium with Jenkins:

You can follow this tutorial to configure Jenkins with Selenium and this other tutorial to configure the Jenkins Slaves.

To sum up,  the main goal of this solution is to use the  Jenkins node "labels" to assign the web browser that will be executed by any node and configure the Jenkins Slaves to execute the Selenium WebDriver without problems (I will write a post about how to configure the Jenkins Slave  properly to execute IE as a service without problems).