Automating test scenarios have become an integral part of any successful product release. We have selenium automation suite covering important feature flows. We run our test suite on a weekly basis as a regression health indicator. We deployed Jenkins, a CI/CD tool, to build, deploy and automate our test suite. In this article, we will explain the steps to set up
Jenkins on our
Ubuntu machine and how we overcame the various challenges we faced.
We collaborate with a USA based large CRM SaaS application to build new features. We also enhance the application. We use Kanban Agile methodology in CRM projects as soon as user stories are ready to work. This process requires lots of deployments every week (viz. features, fixes, enhancements, support requests, etc). Quality is as important as speed to ship stories. For repetitive testing, having automation is paramount. At Kiprosh, we write integration tests with RSpec and Capybara. On every commit in
Github, our CI/CD pipelines run the tests through automatic process. Selenium helps us with our functional and regression testing.
We get a new build deployed to QA every week with user stories, enhancements and bug fixes which are development complete. So we had to run our regression suite very often, manually. We wanted to automate this process. Thus, to run our suite automatically very often, we decided to use
Jenkins as a CI/CD tool
Jenkins installation process was very simple. For
Jenkins installation, we referred link Jenkins Installation On Ubuntu.
Maven project, one needs to download a
Maven Plugin in
Jenkins. And it took us sometime to understand this basic necessity.
Configuring it Right!
Maven Plugin installation, it needed a
Jenkins restart to be able to import the project. Now the next logical step for us was to set Git in Source Code Management.. and hoped all was set to run the project but we got an error
Failed to connect to repository. On further investigation, we discovered there were few more configurations required. An important thing that we learnt is, we need to have the same environment setup in
Jenkins as we have on our local. Set
path to git executable to
Global Tool Configuration under
Manage Jenkins option. A couple of more failed attempts taught us to add
MAVEN_HOME path as well.
Despite the setup being ready,
Jenkins failed to connect to the repository with the error saying
- Command "git ls-remote -h firstname.lastname@example.org: HEAD" returned status code 128: stdout: stderr: Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. And updating our
git credentials, we were able to see our code run...
Adding Ubuntu to the setup process
But within a second it failed throwing an exception
unknown error: Chrome failed to start: exited abnormally. After analyzing the failure, we learned a new thing about setting up
Ubuntu machine - Xvfb!
At the beginning this sounded alien but it's short form of
X virtual framebuffer in
Xvfb set the correct screen resolution. With the help of
Xvfb, selenium is able to identify and manipulate the objects on the screen. Our investigation revealed that
Xvfb is pre-installed. It's a matter of configuring it from
Manage Plugins, and with some trial and error, we got the correct screen size.
We hoped for a smooth run after resolving the above errors. But again ran into errors where the test annotations were not recognized.
Harnessing the Power of POM!
pom.xml file to run the maven project. The
pom.xml will run all the tests in your suite if specific tests to run are not mentioned. Since its a test automation run only, we had some setup classes which need not run each time. We wanted to run only tests specified in
testSuite.xml file. This file contains functional flows for the application. To make this happen we found out about Maven surefire plugin. The link Maven Surefire Plugin says,
The Surefire Plugin can be used during the test phase of the build lifecycle to execute the unit tests of an application. This plugin helped us define the XML file picked up by
pom.xml for execution.
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.12.2</version> <configuration> <suiteXmlFiles> <suiteXmlFile>testSuite.xml</suiteXmlFile> </suiteXmlFiles> </configuration> </plugin> </plugins> </build>
Now we thought we were very close to accomplishing our dream... and one of our maiden runs gave us a new exception aborting the build.
The exception was for
Maven Compilation error [package org.testng.annotations does not exist]
scope is set to
test in the
testng dependency under
pom.xml file. On investigation, we found out that we had to set the
compile rather than
<dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.14.2</version> <scope>compile</scope> </dependency>
With this last change, our setup of the CI/CD pipeline with Jenkins was finally ready. The joy of automation and red/green report is immense.