Automated Tests with Jenkins
I have always been a big fan of automation and having technology run things for me so that I have time to focus on the cool things. Also, in a lot of ways, computers can do these tasks faster and better than me.
Unlike God in the Book of Genesis, we can't simply say "Let there be automation" and have a fully automated work flow and see that it is good. We have to work on it one piece at a time.
This post aims to give a solution to one of those small pieces. Automatically triggering PHPUnit tests on a Git Repository using Jenkins.
The solution I propose here is probably a little rough around the edges. But I think it is a good start and can worked on later.
I will be using a previous test project that has PHPUnit tests set up and is already on Git (you can read about how I did that here.
Installing Jenkins
The first step is to install the actual software. I am installing on an Ubuntu machine and it is actually very straightforward.
I will be honest, I am not 100% sure what these commands do. I just know that the end result is Jenkins being installed.
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
echo deb http://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list
sudo apt-get update
sudo apt-get install jenkins
You can then check it is running (and start and stop it) like you would any other service:
sudo service jenkins status
If all is working, you should be able to go to your server's address on a browser and to port 8080 and see a screen like this:
As it says, you will need to go onto the server and open
/var/lib/jenkins/secrets/initialAdminPassword which contains the initial root password (you will be able to change this later).
Once you have entered the password, you will be invited to install Jenkins with certain plugins:
The most commonly used ones will be installed if you use left setting or you can select which ones to install. If you are not sure yet, use the suggested plugins and you can install plugins later.
The one important plugin that we need is the Github one which was already included in its "suggested" installation option.
Next, you are invited to create the first admin user. This will replace the password that you used earlier:
And that is it! You are presented with the final screen:
I didn't really encounter any surprises or issues following these instructions.
Setting up the Environment
There is a little more setup required for what I want to do. Everyone has different requirements depending on what they want to use Jenkins for.
But the way that I have things set up, Jenkins will run PHPUnit on the same machine. This means that I need to have the environment setup for my project to run the tests.
Luckily, my test project is very simple so I just needed to install PHP7.0, PHP7.0-xml and PHPUnit itself.
One other thing that we will need to do is give our server access to the Github project using SSH keys. You can do this by using the ssh-keygend command and then going onto the project settings on Github and adding the public key.
I think it is possible to use a username and password instead but this way is obviously more secure.
Project Configuration
Now we are ready to create the first project. In Jenkins, projects are called 'jobs' and each job can have one or more steps that lead to a 'build'.
On the home page, you can see the link to create a new job:
Jenkins then gives a bunch of pre configured job types:
I selected the first type, "Freestyle project".
You will see a whole bunch of fields for you to fill out including the name and description.
The interesting one is under the Source Code Management:
Here, I have selected "Git" and given the URL to the repository. I also have to select credentials to use to access it. Since I do not have any yet, I use the Add button to bring up the Jenkins credential creator.
This allows me to create some credentials that can either be used on this Job alone, or are available for other jobs on Jenkins.
The kind that I have used is "SSH Username with Private key", this requires me to give the username and SSH key that I created earlier. I can either point it to the file location or input the key directly.
Once I have added this credential, I am brought back to the job setup where I can then select it from the dropdown list.
Finally, I select the branch that I want to use:
By default, it will use the Master branch and I have left it at this.
Scrolling down you will see the Build section. This is the place where you provide a list of instructions for Jenkins to 'Build' a project. It is up to you how you define a project as being 'built'. In the case of this demo, it is after PHPUnit has successfully run and the tests passed.
Select from the build step dropdown, "Execute shell". You can then give any command as if you were in the terminal inside the Git project.
My command is simple:
It is this simple because of the way Jenkins runs. It first clones the repository using the information we gave earlier and then runs the Build from within the root directory.
As my project has the default settings for PHPUnit, it is enough to simply run phpunit.
The exit code I have set is 1. What this setting means, is that if the shell command gives that code, it will mark the build as failed. In our case, we want to mark it as a failure if any of the unit tests fail. PHPUnit returns an exit code of '1' when any test fails, so that is what we use here.
When I save the job I am taken to the job page, where I can press a button to run a build:
As you can see, it has run correctly (it was my 3rd build by the time I took this screenshot). If you select it, you will be able to see the output from the build and any shell commands you run:
This is especially useful if it fails as you will be able to see the PHPUnit output which will tell you which test has failed.
Trigger Build on Push
This is good. But there is nothing particularly automated about this. I want the tests to run and build to be triggered when something new is pushed to the master branch automatically. You can see how this can be useful if you have a team of people committing code as you can be sure that the merged branch is "stable" automatically.
On the left hand side of the Build page, select "Configure" and you will be brought to the same page as earlier.
Under "Build Triggers", you have a bunch of different options including being done on a schedule:
We actually have to select 2 options. The GitHub hook trigger and the Poll SCM. This actually got me the first time I ran this, as I didn't realise I needed the Poll one too.
Selecting Poll will give you and box to say how often you want to Poll GitHub. Leave this blank as we are using the hook trigger. You can go ahead and save this new configuration.
What this will do, is listen to any requests pushed by GitHub rather than periodically checking which would be a waste of time.
Next we have to go onto the GitHub project settings and add a new webhook. This is will push an event to the specified URL each time the repository receives pushed code. The job on Jenkins was set up to watch the master branch only so only pushes to this branch will actually be acted on.
So on GitHub, you will see these settings:
The payload URL is the Jenkins URL and port with github-webhook on the end. Jenkins is clever enough to direct it to the relevant jobs.
The default option is to trigger with the push event, but you can select a whole bunch of other events to send and thus trigger your builds.
Once we save this webhook. That it is. The build will now run anytime something is pushed to the master branch. To demonstrate this, I recorded a little video of it in action:
One thing you can do that I didn't do in this example is setup what to do after a build is complete. What is particularly useful in this automated world, is the ability to send users an e-mail if a build fails. This way, one you have set Jenkins up, you can leave it to do its thing and it will contact you only if there is a problem.
Summary
Jenkins is incredibly useful and powerful for automating your workflow. I have used it very simply in this example but the options are endless.
Jenkins is only a tool that does the "automation" part. Just a thing that pushes the buttons of one or more other tools and reports to you how they run. In this example, that tool happened to be PHPUnit, but it could have been anything from deployment to syncing data.
My point is that using Jenkins is just one piece in the automated workflow story. The hard part is integrating the other tools themselves.