When we started the Tripolis Dialogue project years ago, we directly created a continuous integration environment with a build server periodically running tests. For a long time, the missing part has been the deployment to test servers. Test server deployments had to be done manually, which was time consuming and error prone. It also meant, that testers did not always have the most current version directly available, leading to a waterfall style of development and testing. It was clear, that we really had to shorten the development/testing cycle and make it easier to support multiple versions of our application.
Part of the technical challenge was, that we had to deploy 5 web-applications (running on Resin as application server) and 4 daemon processes (all Java).
We found a working solution in a combination of a custom Ant plugin for Maven, Maven as build tool, shell scripts and Luntbuild. It took us quiet a while to figure it all out and get it right, but now we are deploying multiple versions each night and can also trigger deployments manually. In this post I am going to show how we solved it.
1. Ant Plugin for Maven
We were not able to find existing Maven plugins for the tasks of copying, unpacking and remotely managing applications, so we decided to write our own Ant Plugin for Maven, which allowed us to use tasks like sshexec and scp.
The plugin basically only needs 3 files: an ant build script, a mapping document and of course Maven’s POM. The image below shows the Eclipse project for the plugin.
The ant script rollout.build.xml performs tasks like SSH commands and SCP and accepts various parameters, which allow us to use it in a very flexible way. In the code example below, you can see two targets, one for the web applications and one for the daemon processes.
In order to use the ant script, a mapping document has to be created to tell Maven how to use our plugin.
Because we use SSH/SCP, ant-jsch and jsch have to be added as dependencies to the pom.xml of the plugin. Here we also specified the more user-friendly prefix ‘rollout’, so that we can reference our new plugin with rollout:daemon and rollout:webapp. Don’t forget to update settings.xml when using the prefix.
If you want to to execute the script automatically as part of one of the Maven lifecycles, you can add the plugin to the pom.xml of your project as described in the Guide to Developing Ant Plugins. Originally we used the ‘site’ lifecycle to create the project sites and deploy the application as well but that slowed the deployment process down too much. Sometime you want to be able to release a new version as fast as possible.
2. Shell scripts for starting/stopping applications
On the test server we are using standard shell scripts to manage the applications. Just as the ant script, these shell scripts accept various parameters, so we can use them for multiple deployments.
In Luntbuild – an open source build automation and management tool – we bring the loose ends together and configure the dynamic parameters for the build and deployment process. The project/scheduler concept of Luntbuild combined with a series of builders make it easy to handle multiple versions of our software.
For the nightly deployment on trunk we use a schedule (screenshot below), which first builds the project (clean install) and then post-build process will setup a database on the test server and deploy all 9 applications.
The builder then uses Maven to execute the deployment goal of our plugin and provides the necessary parameters. As you can see, it is possible to pass parameters from the schedule to the builder and this way it is possible to re-use the builder in many ways. Unfortunately it is not possible to share builders between projects in Luntbuild.
With this workflow we were able to achieve our goal. Deployment is now part of our continuous integration system. Every night each release which is in development is automatically being deployed to our test servers and if necessary we can manually start deployments during the day. All of this saved a lot of time and trouble. It is a very flexible and expandable solutions but the flexibility also added some complexity.
What is missing now are the setup and configuration steps for the test systems and databases. At the moment there are still some manual steps necessary to get a new deployment environment ready.