Having each developer (development team) configure their continuous integration tool’s build jobs is extra work for the team, leads to inconsistencies/mistakes, and the overall build system becomes a pain to manage. Martin Preston covers these issues in depth in his article, “Managing Build Jobs for Continuous Delivery” up on InfoQ.
At UrbanCode, we’re big believers in setting up a form that once filled out, creates a new build project. In AnthillPro, we setup generic processes that would create a new project in source control, and automatically create a build project that pointed at it.
When it came time to build our next generation continuous integration server, uBuild, we organized the tool around this concept. Templates are defined in the nice point and click UI, and new projects are built from templates. Developers “fill in the blanks” on the project details and the new project is available immediately. Developers don’t need to put in much effort, central administration teams get standardization, and the whole system ends up nice and lean.
When we look at software builds, we’re looking, for the most part, at dependency problems. Traditional build scripting tools like Make or Ant are dependency languages. In order to do thing A, do B and C first, etc, etc.
The example below, in order to build the very small program “unique” main.o and strset.o need to be built. Those in turn rely on the input files main.c, strset.h and strset.c.
Understanding this graph is important for performing an incremental build. Continue reading
We include a built in package repository, CodeStation, with AnthillPro, uBuild and uDeploy. They also integrate with third party repos. We place much emphasis on this capability because it is critical for safety, governance and audit.
To explain this, let’s first look at a simplified build and release lifecycle:
Developers submit their work to a source control system. A build is generated from that source and any dependency libraries retrieved by a tool like Maven or CodeStation. That build is submitted to test environments and certified. Finally, it is sent to production.
The following questions are important to be able to answer:
- What is in production?
- Was it certified in test environments (and by whom)?
- How do you know your answers to #1 and #2 are true?
To be able to answer 1 and 2, you need an inventory of what version of something is in an environment or at least logging that indicates the version number. But to truly know that what is in production is what was tested, you have to ensure that not just are the file names the same, but that the exact same files were moved into each environment.
In order to know and prove that the files are the same, one must validate that they are bit-for-bit identical by comparing digital signatures or hashes at deployment time. It helps to actually have the original file around in a tamper resistant location. A good package repository, like the one in uDeploy, will provide that location, the automatic signature generation at build time, and the automatic verification so that you know that what is in production is exactly what was built from known source, and tested in the prior environments.
For more information on package repositories, view our on-demand best practices lesson: 'The Role of Binary Repositories in SCM'
A number of new and updated integrations have rolled out recently, here’s the run-down.
AnthillPro – Jira 4.x
The folks at Atlassian changed up their web services schema. The latest plugin for Jira supports the new Jira web services. You can get it here. If you’re still using the legacy integration rather than the plugin, migrating to Jira 4 is a good time to switch over.
We’re starting to see some adoption of Microsoft Team Foundation Server 2012. Updated integrations for it are now available. They’ll be shipping in uDeploy 4.8. AnthillPro and uBuild integrations are around the corner as well. If you need this integration before the next release, contact support for a patch.
The uDeploy integration for Apache Tomcat has been updated with support for moving over a specialized context.xml with a war file deployment.
uDeploy – Jenkins
This has been available for a little while now, but we should mention that the Jenkins plugin for uDeploy has a couple of enhancements. First, it now supports Jenkins slave builds better. Second, it can be configured to automatically trigger a deployment of the new version in uDeploy.
uDeploy / ServiceNow
This integration was expanded to cover ServiceNow’s CMDB offering.
Tell us. When it comes to integrations, we’re all ears. Ok, ears and nimble coding fingers.
An interesting question came across the AnthillPro mailing list a few days ago: How do you put in place a quote that limits how many builds and tests a single developer can currently run in our build farm? When builds/tests are somewhat costly, it’s reasonable to want to keep individuals from monopolizing a shared pool of machines.
In AnthillPro, the ideal way of restricting access to some resource is with a lock. Usually, a lock represents a shared resource like a database, or network deployer that can only be used by one (or a handful) of processes at a time. In this case, what was needed was a restriction to “my current processes”.
What ended up working was creating an AnthillPro “Lockable Resource” per user. Each user could then be assigned an individual maximum number of current workflows to execute. The build and test workflows are then assigned a scripted resource lookup:
The script would be something like:
return BuildRequestLookup.getCurrent().getRequesterName() + "-quota";
While I can not take the credit for coming up with Continuous Integration (it can be traced back to timeless practices) or for even coining the term, I can take the credit for creating one of the first Open Source (later turned commercial) CI servers — Anthill, released in July of 2001. That has given me an almost unique vantage point for watching the evolution of CI and seeing it take the industry by storm. One of the most interesting phenomenon I have come across was seeing our customers use AnthillPro (back when it had build only functionality) to automate deployments back in 2004. Keep in mind that this was back in the days before the term Continuous Delivery was coined. At first it seemed to me that the wrong tool was being used for the job. But then I realized that the features present in any implementation of CI really advanced Automation from the invention stage to the innovation stage. Continue reading