Dependencies All the Way Down – Builds (2 of 4)

When we look at software builds, we’re looking, for the most part, at dependency problems. Traditional build scripting tools like Make or Ant are dependency languages. In order to do thing A, do B and C first, etc, etc.

The example below, in order to build the very small program “unique” main.o and strset.o need to be built. Those in turn rely on the input files main.c, strset.h and strset.c.

http://www.cs.princeton.edu/courses/archive/spr01/cs217/slides/5.make.pdf

Understanding this graph is important for performing an incremental build. Continue reading

For Audit, control the binaries

We include a built in package repository, CodeStation, with AnthillPro, uBuild and uDeploy. They also  integrate with third party repos.  We place much emphasis on this capability because it is critical for safety, governance and audit.

To explain this, let’s first look at a simplified build and release lifecycle:

 

Code, then build, then deploy to test environments, then deploy to production. Developers submit their work to a source control system. A build is generated from that source and any dependency libraries retrieved by a tool like Maven or CodeStation. That build is submitted to test environments and certified. Finally, it is sent to production.

The following questions are important to be able to answer:

  1. What is in production?
  2. Was it certified in test environments (and by whom)?
  3. How do you know your answers to #1 and #2 are true?

To be able to answer 1 and 2, you need an inventory of what version of something is in an environment or at least logging that indicates the version number. But to truly know that what is in production is what was tested, you have to ensure that not just are the file names the same, but that the exact same files were moved into each environment.

In order to know and prove that the files are the same, one must validate that they are bit-for-bit identical by comparing digital signatures or hashes at deployment time. It helps to actually have the original file around in a tamper resistant location. A good package repository, like the one in uDeploy, will provide that location, the automatic signature generation at build time, and the automatic verification so that you know that what is in production is exactly what was built from known source, and tested in the prior environments.

For more information on package repositories, view our on-demand best practices lesson: 'The Role of Binary Repositories in SCM'

Integration Update

A number of new and updated integrations have rolled out recently, here’s the run-down.

AnthillPro – Jira 4.x

The folks at Atlassian changed up their web services schema. The latest plugin for Jira supports the new Jira web services. You can get it here. If you’re still using the legacy integration rather than the plugin, migrating to Jira 4 is a good time to switch over.

TFS 2012

We’re starting to see some adoption of Microsoft Team Foundation Server 2012. Updated integrations for it are now available. They’ll be shipping in uDeploy 4.8. AnthillPro and uBuild integrations are around the corner as well. If you need this integration before the next release, contact support for a patch.

Tomcat

The uDeploy integration for Apache Tomcat has been updated with support for moving over a specialized context.xml with a war file deployment.

uDeploy – Jenkins

This has been available for a little while now, but we should mention that the Jenkins plugin for uDeploy has a couple of enhancements. First, it now supports Jenkins slave builds better. Second, it can be configured to automatically trigger a deployment of the new version in uDeploy.

uDeploy / ServiceNow

This integration was expanded to cover ServiceNow’s CMDB offering.

What’s Next?

Tell us. When it comes to integrations, we’re all ears. Ok, ears and nimble coding fingers.

Build Farm throttling by user

An interesting question came across the AnthillPro mailing list a few days ago: How do you put in place a quote that limits how many builds and tests a single developer can currently run in our build farm? When builds/tests are somewhat costly, it’s reasonable to want to keep individuals from monopolizing a shared pool of machines.

In AnthillPro, the ideal way of restricting access to some resource is with a lock. Usually, a lock represents a shared resource like a database, or network deployer that can only be used by one (or a handful) of processes at a time. In this case, what was needed was a restriction to “my current processes”.

What ended up working was creating an AnthillPro “Lockable Resource” per user. Each user could then be assigned an individual maximum number of current workflows to execute. The build and test workflows are then assigned a scripted resource lookup:

 

 

 

 

The script would be something like:

return BuildRequestLookup.getCurrent().getRequesterName() + "-quota";

 

Continuous Integration Took Automation from Invention to Innovation

While I can not take the credit for coming up with Continuous Integration (it can be traced back to timeless practices) or for even coining the term, I can take the credit for creating one of the first Open Source (later turned commercial) CI servers — Anthill, released in July of 2001. That has given me an almost unique vantage point for watching the evolution of CI and seeing it take the industry by storm. One of the most interesting phenomenon I have come across was seeing our customers use AnthillPro (back when it had build only functionality) to automate deployments back in 2004. Keep in mind that this was back in the days before the term Continuous Delivery was coined. At first it seemed to me that the wrong tool was being used for the job. But then I realized that the features present in any implementation of CI really advanced Automation from the invention stage to the innovation stage. Continue reading

Customer Success Story: A DevOps Team by Any Other Name

Provides Just as Much Value

I recently visited a customer and was surprised to learn just how sophisticated their IT operation is. They have a private cloud for Dev and Test and automated deployments (courtesy of AnthillPro) across all environments. The development and QA teams may request an environment for a specified period of time and then deploy their build to this environment for testing. This entire system, the environment provisioning and the application deployment is fully automated and turns what used to be a multi-week process into a one that literally takes minutes. BTW, these environments are non trivial as they are made up of multiple virtual machines along with network configuration, firewall rules, load balancing, edge caching, and more. Continue reading