Dependencies All the Way Down – Builds (2 of 4)

When we look at software builds, we’re looking, for the most part, at dependency problems. Traditional build scripting tools like Make or Ant are dependency languages. In order to do thing A, do B and C first, etc, etc.

The example below, in order to build the very small program “unique” main.o and strset.o need to be built. Those in turn rely on the input files main.c, strset.h and strset.c.

http://www.cs.princeton.edu/courses/archive/spr01/cs217/slides/5.make.pdf

Understanding this graph is important for performing an incremental build. Continue reading

For Audit, control the binaries

We include a built in package repository, CodeStation, with AnthillPro, uBuild and uDeploy. They also  integrate with third party repos.  We place much emphasis on this capability because it is critical for safety, governance and audit.

To explain this, let’s first look at a simplified build and release lifecycle:

 

Code, then build, then deploy to test environments, then deploy to production. Developers submit their work to a source control system. A build is generated from that source and any dependency libraries retrieved by a tool like Maven or CodeStation. That build is submitted to test environments and certified. Finally, it is sent to production.

The following questions are important to be able to answer:

  1. What is in production?
  2. Was it certified in test environments (and by whom)?
  3. How do you know your answers to #1 and #2 are true?

To be able to answer 1 and 2, you need an inventory of what version of something is in an environment or at least logging that indicates the version number. But to truly know that what is in production is what was tested, you have to ensure that not just are the file names the same, but that the exact same files were moved into each environment.

In order to know and prove that the files are the same, one must validate that they are bit-for-bit identical by comparing digital signatures or hashes at deployment time. It helps to actually have the original file around in a tamper resistant location. A good package repository, like the one in uDeploy, will provide that location, the automatic signature generation at build time, and the automatic verification so that you know that what is in production is exactly what was built from known source, and tested in the prior environments.

For more information on package repositories, view our on-demand best practices lesson: 'The Role of Binary Repositories in SCM'

Maven’s Strengths and Weaknesses as a Dependency Management System

For many Java teams, switching to Maven will introduce them to formal dependency management. Maven actually does a pretty decent job and is a fantastic system out of the box for informal projects. However, for teams looking to implement rigorous component reuse policies, Maven falls short.

Last month, in our article Chaperoning Promiscuous Software Reuse, we reviewed four elements of a successful strategy for managing components. Briefly, these elements are:

  1. Testing and Validation of 3rd party components Only libraries approved for reuse should be used by developers. If developers want to use additional libraries, they can submit them for screening.
  2. A Definitive Software Library: A team should have an official storage place for reusable, versioned components (both internal and 3rd party).
  3. Using Protection: A team should have an “immune system” that rejects libraries other than those from the DSL.
  4. Rapid impact analysis audit When something goes wrong, it should be easy to determine what versions of which components were used by the project. Continue reading

Request Contexts

Adventures in Advanced AnthillPro Concepts

The idea of a request context is foreign to most AnthillPro users and administrators but is key to understanding how AnthillPro manages build order and consistency along a dependency hierarchy. Administrators using dependencies aggressively would be well served by understanding the request context.

Understanding “Requests”

Before we get into the request context, we should understand the idea of a build request. Immediately after the user presses the Build button, AnthillPro creates a build request. This starts the process towards creating the build, but doesn’t always immediately result in a build. For instance, if the “Force” box isn’t checked, AnthillPro might start by asking source control if there are changes. If there aren’t any changes in source control since the last build, and none of the project’s dependencies have changed the request will terminate having decided that a build is unneeded.  Continue reading

2 Strategies for Managing Run-Time dependencies

Many software projects have dependencies at run time on other projects that are built and deployed by the software team. Some version of another application must be present in order to run properly.
We see this situation appearing most often with Web Services,  J2EE environments where EJB clients and applications must match, and even in the roll-out of related database schemas.

The basic problem these scenarios face, is that deployments must be orchestrated so that any related projects move through the life-cycle at the same time. The strategies for addressing this in AnthillPro are pretty straight-forward.

First create a new AnthillPro project that uses dependency rules to represent the full set of components that get released together. A “Build” of this project is mostly responsible for assembling a set of components that will be deployed together. At this point there are two main paths to follow: Continue reading