Dependencies All the Way Down – Builds (2 of 4)

When we look at software builds, we’re looking, for the most part, at dependency problems. Traditional build scripting tools like Make or Ant are dependency languages. In order to do thing A, do B and C first, etc, etc.

The example below, in order to build the very small program “unique” main.o and strset.o need to be built. Those in turn rely on the input files main.c, strset.h and strset.c.

http://www.cs.princeton.edu/courses/archive/spr01/cs217/slides/5.make.pdf

Understanding this graph is important for performing an incremental build. If only strset.c changes, main.o does not need to be rebuilt. Only strset.o and unique need to be built. The tendency in newer tools is to infer these file level dependencies, or work in a conventions based manner, like Maven does. But at the core, you still have these low level rules.

Because code level reuse is a nightmare, we have established patterns such as Robert Martin’s release-reuse equivilancy principal. These approaches, nudge us towards reusing code in the form of components or libraries. Many project reuse third party libraries for things like logging (log4j / log4net) as well as internally developed projects.

With modern build techniques, managing these library dependencies is a bit more challenging than managing our file by file dependencies. Checking the binaries into source control leads to challenges around updating tightly coupled libraries (sub-projects) as well as keeping track of versions of libraries being used. Instead, we turn to using package repositories such as Maven, NuGet or CodeStation (UrbanCode’s repository included inits build solutions). This requires definitive versions, clear rules for what is needed, and a bill of materials for what went into the build.

A Build Dependency Graph
For us, a file that changes in one of those low level libraries, can result in several hundred builds. 

Applying the build avoidance strategy we looked at above gets more interesting. It now spans a graph not of files, but of builds. Changes to a core library necessitate builds of other libraries/modules, which in turn cause projects and applications to be rebuilt. A fair amount of the complexity buried in the heart of uBuild (and AnthillPro) is managing these complex build graphs.

These cascading builds are critical for continuous integration. Simple CI will rebuild whenever the source code changes to detect quality problems introduced by developers on the project. Advanced CI will take that a step further and rebuild when dependencies change so that blame for quality issues can be pinpointed to libraries maintained by different teams when appropriate.

This is the face of modern build management. “The build” is actually a build of builds. Knowing what is in that package that is being passed to test, means knowing the source code for your project, and also the versioned libraries (and in turn their source code). And knowing what changed means tracking changes across a dependency graph.

I recently presented: 'Managing Build Time Dependencies.' If you want to learn more about this topics, you watch it now.

Part one – Dependencies All the Way Down

Eric Minick

Meet Eric Minick


Eric Minick is the Lead Consultant at UrbanCode