Dependencies all the way down – Run Time & Deployment

This is the third part in our examination of dependencies in build and release.

In part two, we looked at how builds often have both source code and dependencies on libraries. A web application might pull in a third party library that provides XML parsing, and a home grown profanity filter.

At deployment time, we’re worried about a new set of dependencies. That web application probably depends on other components in order to function properly. There may be databases, web services, message buses, third party applications, or content that the application relies on in order to deliver correctly. Continue reading

“Application Release Automation” is a terrible term

The analysts agree, uDeploy and its competitors are “Application Release Automation” tools. We used the same term on our website.  Unfortunately, “Application Release Automation” is a bad description for this class of tools.

Let’s look at the three words in this term and see what fits:

  • Application: Not bad. This scopes things down to software applications rather than hardware or endangered animals. The Application also indicates that we’re working at larger scope than a single tier or component, which is also ok.
  • Release: To release an application, generally means to deploy it into the production environment. Certainly, production is in scope for these tools, but pre-prod environments are as well. Using the same process in prod and testing environments is a best practice. “Release” might be pointing to heavily towards production. In many companies a “Release” is scoped far larger than a single application or application group. A release can contain numerous unrelated applications. That’s not what ARA is about. 

    Similarly, we have shared terminology with release management. Is this tool automating a release managers job? Not really, while it can enforce some quality gates and provide better visibility it doesn’t have a detailed Release Management feature set. At UrbanCode, we even provide a specialized release management tool to focus on those topics.

  • Automation: Not bad. Most people look to a tool like this when they want to greatly improve the speed and quality of their deployments by automating as much as possible.

So the problem is with “Release”. It suggests that we should ignore earlier environments and implies more of a release management role than the tools really fill.

What would be better?

We propose Application Deployment Automation (ADA). Tools like uDeploy are automating the deployment of applications.

Sometimes its a deploying to production – a release. Sometimes its one of the many deployments to the test labs that get us ready for the eventual release. What a tool like uDeploy does extremely well is let you define a standard process that is used in both environments.

Like this sort of thing? Here are other blog posts where I nitpick about jargon:

For Audit, control the binaries

We include a built in package repository, CodeStation, with AnthillPro, uBuild and uDeploy. They also  integrate with third party repos.  We place much emphasis on this capability because it is critical for safety, governance and audit.

To explain this, let’s first look at a simplified build and release lifecycle:

 

Code, then build, then deploy to test environments, then deploy to production. Developers submit their work to a source control system. A build is generated from that source and any dependency libraries retrieved by a tool like Maven or CodeStation. That build is submitted to test environments and certified. Finally, it is sent to production.

The following questions are important to be able to answer:

  1. What is in production?
  2. Was it certified in test environments (and by whom)?
  3. How do you know your answers to #1 and #2 are true?

To be able to answer 1 and 2, you need an inventory of what version of something is in an environment or at least logging that indicates the version number. But to truly know that what is in production is what was tested, you have to ensure that not just are the file names the same, but that the exact same files were moved into each environment.

In order to know and prove that the files are the same, one must validate that they are bit-for-bit identical by comparing digital signatures or hashes at deployment time. It helps to actually have the original file around in a tamper resistant location. A good package repository, like the one in uDeploy, will provide that location, the automatic signature generation at build time, and the automatic verification so that you know that what is in production is exactly what was built from known source, and tested in the prior environments.

For more information on package repositories, view our on-demand best practices lesson: 'The Role of Binary Repositories in SCM'

Deploying Packages not Components

Those who attended the DevOps: IT’s Automation Revolution webcast last week (recording here), won’t be surprised by my favorite slide from Glenn O’Donnell’s portion of the presentation. He discussed the challenge of moving a complex application through environments, and the likelihood of “droping the ball” and missing a piece when you go to production.

To address that challenge, Glenn suggested that you create a package containing all the components and make that the granularity for deployment. With all the components tied together, you can’t forget a piece or deploy versions that haven’t been tested together to production.

A "Package" ties versions of various components into a version of the application.

At UrbanCode, we love this approach. After a few years of helping AnthillPro customers approximate this with the dependency subsystem, we built uDeploy around the concept of an application model, and a Snapshot Deployment. The Snapshot in uDeploy is a virtual package, that ties together versions of all the components, without creating a single monolithic binary.  It was exciting to see a great analyst explain the rationale for this approach so clearly.

Want to learn more? Check out uDeploy or let us know that you’d like a private demo for your team.

Release Snapshots: Critical for release execution and audit

When I work with customers who have even moderately complex deployments, they rarely deploy just a single build at a time. Usually,  a collection of builds, updates and configuration is released in some coordinated fashion. Release manifests help with that coordination.

Release Manifests

A release manifest contains the collection of versioned stuff that is being deployed, configuration settings, and approvals. What, how, where, and who approved it. This is similar to a shipping form listing out the boxes sent, value and contents of the goods, destination, and signatures from sender and receiver. Release manifests are also called deployment manifests or Snapshots.

At execution time, the manifest tells us what to do and ideally how to do it. For instance, we should deploy the database, and two web services, and a mobile front end for our application. The manifest coordinates our deployments so that we no longer have to do it by hand.

Smart Deployments

In a tool like uDeploy the manifest (called Snapshot) describes the full desired state for a target environment. When deployed, only the parts that changed are actually installed. Parts of the system that are already at the right version are left alone. This results in a process that works equally well to make a handful of updates in a frequently changed test environment as it does to update many pieces in production.

Promote what was Tested

A basic idea in most SDLC and Service Transition processes is to test an application and promote what was tested to higher environments and then production. As our software has become more modular, this has become more difficult. We realize that our tests did not execute again “a build”. Rather, they tested a number of inter-related components, configuration and infrastructure. Continue reading

How experiment cultures lead to continuous deployment

In past webinars we’ve talked about Lean startups and what the enterprise can learn from them. We’ve pointed to these guys as examples at the extreme doing crazy things like deploying an app to production 50 times a day.

The standard explanation for lean startups movings towards continuous deployment is that they walked a path of removing wastes. Automated deployments were effective in removing errors and wasted effort. The endeavored to speed features to production because features written but not delivered to the customer is seen as wasteful “inventory”.

This is no doubt at least partly true. But when you read discussions of the practice there’s generally a heavy emphasis on monitoring deployments and rolling them back if Continue reading