Versioning Software Builds Based on Git Tags and Semantic Versioning (SemVer)

Let's start this post by setting the stage first and then move onto the problem. When a build is kicked off for your application/library/etc. on a CI (continuous integration) system like Travis CI or AppVeyor, you are most probably flowing a version number for that build no matter what type of tech stack you use. This is mostly to relate the artifacts, which the build will produce (e.g. Docker images, NuGet packages, .NET assemblies, etc.), with a particular context. This is really useful to be able to communicate and correlate stuff. A few scenarios:

• Hey Mark, please take a look at foobar-1.2.3-rc.657 from our CI Docker registry. That has the issue I have mentioned. You can check it on that image.
• Ow, barfoo-2.2.3-beta.362 NuGet package content misses a few assemblies that should have been there. Let's go back to build logs for this and check what went wrong.

Convinced? Good :) Otherwise, you won't find the rest of the article useful.

The other case is to flow a version number when you actually want to produce a release for your defined environments (e.g. acceptance, staging, production). In this case, you usually don't want to give an arbitrary version to your artifacts because the version will carry the high level information about the changes. There are three important intentions you can give here:

• I am releasing something which has no behavior changes
• I am releasing a new feature which doesn't break my existing consumers
• Dude, brace yourself! I will break the World into half!

You can see Semantic Versioning 2.0.0 for more information about this.

So, what happens here is that we want to let the CI system decide on the version at some cases and take control over which version number to flow in some other cases. Actually, the first statement is not quite correct because you still want to have partial control over what version number to flow for your non-release builds. Here is an example case to highlight what I mean:

• You started developing your application and shipped version 1.0.0.
• Your CI system started flowing prerelease version based on 1.0.0 and also attached the build number to that version (e.g. 1.0.0-beta.54). Notice that it's wrong at this stage because you already shipped v1.0.0. So, it should really be something like 1.0.1-beta-54.
• Now, you are shipping version 1.1.0 as you introduced a new feature.
• After that change, you keep building the software and CI system keeps flowing version 1.1.0 based versions. This is a bit bad as you now don't have the chronological order and version order correlation.

So, what we want here is to assign a version based on the latest release version, which means that you want to have control over this process of assigning a version number. I have seen people having a text file inside the repository to hold the latest release version but that's a bit manual. I assume you kick a release somewhere and you already assign a version at that stage for releases. So, wouldn't it be bad to leverage this?

So, you probably understood my problems here :) Now, let me introduce a few key pieces which will play a role to solve this problem and then later, I will move onto the actual implementation to solve the problem.

Git Tags

Tagging is a feature of Git which allows you to mark specific points in repository's history. As the Git manual also states, people typically use this functionality to mark release points. This is super convenient for our needs here and gets two important things sorted for us:

• A kick-off point for releases. Ultimately, release process will be kicked off when you tag a repository and push that tag to your remote.
• Deciding the base version based on the latest release version.

Semantic Versioning

So, we have the tags. However, it doesn't mean that every tag is a valid version and you can also use Git's tagging feature for some other purposes. This is where SemVer comes into picture and you can safely assume that any tag which is a valid SemVer is for a release. This makes your life so much easier as you can rely on built-in tools like node-semver to help you out (as we will see shortly).

The other thing we have in the mix is to be able to increment the build version after a release. For example, we release version 2.5.6. The next build right after the release should have the version number bigger than 2.5.6. Seems easy as you can just increment the patch version, right? No! 2.5.6-beta is also a valid SemVer. We can go further with 2.5.6-beta.5+736287 which is also a valid SemVer. So, there is a pre-defined spec here and we can again leverage tools like node-semver to work with this domain nicely.

Solution and Bash Implementation

OK, all this information is super useful but how to make it work? Let me walk you through a solution I have introduced recently on a few of the projects I am working on. It's very trivial but that useful at the same time. However, keep in mind that there might be a few things I might have missed as I have been applying this not for a long time. In fact, here might even be better techniques on this that you know. If so, please comment here. I would love to hear them!

I want to example this in two stages and bring them together at the end.

Deciding on a Base Version

When the build is kicked off, one of the first things to do is to decide a base version. This is fairly trivial and here is the flow chart to describe this decision making process:

Here is how the implementation looks like in Bash:

#!/bin/bash

baseVersion=0.0.0-0
if semver "ignorethis" $(git tag -l) &>/dev/null
then
    baseVersion=$(semver $((semver $(git tag -l)) | tail -n1) -i prerelease)
fi

Keep in mind that I am fairly new to Bash. So, there might be wrong/bad usages here.

To explain what happens here with a bit more details:

• We get all the tags for the repository as a list by running git tag -l
• We pass this list to semver command-line tool to filter the invalid SemVer strings. Notice that there is another parameter we pass to semver here called "ignorethis". It's just there to cover cases when there is no tag so that semver command-line tool can return non-zero exit code.
• If semver command-line tool exits with 0, we know that there is at least one tag which is a valid SemVer. So, we run tail -n1 on the semver output to retrieve the latest version and we increment it on its prerelease identifier. This is now our base version.
• If there are no valid SemVer tags on the repository, we set 0.0.0-0 as the base version.

Decide on a Build Version

Now we have a base version and we now need to decide on a build version based on that. This is a bit more involved but again, very trivial to implement. Here is another flow chart to describe this decision making process:

And, here is how the implementation looks like in Bash (specific to Travis CI as it uses Travis CI specific environment variables):

if [ -z "$TRAVIS_TAG" ];
then
    if [ -z "$TRAVIS_BRANCH" ];
    then
        # can add the build metadata to indicate this is pull request build
        echo export PROJECT_BUILD_VERSION="$baseVersion.$TRAVIS_BUILD_NUMBER";
    else
        # can add the build metadata to indicate this is a branch build
        echo export PROJECT_BUILD_VERSION="$baseVersion.$TRAVIS_BUILD_NUMBER";
    fi
else 
    if ! semver $TRAVIS_TAG &>/dev/null
    then
        # can add the build metadata to indicate this is a tag build which is not a SemVer
        echo export PROJECT_BUILD_VERSION="$baseVersion.$TRAVIS_BUILD_NUMBER";
    else
        echo export PROJECT_BUILD_VERSION=$(semver $TRAVIS_TAG);
    fi 
fi

Notice that I am echoing commands rather than directly calling them. This is because of a fact that Travis CI doesn't flow the exports which happens inside a script file. Maybe it does but I was not able to get it working. Anyways, I am calling this script inside my .travis.yml file by evaluating the output like this: eval $(./scripts/set-build-version.sh)

I am not going to separately explain how this works as the flow chart is very easy to grasp (also the Bash script). However, one thing which is worth mentioning is the branch check. After we check if the build is for a branch, we do the same operation no matter what. This is OK for my use case but you can add special metadata to your version in order to indicate which branch the build has happened or whether it was a pull request.

Conclusion

I find this solution very straight forward to pick the version of the build and have a central way of kicking of a release process. I applied this on AspNetCore.Identity.MongoDB project, a MongoDB data store adapter for ASP.NET Core identity. You can also see how I am setting the build version, how I am using it and how I am kicking off a release process.

To bring everything together, here is the entire script to set the build version:

#!/bin/bash

baseVersion=0.0.0-0
if semver "ignorethis" $(git tag -l) &>/dev/null
then
    baseVersion=$(semver $((semver $(git tag -l)) | tail -n1) -i prerelease)
fi

if [ -z "$TRAVIS_TAG" ];
then
    if [ -z "$TRAVIS_BRANCH" ];
    then
        # can add the build metadata to indicate this is pull request build
        echo export PROJECT_BUILD_VERSION="$baseVersion.$TRAVIS_BUILD_NUMBER";
    else
        # can add the build metadata to indicate this is a branch build
        echo export PROJECT_BUILD_VERSION="$baseVersion.$TRAVIS_BUILD_NUMBER";
    fi
else 
    if ! semver $TRAVIS_TAG &>/dev/null
    then
        # can add the build metadata to indicate this is a tag build which is not a SemVer
        echo export PROJECT_BUILD_VERSION="$baseVersion.$TRAVIS_BUILD_NUMBER";
    else
        echo export PROJECT_BUILD_VERSION=$(semver $TRAVIS_TAG);
    fi 
fi

I hope this will be useful to you in some way and as said, if you have a similar technique or a practice that you apply for this case, please share it. Now, go and enjoy this spectacular weekend ;)