Road to Automation: Release Notes

Automation is not a binary thing. We rarely go from manual to fully autonomous with one switch. Just like software development itself, it often requires iterative revisions, with some craftsmanship and never-enough imaginations.

A case I would love to share is our 8-month and counting journey we took to automate our release notes process.

Writing release notes for the software we proudly make isn’t always easy. Even some of the most popular applications don’t get the release notes they deserve (and please don’t learn from them).

Fortunately, we can also get inspiration from some great examples.

Having concise and coherent release notes sure are great, but they often require a lot of effort and time that most developers don’t have.

Thus we initiated a project to help ourselves to relatively effortlessly produce release notes or at least the materials that save time for human editorial.

We started at the source of software changes – commits (at the time of writing, this refers to git commits). We laid out some conventions to encourage good commit and commit message practices. The adaptation of these conventions alone allows our developers to think more on behalf of the end-users, something akin to the concept of having everyone bearing the responsibility of marketing. And for our objective of automation, this gives us a solid starting point due to having a predictable structure of our commit messages.

The next step was to prototype the core capabilities of parsing out commit messages and constructing a markdown formatted release notes. Given our existing practice of utilizing git tags for releases, we quickly made the first usable version.

The first version would require a configuration file to specify the target project git repository and which tags to compare. Both are apparent areas to improve on. So we progressed it to be a portable CLI with automatic tag discovery that works in any git repository (including release itself).

woozyking@runzhoudembp release % release notes --print --skip alpha
## Release Notes: from v1.4.0 to v1.5.0

### devops

* dogfood github release notes after npm publish (f5718f8 by Runzhou Li (woozyking))

### notes

* `--fetch` flag to control whether to run git fetch (28550c9 by Runzhou Li (woozyking))
* `--github` flag to update the head ref associated GitHub release (21d6278 by Runzhou Li (woozyking))
	* omit local file output
	* factor out `utils.exec` to perform `execSync` and `log`
	* requires `$GITHUB_TOKEN`, `$GITHUB_OWNER`, and `$GITHUB_REPO` env vars
	* `$GITHUB_REPO` falls back to local repo toplevel name

The portable CLI nature and its usage simplicity allowed the team to adopt the tool and organically formed a process to perform releases with this tool being a part of it:

1. Create a git tag (at the time of writing, we usually do this through GitHub releases).
2. Sync local repo to obtain the created git tag.
3. Run the release notes command to generate release notes.
4. Copy/paste the output into where we retain the release notes (again, usually the GitHub release from step 1).

The (manual) process highlighted opportunities for further automation. We introduced an option to directly update the GitHub release notes, which eradicated step 4 from above. Then we started integrating the release CLI as a part of the CI/CD (continuous integration/deployment) workflows of our growing number of projects, which eliminated the need for steps 2 and 3. And at the time of writing, we’re attempting to automate away step 1 as well.

At this point, we’d have an established adoption of the tool, with little to no accessibility inhibition and minimal human involvement to initiate a release. So after about six months of active usage, we took our next step into refining its core capabilities. We added an alternative output formatting that we adapted from Keep a Changelog, as well as an NLP (Natural Language Processing) model for labeling the nature of each commit message.

woozyking@runzhoudembp release % release changelog --print --skip alpha
## Changelog: from v2.1.1 to v3.0.0

### CHANGED

* version - v3.0.0 (b2156bd by Tamires)

### FIXED

* lib/parse-subject - fix t2 when no match by default it to 'others' (3593397 by Tamires Lowande)
* lib/baseHandler - cover single tag and no tag cases (281c4c0 by Runzhou Li (woozyking))

### ADDED

* nlp - add model training process as a jupyter notebook (1c57956 by Runzhou Li (woozyking))
* lib/parseCommits - update to the newly trained sub-1MB NLP model (d7c4a1e by Runzhou Li (woozyking))

We initially adopted the NLP model from a random but lucky encounter and then revised it using the same tool made available from Facebook Research. The model is far from perfect, and its training dataset is still lacking. We already have some ideas lined up to seek a better model automatically.

Today, for some software, such as libraries and tools intended for other developers, we think it’s almost sufficient to use the release tool we made to generate their release notes, such as you can see at release’s releases.

But the ones for our end-users, we would still take some time to grey out irrelevant technical details or summarize out the noteworthy highlights based on the generated release notes.

The still needed human involvements are perhaps the next targets to improve on, and the increasingly available and highly accessible tools at our disposal will only make it easier by the day.

The above was a piece of our journey, but if you can identify manual chores among any tasks you do, you may hit on this adventurous road of automation too. It shall be rewarding.