AI Commit Messages and Changelog Automation: Why Beats What

A generated commit message landed in my history reading: refactor: update component and translations. It was accurate. It was also worth nothing, because git show had already told me which two files changed and which lines moved. What no tool could tell me six weeks later was why the translations moved at all — that a merge script owns those files, and my hand edit had been silently reverted by the next run.

This post is about the gap that message sits in. I have wired commit-message generation into two repositories, kept the parts a script can verify, thrown away the parts the model invented, and hit one shell bug that silently deleted a word from a message git then committed without complaint. What follows is what survived, plus the changelog pipeline that hangs off it.

A diff already says what changed

State it plainly: a model handed nothing but git diff can only produce a compressed restatement of its own input. Everything in that summary is recoverable by the reader with git show, and recoverable more accurately, because the diff is the ground truth and the summary is a lossy paraphrase of it. Paying tokens to compress evidence the reader already holds is not automation. It is redundancy with a confidence problem attached.

The value of a commit message is the part that is not in the repository at all. Why this change and not the smaller one. Which of three plausible fixes was chosen, and what ruled the other two out. Whether an ugly conditional is a temporary workaround for a vendor bug or a permanent design decision. A future reader running git blame on that conditional wants exactly that sentence, and the diff cannot supply it, so neither can a generator that reads only the diff.

Conventional Commits is the half a script can check

Conventional Commits v1.0.0 splits a message into a part with a grammar and a part without one. The specification requires a type, allows an optional scope in parentheses, and reserves feat for a commit that adds a feature and fix for a commit that fixes a bug, mapping them to MINOR and PATCH releases under SemVer. A breaking change must be signalled either by an exclamation mark immediately before the colon or by an uppercase BREAKING CHANGE footer, and it triggers a MAJOR release whatever the type says. Footers are word tokens with a colon-space separator that use hyphens in place of whitespace, in the style of Acked-by.

#!/usr/bin/env bash
# .git/hooks/commit-msg — git passes the message file path as $1.
# This validates the HEADER only. Everything after the colon has no
# grammar, so there is nothing here that can judge whether it is TRUE.
set -euo pipefail

header=$(head -n 1 "$1")

# type(scope)!: description — the scope and the "!" are both optional per
# Conventional Commits v1.0.0, and the "!" must sit immediately before the
# colon. The 72-character ceiling is mine, not part of the spec.
pattern='^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\([a-z0-9./-]+\))?!?: .{1,72}$'

if [[ ! $header =~ $pattern ]]; then
  echo "commit-msg: header does not match Conventional Commits v1.0.0" >&2
  echo "  got: $header" >&2
  exit 1
fi

# The spec allows "!" with no footer. I do not: the marker announces that
# something breaks, and the sentence naming WHAT breaks is its whole value.
if [[ $header == *'!:'* ]] && ! grep -qE '^BREAKING[ -]CHANGE: ' "$1"; then
  echo "commit-msg: '!' is set but no BREAKING CHANGE footer explains it" >&2
  exit 1
fi

That is a contract, which means a hook can enforce it and a generator can be measured against it. Everything after the colon — the description — has no grammar at all, and no amount of schema validation tells you whether it is true. So I check the header only, and I check it locally, because a commit-msg hook that rejects a message before it enters history is far cheaper than a CI job that rejects it once the branch is pushed.

Where the why actually lives

The why is not missing. It is written down, just not in the diff. On a normal change it exists in four places before the commit does: the branch name, the issue that opened the work, the pull request description, and the review conversation that changed the approach halfway through. A generator that receives those can write a message worth reading. A generator that receives only git diff cannot, and no amount of prompt engineering repairs an input that does not contain the answer.

#!/usr/bin/env bash
# Assemble the why BEFORE prompting. The diff goes in last and only as file
# names — it is the one input the reader can already reproduce with git show.
set -euo pipefail

branch=$(git rev-parse --abbrev-ref HEAD)
issue=$(echo "$branch" | grep -oE '[0-9]+' | head -n 1 || true)
ctx=$(mktemp -t commit-ctx)

exec > "$ctx"

echo "## Branch"
echo "$branch"
echo

# The issue is the only input here written by the person who HAD the
# problem. Everything else in this file is already somebody's solution.
if [ -n "$issue" ]; then
  echo "## Issue $issue"
  gh issue view "$issue"
  echo
fi

# The review thread holds the approach that was REJECTED. That sentence
# exists nowhere else in the repository, and it is the one a bisect wants.
echo "## Review thread"
gh pr view --comments || true
echo

echo "## Files touched"
git diff --cached --name-status

So the interesting work is the assembly, not the prompt. I collect the context first, write it to a file, and hand the model that file, which also means I can read exactly what it was given whenever the output is wrong. git-cliff arrives at the same conclusion from the other end: it parses remote metadata such as pull request titles, numbers and authors, because commit subjects on their own were not enough to build a changelog from.

The shell ate my backticks and git said nothing

This one cost me a real message. I pasted a suggested commit into git commit -m with the surrounding double quotes intact, and the identifier wrapped in backticks vanished. POSIX is explicit about why: inside double quotes the dollar sign, the backquote and the backslash all retain their special meaning, so a backquoted word is command substitution and gets replaced by that command's standard output. The command did not exist, the output was empty, and the word became nothing at all.

What makes it dangerous is that git exits 0. The shell writes command not found to stderr, git receives a string one word shorter than the one you approved, and the commit lands. I only caught it because the stored subject had two consecutive spaces where the identifier used to be. Here is the exact before and after from my terminal, and the fix I now use for every message longer than a single line:

# Wrong: double quotes leave the backquoted word as command substitution.
$ git commit -m "fix(parser): handle empty `input` arrays"
zsh: command not found: input

$ git log -1 --pretty=%s
fix(parser): handle empty  arrays

# Two spaces where the word used to be. git exited 0. Nothing warned me.

# Right: a quoted heredoc. Quoting the delimiter switches off every
# expansion inside it, so backticks, $VERSION and ${braces} all survive.
$ cat > /tmp/msg.txt <<'MSG'
fix(parser): keep `input` when the array is empty

parseArgs dropped the key entirely, so callers saw undefined instead of an
empty array. Rejected the ${DEFAULT} fallback because it hides a bad payload
rather than surfacing it.

Refs: #412
MSG

$ git commit -F /tmp/msg.txt
# -F is mutually exclusive with -m, which is exactly the point: the message
# is now a file that the shell never re-parsed.

A changelog is only as good as its subjects

Changelog generators read tags and commit subjects. git-cliff builds changelog files by analysing git history and groups commits by conventional-commit type — feat, fix, docs — and that grouping is most of why the output is legible at all. Nothing anywhere in the pipeline improves a subject line. If half the commits between two tags say update component and translations, the release notes say that too, in a nicer font.

# The exact set of subjects that will become release notes. Read this
# BEFORE the tag — afterwards it is a published artefact.
git log "$(git describe --tags --abbrev=0)"..HEAD --pretty='%h %s'

# The subjects that will NOT group, because they carry no Conventional
# Commits type. Every line here lands in an "Other" bucket nobody opens.
git log "$(git describe --tags --abbrev=0)"..HEAD --pretty='%s' |
  grep -vE '^(feat|fix|docs|perf|refactor|revert)(\(.+\))?!?: '

# Breaking changes read from the FOOTER rather than the subject: the "!"
# marker and the footer are separate signals, and only the footer explains.
git log "$(git describe --tags --abbrev=0)"..HEAD \
  --grep='^BREAKING[ -]CHANGE:' --pretty='%h %s%n%b'

Which is why I read the input before tagging rather than the output after it. The commit range between the last tag and HEAD is the exact set of subjects that will be published, and checking it takes under a minute — including the ones that will not group, which land in an Other section nobody opens.

What I never let a model write

Three kinds of sentence stay human, and they happen to be the three a reader is most likely to depend on.

• The breaking-change footer. A generator can spot a changed signature. It cannot know that the old behaviour is load-bearing for one customer integration, and that fact is the entire content of the note.
• The revert explanation. A revert body saying this reverts commit abc1234 says nothing. What broke, where it was observed, and what must be true before the change goes back in are the reasons the revert exists.
• Anything a bisect will read at 2am. If git bisect drops somebody on your commit during an incident, your body text is the handover note. Write it for that reader, not for the changelog.

The split is not about capability, it is about accountability. A header a hook can verify is safe to generate, because a wrong one is caught mechanically within a second. A claim about intent is not, because the only available check is a human who already knows the intent — and a human reviewing that sentence closely enough to catch a wrong one could have written the right one in the same time.

A small win done well, an active harm done badly

I still generate the header. It is a real if boring win — consistent types, consistent scopes, a subject that fits, and a changelog that groups without hand-editing. The 2025 Stack Overflow Developer Survey frames the risk precisely: 66 percent of respondents reported AI solutions that are almost right but not quite, and 45 percent said debugging AI-generated code takes longer. A commit message has the same failure mode with none of the feedback, because nothing crashes when the message is wrong.

That asymmetry is the thing worth ending on. Wrong code fails a test. A wrong commit message passes every check you own, sits in history, and gets read years later by somebody with no way to know it was generated. git log is evidence, and evidence you cannot trust is not evidence — so a terse honest subject is worth more than a fluent invented one.

The rule I ended up with fits in one sentence: automate the part of a commit message that has a grammar, and write the part that has a reason. Feed the branch, the issue and the review thread into the generator so the reason has somewhere to come from. Read the subjects before you tag, not the changelog after. And write the breaking-change note, the revert explanation and the 2am handover yourself, because those are the sentences somebody will rely on when it matters.