Releasing

Notes for hledger release managers and maintainers.

Goals

2023

• Make releasing easy

2022

• Update/consolidate release process docs
• Establish routine monthly release cadence
• Make releasing easy

Release types

hledger major releases happen in the third month of each quarter, normally at or close to the start of the month. Preview releases happen in the other months, if needed. Here's the ideal release schedule:

Glossary

Here are some definitions, useful eg when executing or automating release procedures.

Activities

Release manager activities/responsibilities include:

• Software - selecting changes, packages, release dates; coordinating contributions; ensuring release readiness
• Branch Management - coordinating main and release branch, local and remote repos, CI branches
• Version Bumping - choosing and applying new version numbers and related things like tags, github releases, urls, ghc and dep versions, stackage resolvers, everywhere needed
• Docs - command help, manuals, changelogs, release notes, github release notes, install page, install scripts, announcements, process docs
• Testing - local testing, CI testing, extra release-specific testing
• Artifacts - generating binaries, zip files, github releases etc.
• Publishing - uploading, pushing, making visible, finalising
• Announcing - various announcement stages and channels

These have complex interdependencies and sequencing constraints. Chunk, separate, routinise, document and automate them as far as possible.

General tips

• Update changelogs early and often, eg during/after a PR, to spread the work. See also CHANGELOGS.
• Release (or practice releasing) often.
• Follow RELEASING.md's procedures if helpful. Update a copy (RELEASING2.md) while releasing.
• The release process continually evolves. Don't document procedures in too much detail / prematurely.
• Make things a little better each time through: simpler, more reliable, better documented, more automated, easier, faster, cheaper, higher quality.
• Make releases from a release branch, not from master. Use the tools/release script.
• Before publishing a github release: prepare binaries, release notes, announcements, and tag.
• Before tagging: make binaries for all platforms, from the same commit.
• Before making binaries: do all possible pre-tagging steps and try to get everything finalised. (Binaries' --version shows their git hash, and this should match the release tag.)

Releasing diagram

Here is a map of the value chain, artifacts, tasks and levels of release readiness involved in a hledger release. Arrows indicate dependencies. When making a release, activity moves from the bottom up towards the top. From this we derive the procedure docs below, and gradually automate those more over time.

Procedures

These docs need update.

Release

Copy RELEASING.md

• save a copy of RELEASING.md, make updates in the copy

Check release readiness

• appropriate timing, release manager availability
• master's changelogs are up to date (see CHANGELOGS)
• master or release branch is ready for release
  • clean and synced working copy
  • no pending release-blocking issues/prs
  • tests pass
• Shake is up to date
  • uses same resolver as stack.yaml
  • uses any required workarounds in stack.yaml
  • binary is up to date (./Shake.hs)
  • commit any changes (msg: tools: shake)

Start making cross-platform binaries

• do a preliminary push to origin/binaries
• resolve failures

Update changelogs

in main repo, master branch:

• ./Shake changelogs, clean up CHANGES.md's, ./Shake changelogs -c

See also CHANGELOGS.md.

Prepare release branch

Preview/major release:

• PAUSE=1 ECHO=1 tools/release prep MA.JOR[.99.PREVIEWNUM] (eg 1.24.99.1 for 1.25 preview 1) (XXX seems to go wrong without PAUSE`)
• clean up changelogs, amend changelogs commit (see also CHANGELOGS)
• cherry pick changes from master (if needed)
  • list changes in three side-by-side magit windows
    • 1. NEW IN MASTER: l o MAJORVER-branch..master, M-x magit-toggle-buffer-lock, M-x toggle-window-dedicated (C-c D)
    • 2. HEAD: regular magit status view
    • 3. RELEASE BRANCH: l o MAJORVER-branch, M-x magit-toggle-buffer-lock, M-x toggle-window-dedicated
  • in master window, working from bottom upward, cherry-pick all non-conflicting changes
    • skip changes already in release branch
    • skip changelog, command help, and manuals updates
• release branch testing
  • stack build --test
  • make functest

Bugfix/fixup release:

• cherry pick changes from master

Choose new version

• AKA NEW: MA.JOR, MA.JOR.MINOR, or MA.JOR.MINOR.FIXUP

Set new version

• on release branch do ./Shake setversion -c NEW

Prepare release notes

In site repo:

• update src/release-notes.md
  • copy template, uncomment
  • replace date
  • replace XX with NEW
  • add new content from changelogs, excluding hledger-lib
  • remove any empty sections
  • add contributors, git shortlog -sn OLD..NEW
  • add summary (major release) or remove it (bugfix release)
  • check preview in vs code
  • site repo commit: relnotes: NEW

Prepare announcement

(major/notable bugfix release)

In release branch:

• update doc/ANNOUNCE (major release)
  • summary, contributors from release notes
  • any other edits
  • commit: ;doc: announce

Tag the release

• ensure new version has been set with Shake setversion !
• make tag

Do release build testing

• touch/change Version.hs to encourage recompilation
• stack build
• stack exec -- hledger --version, check version, hash, release date, no '+'
• stack exec -- hledger help | tail, check version, month matches release

Prepare release binaries

• wait for any pending successful builds to complete and update cache
• tools/release bin
• get all platforms built on the same commit
• delete any local downloaded binaries from last release
• download binary artifact zip files (on each successful run: right click, download linked file)

Prepare new site manuals

(major release)

In site repo:

• js/site.js: add NEW, 3 places
• Makefile: add NEW, 2 places
• commit: manuals: add NEW
• make snapshot-NEW (after ensuring main repo has been release-tagged)
• push

Prepare hledger-install script

(major/bugfix/fixup release)

• update hledger-install/hledger-install.sh
  • HLEDGER_INSTALL_VERSION (release date)
  • RESOLVER (same as stack.yaml)
  • EXTRA_DEPS (same as stack.yaml)
  • hledger official packages (NEW)
  • hledger third-party packages (latest version on hackage)
• test ? (won't work until new hledger packages are on hackage) cd; bash ~/src/hledger/hledger-install/hledger-install.sh
• commit: install: NEW

Prepare install page

(major/bugfix/fixup release)

In site repo:

• update install.md
  • query-replace OLD -> NEW in
    • "current hledger release"
    • CI binaries badges/links, including linux-static-arm32v7 if built
    • "building from source"
    • stack install command
    • cabal install command
  • query-replace OLD-brightgreen -> OLD-red
  • only after release binaries are built (preferably after release is published): update --version outputs (search: hledger --version)
  • final output line from hledger test (run in terminal for normal speed)
  • Total count from make functest
  • commit: install: NEW

Draft github release

• copy text from previous similar release, https://github.com/simonmichael/hledger/releases
• create new release, https://github.com/simonmichael/hledger/releases/new
• leave release tag unselected for now
• set title (MA.JOR[...])
• paste & replace with new release notes
• upload CI binaries
• save as draft
• check preview, resolve issues

Pre-release pause

• stop, go afk, take a break
• review time, energy, availability, decide go/no-go

Pre-release tests

• appropriate dates in: changelogs, release notes, --version output, man pages, hledger-install script

Push release branch

in main repo, release branch:

• git push github MA.JOR-branch or magit P p
• wait for CI success (?)

Upload to hackage

in main repo, release branch:

• make hackageupload (major/bugfix/fixup release)

Push release branch and tags

in main repo, release branch:

• git push --tags or magit P t

Publish github release

• edit release
• select release tag (MA.JOR[...])
• click publish button

Update website

• push to github: in site repo, git push github or magit P u
• on hledger.org server:
  • in main repo (both main and site repos on master):
    • git -C site pull && git pull && make site
    • (or to rebuild all pages, also run make -C site all)
  • in /opt/hledger/site/hledger.org.caddy:
    • @oldmanpath: add path for NEW
    • @unversionedmanpath: update to NEW
  • systemctl reload caddy
• check site in browser
  • install page versions, badge colours
  • manuals' versions list in browser. If not updating, try
    • checking https://hledger.org/js/site.js
    • another browser (not safari)
    • shift-reload
    • https://dash.cloudflare.com/f629035917dd3b99b1e37ae20c15ff09/hledger.org/caching/configuration > purge
  • redirects
    • hledger.org/hledger.html redirects to https://hledger.org/NEW/hledger.html

Merge release branch changes to master

• switch back to master
• check out release branch in another working copy (hledger2)
• manually merge release changelogs into master changelogs (see also CHANGELOGS)
• list commits only in release branch: magit l o master..MA.JOR-branch
• cherry-pick any other useful commits

Bump master to next version

(major release)

• ./Shake setversion MA.JOR.99 -c
• ./Shake cmdhelp [-c] (might be empty)
• ./Shake mandates
• ./Shake manuals -c

Push master

in main repo, master branch:

• pass CI checks in dev branch:
  • ensure latest commit will trigger CI (does not begin with semicolon) (?)
  • git push github master:simon or magit P e github/simon, with -f if needed
  • wait for CI success at http://ci.hledger.org
• git push github master or magit P u

Announce release

(major/bugfix release)

• update last release date on plaintextaccounting.org
• share github release link and release notes markdown in #hledger chat
• send ANNOUNCE as email announcement
  • (major release): ANN: hledger NEW to [email protected], [email protected]
  • (bugfix release): brief announcement to [email protected]
• condense to 500 & post at https://fosstodon.org/@simonmic
• maybe condense to 140 & post at https://twitter.com/simonkwmichael

Commit RELEASING.md

• move copy back to RELEASING.md, commit

Post-release followup

• monitor packaging status, update install page
  • docker - expect/merge PR
  • homebrew - expect badge to update soon
  • nix - expect make nix-hledger-version to update after a few days, find and update to that commit hash
  • linux distros - once in a while, follow the links & search for newer versions, update
• provide support, monitor issues
• prepare followup releases if needed
• update process docs and tools

Update packaging

Update homebrew formula

1. ref

   1. helpers: chenrui & stack issue commenters on github, Athas on #haskell

   2. https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request

   3. doc

      If a URL is specified, the SHA-256 checksum of the new download should also be specified. A best effort to determine the SHA-256 and formula name will be made if either or both values are not supplied by the user.

      If a tag is specified, the Git commit revision corresponding to that tag must also be specified.

      Note: this command cannot be used to transition a formula from a URL-and-SHA-256 style specification into a tag-and-revision style specifi- cation, nor vice versa. It must use whichever style specification the for- mula already uses.

2. update homebrew working copy

   1. cd ~/src/DEVTOOLS/homebrew-core

   2. git reset --hard HEAD~1 && git fetch origin && git merge origin/master && git push -f

3. get release tarball checksums

   export V=X.Y; for P in -lib "" -ui -web; do curl -sO https://hackage.haskell.org/package/hledger$P-$V/hledger$P-$V.tar.gz; done && shasum -a256 *.gz

4. update Formula/hledger.rb

   1. update tarball urls, shas

   2. update other version references

   3. maybe update stackage snapshot

   4. possible new process, see: https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request#create-your-pull-request-from-a-new-branch

   5. test (if possible without too many mac/brew hassles)

      1. brew audit --strict Formula/hledger.rb # some tests are post-install, try also after brew upgrade

      2. brew upgrade Formula/hledger.rb -s -n -v # [scrub download cache, dry run, verbose]

         1. super slow, how to speed up ?

            1. comment out stack update

            2. add , "--stack-root=/Users/simon/.stack" to stack install # fails on permissions

      3. brew test Formula/hledger.rb

   6. commit with name "hledger X.Y" and description "Update to hledger X.Y."

5. get merged

   1. git push -f simonmichael

   2. gh pr create -f

   3. monitor: PR CI, PR merge, https://formulae.brew.sh/formula/hledger page

   4. ping brew contributors/maintainers if necessary: @chenrui, @carlocab, @SMillerDev

Update nix install command

1. Wait for the new hledger version to land in nixpkgs

   1. make nix-hledger-version

2. Find the nixpkgs haskell-packages commit post-dating the hledger release:

   1. make nix-view-commits # https://github.com/NixOS/nixpkgs/commits/master/pkgs/development/haskell-modules/hackage-packages.nix

3. Test the nix-env install command with that commit hash

   1. if it fails with "SSL peer certificate or SSH remote key was not OK"

      1. . Users/simon.nix-profile/etc/profile.d/nix.sh

      2. or re-install:

         1. curl -sO https://nixos.org/nix/install

         2. less install

         3. sh install

         4. . Users/simon.nix-profile/etc/profile.d/nix.sh

   2. on linux

   3. on mac

Update stackage

1. update https://github.com/fpco/stackage/blob/master/build-constraints.yaml if needed

2. monitor for new package versions in nightly: check https://www.stackage.org/package/hledger