Issues

The hledger project's issue tracker is on github.

Shortcut urls:

• https://issues.hledger.org - open issues of all types in the github issue tracker
• https://wishes.hledger.org - open WISHes in the github issue tracker
• https://bugs.hledger.org - open BUGs in the github issue tracker
• https://bugs.hledger.org/new - report a new issue
• https://hledger.org/regressions - regression tracking

Misc. issue queries:

• critical bugs, and major bugs affecting many users (open)
• major bugs affecting fewer users (open)
• blocker bugs affecting many users (open)
• bugs without an “affects” impact label
• bugs without an “annoyance” severity label
• issues of unknown type
• assigned open issues
• closed issues with needs-* labels

(Most older issues don’t have impact/severity labels, so those reports are recent-biased)

Open issues grouped by topic/type:

Labels

In the github issue tracker we use labels to categorise issues and PRs:

• A-BUG or A-WISH: the issue’s type, a bug or a wish (feature request)
• affectsN-* and annoyanceN-*: a bug’s impact and severity, used for prioritising
• component/topic labels, like cli, ui, web, journal, csv, balance, queries, packaging, docs: which part of hledger is involved. These names are also used as commit message prefixes and issue title prefixes.
• platform-*: which platforms are involved
• needs-*: what an issue or PR is waiting for, like needs-repro, needs-review, needs-release
• regression, security, bounty, good-first-issue: special cases worth highlighting
• duplicate, invalid, wont-fix, cant-reproduce: why an issue was closed

We sometimes prefix bug titles (especially regressions) with the hledger version(s) affected. This allows searches like new issues in 1.22 and regressions in 1.22.

Prioritising

https://lostgarden.home.blog/2008/05/20/improving-bug-triage-with-user-pain/ describes an interesting method of ranking issues by a single “User Pain” metric. Here’s the simplified version that we are using in the hledger issue tracker:

Two labels can be applied to bug reports, each beginning with the letter a so as to appear near the front of label lists, each with a level from 1 to 5:

Impact

Who may be affected by this bug ?

• affects1-noone: Affects almost no one.
• affects2-devs: Affects packagers or developers.
• affects3-few: Affects just a few users.
• affects4-many: Affects potentially a significant number of users.
• affects5-most: Affects most or all users.

Severity

To people affected, how serious is this bug ?

• annoyance1-trivial: Cleanliness/consistency/developer bug. Only perfectionists care.
• annoyance2-minor: Minor to moderate usability/doc bug, reasonably easy to avoid or tolerate.
• annoyance3-blocker: New user experience or installability bug. A potential user could fail to get started.
• annoyance4-major: Major usability/doc bug, or any regression or crash.
• annoyance5-critical: Any loss of user’s data, privacy, security, or trust.

User Pain

The bug’s User Pain score is Impact * Severity (1 to 25).

Then, practices like these are possible:

• Open bugs can be prioritised in order of User Pain.
• The team can set easy-to-understand quality bars. For example, they could say “In order to release, we must have no open bugs with more than 15 pain.”
• If there are no bugs left above the current quality bar, they can work on feature work.
• If a bug is found that will take more than a week to fix, it can be flagged as a ‘killer’ bug, for special treatment.

Reducing bugs and regressions

Some ideas in 2024-01:

• Maintain ratio of user-visible bugfixes to new features, eg above 10:1 (a new main merge test, human checked)
• A release cycle with no new features
• Alternate bugfix and feature release cycles
• Set bug count targets
• Label all issues for impact/severity/user pain; set max user pain targets
• Gate releases on user pain targets or other bug metrics
• Document and follow more disciplined bug triage/fixing methods
• Identify every new bug early as a regression/non-regression
• Prioritise rapid fixing and releasing for regressions / new bugs
• Cheaper, more frequent bugfix releases
• More intentional systematic tests ? Analyse for weak spots ?
• Property tests ?
• Internal cleanup, architectural improvements, more type safety ?
• Custom issue dashboards (HTMX on hledger.org ?)
• Public list / QA dashboard
• Grow a QA team

Other

Historically we also tried Github projects (2017), Github milestones (2018), and a trello board for wishlist items.