hledger.org

Contributor Guide


IRC #hledger. Also: #hledger-bots, #ledger, #beancount
Mail list list.hledger.org
Twitter #hledger. Also: #plaintextaccounting, @LedgerTips, #ledgercli
Reddit /r/plaintextaccounting
Stack Exchange money.stackexchange.com?hledger
Hacker News hledger mentions
hledger-web demo   demo.hledger.org
hledger-api demo api-demo.hledger.org/api/v1/accounts, api-demo.hledger.org/swagger.json, in swagger editor
unfinished angular sample app (code)
hledger interactive demo https://hledger.alhur.es (hledger compiled to js)
Trello old wishlist planning board
Github simonmichael/hledger (shortcut: code.hledger.org), forks
commits, COMMITS!
open bugs, open wishes, open unknowns, open pull requests, all issues
issues with bounty tag, bountysource bounties, codemill bounties, codefund bounties
stars.hledger.org: our rank among ~30k starred haskell projects:
2016: ->#71, 2017: ->#54, 2018 ->#53)
github projects, waffle.io planning board
Travis CI ...
Appveyor CI ...
Azure CI ...
Hackage packages: hledger-lib, hledger, hledger-ui, hledger-web, hledger-api, hledger-diff, hledger-iadd, hledger-interest, hledger-irr, *hledger*
diffs: hledger-lib, hledger, hledger-ui, hledger-web, hledger-api
build status: hledger-lib, hledger, hledger-ui, hledger-web, hledger-api
reverse deps: hledger-lib, hledger, hledger-ui, hledger-web, hledger-api
bounds status:
... ... ... ... ...
Stackage build-constraints.yaml
open hledger-related issues
packages: hledger-lib, hledger, hledger-ui, hledger-web, hledger-api
versions: hledger-lib, hledger, hledger-ui, hledger-web, hledger-api
...
Repology quick hledger packaging status, detailed *hledger* packaging status
...
Debian source packages: haskell-hledger-lib, bugs, haskell-hledger, bugs, haskell-hledger-ui, bugs, haskell-hledger-web, bugs
binary packages:
  stable: hledger, bugs, hledger-web, bugs
  testing: hledger, bugs, hledger-web, bugs
  unstable: hledger, bugs, hledger-ui, bugs, hledger-web, bugs
  experimental: hledger, bugs, hledger-ui, bugs, hledger-web, bugs
  all: *hledger*
Package Tracking System help
popcon votes: hledger, hledger-ui, hledger-web, hledger active users:
Ubuntu source packages: haskell-hledger-lib, bugs, haskell-hledger, bugs, haskell-hledger-ui, bugs, haskell-hledger-web, bugs
binary packages: *hledger*
Gentoo hledger, hledger-web, *hledger*
Fedora hledger, *hledger*, hledger (package db), Haskell SIG
Void Linux package search -> hledger
Nix *hledger*
Homebrew hledger
Sandstorm hledger web app & reviews, issues
Reference GHC Can I Use

Open issues

COMPONENT BUGS WISHES PRS OTHER
all (simonmichael) bugs (good first issue/easy/other) wishes PRs other
install (hledger-install script) bugs (good first issue/easy/other) wishes PRs other
cli (hledger tool) bugs (good first issue/easy/other) wishes PRs other
ui (hledger-ui tool) bugs (good first issue/easy/other) wishes PRs other
web (hledger-web tool) bugs (good first issue/easy/other) wishes PRs other
api (hledger-api tool) bugs (good first issue/easy/other) wishes PRs other
interest (hledger-interest tool) bugs (good first issue/easy/other) wishes PRs other
journal (journal format) bugs (good first issue/easy/other) wishes PRs other
timeclock (timeclock format) bugs (good first issue/easy/other) wishes PRs other
timedot (timedot format) bugs (good first issue/easy/other) wishes PRs other
csv (CSV format, CSV output) bugs (good first issue/easy/other) wishes PRs other
json (JSON output) bugs (good first issue/easy/other) wishes PRs other
html (HTML output) bugs (good first issue/easy/other) wishes PRs other
accounts (command) bugs (good first issue/easy/other) wishes PRs other
activity (command) bugs (good first issue/easy/other) wishes PRs other
add (command) bugs (good first issue/easy/other) wishes PRs other
balance, bal (command) bugs (good first issue/easy/other) wishes PRs other
balancesheet, bs (command) bugs (good first issue/easy/other) wishes PRs other
cashflow, cf (command) bugs (good first issue/easy/other) wishes PRs other
checkdates (command) bugs (good first issue/easy/other) wishes PRs other
checkdupes (command) bugs (good first issue/easy/other) wishes PRs other
close (command) bugs (good first issue/easy/other) wishes PRs other
import (command) bugs (good first issue/easy/other) wishes PRs other
incomestatement, is (command) bugs (good first issue/easy/other) wishes PRs other
prices (command) bugs (good first issue/easy/other) wishes PRs other
print (command) bugs (good first issue/easy/other) wishes PRs other
printunique (command) bugs (good first issue/easy/other) wishes PRs other
register, reg (command) bugs (good first issue/easy/other) wishes PRs other
registermatch (command) bugs (good first issue/easy/other) wishes PRs other
rewrite (command) bugs (good first issue/easy/other) wishes PRs other
roi (command) bugs (good first issue/easy/other) wishes PRs other
stats (command) bugs (good first issue/easy/other) wishes PRs other
tags (command) bugs (good first issue/easy/other) wishes PRs other
balcmds (bal/bs/bse/cf/is) bugs (good first issue/easy/other) wishes PRs other
budget (balance --budget, budgeting) bugs (good first issue/easy/other) wishes PRs other
periodexpressions (-b, -e, -p, date:) bugs (good first issue/easy/other) wishes PRs other
tags (using tags) bugs (good first issue/easy/other) wishes PRs other
deps (packaging, dependency issues) bugs (good first issue/easy/other) wishes PRs other
doc (documentation, help) bugs (good first issue/easy/other) wishes PRs other
site (website, web presence) bugs (good first issue/easy/other) wishes PRs other
tools (developer tools, infrastructure)   bugs (good first issue/easy/other) wishes PRs other

About the project

Mission

Why was hledger created ?

Mainly:

Also:

What is the hledger project's current mission ?

  1. Provide peace of mind: bring clarity, relief, and peace of mind to folks stressed, confused, overwhelmed by finances.
  2. Educate and empower: help individuals and communities achieve clarity, accountability and mastery with money and time.

Roles and activities

Getting started

New contributors are always welcome in the hledger project. Jump in! Or ask us to help you find a task.

Funder

Become a financial backer to sustain and grow this project, increase your influence, express gratitude, build prosperity consciousness, and help transform world finance!

Tester

When reporting bugs, don't forget to search the tracker for a similar bug report. Otherwise, open a new bug by clicking "New issue", or http://bugs.hledger.org/new.

Enhancement requests are sometimes added to the tracker,but for these consider using the IRC channel and mail list (see Getting help). Both are archived and linkable, so the idea won't be lost. There is also a collection of wishes at the old trello board.

Technical Writer

Graphics Designer

Communicator

Marketing and market understanding is vital.

Maintainer

Help with issue management

Help with packaging

Help with project management

Developer

See Developer workflows

Make

A Makefile is provided to make common developer tasks easy to remember, and to insulate us a little from the ever-evolving Haskell tools ecosystem. Using it is entirely optional, but recommended. You'll need GNU Make installed.

The Makefile contains a fair amount of obsolete cruft and needs cleanup. Some tasks (docs, website) are now handled by the Shake file instead.

The Makefile is self-documenting. Run make to see a list of the main make rules:

$ make
Makefile:37: -------------------- hledger make rules --------------------
Makefile:39: make [help] -- list documented rules in this makefile. make -n RULE shows more detail.
Makefile:204: (INSTALLING:)
Makefile:206: make install -- download dependencies and install hledger executables to ~/.local/bin or equivalent (with stack)
Makefile:231: (BUILDING:)
Makefile:235: make build -- download dependencies and build hledger executables (with stack)
Makefile:304: make hledgerdev -- quickly build the hledger executable (with ghc and -DDEVELOPMENT)
...

To see what a make rule will do without actually doing it, use the -n flag:

$ make build -n
stack build
$ make test -n
(stack test \
        && echo pkgtest PASSED) || echo pkgtest FAILED
(stack exec hledger test \
        && echo builtintest PASSED) || echo builtintest FAILED
(COLUMNS=80 PATH=`pwd`/bin:/home/simon/src/hledger/bin:/home/simon/.local/bin:/home/simon/.cabal/bin:/opt/ghc/7.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/var/lib/gems/1.9.1/bin stack exec -- shelltest --execdir -- -j16 --hide-successes tests \
        && echo functest PASSED) || echo functest FAILED

Shake

Shake.hs in the top directory complements the Makefile; it is used for some more complex tasks, such as building documentation and the web site.

Compile it:

./Shake.hs   # or, make Shake

See help:

./Shake

Code

hledger is a suite of applications, tools and libraries. The main hledger code repository is github.com/simonmichael/hledger (shortcut url code.hledger.org). There are also various hledger addons maintained as separate projects with their own repos.

hledger packages

Within the main repo, there are a number of separate cabal packages, making it easier to pick and choose parts of hledger to install or to package. They are:

hledger-lib

package, code

Core data models, parsing, standard reports, and utilities. Most data types are defined in Hledger.Data.Types, while functions that operate on them are defined in Hledger.Data.TYPENAME. Under Hledger.Read are parsers for the supported input formats. Data files are parsed into a Journal, which contains a list of Transactions, each containing multiple Postings of some MixedAmount (multiple single-CommoditySymbol Amounts) to some AccountName. When needed, the Journal is further processed to derive a Ledger, which contains summed Accounts. In Hledger.Reports there are standard reports, which extract useful data from the Journal or Ledger.

Here's a diagram of the main data model:

diagram

hledger

package, code, manual

hledger's command line interface, and command line options and utilities for other hledger tools.

Try tracing the execution of a hledger command:

  1. Hledger.Cli.Main:main parses the command line to select a command, then
  2. gives it to Hledger.Cli.Utils:withJournalDo, which runs it after doing all the initial parsing.
  3. Parsing code is under hledger-lib:Hledger.Read, eg Hledger.Read.JournalReader.
  4. Commands extract useful information from the parsed data model using hledger-lib:Hledger.Reports, and
  5. render in plain text for console output (or another output format, like CSV).
  6. Everything uses the data types and utilities from hledger-lib:Hledger.Data and hledger-lib:Hledger.Utils.

hledger-ui

package, code, manual

A curses-style text interface.

hledger-web

package, code, manual

A web interface. hledger-web starts a web server built with the yesod framework, and (by default) opens a web browser view on it. It reads the journal file(s) at startup and again whenever they change. It can also write (append) new transactions to the journal file.

There are two main views, which can be filtered with queries:

There is also:

Most of the action is in

Handler module and function names end with R, like the yesod-generated route type they deal with.

Dynamically generated page content is mostly inline hamlet. Lucius/Julius files and widgets generally are not used, except for the default layout.

Here are some ways to run it during development:

$ (cd hledger-web; yesod devel)
$ (cd hledger-web; stack ghci hledger-web)
hledger-web> :main --serve   # restart: ctrl-c, :r, enter, ctrl-p, ctrl-p, enter
$ make ghci-web
ghci> :main --serve

(This rule also creates symbolic links to hledger-web's config, messages, static and templates directories, needed in developer mode, so it can run from the top directory. This may not work on Windows.)

hledger-api

package, code, manual

A web API server. Uses the servant framework.

Quality

Relevant tools include:

Code review

Pull requests

Most contributed hledger code (and some of the project maintainer's code) is submitted and reviewed via Github pull requests. Here are some tips for contributing PRs to hledger.

Code review is important

We aim to improve and sustain hledger's quality and maintainability over the long term.

Many PRs, especially small ones, and even some big ones, can be merged quickly. We love merging good PRs quickly.

Some bigger or more risky PRs can require substantial review, discussion, changes, or re-submission. Sometimes this is a bigger task than the coding. Much valuable design, quality control, and knowledge sharing happens at this time. Some PRs get rejected, but their discussion and exploration can still be a useful contribution. We very much want to avoid wasted work, but it occasionally happens. Our process is evolving and imperfect. All of this is normal.

We hope you'll see it as a golden opportunity to collaborate with experts, share and receive knowledge, refine your design/documentation/code, and practice real-world development and communication skills. Patience and persistence pays.

The pull request

A PR should have a clear purpose, documented in its description. Mention any #ISSUENOs addressed.

Don't tackle too much at once. Smaller/more focussed PRs can be reviewed quicker and accepted (or rejected) quicker.

Consider showing a draft of documentation first (more on this below).

The commit(s)

Commits should be easy to review. Ideally each commit is complete, and has a single clear purpose, which should be documented in the summary (and long description, if needed). #ISSUENOs can be mentioned in summary/description too when appropriate.

Within the above constraint, fewer, larger commits are preferred.

Keep in mind that commit messages are valuable documentation for future developers and troubleshooters. They are also the starting point for package changelogs and hledger release notes. High-quality commit messages makes the release process quicker, and the resulting docs better.

User-impacting commits should mention the user-visible changes, and be described in user-relevant language. Library-user-impacting commits, eg API changes, ideally will also be called out, and can described in more technical language. Commits affecting hledger internals are less important, but you may notice some adhoc conventions if you browse the history. In particular, you can optionally prefix the summary with short component codes (cf Issues) to facilitate history reading and changelog/release note production.

Rewrite and force-push your commits freely (rebase -i, push -f) to clean them up. Unless we decide to squash the PR into one commit, your commits will become part of hledger's history "for all time", so think about future developers trying to understand them, git bisect, etc.

Rebase your commits against latest master for easiest review. Especially if they start to conflict.

The docs

PRs should include appropriate updates to reference documentation, unless otherwise agreed. Typically this means the manual source files (hledger/hledger.m4.md). It can also involve command line option names and descriptions, other --help output, hledger's commands list, hledger-ui's help dialog, hledger-web's help dialog, etc. Sometimes it means the developer docs, at least the ones in the main repo (READMEs).

Reviewers can understand your PR more efficiently once proposed doc changes are provided, and may postpone it otherwise. We are happy to help with the docs if needed - just ask.

Updating rendered manuals (hledger.{1,info,txt,md,html}) is not required, and probably best avoided to reduce conflicts. Updating other docs such as tutorials, how-tos, examples, or screenshots is not required, though it's welcome (may be in a different repo).

Documentation first

hledger follows documentation-driven design. It is in fact highly effective, and highly recommended, to write the new docs (help text/reference manual/haddocks/developer README..) before writing any code. You can share a rough draft on IRC, on the mail list, in an issue comment, or in a "WIP" PR starting with just the proposed docs commit.

This is often the quickest road to getting something merged into hledger. hledger's many parts interact in surprisingly complex ways. The documentation-driven working style lets us discuss, clarify and reach a good-enough consensus economically, after which coding/review/acceptance can go quicker.

http://neilmitchell.blogspot.com/2019/06/the-one-pr-per-day-rule.html

Tests

About testing in the hledger project, as of 201809.

Kinds of tests

  1. Unit tests

    Unit tests exercise small chunks of functionality. In hledger, that means a function. So, many of our functions have one or more unit tests. These are mostly in hledger-lib, with a few in hledger.

    Our unit tests use easytest and some helpers from Hledger.Utils.Test, and follow a consistent pattern which aims to keep them

    • easy to read (clear, concise)
    • easy to write (low boilerplate, low cognitive load)
    • easy to maintain (easy to edit, easy to refactor, robust)
    • easy to associate with the code under test (easy to view/jump between code & test, easy to estimate coverage)
    • and scalable (usable for all devs, easy to run and select, suitable for small/large modules/packages).

    Here's the pattern (let us know if you see a better way):

    Here are some real-world examples.

    The unit tests are shipped as part of the hledger executable, and can always be run via the test command (hledger test).

    Here's the quick way to run unit tests while developing:
    make ghcid-test or make ghcid-test-Hledger.Some.Module.

  2. Doc tests

    Like unit tests, but defined inside functions' haddock documentation, in the style of a GHCI transcript. These test functionality, provide usage examples in the API docs, and test those examples, all at once. They are a bit more finicky and slower than unit tests. See doctest for more.

    doctests do not work on Mac with GHC 8.4+, out of the box. See ghc#15105 for current status and a workaround.

  3. Functional tests

    Functional tests test the overall functioning of the program. For hledger, that means running hledger with various inputs and options and checking for the expected output. This exercises functionality in the hledger and hledger-lib packages. We do this with shelltestrunner. Tests are defined in files under tests/, grouped by component (command or topic name). For more about these, see the README there.

  4. Code tests

    We have some tests aimed at testing eg code quality, generally defined as make rules, such as:

    make haddocktest can haddock process all code docs without error
    make buildtest does all code build warning free with the default GHC version & stackage snapshot
    make buildtestall does the code build warning free with all supported GHC versions/stackage snapshots

    See below for examples.

  5. Package test suites

    Haskell tools like stack and cabal recognise test suites defined in a package's cabal file (or package.yaml file). These can be run via stack test, cabal test etc., and they are required to build and pass by services like Stackage. Here are the currently hledger package test suites:

    package test suite what it runs
    hledger-lib doctests doctests
    hledger-lib easytests unit tests
    hledger test builtin test command (hledger's + hledger-lib's unit tests)
    hledger-ui
    hledger-web
    hledger-api

Coverage

This means how thoroughly the code is tested - both in breadth (are all parts of the code tested at least a little ?) and in depth (are all possible code paths, states, situations tested ?).

Our current test coverage can be summarised like so:

package unit doc functional
hledger-lib X X X
hledger X X
hledger-ui
hledger-web
hledger-api

There are ways to generate detailed coverage reports for haskell unit tests, at least. It would be useful to set this up for hledger.

How to run tests

Run unit tests:

$ make unittest

Run doctests:

$ make doctest

Run functional tests (and unit tests, now):

$ stack install shelltestrunner
$ make functest

Run the package tests (unit tests, maybe doctests, but not functional tests) of all or selected packages.

$ stack test [PKG]

Run "default tests: package plus functional tests":

$ make test

Test generation of haddock docs:

$ make haddocktest

Thorough test for build issues with current GHC:

$ make buildtest

Thorough test for build issues with all supported GHC versions:

$ make buildtestall

Run built-in hledger/hledger-lib unit tests via hledger command:

$ hledger test  # test installed hledger
$ stack build hledger && stack exec -- hledger test  # test just-built hledger
$ hledger test --help
test [TESTPATTERN] [SEED]
  Run the unit tests built in to hledger-lib and hledger,
  printing results on stdout and exiting with success or failure.
  Tests are run in two batches: easytest-based and hunit-based tests.
  If any test fails or gives an error, the exit code will be non-zero.
  If a pattern argument (case sensitive) is provided, only easytests
  in that scope and only hunit tests whose name contains it are run.
  If a numeric second argument is provided, it will set the randomness
  seed for easytests.

Rebuild and rerun hledger/hledger-lib unit tests via ghcid:

$ make ghcid-test

Rebuild and rerun only some tests via ghcid (see hledger test --help):

$ make ghcid-test-TESTPATTERN

See all test-related make rules:

$ make help-test

Benchmarks

Benchmarks are standard performance measurements, which we define using bench declarations in cabal files. There is one in hledger.cabal, with related code and data files in hledger/bench/.

To run the standard hledger benchmark, use stack bench hledger. This installs haskell dependencies (but not system dependencies) and rebuilds as needed, then runs hledger/bench/bench.hs, which by default shows quick elapsed-time measurements for several operations on a standard data file:

$ stack bench hledger
NOTE: the bench command is functionally equivalent to 'build --bench'
...
hledger-0.27: benchmarks
Running 1 benchmarks...
Benchmark bench: RUNNING...
Benchmarking hledger in /Users/simon/src/hledger/hledger with timeit
read bench/10000x1000x10.journal        [1.57s]
print                                   [1.29s]
register                                [1.92s]
balance                                 [0.21s]
stats                                   [0.23s]
Total: 5.22s
Benchmark bench: FINISH

bench.hs has some other modes, which you can use by compiling and running it directly. --criterion reports more detailed and dependable measurements, but takes longer:

$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --criterion
...
Linking bench/bench ...
Benchmarking hledger in /Users/simon/src/hledger/hledger with criterion
benchmarking read bench/10000x1000x10.journal
time                 1.414 s    (1.234 s .. 1.674 s)
                     0.996 R²   (0.989 R² .. 1.000 R²)
mean                 1.461 s    (1.422 s .. 1.497 s)
std dev              59.69 ms   (0.0 s .. 62.16 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking print
time                 1.323 s    (1.279 s .. 1.385 s)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 1.305 s    (1.285 s .. 1.316 s)
std dev              17.20 ms   (0.0 s .. 19.14 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking register
time                 1.995 s    (1.883 s .. 2.146 s)
                     0.999 R²   (0.998 R² .. NaN R²)
mean                 1.978 s    (1.951 s .. 1.995 s)
std dev              25.09 ms   (0.0 s .. 28.26 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking balance
time                 251.3 ms   (237.6 ms .. 272.4 ms)
                     0.998 R²   (0.997 R² .. 1.000 R²)
mean                 260.4 ms   (254.3 ms .. 266.5 ms)
std dev              7.609 ms   (3.192 ms .. 9.638 ms)
variance introduced by outliers: 16% (moderately inflated)

benchmarking stats
time                 325.5 ms   (299.1 ms .. 347.2 ms)
                     0.997 R²   (0.985 R² .. 1.000 R²)
mean                 329.2 ms   (321.5 ms .. 339.6 ms)
std dev              11.08 ms   (2.646 ms .. 14.82 ms)
variance introduced by outliers: 16% (moderately inflated)

--simplebench shows a table of elapsed-time measurements for the commands defined in bench/default.bench. It can also show the results for multiple h/ledger executables side by side, if you tweak the bench.hs code. Unlike the other modes, it does not link with the hledger code directly, but runs the "hledger" executable found in $PATH (so ensure that's the one you intend to test).

$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --simplebench
Benchmarking /Users/simon/.local/bin/hledger in /Users/simon/src/hledger/hledger with simplebench and shell
Using bench/default.bench
Running 4 tests 1 times with 1 executables at 2015-08-23 16:58:59.128112 UTC:
1: hledger -f bench/10000x1000x10.journal print [3.27s]
1: hledger -f bench/10000x1000x10.journal register  [3.65s]
1: hledger -f bench/10000x1000x10.journal balance   [2.06s]
1: hledger -f bench/10000x1000x10.journal stats [2.13s]

Summary (best iteration):

+-----------------------------------------++---------+
|                                         || hledger |
+=========================================++=========+
| -f bench/10000x1000x10.journal print    ||    3.27 |
| -f bench/10000x1000x10.journal register ||    3.65 |
| -f bench/10000x1000x10.journal balance  ||    2.06 |
| -f bench/10000x1000x10.journal stats    ||    2.13 |
+-----------------------------------------++---------+

bench's --simplebench mode is based on a standalone tool, tools/simplebench.hs. simplebench.hs is a generic benchmarker of one or more executables (specified on the command line) against one or more sets of command-line arguments (specified in a file). It has a better command-line interface than bench.hs, so you may find it more convenient for comparing multiple hledger versions, or hledger and ledger. Eg:

$ stack exec -- ghc tools/simplebench
[1 of 1] Compiling Main             ( tools/simplebench.hs, tools/simplebench.o )
Linking tools/simplebench ...
$ tools/simplebench -h
tools/simplebench -h
simplebench: at least one executable needed
bench [-f testsfile] [-n iterations] [-p precision] executable1 [executable2 ...]

Run some functional tests with each of the specified executables,
where a test is "zero or more arguments supported by all executables",
and report the best execution times.

  -f testsfile   --testsfile=testsfile    file containing tests, one per line, default: bench.tests
  -n iterations  --iterations=iterations  number of test iterations to run, default: 2
  -p precision   --precision=precision    show times with this precision, default: 2
  -v             --verbose                show intermediate results
  -h             --help                   show this help

Tips:
- executables may have arguments if enclosed in quotes
- tests can be commented out with #
- results are saved in benchresults.{html,txt}
cd hledger; $ ../tools/simplebench -f bench/default.bench hledger ledger
Using bench/default.bench
Running 4 tests 2 times with 2 executables at 2015-08-24 04:24:37.257068 UTC:

Summary (best iteration):

+-----------------------------------------++---------+--------+
|                                         || hledger | ledger |
+=========================================++=========+========+
| -f bench/10000x1000x10.journal print    ||    3.24 |   0.43 |
| -f bench/10000x1000x10.journal register ||    3.80 |   3.48 |
| -f bench/10000x1000x10.journal balance  ||    2.05 |   0.18 |
| -f bench/10000x1000x10.journal stats    ||    2.10 |   0.19 |
+-----------------------------------------++---------+--------+

Finally, for quick, fine-grained performance measurements when troubleshooting or optimising, I use dev.hs.

Version numbers

Some places version numbers appear:

Some old version numbering goals:

  1. automation, robustness, simplicity, platform independence
  2. cabal versions must be all-numeric
  3. release versions can be concise (without extra .0's)
  4. releases should have a corresponding VCS tag
  5. development builds should have a precise version appearing in --version
  6. development builds should generate cabal packages with non-confusing versions
  7. there should be a way to mark builds/releases as alpha or beta
  8. avoid unnecessary compiling and linking
  9. minimise VCS noise and syncing issues (commits, unrecorded changes)

Current version numbering policy:

Current process:

Sample journals

Synthetic data files like examples/100x100x10.journal are useful for benchmarks and testing. The numbers describe the number of transactions, number of accounts, and maximum account depth respectively. They are generated by tools/generatejournal.hs. They should get built automatically as needed, if not you can use make samplejournals:

$ make samplejournals
ghc tools/generatejournal.hs
[1 of 1] Compiling Main             ( tools/generatejournal.hs, tools/generatejournal.o )
Linking tools/generatejournal ...
tools/generatejournal 100 100 10 >examples/100x100x10.journal
tools/generatejournal 1000 1000 10 >examples/1000x1000x10.journal
tools/generatejournal 1000 10000 10 >examples/1000x10000x10.journal
tools/generatejournal 10000 1000 10 >examples/10000x1000x10.journal
tools/generatejournal 10000 10000 10 >examples/10000x10000x10.journal
tools/generatejournal 100000 1000 10 >examples/100000x1000x10.journal
tools/generatejournal 3 5 5 >examples/ascii.journal
tools/generatejournal 3 5 5 --chinese >examples/chinese.journal
tools/generatejournal 3 5 5 --mixed >examples/mixed.journal

Docs

https://github.com/simonmichael/hledger/tree/master/doc

Project documentation lives in a number of places:

How to prepare changelogs & release notes

Changelogs:

Release notes:

Issues

The hledger project's issue tracker is on github. It contains:

Issue Urls

Labels

Labels are used to categorise:

Components

Issues and the hledger project generally are organised into components: mostly non-overlapping topics, one for each user command, add-on tool, input format, output format, etc. Each component gets a light blue label in the issue tracker. Component names (sometimes abbreviated) are used as a prefix to commit messages, and to organise changelogs and release notes. The current components, and their open issues, can be seen in the Open Issues table above.

Custodians

If you are interested in helping with a particular component for a while, please add yourself as a custodian in Open Issues table above. A custodian's job is to help manage the issues, rally the troops, and drive the open issue count towards zero. The more custodians, the better! By dividing up the work this way, we can scale and make forward progress.

Milestones and Projects

Milestones are used a little bit to plan releases. In 2017 we experimented with projects, but in 2018 milestones are in favour again..

Estimates

You might see some experiments in estimate tracking, where some issue names might have a suffix noting estimated and spent time. Basic format: [ESTIMATEDTOTALTASKTIME|TIMESPENTSOFAR]. Examples: ``` [2] two hours estimated, no time spent [..] half an hour estimated (a dot is ~a quarter hour, as in timedot format) [1d] one day estimated (a day is ~4 hours) [1w] one week estimated (a week is ~5 days or ~20 hours) [3|2] three hours estimated, about two hours spent so far [1|1w|2d] first estimate one hour, second estimate one week, about two days spent so far ``` Estimates are always for the total time cost (not time remaining). Estimates are not usually changed, a new estimate is added instead. Numbers are very approximate, but better than nothing.

Trello

The trello board (trello.hledger.org) is an old collection of wishlist items. This should probably be considered deprecated.

Funding

My vision for the hledger project has always been for it to be "accountable" and "self-sustaining", possibly through new forms of incentivisation. Classic non-monetary FOSS communities are a beautiful and precious thing. Adding money can change their dynamic. Yet, we would enjoy having a lot more issues resolved, and a faster rate of progress. So we experiment, gently.

Currently we use bounties as a way to encourage resolution of issues. There are a few ways to do this:

  1. You or your organisation can offer a bounty simply by saying so on the issue.

  2. You can use Bountysource. A few hledger bounties have been completed there.

  3. You can use the new Open Collective process below.

Issues with bounties of any kind are marked with the bounty label. The Bounty Manager is @simonmichael.

New bounty process

It currently looks like this, and will evolve:

Our bounty amounts are small, and nothing like professional rates in most countries, but they still establish a principle of sustainability, and help us to experiment. You are encouraged to claim your bounties, though you can also choose to transfer them to a new issue of your choice.

Developer workflows

Get developer tools

Ensure stack is installed (or if you’re a cabal expert, feel free to use that.)

Ensure git is installed. On Windows, it comes with stack.

Here are some useful optional tools:

Eg:

stack install ghcid shelltestrunner quickbench hasktags
brew install entr

Get the code

git clone https://github.com/simonmichael/hledger
cd hledger

Review code

Build in place

See also http://hledger.org/download.html#c.-build-the-development-version .

stack build    # hledger hledger-ui ...

This fetches the required GHC version and haskell dependencies from the default stackage snapshot (configured in stack.yaml), then builds all hledger packages. This can take a while! To save time, you can build individual packages, eg just the CLI and TUI.

Note stack does not fetch C libraries such as curses or terminfo, which you might need to install yourself, using your system's package manager. In case of trouble, see download.

If you want to use an older snapshot/GHC for some reason, specify one of the older stack-ghc*.yaml files:

stack --stack-yaml stack-ghc8.2.yaml build

Run in place

stack exec -- hledger     # ARGS...
stack exec -- hledger-ui  # ARGS...
stack exec -- which hledger

Build and install

This builds and also copies the hledger executables to ~/.local/bin or the Windows equivalent (which you should add to your $PATH).

stack install    # hledger hledger-ui ...

Run package tests

Runs any HUnit/doctest/easytest tests defined by each hledger package.

stack test    # hledger ...

Run package benchmarks

Runs any performance reports defined by each hledger package.

stack bench    # hledger ...

Run quickbench benchmarks

Times the end-user commands in bench.sh using quickbench.

make bench

Run functional tests

Runs the shelltestrunner tests defined in tests/, which test the hledger CLI.

stack build hledger
make functest

Run haddock tests

Checks for anything that would break haddock doc generation.

make haddocktest

Checks for the unit-tests embedded in documentation.

make doctest

Simulate Travis tests

Locally runs tests similar to what we run on Travis CI.

make travistest

Test with all supported GHC versions/stackage snapshots

make allsnapshotstest

Use GHCI

GHCI is GHC's REPL, useful for exploring and calling code interactively.

If you try to run GHCI (or things based on it, like ghcid) right after cloning the hledger repo, you might see an error about CPP macros, eg like on #961. To fix this, build the hledger packages once, eg stack build. (Or stack build hledger might be enough, depending what you are doing.)

Get a GHCI prompt for hledger-lib:

cd hledger-lib; stack ghci hledger-lib

Changing into the package directory isn't actually needed, but it enables a custom .ghci script which sets a more useful short prompt.

Get a GHCI prompt for hledger:

cd hledger; stack ghci hledger

Get a GHCI prompt for hledger-ui:

cd hledger-ui; stack ghci hledger-ui

Get a GHCI prompt for hledger-web:

cd hledger-web; stack ghci hledger-web

hledger-web also needs to find some things in its current directory (like the static/ directory). This normally just works, if not please send details.

Add a test

Fix a bug or add a feature

Get your changes accepted

Follow the usual github workflow:

If you're new to this process, help.github.com may be useful.

Add yourself to the contributor list

Work on docs

Most docs tasks are handled by Shake.

List Shake rules:

./Shake

Generate man/info/txt manuals (in hledger*/) and embed in hledger executables:

./Shake manuals
stack build

Generate html manuals and the hledger website (in site/_site/):

./Shake website

To remove all files generated by Shake:

./Shake Clean

Use ghcid for watching GHC/GHCI

ghcid is the most reliable and fastest way to see GHC's feedback, and optionally run tests or a GHCI command, as you edit. We run it via make, for convenience and to watch multiple packages rather than just one. Run make help-ghcid to list related rules.

Watch for compile errors in hledger-lib and hledger:

make ghcid

Watch compile errors and the output of some hledger command:

ghcid -c 'make ghci' -T ':main -f a.j bal --budget -N'

Use --file-watch for watching stack

stack's --file-watch flag will re-run build/test/bench when source files or package.yaml/cabal files change. Eg:

stack test hledger --file-watch

If you find that adding --fast makes this any faster, please update this.

Use entr for watching arbitrary commands

entr is the most robust cross-platform tool for watching files and running a command when they change. Note its first argument must be an executable program, to run a shell command or multiple commands use bash -c "...".

Rerun a single functional test as you change it:

ls tests/budget/budget.test | entr bash -c 'clear; COLUMNS=80 stack exec -- shelltest --execdir tests/budget/budget.test -i12'