hledger and Ledger

2023-01: If you are a Ledger user trying to use hledger with your data, I'd love to hear about your experience and make this easier; please contact me (sm) in the #ledger, #hledger, or #plaintextaccounting chats. And you may want to skip ahead to Interoperating tips or any other topic that seems relevant. See also #1962.

Differences

10000 foot view

How is hledger different from Ledger ? First, the high-order differences:

  • hledger is actively maintained (since 2008)
  • hledger focusses strongly on UX, reliability, and real-world practicality
  • hledger is written in Haskell, which helps with correctness and maintainability
  • hledger tries to reimplement Ledger's best parts in more depth, with improved functionality and robustness.

Compared to Ledger, hledger has

  • a complete and accurate manual
  • standard "financial statement" reports
  • multi-column reports
  • an easier query syntax
  • better depth limiting
  • a battle-tested CSV/SSV/TSV import system
  • and comes with multiple officially-supported user interfaces (CLI, console, TUI, WUI).

Compared to hledger, Ledger has

  • assisted lot tracking for investment transactions
  • more support for embedding small programs in your data to get custom behaviour (value expressions, maybe python expressions ?)
  • a lighter memory footprint and smaller executables

See also:

Features

Over time, features have propagated both ways. Here is a presentation of hledger features and here is a feature comparison as of 2022 (updates welcome):

hledgerLedger
Common features:
journal formatYY
csv formatYY
timeclock formatYY
multiple commoditiesYY
conversion prices and cost reportingYY
market prices and value reportingYY
virtual (unbalanced) postingsYY
automated postingsYY
periodic transactionsYY
budget reportingYY
capital gains reportingYY
report filtering with flags and query argumentsYY
basic output format customisationYY
print, register, balance commandsYY
Features in Ledger only:
automatic revaluation transactions (--revalued)Y
lot reporting (--lots)Y
embedded programming language (value expressions)Y
embedded python snippets / python APIY
probably miscellaneous other things...Y
Features in hledger only:
international number formatsY
timedot formatY
multi-period balance reportsY
account typesY
activity commandY
add commandY
balancesheet commandY
cashflow commandY
check commandY
close commandY
descriptions commandY
diff commandY
files commandY
iadd commandY
import commandY
incomestatement commandY
irr commandY
interest commandY
notes commandY
prices commandY
rewrite commandY
ui commandY
web commandY

Performance

Traditionally, Ledger and hledger performance felt about the same on small files, but Ledger used less memory and was faster with large files - with very large files, up to ~10x faster. That extra speed came partly from providing fewer guarantees, eg Ledger's balance assertions/assignments are not date-aware.

Lately (2021) the performance gap seems to have closed, with hledger outperforming Ledger in some cases - more formal benchmarking needed, please see if you can reproduce. [This was an intel hledger binary running translated on a macbook m1 via Rosetta translation, ie slower than normal]:

$ uname -a
Darwin SMs-slate-mac.local 20.6.0 Darwin Kernel Version 20.6.0: Tue Oct 12 18:33:38 PDT 2021; root:xnu-7195.141.8~1/RELEASE_ARM64_T8101 arm64

$ brew info ledger
...
/opt/homebrew/Cellar/ledger/3.2.1_7 (126 files, 4.7MB) *
  Poured from bottle on 2021-11-18 at 16:04:23
...

$ ledger --version
Ledger 3.2.1-20200518, the command-line accounting tool
...

$ hledger-1.24 --version
hledger 1.24-0-gf0f830e06, mac-x86_64

$ cat bench-ledger.sh 
hledger -f examples/10000x1000x10.journal print
hledger -f examples/10000x1000x10.journal register
hledger -f examples/10000x1000x10.journal balance
hledger -f examples/100000x1000x10.journal balance
hledger -f examples/100000x1000x10.journal balance ff

$ quickbench -f bench-ledger.sh -w ledger,hledger-1.24
Running 5 tests 1 times with 2 executables at 2021-12-09 08:50:10 HST:

Best times:
+-----------------------------------------------++--------+--------------+
|                                               || ledger | hledger-1.24 |
+===============================================++========+==============+
| -f examples/10000x1000x10.journal print       ||   7.08 |         0.84 |
| -f examples/10000x1000x10.journal register    ||  18.16 |        16.65 |
| -f examples/10000x1000x10.journal balance     ||   0.38 |         0.80 |
| -f examples/100000x1000x10.journal balance    ||  29.14 |         6.78 |
| -f examples/100000x1000x10.journal balance ff ||   1.13 |         5.89 |
+-----------------------------------------------++--------+--------------+

$ file /opt/homebrew/bin/ledger /Users/simon/src/hledger/bin/hledger-1.24
/opt/homebrew/bin/ledger:                  Mach-O 64-bit executable arm64
/Users/simon/src/hledger/bin/hledger-1.24: Mach-O 64-bit executable x86_64

In 2022, hledger compiled natively on a macbook air m1 processes 25k transactions per second (which means reporting on a normal year's worth of transactions takes less than a tenth of a second):

$ hledger --version
hledger 1.24.99.2-gba5b0e93f-20220205, mac-aarch64
$ make throughput
date: Tue Feb 8 11:03:50 HST 2022
system: Darwin slate.local 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T8101 arm64
executable: hledger
version: hledger 1.24.99.2-gba5b0e93f-20220205, mac-aarch64
  1000 txns: Run time (throughput)    : 0.07s (15308 txns/s)
  2000 txns: Run time (throughput)    : 0.09s (21121 txns/s)
  3000 txns: Run time (throughput)    : 0.13s (23648 txns/s)
  4000 txns: Run time (throughput)    : 0.17s (23226 txns/s)
  5000 txns: Run time (throughput)    : 0.21s (23647 txns/s)
  6000 txns: Run time (throughput)    : 0.24s (24784 txns/s)
  7000 txns: Run time (throughput)    : 0.29s (24166 txns/s)
  8000 txns: Run time (throughput)    : 0.33s (24450 txns/s)
  9000 txns: Run time (throughput)    : 0.35s (25516 txns/s)
 10000 txns: Run time (throughput)    : 0.41s (24226 txns/s)
100000 txns: Run time (throughput)    : 4.32s (23158 txns/s)
Tue Feb  8 11:03:57 HST 2022

Command line interface

  • hledger does not require a space between command-line flags and their values, eg -fFILE works as well as -f FILE

  • hledger uses --ignore-assertions/-I to disable balance assertions. Ledger uses --permissive for that, and uses -I as the short form of --prices.

  • hledger's -x/--explicit flag makes print show all amounts; Ledger's --explicit flag does something else.

  • hledger's and Ledger's -H/--historical flags are unrelated. hledger's -H makes register and balance-like commands include balances from before the report start date, instead of starting at zero:

    hledger register --help:
    -H --historical           show historical running total/balance
                              (includes postings before report start date)
    
    hledger balance --help:
    -H --historical           show historical ending balance in each period
                              (includes postings before report start date)
    

    Whereas Ledger's -H changes the valuation date used by -V/-X:

    ledger --help:
    --historical (-H)
                              Value commodities at the time of their acquisition.
    
  • hledger's query language is a little less powerful than Ledger's, simpler, and easier to remember. It uses google-like prefixes, such as desc:, payee:, amt:, and not:. Multiple query terms are combined using fixed AND/OR rules. We don't yet support full boolean expressions, so some more advanced queries require two invocations of hledger in a pipe, eg: hledger print QUERY1 | hledger -f- reg QUERY2

  • hledger provides more short flags (-b, -e, -p, -D, -W, -M, -Q, -Y) and the date: query argument for setting report period and interval, and all of these combine nicely.

  • hledger cleans up some old semantic confusion around what "uncleared" means:

    • hledger renames Ledger's "uncleared" status (ie, when the status field is empty) to "unmarked", and the --uncleared/-U flag to --unmarked/-U
    • hledger uses -P as the short form of --pending. Ledger uses -P for grouping by payee.
    • each of hledger's --unmarked/-U, --pending/-P, --cleared/-C flags match only that single status. To match more than one status, the flags can be combined.

    So the hledger equivalent of Ledger's -U flag ("match uncleared") is -UP ("match unmarked or pending").

Journal format

hledger's journal file format is very similar to Ledger's. Some syntactic forms (eg hledger comments vs Ledger comments, or balance assertions) can be interpreted in slightly different ways. A small number of Ledger's syntactic forms are ignored (lot notation) or rejected (value expressions). With some care to restrict yourself to compatible features, or to keep non-compatible features in separate files, it's possible to keep a journal file that works with both hledger and Ledger simultaneously. See also #1752.

Here is a detailed list of Ledger's file format features, from the Ledger manual as of 2022-12, and their status in hledger 1.28, hledger dev, and intended (Yes / Ignored / No).

Supported in hledger ?1.28devNotesGoal
Transactions
5.1 Basic formatYY
5.2 Eliding amountsYY
5.3 Auxiliary datesYY
5.4 CodesYY
5.5 Transaction stateYY
5.6 Transaction notesYY
5.7 MetadataYY
5.7.1 Metadata tagsYYformat is TAG1:, TAG2: not :TAG1:TAG2:
5.7.1.1 Payee metadata tagNN
5.7.2 Metadata valuesYYvalues are terminated by comma, can have multiple tag/values on one line
5.7.3 Typed metadataNNdate:/date2: values are checked as dates, all other tag values are strings
5.8 Virtual postingsYY
5.9 Expression amountsNN
5.10 Balance verificationYY
5.10.1 Balance assertionsYY
5.10.1.1 Special assertion value 0YY
5.10.2 Balance assignmentsYY
5.10.3 Resetting a balanceYY
5.10.4 Balancing transactionsYY
5.11 Posting costYY
5.12 Explicit posting costsYY
5.12.1 Primary and secondary commoditiesNN
5.13 Posting cost expressionsNN
5.14 Total posting costsYY
5.15 Virtual posting costsIIthe parentheses have no effect
5.16 Commodity pricesIIcost basis is not tracked separately from costY
5.16.1 Total commodity pricesIILedger lot notation is ignored, but transactions may fail to balance as a resultY
5.17 Prices versus costsNN
5.18 Fixated prices and costsII
5.19 Lot datesIIY
5.20 Lot notesNIY
5.21 Lot value expressionsNN
5.22 Automated TransactionsYY
5.22.1 Amount multipliersYYdifferent syntax
5.22.2 Accessing the matching posting’s amountNN
5.22.3 Referring to the matching posting’s accountNN
5.22.4 Applying metadata to every matched postingYY
5.22.5 Applying metadata to the generated postingYY
5.22.6 State flagsYY
5.22.7 Effective DatesYYsame as Auxiliary Dates
5.22.8 Periodic TransactionsYY
((Amount valuation expressions))NI
Directives link
P historical (market) pricesYY
= An automated transaction.YY
~ A periodic transaction.YYcertain period expressions can only start on an interval boundary (fixed in dev)
; # % * | comment linesYYbut not % or |
! or @ as a directive prefixnot @Y
account pre-declare account namesYY
account subdirectivesIII/Y
apply account set a default parent accountYY
apply fixed set fixated pricesNI
apply tag assign a tag to transactionsNI
alias rewrite account namesYY
assert error if a value expression failsNIuse hledger check or hledger-check-fancyassertions
bucket/A set a default balancing accountNI
capture replace accounts matched by regex with anotherNIcan be emulated with regex alias
check warn if a value expression failsNIuse hledger check or hledger-check-fancyassertions
comment start multi-line commentsYY
commodity pre-declare commoditiesYY
commodity subdirectivesNIall but format are ignoredI/Y
define define value expressions for future useNI
end/end apply shorthand for ending most recent apply blockNNY
end apply accountYY
end apply fixedNI
end apply tagNI
end apply yearNIY
end tagNI
eval/expr evaluate a value expressionNI
include include another fileYY
payee pre-declare payee namesYY
payee subdirectivesNII/Y
python embed python in journalNI
tag pre-declare tag namesIIY
tag subdirectivesNI
test, a synonym for commentNN
value EXPR set a default valuation function for all commoditiesNI
Y/year/apply year set the year for year-less datesonly YY
N COMM ignore pricing information for a commodityII
D AMT set a default commodity and its formatYY
C AMT1 = AMT2 declare a commodity equivalencyNII/Y
I, i, O, o, b, h timeclock entries in journalNNtimeclock data must be in a separate file (can be included)
--command-line-flags in journalNI

Decoding errors

hledger, like most Haskell programs, exits with a confusing error message if it sees non-ascii data and the system locale is not configured to decode UTF-8. If your data contains non-ascii characters and hledger gives an error such as "invalid byte sequence", "mkTextEncoding: invalid argument" or similar, you must configure your locale.

Tabs and spaces

In places which normally require two or more spaces (or tabs), eg between account name and amount, ledger will also accept a single tab character. But hledger always requires two or more spaces or tabs (ensuring a visually distinct gap). So you might need to add a space in such cases.

Decimal mark

In many countries, comma is used as decimal mark. Without configuration, hledger tries to auto-detect this, to make things just work for everyone. However, it can misparse numbers containing a single digit group mark and no decimal mark, eg parsing 1,000 as 1 when it should be 1000. If you have such numbers in your data, or if you just want to be certain, you should declare the decimal mark being used. The best way is to add a decimal-mark . or decimal-mark , directive in each data file. See Decimal marks, digit group marks for more on this.

Balancing precision

Ledger and hledger can occasionally disagree on whether a transaction is balanced. In this journal, $'s precision (number of decimal places) is 2 in txn1, 4 in txn2, and 4 globally:

2022-01-01 txn1
    expenses                                 AAA 989.02 @ $1.123456  ; $1111.12045312
    checking                                  $-1111.12

2022-01-02 txn2
    expenses                                      $0.1234
    checking

Ledger checks transaction balancedness using local precisions only, so it balances with precision 2, and accepts txn1's $-0.00045312 imbalance.

hledger checks transaction balancedness using global precisions, so it balances with precision 4, and rejects txn1's imbalance. To read these entries with hledger, you have to limit $'s global precision, by adding -c '\$0.00' to the command (easiest when piping) or commodity $0.00 to the file (more permanent, when creating a new file).

More: #1964

Balance assertions / assignments

Ledger calculates balance assignments and checks balance assertions simply in the order they were parsed. hledger calculates balance assignments and checks balance assertions in date order and then (for postings on the same date) parse order. This ensures correct, reliable behaviour independent of the ordering of journal entries and files.

hledger correctly handles multiple balance assignments/assertions within a single transaction.

hledger adds a restriction on balance assignments: it does not allow balance assignments on accounts affected by auto posting rules (since in general this can make balancing the journal impossible).

Directive scope

The region affected by directives, and their behaviour with included files or sibling files, is sometimes different in hledger. This is to ensure robust, predictable behaviour. Here are hledger's Directive effects and Directives and multiple files behaviour. You might need to move directives and/or rearrange your files.

Periodic transactions

hledger understands most Ledger periodic transactions, but if you find some variants that are not supported, please report.

hledger 1.28 tends to require periodic transactions to start on a natural period boundary, unless you use one of these syntaxes. This will be fixed in 1.29.

When you do specify a custom start date, hledger will start the transactions on that date. Ledger seems to always generate them on the period boundaries.

Amount expressions

hledger does not support value expressions, Ledger's embedded programming language. In particular, parenthesised amount expressions like ($10 / 3) are not supported; these must be converted to explicit amounts. Here are the known ways:

  • Convert each expression manually, eg replace ($10 / 3) with $3.333.

  • Convert each expression with ledger. Eg in emacs, select the parenthesised expression and enter C-u M-| xargs ledger eval (and remove the newline). This might not work in all cases.

  • Convert all expressions with beancount. This is a lossy conversion, but it might be good enough. After installing ledger2beancount, beancount, and beancount2ledger (see #33), try:

    $ ledger2beancount file.ledger > file.beancount
    $ beancount2ledger file.beancount > file.journal
    

Lot notation

hledger currently does not provide automatic lot selection or a --lots report; instead you must track them manually, recording cost basis with @ and using explicit per-lot subaccounts and gain/loss postings (see https://hledger.org/investments.html).

More importantly, hledger ignores Ledger's lot notation, like -5 AAPL {$50.00} [2012/04/10] (Oh my!) @@ $375.00. (Any of {LOTUNITCOST}, {{LOTTOTALCOST}}, {=FIXEDLOTUNITCOST}, {{=FIXEDLOTTOTALCOST}}, [LOTDATE], (LOTNOTE) after a posting amount). This can disrupt transaction balancing, making files unreadable. (#1084) For now the only true workaround is to rewrite such entries to use hledger-style explicit lot notation.

Other differences

  • hledger's input data formats (journal, timeclock, timedot, ...) are separate; you can't mix timeclock entries and journal entries in one file as in Ledger. (Though a journal file can include a timeclock file.) This helps implement more useful error messages.

  • hledger supports international number formats, auto-detecting the decimal mark (comma or period), digit group mark (period, comma, or space) and digit group sizes to use for each commodity. Or, these can be declared explicitly with commodity directives.

  • hledger's default commodity directive (D) sets the commodity to be used for subsequent commodityless amounts, and also sets that commodity's display settings if such an amount is the first seen. Ledger uses D only for commodity display settings and for the entry command.

  • hledger auto postings allow only minimal customisation of the amount (just multiplying the matched amount by a constant), not a full embedded expression language like Ledger. (And we call them "auto" to avoid "automatic" vs "automated" confusion.)

  • In multi-period reports, hledger expands the report start/end dates to span whole periods.

  • hledger's print command always shows both the primary transaction date and any secondary date, in their usual positions. Ledger's print command with --aux-date replaces the primary date with any secondary date.

  • hledger always shows time balances (from timeclock or timedot data) in hours.

  • hledger always splits multi-day time sessions at midnight, to show the per-day amounts. Ledger does this only with the --day-break flag.

  • hledger's CSV/TSV/SSV-reading and import system is more mature and flexible than Ledger's convert command.

  • Ledger can report multiple errors at once; hledger reports only one error at a time.

  • Ledger can also output warnings. hledger does not print warnings; it either succeeds or fails.

  • hledger will complain if transaction or posting comments contain date: or date2: not followed by a valid date tag value.

Interoperating tips

The core of hledger's and Ledger's journal formats is the same, so you can use both tools on the same files, if you avoid syntax that is specific to one or the other.

However if you are a long-time Ledger user, you will certainly have Ledger-specific syntax, so for most Ledger users the quickest way to tap into hledger reports is some variant of

$ ledger print --raw | hledger -f- CMD

The print command omits directives. --raw prevents decimal zeroes being added to amounts and disrupting transaction balancing. The output may be journal entries readable by hledger. If so you can do quick reporting like so:

$ ledger print --raw | hledger -f- check       # check for problems
$ ledger print --raw | hledger -f- stats       # show journal statistics
$ ledger print --raw | hledger -f- is -MAS -2  # summarise monthly revenues/expenses
$ ledger print --raw | hledger -f- web         # view journal in hledger-web WUI
$ hledger-ui -f <(ledger print --raw)          # view journal in hledger-ui TUI (works in bash)

This does not help with amount expressions, like ($10 / 3). If you have those, see the workarounds above.

Nor does it help with lot notation, like -5 AAPL {$50.00} [2012/04/10] (Oh my!) @@ $375.00. This is the most difficult Ledger-hledger interop issue. For now the only true workaround is to rewrite such entries to use hledger-style lot notation (see link above).

An alternative is to segregate problematic or tool-specific data into separate tool-specific files, keeping as much data as possible in a shared common file. Then select the appropriate files for each tool, using multiple -f options, or include directives.

Another way is to do a one-way conversion to hledger format, perhaps periodically, doing whatever edits and transformations are necessary and feasible. sed, perl and/or a powerful editor with macros, like Emacs, can be a big help. Try to automate the steps as a script so you can easily redo the conversion when needed.

History

I (Simon) discovered John Wiegley's Ledger in 2006, and was very happy to find this efficient command-line reporting tool with a transparent data format. Initially, I used it to generate time reports for my job. Before long I wanted some improvements - splitting sessions at day boundaries, reporting in hours, etc.

Meanwhile, John was now busy elsewhere. For a long time the Ledger project remained stalled, with unfixed functionality/documentation bugs and an ever-looming v3 release making life hard for new users and creating friction for community growth. I did what I could to help - reporting bugs, providing support, contributing a domain and website - but I didn't want to invest in learning C++.

I was learning and investing time in Haskell, and I felt Ledger could be perhaps implemented well, and perhaps more effectively in the long run, in this language. I urgently needed a rock solid, hassle-free and enjoyable accounting tool. Also, I wanted a more active project and some way to make progress on the roadbumps and confusion facing other newcomers.

Of course I tried a little shiny-tech salesmanship on John, but couldn't expect him to start over. (At that time he was deeply in the C++ world; nowadays he is a Haskell expert!)

So in 2007 I began experimenting. I built a toy parser in a few different languages, and it was easiest in Haskell. I kept tinkering. Goals included:

  1. to get better at Haskell by building something useful to me
  2. to implement at least the basic core of Ledger, adapted for my needs
  3. to learn how well Haskell could work for real-world applications

And later:

  1. to provide a new highly-compatible implementation of at least the basics of Ledger, useful to others, with a greater focus on ease of use, reliability, documentation and web presence
  2. to experiment with new user interfaces, APIs, etc.

Before too long I had a tool that was useful to me. With Ledger still installed, and by maintaining high compatibility, I now had two implementations which could be compared at times of confusion about functionality or suspected bugs/bookkeeping errors, which was quite valuable.

Later, John returned for a while and finished Ledger version 3, the Ledger project attracted new contributors and maintainers, and incremental improvements resumed. I continued sharing discoveries and design discussions, and we have seen many ideas propagating in both directions. I think having independent but compatible implementations has been quite helpful for troubleshooting, exploring the design space, and growing the community. For a while I ran LedgerTips on twitter.

hledger shared #ledger's IRC channel until 2014, when I created the #hledger channel (now accessible on Libera IRC and Matrix).

In 2016 I set up https://plaintextaccounting.org as a common entry point and information hub.

The further adventures in hledger's development are not yet told, other than in the commit log, issue tracker and mail list, but other contributors joined the project and CREDITS notes some of their work.