AL Runner: Testing Against Real Cronus Data, Debugging Restored, and Mutation-Ready Coverage

Business Central AL AL Runner Testing Open Source

I last wrote about AL Runner on August 20, covering v2.2 and v2.3. Seven releases have shipped since then, through today’s v2.10.0, and the headline one connects directly to the post I wrote this morning about bcdb : you can now point AL Runner at a real BC backup and test against the actual data in it, starting with the Cronus demo database every BC sandbox artifact ships with.

Testing against real Cronus data

Up to now, AL Runner’s in-memory database started empty and stayed empty except for whatever your own test code inserted. That’s fine for testing your own logic in isolation, but it misses an entire class of bug: code that assumes a setup table has a row, or that a No. Series is configured, or that a base-app default exists — because in a real BC company, it does. Those bugs only show up against real data.

--test-data fixes that. It hydrates the runner’s in-memory database from the BC backup shipped in the BC artifact cache — the same Cronus backup Microsoft ships with every sandbox — before your install triggers run, so your tests find the setup records a real environment actually has.

This is the payoff of bcdb, the tool I introduced this morning. bcdb is what reads the rows out of the .bak file; --test-data is what feeds them into the runner. On BC 28.1 W1’s Cronus backup, as of v2.10.0 that’s 344 tables and 39,231 rows, including table-extension fields merged in from their companion tables — 68 of those tables were being skipped entirely before that merge landed.

A few things about how it behaves that matter if you use it:

  • It’s off by default. No flag, no backup opened, no change to caching. Existing runs are untouched.
  • A row goes in through the same value codec BC itself uses, not through AL Insert — a real restore fires no OnInsert and runs no validation, so neither does this.
  • Anything it can’t rebuild faithfully, it refuses rather than guesses. That list started long — Date, DateTime, Time, DateFormula, Blob, Media, MediaSet, RecordId all refused whole tables in the first version — but each type got transcribed from BC’s own decompiled reader (NavSqlCommand.CreateNavValueFromReader in Microsoft.Dynamics.Nav.Ncl.dll, byte-identical across the 27.5 and 28.1 artifacts) rather than guessed at, and each one closed a batch of refusals. As of v2.10.0 only 12 of Cronus’s 356 tables still refuse, and none of them are a value-type problem anymore — every remaining refusal is a bare backup column this build’s AL table has no matching field for.
  • A backup with more than one company and no company named fails the run rather than picking one for you.

--country shipped in the same release, so --test-data isn’t limited to the W1 Cronus backup — it auto-provisions the localized artifact set for whichever country’s demo data you actually need.

Debugging is back

--dap restores real breakpoint debugging on the v2 architecture: set a breakpoint, hit it, inspect locals, step through AL line by line — next, stepIn, stepOut all work with real per-statement granularity, not just “stopped somewhere in this codeunit.” It runs as an actual Debug Adapter Protocol server over TCP, and a stdio transport shipped a release later so VS Code can launch the adapter directly instead of you managing a port yourself.

The interesting part is where the pause boundary actually is. BC instruments every AL statement with a StmtHit call before that statement’s own side effect runs — the same mechanism --coverage already reads. --dap hooks that same call to block the AL thread when a hit statement matches a registered breakpoint. “Stopped at line L” then means exactly what you’d expect from any other debugger: line L-1’s effects already happened, line L’s haven’t yet. Getting that boundary right mattered enough that it caught a real bug during development — the paused frame’s own statement-number field was still reporting the previous statement at the instant the hook fired, which would have shown you the wrong line number while correctly stopping at the right one.

--dap exists largely because ALchemist — a VS Code extension by Torben Leth that runs your AL through AL Runner on every save and shows the results inline, Quokka.js-style — needed it. He filed the request the day --dap’s first cut landed, and the back-and-forth from there shaped a good chunk of what came after it: the stdio transport (so ALchemist can launch the adapter directly instead of managing a TCP port), per-execution captured values instead of a single end-of-test snapshot, and a real bug he caught where Message() output and per-iteration values silently vanished on the execute path. As of --dap stdio, the raw protocol handshake is verified working against ALchemist’s own client code; wiring it into VS Code’s actual breakpoint UI is still ahead of it.

Coverage that knows which test covered which line

--coverage already told you which statements your test suite executed. As of v2.8.0, a new perTestCoverage field on the server protocol tells you which statements each individual test executed, keyed by test name.

This exists for mutation testing. Without per-test attribution, a mutation testing tool has to run every mutant against your entire test suite to find out which tests, if any, would catch it — one measured run put that at 345 mutants against 61 tests, roughly five hours. With per-test coverage, a mutation tool can narrow each mutant down to only the tests that actually touched the mutated line, since a test that never executed a statement can’t possibly fail when that statement changes. The existing coverage field is untouched, so nothing that already consumes it needs to change.

That measured run is LethAL , Torben Leth’s mutation testing tool for AL — it deliberately breaks small pieces of your code (flips a < to <=, drops a TestField, empties a block) and checks whether your tests notice. AL Runner is one of its two backends, and its own docs are upfront about the tradeoff: the authoritative backend is a real BC container over OData, because AL Runner’s asserterror doesn’t yet fail a test the way real BC does, so a mutant that only dies to that check comes back a false survivor there. AL Runner is the fast path, not the one you’d quote a mutation score from — but per-test coverage is what makes it useful as one at all.

TDD mode, and now it works under watch

The driver for this is agentic development. You can tell a coding agent to write only test cases in one pass, then constrain it to only touch implementation in the next — the same discipline as red-green TDD, enforced instead of just requested. That only works if the agent can actually run the red half without touching implementation at all. Normally it can’t: a test that calls a procedure or field the app doesn’t have yet is a compile error in AL, and BC’s own ContinueBuildOnError doesn’t cover a method body referencing a missing symbol — the whole app group gets dropped and the run exits with code 3 and zero test results. Not a red test. No test at all.

--tdd (off by default) changes that. Every [Test] procedure that references a missing symbol now reports as a failed test naming the exact diagnostic, instead of vanishing along with its whole object. The first version refused to guess at what the missing member should look like — a wrong guess would compile and produce a test that’s red for the wrong reason, worse than not running it at all. A later release added actual inference: for a member with an inferable signature, the runner generates a stub so your test compiles and fails on the assertion, not on a missing symbol. A member it can’t confidently infer still gets refused rather than guessed at.

What shipped in this window is combining that with --watch, which previously wasn’t allowed. Now it is, so you get the real loop: write the test, save, watch it fail against a generated stub, implement the real thing, save again, watch it pass, no restart in between.

A change worth knowing about before you upgrade

v2.8.0 includes a real behavior change, not just a fix: compilation now fails when BC’s compiler reports an error, even if at least one object in the bundle still emitted successfully. The old behavior kept going in that case; the new one stops and returns exit code 3 with the diagnostics, on every entry point — the default bundled path, --per-suite, --server, and --precompile.

AL that starts failing here was never valid in the first place — a real BC service tier would have refused to publish it — but the failure itself is new. If you’re driving the runner through --server from an editor integration, this now reaches you as an exitCode: 3 response with compilationErrors in a spot where tests used to just run.

The same release fixed NumberSequence (previously a null reference on any AL calling it) and query columns using Sum/Count/Average/Min/Max, which used to silently return the raw ungrouped rows instead of aggregating — a test asserting a total got a plausible wrong number instead of a failure, which is the worse of the two ways to be wrong.

Where things stand

As of v2.10.0: --test-data is hydrating real Cronus rows through the same reader that powers bcdb, real breakpoint debugging exists again with the protocol verified end-to-end, coverage can drive mutation testing, and TDD mode works the way you’d actually want to use it day to day. The full list of what shipped in each release, including the smaller fixes, is in the changelog .