Overview

Mutation testing is a powerful technique to evaluate the quality of your test suite. Unlike code coverage, which only tells you which lines of code were executed, mutation testing tells you how well those lines are tested.

It works by automatically making small changes (mutations) to your code — like changing + to - or removing an assert — and checking if your tests fail. If the tests still pass (the mutant "survives"), it means your tests didn't catch that change, revealing a potential gap in your test suite.

Why Mutation Testing?

Code coverage can be misleading. Consider this example:

fun add(a: int, b: int): int {
    assert (a > 0) throw 5; // Validation
    return a + b;
}

If you write a test add(1, 2), you might achieve 100% line coverage. However, what if you forgot to test the validation logic?

Mutation testing will try to remove the assert (a > 0) throw 5 statement. If your test suite doesn't have a test case like add(-1, 2) that expects a failure, the mutant will survive. This tells you that even though the line is "covered", the logic within it is not actually verified.

This is especially critical for smart contracts where assertions guard security invariants. If an assertion can be removed without any test failing, that security check might be effectively untested.

Beyond testing gaps, mutation testing can also reveal implementation issues. For example, if you have two storage.save() calls and removing the second one doesn't cause tests to fail, it might indicate that the second call is redundant and could be removed to optimize gas usage.

How It Works

1. Mutant Generation: Acton scans your contract code and identifies places where it can introduce bugs (mutants).
2. Compilation: Each mutant is compiled. If a mutant doesn't compile (e.g., syntax error), it's discarded.
3. Testing: Acton runs your test suite against each valid mutant.
4. Reporting:

• Killed: The tests failed (good!).
• Survived: The tests passed (bad!). This indicates a gap in testing.
• Compile errors: Mutants that fail to compile are reported separately and excluded from the mutation score.

Running Mutation Tests

To run mutation tests, you need to specify the contract you want to mutate using the --mutate-contract flag. This corresponds to the contract name defined in your Acton.toml.

acton test --mutate --mutate-contract Wallet

Options

Parallel Workers

Mutation testing can spawn a lot of compile and test workers. By default, Acton uses the host's available parallelism and keeps one isolated mutation workspace per worker.

Use --mutation-workers <N> when you want to cap or raise concurrency for a specific run:

acton test --mutate --mutate-contract Wallet --mutation-workers 4

This is useful when you want to reduce CPU, disk, or process pressure on a laptop or CI machine. Results are still reported incrementally as workers finish, while output stays ordered by mutation ID.

Scoping Mutation Runs

Mutation testing can be expensive because each valid mutant is compiled and tested separately. In practice, most local runs should use one or more filters to keep the scope small and the feedback fast.

Changed-line modes

Use --mutation-diff to mutate only code that intersects changed lines.

• worktree: compares the current worktree with HEAD. This is the best mode for uncommitted local changes. Untracked files are treated as fully changed.
• ref: compares the current checkout against an explicit ref, tag, or commit. This mode requires --mutation-diff-ref <REF>.
• branch: compares the current branch against the merge-base with its upstream branch. If the branch has no upstream, pass --mutation-diff-ref <REF> to choose the base explicitly, for example origin/main.

Examples:

# Only mutate lines changed in the current worktree
acton test --mutate --mutate-contract Wallet --mutation-diff worktree

# Compare against an explicit ref or commit
acton test --mutate --mutate-contract Wallet --mutation-diff ref --mutation-diff-ref HEAD~1

# Compare the current branch against its upstream merge-base
acton test --mutate --mutate-contract Wallet --mutation-diff branch

# Compare the current branch against a specific branch instead of upstream
acton test --mutate --mutate-contract Wallet --mutation-diff branch --mutation-diff-ref origin/main

Level and rule filters

Use --mutation-levels to restrict the run to selected rule levels:

acton test --mutate --mutate-contract Wallet --mutation-levels critical,major

Use --mutation-disable-rules to exclude noisy or intentionally ignored rules:

acton test --mutate --mutate-contract Wallet --mutation-disable-rules replace_plus_with_minus --mutation-disable-rules replace_minus_with_plus

All mutation filters compose. For example, this command runs only critical mutations on lines changed in the current worktree and skips one specific rule:

acton test --mutate --mutate-contract Wallet --mutation-diff worktree --mutation-levels critical --mutation-disable-rules remove_logical_not

The reported mutation score always reflects only the mutants that remain after all filters are applied.

Custom JSON rules

Use --mutation-rules-file when you want to add project-specific query-based mutations without recompiling Acton.

Custom rules are merged with the built-in set. If a custom rule uses the same rule ID as a built-in rule, the custom rule overrides it.

Example:

[
  {
    "name": "replace_plus_with_multiply_custom",
    "description": "Replace + with *",
    "explanation": "Custom arithmetic mutation loaded from JSON.",
    "level": "major",
    "group": "arithmetic",
    "matcher": {
      "type": "query",
      "query": "(binary_operator operator_name: \"+\" @op)",
      "capture": "op"
    },
    "edit": {
      "type": "replace",
      "replacement": "*"
    }
  }
]

Then run:

acton test --mutate --mutate-contract Wallet --mutation-rules-file mutation-rules.json

Print the JSON schema for editor integration or validation with:

acton meta get-schema mutation-rules

Currently, JSON rules support only the serializable query matcher and remove or replace edits.

Session logs and resume

Each mutation run gets a session ID. If you do not pass one explicitly, Acton generates a hash-like ID and starts writing append-only JSON Lines progress to .acton/mutation-sessions/<SESSION_ID>.jsonl.

Use --mutation-session-id when you want to keep that ID stable and continue the same run later:

# Start or resume a session for the current worktree changes
acton test --mutate --mutate-contract Wallet --mutation-diff worktree --mutation-session-id wallet-worktree

To resume successfully, keep the same contract and the same mutation filters. Acton validates the stored session metadata before reusing the file.

If you interrupt the run with Ctrl+C, Acton stops without marking the session as finished and prints the exact resume command for that session.

Re-running specific mutants

Use --mutation-id when you want to focus on one or more mutants from a previous run. Pass the mutation numbers shown in the report and keep the same mutation filters you used when those IDs were produced.

Examples:

# Re-run one specific mutant
acton test --mutate --mutate-contract Wallet --mutation-id 2

# Re-run several mutants from the same filtered run
acton test --mutate --mutate-contract Wallet --mutation-id 1,3

If you also use filters such as --mutation-diff, --mutation-levels, or --mutation-disable-rules, the IDs must still exist after those filters are applied. When you combine this with --mutation-session-id, the session must be created with the same filtered mutation set.

Minimum mutation score

Use --mutation-minimum-percent to turn mutation testing into a CI quality gate:

acton test --mutate --mutate-contract Wallet --mutation-minimum-percent 85

The threshold is applied to the final mutation score after filters are applied. Compile errors are reported separately and excluded from that score.

Config defaults

You can store the same filters in Acton.toml:

[test.mutation]
diff = "branch"
diff-ref = "origin/main"
mutation-levels = ["critical", "major"]
minimum-percent = 85
disable-rules = ["replace_plus_with_minus"]

CLI flags override [test.mutation] for the current run.

Requirements

For mutation testing to work correctly, your tests must build contracts using their name from Acton.toml, not direct file paths:

// ✅ Correct — uses contract name
val wallet = build("JettonWallet");

// ❌ Incorrect — uses direct file path
val wallet = build("JettonWallet", "./contracts/JettonWallet.tolk");

Mutation testing analyzes the contract specified by --mutate-contract and its dependencies. When tests use direct file paths, the mutated code won't be used, making mutation testing inefficient.

Mutation Rules

Acton currently includes 49 built-in mutation rules for assertions, storage and upgrade calls, external-message handling, control flow, arithmetic, comparisons, booleans, logical operators, and bitwise or shift operators.

See Mutation Rules for complete list.

Interpreting Results

After running mutation tests, you will see a summary of the results:

Mutation Testing
────────────────────────────────────────────────────────────
Session:  4949ed3b0ae13461
Contract: JettonWallet
Source:   contracts/JettonWallet.tolk
Files:    6
Mutants:  51

  ◉ Mutation 1/51 contracts/JettonWallet.tolk:36:17 Remove assert statements KILLED
  ◉ Mutation 2/51 contracts/JettonWallet.tolk:77:13 Remove assert statements SURVIVED
  ◉ Mutation 3/51 contracts/JettonWallet.tolk:78:13 Remove assert statements KILLED
  ◉ Mutation 4/51 contracts/JettonWallet.tolk:81:13 Remove assert statements KILLED
  ◉ Mutation 5/51 contracts/JettonWallet.tolk:82:13 Remove assert statements KILLED
  ◉ Mutation 6/51 contracts/JettonWallet.tolk:87:13 Remove assert statements KILLED
  ◉ Mutation 7/51 contracts/JettonWallet.tolk:113:13 Remove assert statements KILLED
  ◉ Mutation 8/51 contracts/JettonWallet.tolk:114:13 Remove assert statements KILLED
  ◉ Mutation 9/51 contracts/JettonWallet.tolk:134:13 Remove assert statements SURVIVED
  ...

• KILLED: Your tests successfully detected the bug.
• SURVIVED: Your tests failed to detect the bug. You should add a test case to cover this scenario.
• Session log: Progress is written incrementally to .acton/mutation-sessions/<SESSION_ID>.jsonl, which makes interrupted runs resumable.
• Mutation IDs: The number in Mutation 2/51 or Mutation #2 can be used later with --mutation-id 2 to re-run that specific mutant.

For survived mutants, Acton provides a diff showing exactly what was changed and explains why it's an issue:

  ✗ Mutation #2
  Rule:  Remove assert statements [remove_assert]
  Level: critical
  Group: assertion
  at contracts/JettonMinter.tolk:46:13
    44 │
    45 │         RequestWalletAddress => {
    46 │             assert (
    47 │                 in.valueCoins > in.originalForwardFee + MINIMAL_MESSAGE_VALUE_BOUND
    48 │             ) throw ERR_NOT_ENOUGH_AMOUNT_TO_RESPOND;
    49 │
    50 │             var respondOwnerAddress: Cell<address>? = msg.includeOwnerAddress

  Why it's bad: This assertion is not covered by tests. This could lead to security vulnerabilities if the condition is not enforced.