scruple

An agent says: I win 97% of disputes. Promote my new rule. Scruple answers: No. You filed 128 disputes you could never win.

the blue marble

drawn in characters, a different view each visit

01 / HOW IT WORKSTest what an agent learned before it touches money.

A simulator and a set of verifiers for agents that move money. Every stage is deterministic and leaves a record.

  • SimulateSeeded disputes and a double-entry ledger, 8 case types.
  • VerifyCost read from ledger postings, scored per case type.
  • GateReplay, canary, a named approver, rollback.

Train on it.Evaluate on it.Ship through it.

scruple demo, abridgedsynthetic data

02 / PROOFThe average said yes. One case type said no.

Two learned rules with the same condition. The bad one lowered the average loss by $10.94 a case, and made late-evidence disputes worse. Blocked.

Mean loss per case, current policy against the bad rule3,000 held-out cases

Good ruleBad rule
ConditionContest "item not received" disputes when delivery was confirmed
RunsAfter existing safeguardsBefore existing safeguards
Mean loss per case$38.43 → $26.85$38.43 → $27.49
Worst case typeNone got worseLate evidence: $0.00 → $10.79
Policy violations0128
ResultPromoted, canary passed, approvedBlocked. Nothing went live.

03 / RED TEAMSeven ways to cheat. None got through.

Each attack had to fool a naive verifier first, so we know it is a real attack.

  • Improve the average, break a rare caseBlocked
  • Accept small disputes, refund twiceBlocked
  • Escalate everything to humansBlocked
  • Memorize training casesHeld
  • Report zero lossIgnored
  • Report zero debits959 mismatches
  • Delete ledger entries185 mismatches

Still open, and pinned as tests: many submissions probing one held-out set, live drift, harm across case-type boundaries.

7 attacks, 0 throughscruple redteam

04 / WHO IT'S FOROne loop, three buyers.

The same environments and verifiers train an agent, test its releases, and give a bank evidence to approve it.

  • AI labsTrain agents on money workflows with rewards they cannot game.
  • BuildersTest every release of a finance agent against the same cases.
  • BanksApprove agent changes with evidence a model-risk reviewer can read.

Today: simulator, verifiers, gate for learned rules. Next: step-by-step RL environments, model and prompt changes.

One loop: train, evaluate, shipsame verifiers each time

05 / AUDIT TRAILChange one entry and the chain breaks.

Every decision is hashed into a chain. Try it: edit an entry and watch every later link fail.

Integrity OK. 20 entries, head f83678b2730cb21c.

Audit chain, entries 12 to 17 from the same runSHA-256

06 / STATUSRunning today, next on the bench.

  • Dispute simulator, 8 case types, double-entry ledgershipped
  • Ledger, outcome and policy verifiers, 10 attack testsshipped
  • Replay, canary, approval, rollback, audit, reportsshipped
  • Step-by-step environments for RL trainingnext
  • Reconciliation workflownext
  • Gate model, prompt and tool changesnext

Reference implementation v0.1.0, 93 tests. The CLI answers to scruple and agent-gate.

Ledger rain, debits and credits in replaysynthetic