An agent says: I win 97% of disputes. Promote my new rule. Scruple answers: No. You filed 128 disputes you could never win.
the blue marble
drawn in characters, a different view each visit
01 / HOW IT WORKSTest what an agent learned before it touches money.
A simulator and a set of verifiers for agents that move money. Every stage is deterministic and leaves a record.
- SimulateSeeded disputes and a double-entry ledger, 8 case types.
- VerifyCost read from ledger postings, scored per case type.
- GateReplay, canary, a named approver, rollback.
Train on it.Evaluate on it.Ship through it.
scruple demo, abridgedsynthetic data
02 / PROOFThe average said yes. One case type said no.
Two learned rules with the same condition. The bad one lowered the average loss by $10.94 a case, and made late-evidence disputes worse. Blocked.
Mean loss per case, current policy against the bad rule3,000 held-out cases
| Good rule | Bad rule | |
|---|---|---|
| Condition | Contest "item not received" disputes when delivery was confirmed | |
| Runs | After existing safeguards | Before existing safeguards |
| Mean loss per case | $38.43 → $26.85 | $38.43 → $27.49 |
| Worst case type | None got worse | Late evidence: $0.00 → $10.79 |
| Policy violations | 0 | 128 |
| Result | Promoted, canary passed, approved | Blocked. Nothing went live. |
03 / RED TEAMSeven ways to cheat. None got through.
Each attack had to fool a naive verifier first, so we know it is a real attack.
- Improve the average, break a rare caseBlocked
- Accept small disputes, refund twiceBlocked
- Escalate everything to humansBlocked
- Memorize training casesHeld
- Report zero lossIgnored
- Report zero debits959 mismatches
- Delete ledger entries185 mismatches
Still open, and pinned as tests: many submissions probing one held-out set, live drift, harm across case-type boundaries.
7 attacks, 0 throughscruple redteam
04 / WHO IT'S FOROne loop, three buyers.
The same environments and verifiers train an agent, test its releases, and give a bank evidence to approve it.
- AI labsTrain agents on money workflows with rewards they cannot game.
- BuildersTest every release of a finance agent against the same cases.
- BanksApprove agent changes with evidence a model-risk reviewer can read.
Today: simulator, verifiers, gate for learned rules. Next: step-by-step RL environments, model and prompt changes.
One loop: train, evaluate, shipsame verifiers each time
05 / AUDIT TRAILChange one entry and the chain breaks.
Every decision is hashed into a chain. Try it: edit an entry and watch every later link fail.
Integrity OK. 20 entries, head f83678b2730cb21c.
Audit chain, entries 12 to 17 from the same runSHA-256
06 / STATUSRunning today, next on the bench.
- Dispute simulator, 8 case types, double-entry ledgershipped
- Ledger, outcome and policy verifiers, 10 attack testsshipped
- Replay, canary, approval, rollback, audit, reportsshipped
- Step-by-step environments for RL trainingnext
- Reconciliation workflownext
- Gate model, prompt and tool changesnext
Reference implementation v0.1.0, 93 tests. The CLI answers to scruple and agent-gate.
Ledger rain, debits and credits in replaysynthetic