One consistent set of rules across the bank's policies and procedures.
A synthetic regional bank named Larkhaven Bank
Ingested185 documents and 1,482 clauses
Extracted359 rules read by code and 145 clauses for the AI
Paired1,552 pairs that state one rule
Compared47 contradictions and 20 variations left alone
Classified5 pairs the AI settled
Ready for review137 findings ranked by risk
The library
Larkhaven Bank keeps 185 documents in its library. They hold 1,482 clauses across policies, procedures, job aids, FAQ pages, bulletins, and state addenda, and 14 of them are older versions that are still published. Different teams wrote them over many years, and nobody can say for sure whether they still agree.
Run the review to read every document, compare every statement of every rule, and rank what needs fixing.
Contradictions82Code 47 · rule or AI 35
Duplicate steps41Copied runs of steps
Outdated versions14Old versions in use
Variations left alone20Narrowed on purpose
Open decisions0Where experts disagree
Held items0Failed a check
Hours and cost
A manual review means someone reads every single document against the rest of the library by hand. The review leaves a specialist only the findings to confirm with both clauses shown side by side.
Every number here is an assumption you can change, and only the review's own time and AI cost are measured.
Change a rule
An engineer changes the branch manager fee waiver limit in the Service Fee Waiver Policy from $50 to $40, and the review reruns only what that change touches.
Press Make the change to see exactly what the review reruns.
The review read 1 of 624 sections again, re-extracted 2 clauses, and compared 1 rule again. It reused the other 58 findings as they were and took 1.2 milliseconds against 115 for a full run.
The change created 5 contradictions because every other document still states $50.
Risk is the category's weight times the sum of three things. They're the topic's severity, the customer weight when the topic faces customers, and the breadth weight times the share of the bank's states and business lines the finding reaches. Change any weight above and the whole list ranks again at once.
Rank
Category
Rule
Keep and change
Decided by
Risk
Why these weren't flagged
20 clauses set a different value on purpose inside a narrower scope they state, and 20 planted look-alikes resemble a tracked rule. None of them is a contradiction, and staying quiet about them is half of what makes a review trustworthy. Every false alarm costs a specialist time on a rule that already works as written.
A state addendum sets its own value for its state. It differs from Account Opening Desk Card, Customer Identification Policy, Deposit Account Opening Policy, Identification Questions from the Branches, and New Account Quick Reference.
Colorado Branch Addendum
State addendum · version 1 · effective Aug 14, 2026
Applies to Colorado branches. All business lines.
Colorado requirements
In Colorado branches, accounts open fewer than 90 days are new accounts for hold and limit purposes.
ADD-CO-01@v1/s2/c2
CodeNew account period90 days
VariationLarge-cash review threshold$9,000Handling a Cash Shortage or Overage
One clause narrows the scope on purpose (For commercial accounts). It differs from Balancing a Teller Drawer, Branch Bulletin 2026-11: Large-cash review threshold, Cash Handling FAQ, Cash Handling Policy, Counting Cash Under Dual Control, Handling a Cash Shortage or Overage, Large-Cash Review Quick Reference, Logging Transactions for Large-Cash Review, Ordering Cash Shipments, Selling Excess Cash to the Vault, and Teller Cash Limits Card.
Handling a Cash Shortage or Overage
Procedure · version 3 · effective May 21, 2026
Applies to all branches. All business lines.
Steps
For commercial accounts, cash transactions of $9,000 or more go to the large-cash review queue the same day.
A state addendum sets its own value for its state. It differs from Branch Bulletin 2025-08: Vault combination change interval, Branch Security and Opening Policy, Changing Vault Combinations, Closing Checklist, Handling a Late Opener, Opening Checklist, Testing the Alarm System, and Vault Security Card.
Oklahoma Branch Addendum
State addendum · version 3 · effective Aug 22, 2026
Applies to Oklahoma branches. All business lines.
Oklahoma requirements
In Oklahoma branches, vault combinations are changed every 3 months and whenever an employee who knows the combination leaves.
ADD-OK-01@v3/s2/c2
CodeVault combination change interval3 months
VariationComplaint acknowledgment deadline3 business daysResponding to a Complaint by Phone
One clause narrows the scope on purpose (For commercial accounts). It differs from Complaint Escalation Questions, Complaint Handling FAQ, Complaint Management Policy, Escalating a Sensitive Complaint, Logging a Customer Complaint, Resolving and Closing a Complaint, and Reviewing Complaint Trends Each Quarter.
Responding to a Complaint by Phone
Procedure · version 3 · effective Nov 24, 2024
Applies to all branches. All business lines.
Steps
For commercial accounts, send the customer an acknowledgment no later than 3 business days after a written complaint arrives.
PRO-CMP-06@v3/s3/c10
CodeComplaint acknowledgment deadline3 business days
VariationComplaint resolution deadline10 business daysKansas Branch Addendum
A state addendum sets its own value for its state. It differs from Acknowledging a Written Complaint, Branch Bulletin 2026-15: Complaint resolution deadline, Complaint Escalation Checklist, Complaint Escalation Questions, Complaint Handling FAQ, Complaint Management Policy, and Responding to a Complaint by Phone.
Kansas Branch Addendum
State addendum · version 1 · effective Dec 14, 2025
Applies to Kansas branches. All business lines.
Kansas requirements
In Kansas branches, resolve complaints within 10 business days of receipt, or tell the customer when to expect a resolution.
ADD-KS-01@v1/s2/c3
AI classifierComplaint resolution deadline10 business days
VariationComplaint resolution deadline10 business daysOklahoma Branch Addendum
A state addendum sets its own value for its state. It differs from Acknowledging a Written Complaint, Branch Bulletin 2026-15: Complaint resolution deadline, Complaint Escalation Checklist, Complaint Escalation Questions, Complaint Handling FAQ, Complaint Management Policy, and Responding to a Complaint by Phone.
Oklahoma Branch Addendum
State addendum · version 3 · effective Aug 22, 2026
Applies to Oklahoma branches. All business lines.
Oklahoma requirements
In Oklahoma branches, complaints should be closed within 10 business days of the day they arrive.
A state addendum sets its own value for its state. It differs from Card Services Quick Reference, Debit Card Dispute Policy, Dispute Timelines Card, and Provisional Credit Questions.
Kansas Branch Addendum
State addendum · version 1 · effective Dec 14, 2025
Applies to Kansas branches. All business lines.
Kansas requirements
In Kansas branches, customers have 30 days from the statement date to dispute a debit card transaction.
A state addendum sets its own value for its state. It differs from Charging Dormant Account Fees, Dormancy Notice Checklist, Dormant Account FAQ, Dormant Account Policy, Identifying Dormant Accounts, and Reactivation Questions.
Texas Branch Addendum
State addendum · version 3 · effective Feb 10, 2026
Applies to Texas branches. All business lines.
Texas requirements
In Texas branches, the monthly fee for a dormant account is $10.
ADD-TX-01@v3/s2/c2
CodeDormant account fee$10
VariationNotice before dormancy60 daysOklahoma Branch Addendum
A state addendum sets its own value for its state. It differs from Branch Bulletin 2026-25: Notice before dormancy, Charging Dormant Account Fees, Dormancy Notice Checklist, Dormant Account FAQ, Dormant Account Policy, Dormant Account Quick Reference, Identifying Dormant Accounts, Reactivating a Dormant Account, and Sending Dormancy Notices.
Oklahoma Branch Addendum
State addendum · version 3 · effective Aug 22, 2026
Applies to Oklahoma branches. All business lines.
Oklahoma requirements
In Oklahoma branches, customers receive written notice 60 days before their account becomes dormant.
ADD-OK-01@v3/s2/c3
CodeNotice before dormancy60 days
VariationNotice before dormancy45 daysTexas Branch Addendum
A state addendum sets its own value for its state. It differs from Branch Bulletin 2025-10: Notice before dormancy, Branch Bulletin 2026-25: Notice before dormancy, Charging Dormant Account Fees, Dormant Account FAQ, Dormant Account Policy, Dormant Account Quick Reference, Identifying Dormant Accounts, Reactivating a Dormant Account, and Sending Dormancy Notices.
Texas Branch Addendum
State addendum · version 3 · effective Feb 10, 2026
Applies to Texas branches. All business lines.
Texas requirements
In Texas branches, customers receive written notice 45 days before their account becomes dormant.
A state addendum sets its own value for its state. It differs from Branch Bulletin 2025-13: Fee refund request window, Processing a Fee Waiver Request, Reversing a Service Fee, Service Fee Waiver Policy, and Waiver Authority Card.
Colorado Branch Addendum
State addendum · version 1 · effective Aug 14, 2026
Applies to Colorado branches. All business lines.
Colorado requirements
In Colorado branches, customers may request a fee refund within 90 days of the statement date.
ADD-CO-01@v1/s2/c1
CodeFee refund window90 days
VariationFee waivers per account per yearthreeReversing a Service Fee
One clause narrows the scope on purpose (For small business accounts). It differs from Documenting Fee Waivers in Account Notes, Fee Schedule at a Glance, Fee Waiver Quick Reference, Handling Fee Refund Requests, Service Fee Waiver Policy, and Waiver Authority Card.
Reversing a Service Fee
Procedure · version 2 · effective Oct 19, 2025
Applies to all branches. Consumer and small business accounts.
Steps
For small business accounts, limit fee waivers to three per account in each calendar year.
One clause narrows the scope on purpose (For commercial accounts). It differs from Loan Pricing Exception Policy, Pricing Exception Questions, Rate Exception Authority Card, and Requesting a Rate Exception.
Loan Pricing Exception Policy
Policy · version 4 · effective Jun 19, 2025
Applies to all branches. All business lines.
Policy statements
For commercial accounts, A branch manager can lower a loan rate by no more than 0.15% as an exception.
POL-LND-01@v4/s3/c2
CodeBranch manager rate exception limit0.15%
VariationDays overdrawn before charge-off30 daysTexas Branch Addendum
A state addendum sets its own value for its state. It differs from Closing an Overdrawn Account, Overdraft FAQ, Overdraft Policy, Overdraft Quick Reference, Referring Accounts for Charge-Off, and Reviewing Overdrawn Accounts Each Morning.
Texas Branch Addendum
State addendum · version 3 · effective Feb 10, 2026
Applies to Texas branches. All business lines.
Texas requirements
In Texas branches, accounts that remain overdrawn for 30 days are referred for charge-off.
ADD-TX-01@v3/s2/c3
CodeDays overdrawn before charge-off30 days
VariationOverdraft fee per item$29Closing an Overdrawn Account
One clause narrows the scope on purpose (For small business accounts). It differs from Assessing and Reversing Overdraft Fees, Branch Bulletin 2025-39: Overdraft fee per item, Closing an Overdrawn Account, Overdraft Fee Desk Card, Overdraft Fee Questions from Customers, Overdraft Policy, Overdraft Quick Reference, and Referring Accounts for Charge-Off.
Closing an Overdrawn Account
Procedure · version 1 · effective Jul 19, 2025
Applies to all branches. Consumer and small business accounts.
Steps
For small business accounts, an overdraft fee of $29 applies to each item the bank pays when the balance is not enough to cover it.
PRO-ODR-05@v1/s3/c10
CodeOverdraft fee per item$29
VariationClosed account record retention10 yearsColorado Branch Addendum
A state addendum sets its own value for its state. It differs from Approving Records Destruction, Destruction Approval Checklist, Legal Hold Questions, Records Retention Policy, Records Retention Schedule Card, Releasing a Legal Hold, and Shipping Records to Offsite Storage.
Colorado Branch Addendum
State addendum · version 1 · effective Aug 14, 2026
Applies to Colorado branches. All business lines.
Colorado requirements
In Colorado branches, retain closed account records for 10 years from the closing date.
ADD-CO-01@v1/s2/c3
CodeClosed account record retention10 years
VariationClosed account record retention10 yearsKansas Branch Addendum
A state addendum sets its own value for its state. It differs from Approving Records Destruction, Destruction Approval Checklist, Legal Hold Questions, Records Retention Policy, Records Retention Schedule Card, Releasing a Legal Hold, and Shipping Records to Offsite Storage.
Kansas Branch Addendum
State addendum · version 1 · effective Dec 14, 2025
Applies to Kansas branches. All business lines.
Kansas requirements
In Kansas branches, account records are kept for 10 years after the account closes.
ADD-KS-01@v1/s2/c1
CodeClosed account record retention10 years
VariationClosed account record retention5 yearsReleasing a Legal Hold
One clause narrows the scope on purpose (For commercial accounts). It differs from Approving Records Destruction, Legal Hold Questions, Records Retention Policy, and Records Retention Schedule Card.
Releasing a Legal Hold
Procedure · version 3 · effective Jan 27, 2025
Applies to all branches. All business lines.
Exceptions and escalation
For commercial accounts, retain closed account records for 5 years from the closing date.
One clause narrows the scope on purpose (For commercial accounts). It differs from Callback Checklist, Completing Wire Callback Verification, Recalling or Cancelling a Wire, Wire Cutoff Questions, and Wire Transfer Policy.
Processing Outgoing Commercial Wires
Procedure · version 4 · effective Mar 11, 2026
Applies to all branches. Commercial accounts.
Steps
For commercial accounts, call the customer back at a phone number already on file before releasing any outgoing wire of $5,000 or more.
PRO-WIR-02@v4/s3/c8
CodeWire callback threshold$5,000
Planted look-alikes
These come from the evaluation's answer key, and code never reads the key. Each one shows what code read instead.
The bank charges $3 for each withdrawal at an ATM outside its network.Assessing and Reversing Overdraft Fees
Code read this as the rule “Non-network ATM fee” and never compared it with “Overdraft fee per item” because those are different rules.
Incoming wires received by 4:30 PM Central are posted to the customer's account the same day.Second Approval for Large Wires
Code read this as the rule “Incoming wire posting time” and never compared it with “Outgoing wire cutoff” because those are different rules.
The daily ATM withdrawal limit for consumer debit cards is $800.Cash Handling FAQ
Code read this as the rule “Daily ATM withdrawal limit” and never compared it with “Teller drawer limit” because those are different rules.
Debit card authorization holds drop off within 3 business days if the merchant does not settle the charge.Funds Availability Policy
Code read this as the rule “Card authorization hold” and never compared it with “Standard check hold” because those are different rules.
Tell customers a replacement card will arrive within 7 business days.Investigating a Dispute
Code read this as the rule “Replacement card delivery” and never compared it with “Provisional credit deadline” because those are different rules.
Send the visit survey within 5 business days after the customer's branch visit.Escalating a Sensitive Complaint
Code read this as the rule “Survey timing” and never compared it with “Complaint acknowledgment deadline” because those are different rules.
Send dormant accounts one statement every 12 months.Dormant Account Policy
Code read this as the rule “Dormant account statements” and never compared it with “Inactivity before dormancy” because those are different rules.
Records under a legal hold must not be destroyed until Compliance releases the hold in writing.Funds Availability Policy
Code read this as the rule “Legal hold” and never compared it with “Standard check hold” because those are different rules.
The bank charges $3 for each withdrawal at an ATM outside its network.Referring Accounts for Charge-Off
Code read this as the rule “Non-network ATM fee” and never compared it with “Overdraft fee per item” because those are different rules.
Incoming wires received by 4:30 PM Central are posted to the customer's account the same day.Completing Wire Callback Verification
Code read this as the rule “Incoming wire posting time” and never compared it with “Outgoing wire cutoff” because those are different rules.
The daily ATM withdrawal limit for consumer debit cards is $800.Cash Handling FAQ
Code read this as the rule “Daily ATM withdrawal limit” and never compared it with “Teller drawer limit” because those are different rules.
An authorization hold that the merchant never settles releases within 3 business days.Explaining Holds to Customers
Code read this as the rule “Card authorization hold” and never compared it with “Standard check hold” because those are different rules.
Tell customers a replacement card will arrive within 7 business days.Provisional Credit Questions
Code read this as the rule “Replacement card delivery” and never compared it with “Provisional credit deadline” because those are different rules.
Send the visit survey within 5 business days after the customer's branch visit.Complaint Management Policy
Code read this as the rule “Survey timing” and never compared it with “Complaint acknowledgment deadline” because those are different rules.
Dormant accounts receive a statement every 12 months.Identifying Dormant Accounts
Code read this as the rule “Dormant account statements” and never compared it with “Inactivity before dormancy” because those are different rules.
Records under a legal hold must not be destroyed until Compliance releases the hold in writing.Check Hold Questions from Customers
Code read this as the rule “Legal hold” and never compared it with “Standard check hold” because those are different rules.
Withdrawals at ATMs outside the network are charged $3 each.Branch Bulletin 2025-39: Overdraft fee per item
Code read this as the rule “Non-network ATM fee” and never compared it with “Overdraft fee per item” because those are different rules.
Post incoming wires the same day when they arrive by 4:30 PM Central.Wire Cutoff Questions
Code read this as the rule “Incoming wire posting time” and never compared it with “Outgoing wire cutoff” because those are different rules.
Consumer debit cards can withdraw up to $800 a day at ATMs.Cash Handling FAQ
Code read this as the rule “Daily ATM withdrawal limit” and never compared it with “Teller drawer limit” because those are different rules.
Debit card authorization holds drop off within 3 business days if the merchant does not settle the charge.Funds Availability Policy
Code read this as the rule “Card authorization hold” and never compared it with “Standard check hold” because those are different rules.
Where the experts disagree
An operations lead and a compliance officer labeled every evaluation pair, and both are synthetic. They agree on the clear cases and disagree in four groups where reasonable people do. Each group gets one decision with its reason, and the decision becomes a rule code applies, a new version of the answer key, and a test.
Their agreement beyond chance is a kappa of 0.874 before any decision and 1.0 with 4 of 4 groups decided.
An exception one document leaves out
6 pairs
One document states a rule with an exception path and another states it without one. Is that a conflict?
Verifying Customer Identification, Procedure version 4
Every signer must present two separate identification documents before the account is opened.
Deposit Account Opening Policy, Policy version 2
Do not open an account until you have verified two forms of identification for each signer, or one form plus a verified bank reference when the branch manager approves.
Marisol OrtegaCompliance officer
conflict
Staff who follow the procedure will refuse requests the policy allows, and that means the two documents disagree in practice.
Dana WhitfieldOperations lead
consistent
The procedure covers the everyday case, and the exception lives in the policy where it belongs.
Decided
It's a conflict. Staff follow the procedure in front of them, and when it leaves out an exception the policy grants they end up refusing customers the policy allows.
It now settles every pair in this group by rule, it labels them in version 2 of the answer key, and a test holds code to it.
Days against business days
6 pairs
One document gives a deadline in days and another gives the same number in business days. Is that a conflict?
Funds Availability Policy, Policy version 5
Place a standard hold of 2 business days on deposited checks that don't qualify for next-day availability.
Funds Availability FAQ, FAQ version 2
Place a standard hold of 2 days on deposited checks that don't qualify for next-day availability.
Marisol OrtegaCompliance officer
conflict
Days and business days can differ by four days, and customers are told the shorter one.
Dana WhitfieldOperations lead
consistent
Everyone in the branches reads this deadline as business days.
Decided
It's a conflict. Ten days and ten business days can be four days apart, and a deadline staff give a customer has to mean one thing everywhere it's written.
It now settles every pair in this group by rule, it labels them in version 2 of the answer key, and a test holds code to it.
A general role against a specific one
6 pairs
One document names a specific role and another says “a manager.” Does the general wording widen who can approve?
Records Retention Policy, Policy version 5
The records custodian must approve the destruction of any records in writing.
Shipping Records to Offsite Storage, Procedure version 3
Records may be destroyed only with written approval from a manager.
Marisol OrtegaCompliance officer
conflict
An assistant manager or a head teller could read a general role as permission, and that widens who can approve.
Dana WhitfieldOperations lead
consistent
In a branch the manager is the branch manager.
Decided
It's a conflict. Approval authority has to name one role, and a general word like manager lets an assistant manager or a head teller approve when the policy only meant the branch manager.
It now settles every pair in this group by rule, it labels them in version 2 of the answer key, and a test holds code to it.
An older version still published
3 pairs
An older version of a document is still published beside the current one. Is that a live conflict or a cleanup task?
Closing Checklist, Job aid version 4
Vault combination change interval | 6 months
Closing Checklist, Job aid version 3
Vault combination change interval | 12 months
Marisol OrtegaCompliance officer
conflict
The older version is still published where a branch can follow it today, and that keeps it a live conflict until it's archived.
Dana WhitfieldOperations lead
superseded
The newer version replaced it, and that makes this a cleanup task.
Decided
It's a conflict. A branch that can still open the older version can still follow it, and that makes it a live conflict until it's archived.
It now settles every pair in this group by rule, it labels them in version 2 of the answer key, and a test holds code to it.
Consolidated drafts
Each contradicted rule gets one consolidated section that replaces every clause that disagrees with the kept one. 34 of 34 rules have a draft so far.
Each value in a draft is a citation that code fills from its clause, and a gate fails any draft with a value it can't trace. A specialist approves every draft.
The judge
The judge comes from a different model family than the drafter, and it grades every draft on four criteria. It's trusted on a criterion only where it agreed with both experts on drafts with known defects, and a person checks every other criterion. Each draft below shows the judge's grades and the words it quoted.
Criterion
Agreed with the experts
Who grades it
Keeps every rule
30 of 30
Graded by the judge
Adds no rule
30 of 30
Graded by the judge
Keeps every scope
30 of 30
Graded by the judge
Reads plainly
27 of 27
Graded by the judge
Approver for identification exceptions
Passed the gateApproved by the specialist
Approving identification exceptions
Only the branch manager can approve an exception to the identification requirements.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Only the {clause:POL-AOP-01@v2/s3/c1#value} can approve an exception to the identification requirements. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Any exception to the identification requirements must be approved by a manager.Only theregionalbranchmanager can approve an exception to the identification requirements.Approver for identification exceptions | regional manager
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 4 clauses behind this draft
Deposit Account Opening Policy Only the branch manager can approve an exception to the identification requirements.
Branch Bulletin 2025-30: Approver for identification exceptions Any exception to the identification requirements must be approved by a manager.
Identification Questions from the Branches Only the regional manager can approve an exception to the identification requirements.
Identification Checklist Approver for identification exceptions | regional manager
Forms of identification per signer
Passed the gateApproved by the specialist
Identification for each signer
Do not open an account until you have verified two forms of identification for each signer. When the branch manager approves, you can instead accept one form of identification plus a verified bank reference.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Do not open an account until you have verified {clause:POL-AOP-02@v2/s4/c1#value} forms of identification for each signer. When the branch manager approves, you can instead accept {clause:POL-AOP-02@v2/s4/c1#v2} form of identification plus a verified bank reference. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Collect two forms of identification from each signer before opening the account.Do not open an account until you have verified two forms of identification for each signer.FormsWhen the branch manager approves, you can instead accept one formof identificationper signer | two Every signer must present two separate identification documents before the account is opened. Do not open an account until you haveplus averifiedtwo forms of identification for each signer. Every signer must present one separate identification documents before the account is opened. Every signer must present two separate identification documents before the account is opened.bank reference.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 8 clauses behind this draft
Customer Identification Policy Do not open an account until you have verified two forms of identification for each signer, or one form plus a verified bank reference when the branch manager approves.
Account Opening FAQ Collect two forms of identification from each signer before opening the account.
Identification Questions from the Branches Do not open an account until you have verified two forms of identification for each signer.
Identification Checklist Forms of identification per signer | two
Opening a Consumer Checking Account Every signer must present two separate identification documents before the account is opened.
Opening a Savings Account Do not open an account until you have verified two forms of identification for each signer.
Opening a Small Business Account Every signer must present one separate identification documents before the account is opened.
Verifying Customer Identification Every signer must present two separate identification documents before the account is opened.
Minimum opening deposit
Passed the gateApproved by the specialist
Minimum deposit for consumer checking
The minimum opening deposit for a consumer checking account is $25.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
The minimum opening deposit for a consumer checking account is {clause:POL-AOP-01@v2/s3/c2#value}. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
The minimum opening deposit for a consumer checking account is$100. Minimum opening deposit for consumer checking | $50$25.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Deposit Account Opening Policy The minimum opening deposit for a consumer checking account is $25.
Account Opening FAQ The minimum opening deposit for a consumer checking account is $100.
An account that has been open fewer than 30 days is a new account for hold and limit purposes.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
An account that has been open fewer than {clause:POL-AOP-02@v2/s3/c1#value} is a new account for hold and limit purposes. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
AccountsAn account that has beenopen fewer than9030daysareis anewaccountsaccountfor hold and limit purposes.An account is treated as a new account for the first 60 days after it opens.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Customer Identification Policy Accounts open fewer than 30 days are new accounts for hold and limit purposes.
Branch Bulletin 2026-29: How long an account counts as new Accounts open fewer than 90 days are new accounts for hold and limit purposes.
Identification Questions from the Branches An account is treated as a new account for the first 60 days after it opens.
Dual control at opening
Passed the gateApproved by the specialist
Dual control when opening the branch
two employees must open the branch together. No employee opens the branch alone.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
{clause:POL-BOP-01@v5/s3/c1#v2} employees {clause:POL-BOP-01@v5/s3/c1#value} open the branch together. No employee opens the branch alone. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
One employee maytwo employees mustopen the branchwhentogether. No employee opensthesecond opener is running late, as long as the alarm log is noted.branch alone.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Branch Security and Opening Policy The branch is opened by two employees together, never by one.
Branch Opening FAQ One employee may open the branch when the second opener is running late, as long as the alarm log is noted.
Teller drawer limit
Passed the gateApproved by the specialist
Teller drawer cash limit
Keep no more than $15,000 in a teller drawer at any time.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Keep no more than {clause:POL-CSH-01@v3/s3/c1#value} in a teller drawer at any time. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
AKeep no more than $15,000 in ateller drawermay not hold more than $20,000at any time.Sell excess cash to the vault whenever a teller drawer goes above $20,000.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Cash Handling Policy A teller drawer may not hold more than $15,000 at any time.
Large-Cash Review Questions A teller drawer may not hold more than $20,000 at any time.
Opening and Closing the Vault Sell excess cash to the vault whenever a teller drawer goes above $20,000.
Dual control for the vault
Passed the gateApproved by the specialist
Dual control for the vault
The vault must be opened under dual control with two employees present.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
The vault {clause:POL-CSH-01@v3/s4/c2#value} be opened under dual control with {clause:POL-CSH-01@v3/s4/c2#v2} employees present. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
DualThe vault must be opened under dualcontrolfor the vault | Optionalwith two employees present.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Cash Handling Policy The vault must be opened under dual control, with two employees present.
Teller Cash Limits Card Dual control for the vault | Optional
Vault combination change interval
Passed the gateApproved by the specialist
Changing vault combinations
Vault combinations are changed every 6 months. They are also changed whenever an employee who knows the combination leaves.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Vault combinations are changed every {clause:POL-BOP-01@v5/s4/c1#value}. They are also changed whenever an employee who knows the combination leaves. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
ChangeVault combinations are changed every 6 months. They are also changed whenever an employee who knowsthevaultcombinationat least every 12 months. Change the vault combination at least every 12 months.leaves.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Branch Security and Opening Policy Vault combinations are changed every 6 months and whenever an employee who knows the combination leaves.
Branch Bulletin 2025-08: Vault combination change interval Change the vault combination at least every 12 months.
Testing the Alarm System Change the vault combination at least every 12 months.
Complaint acknowledgment deadline
Passed the gateApproved by the specialist
Acknowledging written complaints
Staff send the customer an acknowledgment no later than 2 business days after a written complaint arrives.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Staff send the customer an acknowledgment no later than {clause:POL-CMP-01@v5/s4/c2#value} after a written complaint arrives. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Complaint acknowledgment deadline | 3 business days SendStaff sendthe customer an acknowledgment no later than 2businessdays after a written complaint arrives.Acknowledge every written complaint within 3 business days.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 4 clauses behind this draft
Complaint Management Policy Send the customer an acknowledgment no later than 2 business days after a written complaint arrives.
Complaint Escalation Checklist Complaint acknowledgment deadline | 3 business days
Logging a Customer Complaint Send the customer an acknowledgment no later than 2 days after a written complaint arrives.
Acknowledging a Written Complaint Acknowledge every written complaint within 3 business days.
Unfair treatment escalation
Passed the gateApproved by the specialist
Escalating unfair treatment complaints
Staff escalate every complaint that alleges unfair treatment to the regional manager the same day.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Staff escalate every complaint that alleges unfair treatment to the {clause:POL-CMP-01@v5/s4/c1#value} the same day. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
ComplaintsStaff escalate every complaintthatallegeallegesunfair treatmentare escalated to management the same day. Complaints that allege unfair treatment are escalatedto thebranchregionalmanager the same day.Complaints that allege unfair treatment are escalated to management the same day.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 4 clauses behind this draft
Complaint Management Policy Complaints that allege unfair treatment are escalated to the regional manager the same day.
Branch Bulletin 2025-32: Escalation for unfair treatment complaints Complaints that allege unfair treatment are escalated to management the same day.
Complaint Escalation Questions Complaints that allege unfair treatment are escalated to the branch manager the same day.
Responding to a Complaint by Phone Complaints that allege unfair treatment are escalated to management the same day.
Complaint resolution deadline
Passed the gateApproved by the specialist
Complaint resolution deadline
Staff resolve each complaint within 15 business days of receipt or tell the customer when to expect a resolution.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Staff resolve each complaint within {clause:POL-CMP-01@v5/s3/c1#value} of receipt or tell the customer when to expect a resolution. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Complaints should be closed within 15 days of the day they arrive. Complaints should be closed within 15 days of the day they arrive. Complaints should be closedStaff resolve each complaintwithin 15 business days ofreceipt or telltheday they arrive. Complaint resolution deadline | 10 business days Complaint resolution deadline | 15 business dayscustomer when to expect a resolution.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 6 clauses behind this draft
Complaint Management Policy Resolve complaints within 15 business days of receipt, or tell the customer when to expect a resolution.
Branch Bulletin 2026-15: Complaint resolution deadline Complaints should be closed within 15 days of the day they arrive.
Complaint Handling FAQ Complaints should be closed within 15 days of the day they arrive.
Complaint Escalation Questions Complaints should be closed within 15 business days of the day they arrive.
Complaint Handling Card Complaint resolution deadline | 10 business days
Complaint Escalation Checklist Complaint resolution deadline | 15 business days
Provisional credit deadline
Passed the gateApproved by the specialist
Provisional credit deadline
Staff credit the customer provisionally no later than 10 business days after the dispute form is complete.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Staff credit the customer provisionally no later than {clause:POL-DSP-01@v3/s4/c1#value} after the dispute form is complete. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
ProvisionalStaffcreditthe customer provisionally no later than 10 business days after the dispute formisissued within 10 days of receiving a completed dispute.complete.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Debit Card Dispute Policy Credit the customer provisionally no later than 10 business days after the dispute form is complete.
Investigating a Dispute Provisional credit is issued within 10 days of receiving a completed dispute.
Dispute review threshold
Passed the gateApproved by the specialist
Specialist review of larger disputes
A card services specialist reviews every dispute over $500 before credit is issued.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
A card services specialist reviews every dispute over {clause:POL-DSP-01@v3/s3/c1#value} before credit is issued. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Dispute amount that needs card services review | $1,000 Disputes over $1,000 are reviewed by aAcard services specialistreviews every dispute over $500before credit is issued.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Debit Card Dispute Policy Disputes over $500 are reviewed by a card services specialist before credit is issued.
Taking a Debit Card Dispute Disputes over $1,000 are reviewed by a card services specialist before credit is issued.
Card dispute window
Passed the gateApproved by the specialist
Debit card dispute window
Customers have 60 days from the statement date to dispute a debit card transaction. The window is longer when the customer was traveling or hospitalized.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Customers have {clause:POL-DSP-01@v3/s4/c2#value} from the statement date to dispute a debit card transaction. The window is longer when the customer was traveling or hospitalized. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Accept debit card disputes filed within 30 days of the statement that first showed the charge.Customers have 60 days from the statement date to dispute a debit card transaction.Debit card disputeThewindow| 60 days Debit card dispute window | 60 daysis longer when the customer was traveling or hospitalized.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 5 clauses behind this draft
Debit Card Dispute Policy Customers have 60 days from the statement date to dispute a debit card transaction, or longer when the customer was traveling or hospitalized.
Branch Bulletin 2026-20: Debit card dispute window Accept debit card disputes filed within 30 days of the statement that first showed the charge.
Provisional Credit Questions Customers have 60 days from the statement date to dispute a debit card transaction.
Dispute Timelines Card Debit card dispute window | 60 days
An account with no activity initiated by the customer for 12 months is classified as dormant.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
An account with no activity initiated by the customer for {clause:POL-DRM-01@v6/s4/c1#value} is classified as dormant. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Inactivity before anAnaccountwith no activity initiated by the customer for 12 monthsisdormant | 18 monthsclassified as dormant.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Dormant Account Policy An account with no customer-initiated activity for 12 months is classified as dormant.
Dormant Account Quick Reference Inactivity before an account is dormant | 18 months
Notice before dormancy
Passed the gateApproved by the specialist
Notice before an account becomes dormant
Customers receive written notice 30 days before their account becomes dormant.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Customers receive written notice {clause:POL-DRM-01@v6/s4/c2#value} before their account becomes dormant. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Customers receive written notice6030days before their account becomes dormant.Notice before dormancy | 45 days
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Dormant Account Policy Customers receive written notice 30 days before their account becomes dormant.
Branch Bulletin 2025-10: Notice before dormancy Customers receive written notice 60 days before their account becomes dormant.
Dormancy Notice Checklist Notice before dormancy | 45 days
Fee refund window
Passed the gateApproved by the specialist
Fee refund request window
Customers may request a fee refund within 60 days of the statement date.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Customers may request a fee refund within {clause:POL-FEE-01@v2/s4/c1#value} of the statement date. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Fee refund requests are accepted for 30 days after the statement date.Customers may request a fee refund within9060days of the statement date.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Service Fee Waiver Policy Customers may request a fee refund within 60 days of the statement date.
Branch Bulletin 2025-13: Fee refund request window Fee refund requests are accepted for 30 days after the statement date.
Explaining Fees to a New Customer Customers may request a fee refund within 90 days of the statement date.
Branch manager fee waiver limit
Passed the gateApproved by the specialist
Branch manager fee waivers
A branch manager may waive up to $50 in fees for one account in a calendar year.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
A branch manager may waive up to {clause:POL-FEE-01@v2/s3/c2#value} in fees for one account in a calendar year. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
The most aAbranch managercanmaywaiveup to $50 in feesfor one account in a calendaryear is $75. The most a branch manager can waive for one account in a calendar year is $100.year.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Service Fee Waiver Policy The most a branch manager can waive for one account in a calendar year is $50.
Fee Refund Questions The most a branch manager can waive for one account in a calendar year is $75.
Processing a Fee Waiver Request The most a branch manager can waive for one account in a calendar year is $100.
Personal banker fee waiver limit
Passed the gateApproved by the specialist
Personal banker fee waivers
Personal bankers may waive service fees up to $25 per account in a calendar year. With branch manager approval noted in the account they may waive up to $50.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Personal bankers may waive service fees up to {clause:POL-FEE-01@v2/s3/c1#value} per account in a calendar year. With branch manager approval noted in the account they may waive up to {clause:POL-FEE-01@v2/s3/c1#v2}. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
A personal banker can waive no more than $25 for one account in a calendar year. Personal banker fee waiver limit | $25 Personal banker fee waiver limit | $15 A personal banker can waive no more than $25 for one account in a calendar year.Personal bankers may waive service fees up to $25 per account in a calendar year.With branch manager approval noted in the account they may waive up to $50.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 6 clauses behind this draft
Service Fee Waiver Policy Personal bankers may waive service fees up to $25 per account in a calendar year, or up to $50 with branch manager approval noted in the account.
Fee Refund Questions A personal banker can waive no more than $25 for one account in a calendar year.
Fee Schedule at a Glance Personal banker fee waiver limit | $25
Waiver Authority Card Personal banker fee waiver limit | $15
Reviewing Waiver Activity Each Month A personal banker can waive no more than $25 for one account in a calendar year.
Documenting Fee Waivers in Account Notes Personal bankers may waive service fees up to $25 per account in a calendar year.
Fee waivers per account per year
Passed the gateApproved by the specialist
Fee waivers per account each year
Each account may receive no more than two fee waivers in a calendar year.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Each account may receive no more than {clause:POL-FEE-01@v2/s4/c2#value} fee waivers in a calendar year. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Limit fee waivers to one per account in each calendar year.Each account may receive no more thanonetwofee waivers in a calendar year.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Service Fee Waiver Policy Each account may receive no more than two fee waivers in a calendar year.
Handling Fee Refund Requests Limit fee waivers to one per account in each calendar year.
Documenting Fee Waivers in Account Notes Each account may receive no more than one fee waivers in a calendar year.
Large deposit hold threshold
Passed the gateApproved by the specialist
Extended holds on large check deposits
Check deposits over $6,000 in one day are eligible for an extended hold on the amount over that figure.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Check deposits over {clause:POL-HLD-01@v5/s4/c2#value} in {clause:POL-HLD-01@v5/s4/c2#v2} are eligible for an extended hold on the amount over that figure. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Check deposits over$5,000$6,000in one day are eligible for an extended hold on the amount over that figure.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Funds Availability Policy Check deposits over $6,000 in one day are eligible for an extended hold on the amount over that figure.
Extending a Hold on a Large Deposit Check deposits over $5,000 in one day are eligible for an extended hold on the amount over that figure.
New account check hold
Passed the gateApproved by the specialist
Check holds on new accounts
Checks deposited to a new account can be held for as long as 7 business days.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Checks deposited to a new account can be held for as long as {clause:POL-HLD-01@v5/s4/c1#value}. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Longest check hold on a new account | 9 business daysChecks deposited to a new accountmaycanbe held forup to 7 days. New accounts can have check holds ofas long as97business days.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 4 clauses behind this draft
Funds Availability Policy New accounts can have check holds of as long as 7 business days.
Hold Calendar Quick Reference Longest check hold on a new account | 9 business days
Holds on New Accounts Checks deposited to a new account may be held for up to 7 days.
Holds on Mobile Deposits New accounts can have check holds of as long as 9 business days.
Standard check hold
Passed the gateApproved by the specialist
Standard check hold
Place a standard hold of 2 business days on deposited checks that don't qualify for next day availability.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Place a standard hold of {clause:POL-HLD-01@v5/s3/c1#value} on deposited checks that don't qualify for next day availability. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Place a standard hold of 2businessdays on deposited checks that don't qualify fornext-daynext dayavailability.Funds from deposited checks are available within 3 business days unless a longer hold applies. Place a standard hold of 1 business day on deposited checks that don't qualify for next-day availability.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 4 clauses behind this draft
Funds Availability Policy Place a standard hold of 2 business days on deposited checks that don't qualify for next-day availability.
Funds Availability FAQ Place a standard hold of 2 days on deposited checks that don't qualify for next-day availability.
Explaining Holds to Customers Funds from deposited checks are available within 3 business days unless a longer hold applies.
Holds on Mobile Deposits Place a standard hold of 1 business day on deposited checks that don't qualify for next-day availability.
Documenting pricing exceptions
Passed the gateApproved by the specialist
Documenting pricing exceptions
Staff must record the reason and the approver for every pricing exception in the loan file.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Staff {clause:POL-LND-01@v4/s3/c1#value} record the reason and the approver for every pricing exception in the loan file. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
DocumentingStaff must record the reason and the approver for everypricingexceptions | Optional Pricing exceptions may be documentedexceptionin the loanfile after closing if time is short.file.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Loan Pricing Exception Policy Record the reason and the approver for every pricing exception in the loan file.
Documenting Pricing Exceptions Pricing exceptions may be documented in the loan file after closing if time is short.
Branch manager rate exception limit
Passed the gateApproved by the specialist
Branch manager rate exceptions
A branch manager can lower a loan rate by no more than 0.25% as an exception.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
A branch manager can lower a loan rate by no more than {clause:POL-LND-01@v4/s4/c1#value} as an exception. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Rate exceptions largerA branch manager can lower a loan rate by no morethan0.15% need approval above the branch manager.0.25% as an exception.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Loan Pricing Exception Policy A branch manager can lower a loan rate by no more than 0.25% as an exception.
Branch Bulletin 2025-26: Branch manager rate exception limit Rate exceptions larger than 0.15% need approval above the branch manager.
Regional manager rate exception limit
Passed the gateApproved by the specialist
Regional manager rate exceptions
A regional manager may approve a rate exception as large as 0.50%.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
A regional manager may approve a rate exception as large as {clause:POL-LND-01@v4/s4/c2#value}. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Regional managers may approve rate exceptions of up to 0.75%.A regional managercanmayapprove a rate exception as large as0.75%.0.50%.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Loan Pricing Exception Policy A regional manager can approve a rate exception as large as 0.50%.
Rate Exception FAQ Regional managers may approve rate exceptions of up to 0.75%.
Requesting a Rate Exception A regional manager can approve a rate exception as large as 0.75%.
Days overdrawn before charge-off
Passed the gateApproved by the specialist
Referring overdrawn accounts for charge-off
Refer an account for charge-off once it has been negative for 45 days.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Refer an account for charge-off once it has been negative for {clause:POL-ODR-01@v4/s3/c1#value}. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Accounts that remain overdrawnRefer an accountfor60 days are referredcharge-off once it has been negativeforcharge-off.45 days.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Overdraft Policy Refer an account for charge-off once it has been negative for 45 days.
Referring Accounts for Charge-Off Accounts that remain overdrawn for 60 days are referred for charge-off.
Records destruction approver
Passed the gateApproved by the specialist
Approving records destruction
The records custodian must approve the destruction of any records in writing.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
The {clause:POL-REC-01@v5/s4/c1#value} must approve the destruction of any records in writing. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Records may be destroyed only with written approval from a manager.The records custodian must approve the destruction of any records in writing.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 2 clauses behind this draft
Records Retention Policy The records custodian must approve the destruction of any records in writing.
Shipping Records to Offsite Storage Records may be destroyed only with written approval from a manager.
Closed account record retention
Passed the gateApproved by the specialist
Keeping closed account records
Account records are kept for 7 years after the account closes.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Account records are kept for {clause:POL-REC-01@v5/s3/c1#value} after the account closes. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Retention for closed account records | 5 yearsAccount records are kept for57years after the account closes.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Records Retention Policy Account records are kept for 7 years after the account closes.
Destruction Approval Checklist Retention for closed account records | 5 years
Shipping Records to Offsite Storage Account records are kept for 5 years after the account closes.
Wire callback threshold
Passed the gateApproved by the specialist
Callbacks for outgoing wires
Outgoing wires of $10,000 or more require a callback to the customer at a number already on file, unless the wire goes to an account the customer has used in the past year.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Outgoing wires of {clause:POL-WIR-01@v6/s4/c2#value} or more require a callback to the customer at a number already on file, unless the wire goes to an account the customer has used in the past year. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Outgoing wires of $5,000 or more require a callback to the customer at a number already on file.Outgoing wires of $10,000 or more require a callback to the customer at a number already onfile. Outgoing wires of $10,000 or more require a callback that afile, unless thewirespecialist completes. Callback threshold for outgoing wires | $10,000 Callgoes to an accountthe customerback at a phone number already on file before releasing any outgoing wire of $10,000 or more. Outgoing wires of $10,000 or more require a callback tohas used inthecustomer at a number already on file.past year.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 7 clauses behind this draft
Wire Transfer Policy Outgoing wires of $10,000 or more require a callback to the customer at a number already on file, unless the wire goes to an account the customer has used in the past year.
Branch Bulletin 2026-11: Callback threshold for outgoing wires Outgoing wires of $5,000 or more require a callback to the customer at a number already on file.
Wire Cutoff Questions Outgoing wires of $10,000 or more require a callback to the customer at a number already on file.
Wire Cutoff Questions Outgoing wires of $10,000 or more require a callback that a wire specialist completes.
Callback Checklist Callback threshold for outgoing wires | $10,000
Processing Outgoing Consumer Wires Call the customer back at a phone number already on file before releasing any outgoing wire of $10,000 or more.
Recalling or Cancelling a Wire Outgoing wires of $10,000 or more require a callback to the customer at a number already on file.
Who completes wire callbacks
Passed the gateApproved by the specialist
Who completes wire callbacks
Every wire callback is completed by a wire specialist. The employee who took the wire request never completes the callback.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Every wire callback is completed by a {clause:POL-WIR-01@v6/s4/c3#value}. The employee who took the wire request never completes the callback. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Only an operations employee may complete theEvery wirecallbackfor an outgoing wire. Callbacks areiscompleted byan operations employee, never by thea wire specialist. Theemployee who took therequest. Whowire request nevercompleteswire callbacks | tellerthe callback.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 4 clauses behind this draft
Wire Transfer Policy Callbacks are completed by a wire specialist, never by the employee who took the request.
Branch Bulletin 2025-30: Who completes wire callbacks Only an operations employee may complete the callback for an outgoing wire.
Wire Cutoff Questions Callbacks are completed by an operations employee, never by the employee who took the request.
Wire Desk Quick Reference Who completes wire callbacks | teller
Outgoing wire cutoff
Passed the gateApproved by the specialist
Consumer and small business wire cutoff
Submit outgoing consumer and small business wires by 3:00 PM Central time for processing on the same day.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
Submit outgoing consumer and small business wires by {clause:POL-WIR-01@v6/s4/c1#value} Central time for processing on the same day. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
Submit outgoing consumer and small business wires by2:003:00PM Centraltimeforsame-day processing. Submit outgoing consumer and small business wires by 2:00 PM Central for same-day processing.processing on the same day.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Wire Transfer Policy Submit outgoing consumer and small business wires by 3:00 PM Central for same-day processing.
Wire Cutoff Questions Submit outgoing consumer and small business wires by 2:00 PM Central for same-day processing.
Recalling or Cancelling a Wire Submit outgoing consumer and small business wires by 2:00 PM Central for same-day processing.
Outgoing wire cutoff
Passed the gateApproved by the specialist
Commercial wire cutoff
The cutoff for outgoing commercial wires is 4:00 PM Central time.
Inject an untraced value
This is a labeled test. It adds one value outside any citation, and the gate fails the draft until it's removed.
The cutoff for outgoing commercial wires is {clause:POL-WIR-01@v6/s3/c2#value} Central time. On weekends the limit is $999.
the value "$999" isn't traced to a clause
What changes in the replaced clauses
The cutoff for outgoing commercial wires is3:304:00PM Central time.The cutoff for outgoing commercial wires is 3:30 PM Central time.
The judge's grades
Keeps every ruleGraded by the judge
Adds no ruleGraded by the judge
Keeps every scopeGraded by the judge
Reads plainlyGraded by the judge
The 3 clauses behind this draft
Wire Transfer Policy The cutoff for outgoing commercial wires is 4:00 PM Central time.
Completing Wire Callback Verification The cutoff for outgoing commercial wires is 3:30 PM Central time.
Recalling or Cancelling a Wire The cutoff for outgoing commercial wires is 3:30 PM Central time.
The assistant before and after
An employee's question retrieves the five best clauses by keyword search, and the assistant answers from those alone with a citation for every value. Before reconciliation the library disagrees with itself, and the answer has to say so. After reconciliation the answer cites the one consolidated rule and nothing else.
How much can a branch manager waive in fees for one account in a year?
Before reconciliation
Sources disagree
The Service Fee Waiver Policy sets the limit at $50 per account per calendar year, and the policy outranks the other sources. The sources disagree, since the Processing a Fee Waiver Request procedure states $100, the Fee Refund Questions FAQ states $75, and the Branch Bulletin and Fee Waiver FAQ both state $50.
POL-FEE-01@v2/s3/c2Service Fee Waiver Policy
PRO-FEE-01@v1/s3/c5Processing a Fee Waiver Request
A branch manager may waive up to $50 in fees for one account in a calendar year. The Service Fee Waiver Policy states this limit directly, and the other clauses retrieved for this question give the same figure.
POL-FEE-01@v2/s3/c2Service Fee Waiver Policy
How long can checks deposited to a new account be held?
Before reconciliation
Sources disagree
The sources disagree. The Funds Availability Policy allows holds of up to 7 business days on new accounts. The Holds on New Accounts procedure says 7 days without naming business days, and the Holds on Mobile Deposits procedure says 9 business days, but the policy outranks both procedures and 7 business days is the value to apply.
POL-HLD-01@v5/s4/c1Funds Availability Policy
PRO-HLD-03@v3/s3/c2Holds on New Accounts
PRO-HLD-06@v2/s3/c4Holds on Mobile Deposits
After reconciliation
Checks deposited to a new account can be held for as long as 7 business days under the Funds Availability Policy, which took effect on 2026-05-20. The general FAQ's 2-business-day availability applies only when no longer hold is in place. So it does not change the new-account limit.
POL-HLD-01@v5/s4/c1Funds Availability Policy
How long does the bank have to resolve a complaint?
Before reconciliation
None of the retrieved clauses sets a time limit for resolving a complaint. The Complaint Handling Card only says to log the complaint in the complaint system, even when it is resolved on the spot, and the other clauses cover account newness and check holds. The retrieved clauses do not answer this question.
JA-CMP-01@v1/s2/c2Complaint Handling Card
After reconciliation
Staff must resolve each complaint within 15 business days of receipt, or tell the customer when to expect a resolution. The only clause retrieved for this is a consolidated draft with no effective date and its status as current policy is unconfirmed.
draft:g-46a9b4deComplaint resolution deadline
Which disputes need card services review?
Before reconciliation
Disputes over $500 need review by a card services specialist before credit is issued. The Debit Card Dispute Policy sets this rule, and the branch bulletin and both FAQ pages state the same threshold. The sources therefore do not conflict. The debit card dispute procedure only sets the order in which specialists review disputes as they arrive and does not change which disputes need review.
Disputes over $500 need review by a card services specialist before credit is issued. The Debit Card Dispute Policy states this rule, and the branch bulletin and two FAQ pages give the same $500 threshold.
The sources disagree. The Records Retention Policy sets 7 years after the account closes and the Shipping Records to Offsite Storage procedure sets 5 years, and since a policy outranks a procedure, 7 years is the value to follow. The Kansas Branch Addendum separately sets 10 years for Kansas branches.
POL-REC-01@v5/s3/c1Records Retention Policy
PRO-REC-05@v3/s3/c3Shipping Records to Offsite Storage
ADD-KS-01@v1/s2/c1Kansas Branch Addendum
After reconciliation
Closed account records are kept for 7 years after the account closes. The Records Retention Policy sets this period, and the Approving Records Destruction procedure and the Records Retention Schedule Card state the same 7 years.
POL-REC-01@v5/s3/c1Records Retention Policy
PRO-REC-02@v1/s3/c2Approving Records Destruction
JA-REC-01@v1/s1/c1Records Retention Schedule Card
How long is an account considered a new account?
Before reconciliation
Sources disagree
The sources disagree. The New Account Quick Reference and the Account Opening Desk Card both say an account counts as new for 30 days, while the Identification Questions from the Branches FAQ says 60 days. No policy or procedure in the retrieved clauses sets this period, and the bank's ranking does not order a job aid against an FAQ, which leaves neither value ranked above the other.
JA-AOP-01@v3/s1/c2New Account Quick Reference
JA-AOP-03@v4/s1/c1Account Opening Desk Card
FAQ-AOP-02@v2/s3/c1Identification Questions from the Branches
After reconciliation
An account is treated as a new account for the first 30 days after it opens. The Deposit Account Opening Policy sets this period, and both account opening job aids state the same 30 days.
POL-AOP-01@v2/s4/c1Deposit Account Opening Policy
JA-AOP-01@v3/s1/c2New Account Quick Reference
JA-AOP-03@v4/s1/c1Account Opening Desk Card
Do pricing exceptions have to be documented?
Before reconciliation
Sources disagree
The sources disagree. The Documenting Pricing Exceptions procedure allows documentation in the loan file after closing when time is short, and the branch bulletin and the Exception Documentation Checklist say every pricing exception must be documented there with the reason and the approver. The Rate Exception Authority Card lists documentation as optional, but the procedure outranks a job aid. So the requirement to document every exception is the one to follow.
Yes. Every pricing exception must be documented in the loan file with the reason and the approver, as stated in the Approving a Rate Exception procedure, the Rate Exception Authority Card and Branch Bulletin 2026-22, and the Exception Documentation Checklist marks this documentation as required. The sources that state the rule agree, and no conflict needs to be flagged.
An account becomes dormant after 18 months of inactivity, according to the Dormant Account Quick Reference (version 4, effective 2026-07-23). That source is a job aid, and none of the retrieved clauses from a policy or procedure states the period. So the figure rests on the quick reference alone.
JA-DRM-01@v4/s1/c3Dormant Account Quick Reference
After reconciliation
None of the retrieved clauses states when an account becomes dormant. So the time period cannot be confirmed from them. The Identifying Dormant Accounts procedure only says to verify the customer's identity before reactivating a dormant account. The Texas Branch Addendum sets a $10 monthly fee for dormant accounts in Texas branches, but it does not give a time period either.
PRO-DRM-01@v1/s3/c6Identifying Dormant Accounts
ADD-TX-01@v3/s2/c2Texas Branch Addendum
What is the monthly fee on a dormant account?
Before reconciliation
Sources disagree
The sources disagree. The Dormant Account Quick Reference (v4, effective 2026-07-23) sets the monthly dormant account fee at $10, and the Texas Branch Addendum (v3) sets $10 for Texas branches. The Dormancy Notice Checklist (v2) and the two Dormant Account FAQs give $5, but all three predate the July 2026 job aid. Job aids and FAQs rank at the same level, which leaves the newer $10 figure as the most current value while the $5 sources are corrected.
JA-DRM-01@v4/s1/c2Dormant Account Quick Reference
ADD-TX-01@v3/s2/c2Texas Branch Addendum
JA-DRM-02@v2/s1/c2Dormancy Notice Checklist
FAQ-DRM-01@v1/s2/c1Dormant Account FAQ
FAQ-DRM-02@v1/s2/c1Reactivation Questions
After reconciliation
Sources disagree
The Dormant Account Policy sets the monthly dormant account fee at $5 with no location limit. The Texas Branch Addendum sets it at $10 for dormant accounts in Texas branches, and the policy does not mention that exception. So the sources disagree for Texas. The $10 figure applies only to Texas branches, and the $5 figure applies everywhere the policy governs.
POL-DRM-01@v6/s3/c1Dormant Account Policy
ADD-TX-01@v3/s2/c2Texas Branch Addendum
Who can approve an exception to the identification requirements?
Before reconciliation
Sources disagree
The sources disagree. The Deposit Account Opening Policy says only the branch manager can approve an exception to the identification requirements, and the Account Opening FAQ agrees, but the Identification Questions from the Branches FAQ says only the regional manager can. The policy outranks the FAQ. So the branch manager is the approver to follow.
POL-AOP-01@v2/s3/c1Deposit Account Opening Policy
FAQ-AOP-01@v4/s3/c1Account Opening FAQ
FAQ-AOP-02@v2/s2/c1Identification Questions from the Branches
After reconciliation
Only the branch manager can approve an exception to the identification requirements. The Deposit Account Opening Policy and the Account Opening FAQ both state this, and they agree.
POL-AOP-01@v2/s3/c1Deposit Account Opening Policy
FAQ-AOP-01@v4/s3/c1Account Opening FAQ
Readiness report
Is the review ready for a supervised pilot?
Code writes this report from the results files and no AI writes any of it. That keeps it from overstating.
Ready for a supervised pilot in which every change goes to a specialist before it publishes.
What was reviewed
The demo library holds 185 synthetic documents with 1,482 clauses in 3 formats across 13 topics.
How it was checked
Every problem in the library was planted on purpose and recorded in an answer key that froze before any detector existed. The engine was tuned on two development libraries and then frozen. It was scored once on three libraries it had never seen, and no later version of the engine was ever scored on them again.
Results on held-out data
On the three held-out libraries code alone caught 262 of 270 planted problems.
What was planted
Result
Contradictions
104 of 108
Contradictions in different words
38 of 42
Duplicate steps
75 of 75
Outdated versions
45 of 45
False alarms on deliberate variations
0 of 60
False alarms on look-alikes
0 of 60
The experts' agreement and the decisions
The two experts agree beyond chance at a kappa of 0.874 before any decision and 1.0 with 4 of 4 groups decided. Each decision becomes a rule code applies.
The judge and where people still review
The judge is trusted on 4 of 4 criteria where it matched both experts on drafts with known defects. A specialist still approves every draft.
Cost and time
Code reviews the whole demo library in 0.1 seconds. The recorded AI steps cost $1.73 across 62 calls.
Known limits
Scanned PDFs and tables inside PDFs aren't read. The parser reads Word, web pages, and Markdown.
Three planted problems span several clauses, and in each one three rules together leave no one able to act. Pairwise comparison can't see them.
Code can't read every policy statement. When it misses one a weaker document can become the authority until the rule extractor reads the clause.
The held-out libraries use the same phrasing families as the development ones. The hand-written clauses test new wording, and code alone reads few of them.
Masking uses patterns. A production deployment would add a dedicated detector for names and addresses.
The judge's calibration drafts carry clear-cut planted defects. On the 34 real drafts Gemini 3.1 Pro flagged five the specialist had approved, and the routed judge flagged none. The specialist ruled three harmless, one a gap in the source policy, and one real. That's why a person still approves every single draft.
Keyword search often misses a question's clause before reconciliation. Embeddings found some and lost others, and a ten-clause hybrid is next.
The model comparison runs each model on one library's 180 extraction clauses and a subset of 69 pairs, 8 drafts, 30 calibration drafts, and 10 questions. On the assistant a single answer separates the route from the rest. The candidates were fixed on 10/28, and a newer model such as Gemini 4 Argon joins with one owner run.
Every model is priced at list price without the prompt caching a deployment would add, and each model's thinking counts as output the way its vendor bills it.
A model's result on the same clauses can move between runs by more than the extraction bar. Close routes need repeated runs or a person's hold.
An extractor that reads a conditional approval as a general rule can make the wrong clause a rule's authority. Approval caught it on the demo.
Every bank, person, policy, and number here is synthetic.
Recommendation
Ready for a supervised pilot in which every change goes to a specialist before it publishes.
How the review works
Code reads the library, compares every rule, and makes every decision it can. Two AI steps read only what code can't, people settle what the evidence can't, and three more AI steps write the fix under code's checks. Open any box to read the code behind it on GitHub. It's the same code that built every number on this page.
Where it's deterministic, probabilistic, and human
Code settles every rule it can read, and the AI never sees those. Code checks every AI output, and people own what the evidence can't decide.
Step
Done by
Type
How it's kept honest
Parse documents into sections and clauses with stable IDs
Code
Deterministic
Byte-identical rebuilds and a fingerprint for every library
Read clear values like dollar amounts, times, deadlines, percentages, counts, roles, and must or may
Code
Deterministic
Unit tests on every pattern
Read rules from free text into a fixed form
AI step A1
Probabilistic
Every value quoted from the clause, roles from a fixed list, inferred fields flagged, and one repair
Find the statements of the same rule
Code
Deterministic
The rule vocabulary within each topic was tuned on development seeds and scored on held-out seeds
Compare values on the same rule
Code
Deterministic
Exact and explainable with unit tests
Apply scope and dates
Code
Deterministic
A deliberate variation with a stated scope is never flagged
Judge what code can't settle
AI step A2
Probabilistic
A label from a fixed list, both clauses quoted, and scored against the frozen key
Apply precedence
Code
Deterministic
The rules come from the decision records
Spot where the experts disagree
Code
Deterministic
Label comparison and Cohen's kappa
Settle a disagreement
People
Human
A decision record that becomes a rule and a test
Draft the consolidated section
AI step A3
Probabilistic
Values only through citations, a gate for untraced values, a diff by code, and a specialist's approval
Grade drafts at scale
AI step A4
Probabilistic
Calibrated against the experts by criterion and never from the drafter's family
Answer employees' questions
AI step A5
Probabilistic
Every value in a cited clause and conflicting sources flagged
Choose each AI step's model
Code
Deterministic
The cheapest model that clears the step's bar within the approved vendors
Mask account numbers and identifiers before any model call
Code
Deterministic
A test that no recorded prompt holds an unmasked pattern
Publish approved changes to the assistant's index
Code
Deterministic
Lineage from each change to its finding and its approver
The five AI steps
A1 the rule extractor
May read a clause code queued and propose the rules it states.
Never invents a value, a role, or a scope the clause doesn't state, and it never sees the answer key.
A2 the conflict classifier
May label a pair code couldn't settle and quote the words that decided it.
Never makes a finding on its own. Only a conflict label becomes one, and code checks every quote.
A3 the drafter
May write one consolidated section for a contradicted rule.
Never states a value except through a citation, and it can't publish anything.
A4 the judge
May grade drafts on the criteria where it agreed with the experts.
Never grades a draft from its own model family or decides a criterion it wasn't trusted on.
A5 the assistant
May answer an employee's question from the retrieved clauses with a citation for every value.
Never states a value no cited clause holds, and it never hides a disagreement between sources.
Results
Every result here is read from a results file the build or a scored run wrote. Each held-out library was scored once after the code and prompts it measures were frozen.
What ran
Planted problems caught
False alarms
Code alone on three held-out libraries
262 of 270
0 on 120 deliberate variations and look-alikes
Code alone on three new held-out libraries
265 of 267
0
Code with the AI steps on the same three libraries
266 of 267
0
Model routing
Each step runs on the cheapest model that clears its bar among the client's approved vendors, and the rest that clear it become fallbacks. The judge never comes from the drafter's vendor, and a drafter must pass both the judge and the gate. Choose fewer vendors to see what a narrower list costs and where quality drops.
On these routes a review of the 1,853-document library costs about 6% less in model calls than on the models each step started with.
A2 the conflict classifier moves to Claude Sonnet 5.5 on Jan 1, 2027 when Gemini 3.8 Flash's introductory price ends.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
GPT-6 Luna
97.8%
$0.097
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Claude Haiku 5.5
97.8%
$0.20
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Claude Sonnet 5.5
98.9%
$2.39
Held as the route after the engineer read its drafts
A1 the rule extractor
Gemini 3.8 Flash
100%
$3.27
Clears the bar
A1 the rule extractor
Claude Opus 5.5
98.9%
$4.68
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Gemini 3.1 Pro
98.3%
$6.16
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
GPT-6 Astra
96.7%
$6.17
Misses the bar. More than one point below the best model on exact readings
A2 the conflict classifier
GPT-6 Luna
92.2%
$0.12
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Claude Haiku 5.5
93.8%
$0.38
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Gemini 3.8 Flash
98.4%
$3.80
The route
A2 the conflict classifier
Claude Sonnet 5.5
96.8%
$3.98
Clears the bar
A2 the conflict classifier
Claude Opus 5.5
98.4%
$6.27
Clears the bar
A2 the conflict classifier
GPT-6 Astra
98.4%
$8.03
Clears the bar
A2 the conflict classifier
Gemini 3.1 Pro
93.8%
$11.85
Misses the bar. More than two points below the best model's accuracy
A3 the drafter
GPT-6 Luna
87.5%
$0.24
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Claude Haiku 5.5
75%
$0.47
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Gemini 3.8 Flash
100%
$3.05
Clears the bar
A3 the drafter
Claude Sonnet 5.5
100%
$4.82
Clears the bar
A3 the drafter
Claude Opus 5.5
100%
$8.72
Held as the route after the engineer read its drafts
A3 the drafter
GPT-6 Astra
100%
$12.97
Clears the bar
A3 the drafter
Gemini 3.1 Pro
100%
$13.23
Clears the bar
A4 the judge
GPT-6 Luna
70%
$0.17
Misses the bar. Not trusted on every criterion against the experts
A4 the judge
Claude Haiku 5.5
76.7%
$0.30
Sits out because it comes from the drafter's vendor
A4 the judge
Gemini 3.8 Flash
100%
$1.44
The route
A4 the judge
Claude Sonnet 5.5
100%
$2.92
Sits out because it comes from the drafter's vendor
A4 the judge
Claude Opus 5.5
92.6%
$6.29
Sits out because it comes from the drafter's vendor
A4 the judge
Gemini 3.1 Pro
100%
$6.45
Clears the bar
A4 the judge
GPT-6 Astra
85.2%
$7.07
Misses the bar. Not trusted on every criterion against the experts
A5 the assistant
GPT-6 Luna
75%
$0.26
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Claude Haiku 5.5
100%
$0.95
The route
A5 the assistant
Claude Sonnet 5.5
87.5%
$9.68
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Gemini 3.8 Flash
87.5%
$10.07
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Claude Opus 5.5
87.5%
$16.29
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
GPT-6 Astra
75%
$20.36
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Gemini 3.1 Pro
87.5%
$28.86
Misses the bar. More than one question in ten below the best model on the questions search could answer
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
Claude Sonnet 5.5
Held on this model after reading its output The engineer held extraction on Sonnet after GPT-6 Luna read rules into the demo's clauses that don't state them, and that hold stayed when the 180-clause comparison favored Gemini 3.8 Flash. Sonnet carries the held-out scores and the approved demo, and its two misses there were ambiguous clauses it read exactly in its scored run. Gemini 3.8 Flash would route without the hold.
Gemini 3.8 Flash
GPT-6 LunaMore than one point below the best model on exact readings
Claude Haiku 5.5More than one point below the best model on exact readings
$3.40
A2 the conflict classifier
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Claude Sonnet 5.5, Claude Opus 5.5, GPT-6 Astra
GPT-6 LunaMore than two points below the best model's accuracy
Claude Haiku 5.5More than two points below the best model's accuracy
$0.21
A3 the drafter
Claude Opus 5.5
Held on this model after reading its output The specialist read and approved Opus's 34 demo drafts, and neither the gate nor the judge can tell how a draft reads. Gemini 3.8 Flash would route without the hold.
Gemini 3.8 Flash, Claude Sonnet 5.5, GPT-6 Astra, Gemini 3.1 Pro
GPT-6 LunaThe gate or the judge failed more of its drafts than the best model's
Claude Haiku 5.5The gate or the judge failed more of its drafts than the best model's
$0.46
A4 the judge
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Gemini 3.1 Pro
GPT-6 LunaNot trusted on every criterion against the experts
$0.076
A5 the assistant
Claude Haiku 5.5
The cheapest approved model that clears the bar
None
GPT-6 LunaMore than one question in ten below the best model on the questions search could answer
$0.95 per 1,000 questions
On these routes a review of the 1,853-document library costs about 6% less in model calls than on the models each step started with.
A2 the conflict classifier moves to Claude Sonnet 5.5 on Jan 1, 2027 when Gemini 3.8 Flash's introductory price ends.
Against every vendor approved
Every step keeps the quality it has with every vendor approved.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
Claude Haiku 5.5
97.8%
$0.20
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Claude Sonnet 5.5
98.9%
$2.39
Held as the route after the engineer read its drafts
A1 the rule extractor
Gemini 3.8 Flash
100%
$3.27
Clears the bar
A1 the rule extractor
Claude Opus 5.5
98.9%
$4.68
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Gemini 3.1 Pro
98.3%
$6.16
Misses the bar. More than one point below the best model on exact readings
A2 the conflict classifier
Claude Haiku 5.5
93.8%
$0.38
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Gemini 3.8 Flash
98.4%
$3.80
The route
A2 the conflict classifier
Claude Sonnet 5.5
96.8%
$3.98
Clears the bar
A2 the conflict classifier
Claude Opus 5.5
98.4%
$6.27
Clears the bar
A2 the conflict classifier
Gemini 3.1 Pro
93.8%
$11.85
Misses the bar. More than two points below the best model's accuracy
A3 the drafter
Claude Haiku 5.5
75%
$0.47
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Gemini 3.8 Flash
100%
$3.05
Clears the bar
A3 the drafter
Claude Sonnet 5.5
100%
$4.82
Clears the bar
A3 the drafter
Claude Opus 5.5
100%
$8.72
Held as the route after the engineer read its drafts
A3 the drafter
Gemini 3.1 Pro
100%
$13.23
Clears the bar
A4 the judge
Claude Haiku 5.5
76.7%
$0.30
Sits out because it comes from the drafter's vendor
A4 the judge
Gemini 3.8 Flash
100%
$1.44
The route
A4 the judge
Claude Sonnet 5.5
100%
$2.92
Sits out because it comes from the drafter's vendor
A4 the judge
Claude Opus 5.5
92.6%
$6.29
Sits out because it comes from the drafter's vendor
A4 the judge
Gemini 3.1 Pro
100%
$6.45
Clears the bar
A5 the assistant
Claude Haiku 5.5
100%
$0.95
The route
A5 the assistant
Claude Sonnet 5.5
87.5%
$9.68
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Gemini 3.8 Flash
87.5%
$10.07
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Claude Opus 5.5
87.5%
$16.29
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Gemini 3.1 Pro
87.5%
$28.86
Misses the bar. More than one question in ten below the best model on the questions search could answer
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
Claude Sonnet 5.5
Held on this model after reading its output The engineer held extraction on Sonnet after GPT-6 Luna read rules into the demo's clauses that don't state them, and that hold stayed when the 180-clause comparison favored Gemini 3.8 Flash. Sonnet carries the held-out scores and the approved demo, and its two misses there were ambiguous clauses it read exactly in its scored run. Gemini 3.8 Flash would route without the hold.
Gemini 3.8 Flash
Claude Haiku 5.5More than one point below the best model on exact readings
$3.40
A2 the conflict classifier
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Claude Sonnet 5.5, Claude Opus 5.5
Claude Haiku 5.5More than two points below the best model's accuracy
$0.21
A3 the drafter
Claude Opus 5.5
Held on this model after reading its output The specialist read and approved Opus's 34 demo drafts, and neither the gate nor the judge can tell how a draft reads. Gemini 3.8 Flash would route without the hold.
Gemini 3.8 Flash, Claude Sonnet 5.5, Gemini 3.1 Pro
Claude Haiku 5.5The gate or the judge failed more of its drafts than the best model's
$0.46
A4 the judge
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Gemini 3.1 Pro
None
$0.076
A5 the assistant
Claude Haiku 5.5
The cheapest approved model that clears the bar
None
None
$0.95 per 1,000 questions
On these routes a review of the 1,853-document library costs about 8% less in model calls than on the models each step started with.
Against every vendor approved
A2 the conflict classifier drops from 98.4% to 96.8%.
A4 the judge goes to a person.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
GPT-6 Luna
97.8%
$0.097
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Claude Haiku 5.5
97.8%
$0.20
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Claude Sonnet 5.5
98.9%
$2.39
Held as the route after the engineer read its drafts
A1 the rule extractor
Claude Opus 5.5
98.9%
$4.68
Clears the bar
A1 the rule extractor
GPT-6 Astra
96.7%
$6.17
Misses the bar. More than one point below the best model on exact readings
A2 the conflict classifier
GPT-6 Luna
92.2%
$0.12
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Claude Haiku 5.5
93.8%
$0.38
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Claude Sonnet 5.5
96.8%
$3.98
The route
A2 the conflict classifier
Claude Opus 5.5
98.4%
$6.27
Clears the bar
A2 the conflict classifier
GPT-6 Astra
98.4%
$8.03
Clears the bar
A3 the drafter
GPT-6 Luna
87.5%
$0.24
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Claude Haiku 5.5
75%
$0.47
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Claude Sonnet 5.5
100%
$4.82
Clears the bar
A3 the drafter
Claude Opus 5.5
100%
$8.72
Held as the route after the engineer read its drafts
A3 the drafter
GPT-6 Astra
100%
$12.97
Clears the bar
A4 the judge
GPT-6 Luna
70%
$0.17
Misses the bar. Not trusted on every criterion against the experts
A4 the judge
Claude Haiku 5.5
76.7%
$0.30
Sits out because it comes from the drafter's vendor
A4 the judge
Claude Sonnet 5.5
100%
$2.92
Sits out because it comes from the drafter's vendor
A4 the judge
Claude Opus 5.5
92.6%
$6.29
Sits out because it comes from the drafter's vendor
A4 the judge
GPT-6 Astra
85.2%
$7.07
Misses the bar. Not trusted on every criterion against the experts
A5 the assistant
GPT-6 Luna
75%
$0.26
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Claude Haiku 5.5
100%
$0.95
The route
A5 the assistant
Claude Sonnet 5.5
87.5%
$9.68
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Claude Opus 5.5
87.5%
$16.29
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
GPT-6 Astra
75%
$20.36
Misses the bar. More than one question in ten below the best model on the questions search could answer
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
Claude Sonnet 5.5
Held on this model after reading its output The engineer held extraction on Sonnet after GPT-6 Luna read rules into the demo's clauses that don't state them, and that hold stayed when the 180-clause comparison favored Gemini 3.8 Flash. Sonnet carries the held-out scores and the approved demo, and its two misses there were ambiguous clauses it read exactly in its scored run. Claude Sonnet 5.5 would route without the hold.
Claude Opus 5.5
GPT-6 LunaMore than one point below the best model on exact readings
Claude Haiku 5.5More than one point below the best model on exact readings
$3.40
A2 the conflict classifier
Claude Sonnet 5.5
The cheapest approved model that clears the bar
Claude Opus 5.5, GPT-6 Astra
GPT-6 LunaMore than two points below the best model's accuracy
Claude Haiku 5.5More than two points below the best model's accuracy
$0.22
A3 the drafter
Claude Opus 5.5
Held on this model after reading its output The specialist read and approved Opus's 34 demo drafts, and neither the gate nor the judge can tell how a draft reads. Claude Sonnet 5.5 would route without the hold.
Claude Sonnet 5.5, GPT-6 Astra
GPT-6 LunaThe gate or the judge failed more of its drafts than the best model's
Claude Haiku 5.5The gate or the judge failed more of its drafts than the best model's
$0.46
A4 the judge
A person
No approved model clears this step's bar so a person does it
None
GPT-6 LunaNot trusted on every criterion against the experts
GPT-6 AstraNot trusted on every criterion against the experts
A person
A5 the assistant
Claude Haiku 5.5
The cheapest approved model that clears the bar
None
GPT-6 LunaMore than one question in ten below the best model on the questions search could answer
$0.95 per 1,000 questions
On these routes a review of the 1,853-document library costs about 14% more in model calls than on the models each step started with.
Against every vendor approved
A4 the judge goes to a person.
A5 the assistant drops from 100% to 87.5%.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
GPT-6 Luna
97.8%
$0.097
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Gemini 3.8 Flash
100%
$3.27
The route
A1 the rule extractor
Gemini 3.1 Pro
98.3%
$6.16
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
GPT-6 Astra
96.7%
$6.17
Misses the bar. More than one point below the best model on exact readings
A2 the conflict classifier
GPT-6 Luna
92.2%
$0.12
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Gemini 3.8 Flash
98.4%
$3.80
The route
A2 the conflict classifier
GPT-6 Astra
98.4%
$8.03
Clears the bar
A2 the conflict classifier
Gemini 3.1 Pro
93.8%
$11.85
Misses the bar. More than two points below the best model's accuracy
A3 the drafter
GPT-6 Luna
87.5%
$0.24
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Gemini 3.8 Flash
100%
$3.05
The route
A3 the drafter
GPT-6 Astra
100%
$12.97
Clears the bar
A3 the drafter
Gemini 3.1 Pro
100%
$13.23
Clears the bar
A4 the judge
GPT-6 Luna
70%
$0.17
Misses the bar. Not trusted on every criterion against the experts
A4 the judge
Gemini 3.8 Flash
100%
$1.44
Sits out because it comes from the drafter's vendor
A4 the judge
Gemini 3.1 Pro
100%
$6.45
Sits out because it comes from the drafter's vendor
A4 the judge
GPT-6 Astra
85.2%
$7.07
Misses the bar. Not trusted on every criterion against the experts
A5 the assistant
GPT-6 Luna
75%
$0.26
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Gemini 3.8 Flash
87.5%
$10.07
The route
A5 the assistant
GPT-6 Astra
75%
$20.36
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Gemini 3.1 Pro
87.5%
$28.86
Clears the bar
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
Gemini 3.8 Flash
The cheapest approved model that clears the bar
None
GPT-6 LunaMore than one point below the best model on exact readings
$4.65
A2 the conflict classifier
Gemini 3.8 Flash
The cheapest approved model that clears the bar
GPT-6 Astra
GPT-6 LunaMore than two points below the best model's accuracy
$0.21
A3 the drafter
Gemini 3.8 Flash
The cheapest approved model that clears the bar
GPT-6 Astra, Gemini 3.1 Pro
GPT-6 LunaThe gate or the judge failed more of its drafts than the best model's
$0.16
A4 the judge
A person
No approved model clears this step's bar so a person does it
None
GPT-6 LunaNot trusted on every criterion against the experts
GPT-6 AstraNot trusted on every criterion against the experts
A person
A5 the assistant
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Gemini 3.1 Pro
GPT-6 LunaMore than one question in ten below the best model on the questions search could answer
$10.07 per 1,000 questions
On these routes a review of the 1,853-document library costs about 8% less in model calls than on the models each step started with.
Against every vendor approved
A2 the conflict classifier drops from 98.4% to 96.8%.
A4 the judge goes to a person.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
Claude Haiku 5.5
97.8%
$0.20
Misses the bar. More than one point below the best model on exact readings
A1 the rule extractor
Claude Sonnet 5.5
98.9%
$2.39
Held as the route after the engineer read its drafts
A1 the rule extractor
Claude Opus 5.5
98.9%
$4.68
Clears the bar
A2 the conflict classifier
Claude Haiku 5.5
93.8%
$0.38
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
Claude Sonnet 5.5
96.8%
$3.98
The route
A2 the conflict classifier
Claude Opus 5.5
98.4%
$6.27
Clears the bar
A3 the drafter
Claude Haiku 5.5
75%
$0.47
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
Claude Sonnet 5.5
100%
$4.82
Clears the bar
A3 the drafter
Claude Opus 5.5
100%
$8.72
Held as the route after the engineer read its drafts
A4 the judge
Claude Haiku 5.5
76.7%
$0.30
Sits out because it comes from the drafter's vendor
A4 the judge
Claude Sonnet 5.5
100%
$2.92
Sits out because it comes from the drafter's vendor
A4 the judge
Claude Opus 5.5
92.6%
$6.29
Sits out because it comes from the drafter's vendor
A5 the assistant
Claude Haiku 5.5
100%
$0.95
The route
A5 the assistant
Claude Sonnet 5.5
87.5%
$9.68
Misses the bar. More than one question in ten below the best model on the questions search could answer
A5 the assistant
Claude Opus 5.5
87.5%
$16.29
Misses the bar. More than one question in ten below the best model on the questions search could answer
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
Claude Sonnet 5.5
Held on this model after reading its output The engineer held extraction on Sonnet after GPT-6 Luna read rules into the demo's clauses that don't state them, and that hold stayed when the 180-clause comparison favored Gemini 3.8 Flash. Sonnet carries the held-out scores and the approved demo, and its two misses there were ambiguous clauses it read exactly in its scored run. Claude Sonnet 5.5 would route without the hold.
Claude Opus 5.5
Claude Haiku 5.5More than one point below the best model on exact readings
$3.40
A2 the conflict classifier
Claude Sonnet 5.5
The cheapest approved model that clears the bar
Claude Opus 5.5
Claude Haiku 5.5More than two points below the best model's accuracy
$0.22
A3 the drafter
Claude Opus 5.5
Held on this model after reading its output The specialist read and approved Opus's 34 demo drafts, and neither the gate nor the judge can tell how a draft reads. Claude Sonnet 5.5 would route without the hold.
Claude Sonnet 5.5
Claude Haiku 5.5The gate or the judge failed more of its drafts than the best model's
$0.46
A4 the judge
A person
Every approved judge comes from the drafter's vendor so a person checks the drafts
None
None
A person
A5 the assistant
Claude Haiku 5.5
The cheapest approved model that clears the bar
None
None
$0.95 per 1,000 questions
On these routes a review of the 1,853-document library costs about 14% more in model calls than on the models each step started with.
Against every vendor approved
A4 the judge goes to a person.
A5 the assistant drops from 100% to 87.5%.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
Gemini 3.8 Flash
100%
$3.27
The route
A1 the rule extractor
Gemini 3.1 Pro
98.3%
$6.16
Misses the bar. More than one point below the best model on exact readings
A2 the conflict classifier
Gemini 3.8 Flash
98.4%
$3.80
The route
A2 the conflict classifier
Gemini 3.1 Pro
93.8%
$11.85
Misses the bar. More than two points below the best model's accuracy
A3 the drafter
Gemini 3.8 Flash
100%
$3.05
The route
A3 the drafter
Gemini 3.1 Pro
100%
$13.23
Clears the bar
A4 the judge
Gemini 3.8 Flash
100%
$1.44
Sits out because it comes from the drafter's vendor
A4 the judge
Gemini 3.1 Pro
100%
$6.45
Sits out because it comes from the drafter's vendor
A5 the assistant
Gemini 3.8 Flash
87.5%
$10.07
The route
A5 the assistant
Gemini 3.1 Pro
87.5%
$28.86
Clears the bar
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
Gemini 3.8 Flash
The cheapest approved model that clears the bar
None
None
$4.65
A2 the conflict classifier
Gemini 3.8 Flash
The cheapest approved model that clears the bar
None
None
$0.21
A3 the drafter
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Gemini 3.1 Pro
None
$0.16
A4 the judge
A person
Every approved judge comes from the drafter's vendor so a person checks the drafts
None
None
A person
A5 the assistant
Gemini 3.8 Flash
The cheapest approved model that clears the bar
Gemini 3.1 Pro
None
$10.07 per 1,000 questions
On these routes a review of the 1,853-document library costs about 71% less in model calls than on the models each step started with.
Against every vendor approved
A1 the rule extractor drops from 98.9% to 97.8%.
A4 the judge goes to a person.
A5 the assistant drops from 100% to 75%.
Quality against cost for every modelThere's one chart per step since each step has its own measure. The ringed dot is the route, and the shaded band clears the bar.
The route
Clears the bar
Misses the bar
Sits out as the drafter's vendor
The bar and the band that clears it
Cost per 1,000 items at each vendor's list price on a log scale.
Show every measurement as a table
Step
Model
Measure
Cost per 1,000 items
Result
A1 the rule extractor
GPT-6 Luna
97.8%
$0.097
The route
A1 the rule extractor
GPT-6 Astra
96.7%
$6.17
Misses the bar. More than one point below the best model on exact readings
A2 the conflict classifier
GPT-6 Luna
92.2%
$0.12
Misses the bar. More than two points below the best model's accuracy
A2 the conflict classifier
GPT-6 Astra
98.4%
$8.03
The route
A3 the drafter
GPT-6 Luna
87.5%
$0.24
Misses the bar. The gate or the judge failed more of its drafts than the best model's
A3 the drafter
GPT-6 Astra
100%
$12.97
The route
A4 the judge
GPT-6 Luna
70%
$0.17
Sits out because it comes from the drafter's vendor
A4 the judge
GPT-6 Astra
85.2%
$7.07
Sits out because it comes from the drafter's vendor
A5 the assistant
GPT-6 Luna
75%
$0.26
The route
A5 the assistant
GPT-6 Astra
75%
$20.36
Clears the bar
Step
Model
Why
Fallbacks
Cheaper models not chosen
Cost at scale
A1 the rule extractor
GPT-6 Luna
The cheapest approved model that clears the bar
None
None
$0.14
A2 the conflict classifier
GPT-6 Astra
The cheapest approved model that clears the bar
None
GPT-6 LunaMore than two points below the best model's accuracy
$0.44
A3 the drafter
GPT-6 Astra
The cheapest approved model that clears the bar
None
GPT-6 LunaThe gate or the judge failed more of its drafts than the best model's
$0.69
A4 the judge
A person
Every approved judge comes from the drafter's vendor so a person checks the drafts
None
None
A person
A5 the assistant
GPT-6 Luna
The cheapest approved model that clears the bar
GPT-6 Astra
None
$0.26 per 1,000 questions
Every model is priced on its own tokens at its list price as of 10/30/2026 without the 4,936 tokens of client context each recorded Claude call carries.
What the judge caught in the comparison's drafts
Every model passed the gate on all eight drafts. The gate proves values and nothing else, and these are the drafts the judge failed.
Model
Rule
Criterion
The judge's evidence
GPT-6 Luna
Unfair treatment escalation
Keeps every rule
the same day
Claude Haiku 5.5
Forms of identification per signer
Keeps every rule
when the branch manager approves
Claude Haiku 5.5
Unfair treatment escalation
Keeps every rule
the same day
The judge also flagged 3 drafts for adding Staff as the subject, and the specialist ruled that harmless.
Do embeddings earn their place?
Embeddings are the usual first answer when a system has to find related text. Here they had to beat what the pipeline already does on held-out libraries with their threshold tuned on development libraries alone, and they're kept only where they win under a rule that was set before the first result came in.
Pairing two statements of one rule
How a queued clause gets its rule
Paraphrased contradictions paired
Wrong rules added
Added cost per library
Added model time per library
Code alone
79 of 84
0
$0
0 s
Code and embeddings
80 of 84
3
$0.0004
7 s
Code and the rule extractor
84 of 84
6
$0.30
110 s
All three
84 of 84
6
$0.30
117 s
Embeddings pair nothing the rule extractor misses, and on their own they pair fewer. The pipeline keeps keywords and the extractor.
Finding the clause a question needs
Search
The question's clause in what the assistant reads
Its clause and the one that contradicts it
Top five by keywords
34 of 60
19 of 60
Top ten by keywords
41 of 60
37 of 60
Top five by keywords and embeddings
39 of 60
25 of 60
Top ten by keywords and embeddings (exploratory)
48 of 60
42 of 60
The hybrid doesn't beat keywords at both depths without losing a question. The assistant keeps keyword search.
A ten-clause hybrid tried after the choice found both clauses for 42 of 60 questions against 37 for keywords at ten and lost none. It's the next test.
Vectors from text-embedding-3-small on masked text. The threshold of 0.76 came from development libraries 11 and 23, and the results are from 6 held-out libraries.
Cost and time at scale
Code runs every deterministic stage on the scale library of 1,853 documents and 14,853 clauses in 2.63 seconds.
Step
Items at scale
Model
Tokens in and out per item
Estimated cost
A2 the conflict classifier
55
google-b
452 and 1,206
$0.27
A3 the drafter
53
claude-opus
1,155 and 222
$0.48
A1 the rule extractor
1,423
claude-sonnet
409 and 105
$2.66
The AI steps would cost about $3.41 for the whole scale library. That's an estimate from the demo library's tokens per item at 10/30/2026 list prices.
A real sample of 30 clauses from the scale library measured +7% against the estimate's tokens per item.
What a specialist can do alone
This answers which capabilities belong in shared services and which need an engineer.
A policy specialist alone
Search the library and compare two sections.
Open a finding and see both sources side by side.
Record a decision with its reason.
Edit and approve a consolidated draft.
Ask the assistant a question.
Needs an engineer
A new document format.
A new rule type or comparator.
A change to the precedence code.
A new version of the answer key.
Model routing and deployment.
What keeps self-serve safe
Code checks every AI output before it's used.
Decisions are versioned and each one has a test.
A draft publishes only after a specialist approves it.
The CI gate fails any regression.
Known limits
Scanned PDFs and tables inside PDFs aren't read. The parser reads Word, web pages, and Markdown.
Three planted problems span several clauses, and in each one three rules together leave no one able to act. Pairwise comparison can't see them.
Code can't read every policy statement. When it misses one a weaker document can become the authority until the rule extractor reads the clause.
The held-out libraries use the same phrasing families as the development ones. The hand-written clauses test new wording, and code alone reads few of them.
Masking uses patterns. A production deployment would add a dedicated detector for names and addresses.
The judge's calibration drafts carry clear-cut planted defects. On the 34 real drafts Gemini 3.1 Pro flagged five the specialist had approved, and the routed judge flagged none. The specialist ruled three harmless, one a gap in the source policy, and one real. That's why a person still approves every single draft.
Keyword search often misses a question's clause before reconciliation. Embeddings found some and lost others, and a ten-clause hybrid is next.
The model comparison runs each model on one library's 180 extraction clauses and a subset of 69 pairs, 8 drafts, 30 calibration drafts, and 10 questions. On the assistant a single answer separates the route from the rest. The candidates were fixed on 10/28, and a newer model such as Gemini 4 Argon joins with one owner run.
Every model is priced at list price without the prompt caching a deployment would add, and each model's thinking counts as output the way its vendor bills it.
A model's result on the same clauses can move between runs by more than the extraction bar. Close routes need repeated runs or a person's hold.
An extractor that reads a conditional approval as a general rule can make the wrong clause a rule's authority. Approval caught it on the demo.
Every bank, person, policy, and number here is synthetic.
How I'd start an engagement like this
Evaluation comes first so every later step is measured against something the client's own experts agreed on.
Week one. Gather representative documents and real questions with the client's experts, and freeze an answer key before building anything.
Week two. Build a code-only baseline and measure it against the key.
Week three. Add AI only where code can't decide, and score it once on held-out documents.
Week four. Settle the experts' disagreements with decision records and write the readiness report from the results.
The production kit
Every piece is built, and this table says plainly which ones ran and when.
The CI gate
Every test, the copy lint, the build with its strip check, the key scan, and the regression thresholds
Ran and passed on 2026-11-11
The audit log
One line for every model call with its hashes, tokens, cost, time, and validation
810 calls logged, and every one matches its recording
Masking
Account numbers, ID numbers, phone numbers, email addresses, and customer names masked before any call
0 customer details masked so far
Secrets
This public copy holds no keys at all, and the key scan proves it
The key scan finds no key in any file
Roles
A deployment signs people in as a viewer, a policy specialist who approves changes, or an engineer who changes routing and rules, and no one holds both rights
Designed for a deployment and left out of this demo
The audit log
Time
Step
Run
Model
Tokens in
Tokens out
Problems
Repair
2026-10-30 17:19
assist
cmp-6101
claude-haiku
6,180
1,073
0
2026-10-30 17:19
assist
cmp-6101
claude-haiku
6,138
895
0
2026-10-30 17:19
assist
cmp-6101
claude-haiku
6,152
1,198
0
2026-10-30 17:19
assist
cmp-6101
claude-haiku
6,174
1,481
1
2026-10-30 17:19
assist
cmp-6101
claude-haiku
6,230
2,354
0
Repair
2026-10-30 17:19
extract
cmp-6101
openai-a
2,754
686
0
2026-10-30 17:19
extract
cmp-6101
openai-a
2,693
728
0
2026-10-30 17:19
extract
cmp-6101
openai-a
2,631
790
0
2026-10-30 17:19
extract
cmp-6101
openai-a
2,678
785
0
2026-10-30 17:19
extract
cmp-6101
openai-a
2,672
709
0
2026-10-30 17:19
extract
cmp-6101
openai-a
2,660
420
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,943
767
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,878
883
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,897
834
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,856
624
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,923
602
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,820
646
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,897
648
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,039
376
0
2026-10-30 17:20
classify
cmp-6101
openai-a
2,876
680
0
2026-10-30 17:20
draft
cmp-6101
openai-a
2,191
156
0
2026-10-30 17:20
draft
cmp-6101
openai-a
2,924
193
0
2026-10-30 17:20
draft
cmp-6101
openai-a
2,149
274
0
2026-10-30 17:20
judge
cmp-6101
openai-a
982
397
0
2026-10-30 17:20
judge
cmp-6101
openai-a
987
550
0
2026-10-30 17:20
judge
cmp-6101
openai-a
1,035
546
0
2026-10-30 17:20
judge
cmp-6101
openai-a
1,000
487
0
2026-10-30 17:20
judge
cmp-6101
openai-a
971
418
0
2026-10-30 17:20
judge
cmp-6101
openai-a
1,036
641
0
2026-10-30 17:20
assist
cmp-6101
openai-a
857
284
0
2026-10-30 17:20
assist
cmp-6101
openai-a
870
267
0
2026-10-30 17:20
assist
cmp-6101
openai-a
861
76
1
2026-10-30 17:20
assist
cmp-6101
openai-a
894
175
0
Repair
2026-10-30 17:20
assist
cmp-6101
openai-a
869
173
0
2026-10-30 17:20
assist
cmp-6101
openai-a
852
52
0
2026-10-30 17:20
assist
cmp-6101
openai-a
874
274
0
2026-10-30 17:20
assist
cmp-6101
openai-a
868
81
0
2026-10-30 17:20
assist
cmp-6101
openai-a
881
210
0
2026-10-30 17:20
assist
cmp-6101
openai-a
870
60
0
2026-10-30 17:21
assist
cmp-6101
openai-a
884
106
1
2026-10-30 17:21
assist
cmp-6101
openai-a
916
215
1
Repair
2026-10-30 17:21
extract
cmp-6101
openai-b
2,631
1,296
0
2026-10-30 17:21
extract
cmp-6101
openai-b
2,754
1,508
0
2026-10-30 17:21
extract
cmp-6101
openai-b
2,693
1,435
0
2026-10-30 17:21
extract
cmp-6101
openai-b
2,660
275
0
2026-10-30 17:21
extract
cmp-6101
openai-b
2,672
1,268
0
2026-10-30 17:21
extract
cmp-6101
openai-b
2,678
2,036
0
2026-10-30 17:21
classify
cmp-6101
openai-b
2,943
1,250
0
2026-10-30 17:21
classify
cmp-6101
openai-b
2,897
1,377
0
2026-10-30 17:21
classify
cmp-6101
openai-b
2,878
1,436
0
2026-10-30 17:22
classify
cmp-6101
openai-b
2,923
879
0
2026-10-30 17:22
classify
cmp-6101
openai-b
2,856
1,120
0
2026-10-30 17:22
classify
cmp-6101
openai-b
2,897
1,192
0
2026-10-30 17:22
classify
cmp-6101
openai-b
2,820
2,321
0
2026-10-30 17:22
classify
cmp-6101
openai-b
2,039
598
0
2026-10-30 17:22
classify
cmp-6101
openai-b
2,876
1,736
0
2026-10-30 17:22
draft
cmp-6101
openai-b
2,191
276
0
2026-10-30 17:22
draft
cmp-6101
openai-b
2,149
1,010
0
2026-10-30 17:22
draft
cmp-6101
openai-b
2,924
1,104
0
2026-10-30 17:22
judge
cmp-6101
openai-b
982
911
0
2026-10-30 17:22
judge
cmp-6101
openai-b
987
1,388
0
2026-10-30 17:22
judge
cmp-6101
openai-b
1,035
1,559
0
2026-10-30 17:22
judge
cmp-6101
openai-b
1,000
1,026
0
2026-10-30 17:23
judge
cmp-6101
openai-b
971
1,553
0
2026-10-30 17:23
judge
cmp-6101
openai-b
1,036
861
1
2026-10-30 17:23
judge
cmp-6101
openai-b
1,078
1,348
0
Repair
2026-10-30 17:23
assist
cmp-6101
openai-b
861
183
0
2026-10-30 17:23
assist
cmp-6101
openai-b
870
413
0
2026-10-30 17:23
assist
cmp-6101
openai-b
869
246
0
2026-10-30 17:23
assist
cmp-6101
openai-b
857
363
1
2026-10-30 17:23
assist
cmp-6101
openai-b
888
223
0
Repair
2026-10-30 17:23
assist
cmp-6101
openai-b
852
138
0
2026-10-30 17:23
assist
cmp-6101
openai-b
874
396
0
2026-10-30 17:23
assist
cmp-6101
openai-b
868
151
0
2026-10-30 17:23
assist
cmp-6101
openai-b
881
393
0
2026-10-30 17:23
assist
cmp-6101
openai-b
870
114
0
2026-10-30 17:23
assist
cmp-6101
openai-b
884
215
1
2026-10-30 17:23
assist
cmp-6101
openai-b
916
218
1
Repair
2026-10-30 17:23
extract
cmp-6101
google-a
2,942
3,712
0
2026-10-30 17:23
extract
cmp-6101
google-a
3,001
4,452
0
2026-10-30 17:23
extract
cmp-6101
google-a
3,073
5,260
0
2026-10-30 17:24
extract
cmp-6101
google-a
2,984
3,429
0
2026-10-30 17:24
extract
cmp-6101
google-a
2,982
5,126
0
2026-10-30 17:24
extract
cmp-6101
google-a
2,963
4,655
0
2026-10-30 17:24
classify
cmp-6101
google-a
3,337
4,861
0
2026-10-30 17:25
classify
cmp-6101
google-a
3,323
5,266
0
2026-10-30 17:25
classify
cmp-6101
google-a
3,387
7,545
0
2026-10-30 17:25
classify
cmp-6101
google-a
3,367
4,538
0
2026-10-30 17:25
classify
cmp-6101
google-a
3,240
7,805
0
2026-10-30 17:25
classify
cmp-6101
google-a
3,277
8,726
0
2026-10-30 17:26
classify
cmp-6101
google-a
2,320
3,293
0
2026-10-30 17:26
classify
cmp-6101
google-a
3,346
9,328
0
2026-10-30 17:26
classify
cmp-6101
google-a
3,326
11,962
0
2026-10-30 17:27
draft
cmp-6101
google-a
2,520
1,924
0
2026-10-30 17:27
draft
cmp-6101
google-a
3,342
2,898
0
2026-10-30 17:27
draft
cmp-6101
google-a
2,402
2,622
0
2026-10-30 17:27
judge
cmp-6101
google-a
1,059
2,231
0
2026-10-30 17:27
judge
cmp-6101
google-a
1,060
2,302
0
2026-10-30 17:27
judge
cmp-6101
google-a
1,108
2,833
0
2026-10-30 17:27
judge
cmp-6101
google-a
1,123
2,116
0
2026-10-30 17:27
judge
cmp-6101
google-a
1,077
2,889
0
2026-10-30 17:27
judge
cmp-6101
google-a
1,049
2,684
0
2026-10-30 17:27
assist
cmp-6101
google-a
986
2,473
0
2026-10-30 17:28
assist
cmp-6101
google-a
968
2,972
0
2026-10-30 17:28
assist
cmp-6101
google-a
964
808
1
2026-10-30 17:28
assist
cmp-6101
google-a
1,001
2,038
0
Repair
2026-10-30 17:28
assist
cmp-6101
google-a
981
2,014
0
2026-10-30 17:28
assist
cmp-6101
google-a
954
929
0
2026-10-30 17:28
assist
cmp-6101
google-a
978
3,209
0
2026-10-30 17:28
assist
cmp-6101
google-a
994
1,737
0
2026-10-30 17:28
assist
cmp-6101
google-a
978
2,710
0
2026-10-30 17:28
assist
cmp-6101
google-a
986
888
0
2026-10-30 17:28
assist
cmp-6101
google-a
982
2,476
0
2026-10-30 17:29
extract
cmp-6101
google-b
3,001
6,173
0
2026-10-30 17:29
extract
cmp-6101
google-b
2,942
6,631
0
2026-10-30 17:29
extract
cmp-6101
google-b
2,982
4,186
0
2026-10-30 17:30
extract
cmp-6101
google-b
2,963
1,988
0
2026-10-30 17:30
extract
cmp-6101
google-b
2,984
7,757
0
2026-10-30 17:30
extract
cmp-6101
google-b
3,073
18,205
0
2026-10-30 17:31
classify
cmp-6101
google-b
3,387
4,720
0
2026-10-30 17:31
classify
cmp-6101
google-b
3,337
4,840
0
2026-10-30 17:31
classify
cmp-6101
google-b
3,323
5,120
0
2026-10-30 17:31
classify
cmp-6101
google-b
3,367
4,204
0
2026-10-30 17:32
classify
cmp-6101
google-b
3,346
2,952
0
2026-10-30 17:32
classify
cmp-6101
google-b
3,277
11,121
0
2026-10-30 17:32
classify
cmp-6101
google-b
3,326
5,816
0
2026-10-30 17:32
classify
cmp-6101
google-b
2,320
4,278
0
2026-10-30 17:33
classify
cmp-6101
google-b
3,240
21,151
0
2026-10-30 17:33
draft
cmp-6101
google-b
2,520
1,177
0
2026-10-30 17:33
draft
cmp-6101
google-b
2,402
1,142
0
2026-10-30 17:34
draft
cmp-6101
google-b
3,342
2,537
0
2026-10-30 17:34
judge
cmp-6101
google-b
1,060
1,124
0
2026-10-30 17:34
judge
cmp-6101
google-b
1,059
1,784
0
2026-10-30 17:34
judge
cmp-6101
google-b
1,108
2,144
0
2026-10-30 17:34
judge
cmp-6101
google-b
1,077
1,680
0
2026-10-30 17:34
judge
cmp-6101
google-b
1,049
1,484
0
2026-10-30 17:34
judge
cmp-6101
google-b
1,123
1,982
0
2026-10-30 17:34
assist
cmp-6101
google-b
968
1,529
0
2026-10-30 17:34
assist
cmp-6101
google-b
986
2,233
0
2026-10-30 17:34
assist
cmp-6101
google-b
964
504
1
2026-10-30 17:34
assist
cmp-6101
google-b
1,001
1,893
0
Repair
2026-10-30 17:34
assist
cmp-6101
google-b
981
2,081
0
2026-10-30 17:34
assist
cmp-6101
google-b
954
550
0
2026-10-30 17:35
assist
cmp-6101
google-b
978
2,903
0
2026-10-30 17:35
assist
cmp-6101
google-b
978
1,803
0
2026-10-30 17:35
assist
cmp-6101
google-b
982
1,617
0
2026-10-30 17:35
assist
cmp-6101
google-b
994
4,217
0
2026-10-30 17:35
assist
cmp-6101
google-b
986
5,377
0
2026-10-30 17:55
judge
drafts-6101
google-a
930
2,485
0
2026-10-30 17:55
judge
drafts-6101
google-a
931
2,480
0
2026-10-30 17:55
judge
drafts-6101
google-a
943
2,702
0
2026-10-30 17:55
judge
drafts-6101
google-a
855
1,749
0
2026-10-30 17:55
judge
drafts-6101
google-a
941
2,149
0
2026-10-30 17:55
judge
drafts-6101
claude-sonnet
5,730
127
0
2026-10-30 17:55
judge
drafts-6101
claude-sonnet
6,144
604
0
2026-10-30 17:55
judge
drafts-6101
claude-sonnet
6,148
940
0
2026-10-30 18:35
draft
2026
claude-opus
7,360
417
0
2026-10-30 18:35
judge
2026
google-a
614
629
0
2026-10-30 19:45
extract
2026
openai-b
2,620
564
0
2026-10-30 19:45
extract
2026
openai-b
2,760
1,067
0
2026-10-30 19:45
extract
2026
openai-b
2,660
1,096
0
2026-10-30 19:46
extract
2026
openai-b
2,691
466
0
2026-10-30 19:46
extract
2026
openai-b
2,695
1,033
0
2026-10-30 19:46
extract
2026
openai-b
2,673
872
0
2026-10-30 19:46
extract
2026
openai-b
2,692
1,984
0
2026-10-30 19:46
extract
2026
openai-b
2,643
276
0
2026-10-30 19:46
extract
2026
openai-b
2,647
790
0
2026-10-30 19:46
extract
2026
openai-b
2,638
282
0
2026-10-30 19:46
extract
2026
openai-b
2,638
383
0
2026-10-30 19:46
extract
2026
openai-b
2,666
648
0
2026-10-30 19:46
extract
2026
openai-b
2,215
523
0
2026-10-30 19:46
extract
2026
openai-b
2,684
1,197
0
2026-10-30 19:46
extract
2026
openai-b
2,657
1,179
3
2026-10-30 19:46
extract
2026
openai-b
2,118
584
0
Repair
2026-10-30 19:47
classify
2026
google-b
2,711
7,234
0
2026-10-30 19:47
draft
2026
claude-opus
7,392
528
0
2026-10-30 19:47
draft
2026
claude-opus
9,299
803
0
2026-10-30 19:47
draft
2026
claude-opus
9,195
1,063
0
2026-10-30 19:47
judge
2026
google-b
912
1,168
0
2026-10-30 19:47
judge
2026
google-b
864
1,644
0
2026-10-30 19:48
judge
2026
google-b
900
2,541
0
2026-10-30 19:48
judge
2026
google-b
897
1,142
0
2026-10-30 19:48
judge
2026
google-b
915
1,208
0
2026-10-30 19:48
judge
2026
google-b
917
971
0
2026-10-30 19:48
judge
2026
google-b
872
1,107
0
2026-10-30 19:48
extract
sample-9001
openai-b
2,779
628
0
2026-10-30 19:48
extract
sample-9001
openai-b
2,777
1,049
0
2026-10-30 19:48
extract
sample-9001
openai-b
2,805
1,944
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
9,000
1,649
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
9,025
1,639
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,948
2,172
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,853
1,451
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,923
1,986
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,933
1,756
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,954
2,331
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,970
1,508
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,885
556
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,899
337
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,960
1,731
0
2026-10-30 21:16
extract
cmp-6101
claude-opus
8,936
2,261
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
9,005
1,657
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
9,030
1,598
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,953
1,831
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,928
2,481
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,857
1,497
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,956
2,451
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,937
1,790
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,973
1,858
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,962
1,592
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,890
326
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,904
328
0
2026-10-30 21:17
extract
cmp-6101
claude-sonnet
8,940
2,019
0
2026-10-30 21:17
extract
cmp-6101
claude-haiku
9,049
2,947
0
2026-10-30 21:17
extract
cmp-6101
claude-haiku
9,026
3,236
0
2026-10-30 21:17
extract
cmp-6101
claude-haiku
8,971
3,262
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,879
2,724
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,948
4,347
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,977
5,212
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,958
1,294
1
2026-10-30 21:18
extract
cmp-6101
claude-haiku
7,797
821
0
Repair
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,995
3,314
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,911
1,277
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,983
4,192
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,962
4,187
0
2026-10-30 21:18
extract
cmp-6101
claude-haiku
8,923
1,449
0
2026-10-30 21:18
extract
cmp-6101
openai-a
2,705
686
0
2026-10-30 21:18
extract
cmp-6101
openai-a
2,762
766
0
2026-10-30 21:18
extract
cmp-6101
openai-a
2,680
878
0
2026-10-30 21:18
extract
cmp-6101
openai-a
2,633
671
0
2026-10-30 21:18
extract
cmp-6101
openai-a
2,676
699
0
2026-10-30 21:18
extract
cmp-6101
openai-a
2,675
949
0
2026-10-30 21:19
extract
cmp-6101
openai-a
2,692
908
0
2026-10-30 21:19
extract
cmp-6101
openai-a
2,698
734
0
2026-10-30 21:19
extract
cmp-6101
openai-a
2,655
235
0
2026-10-30 21:19
extract
cmp-6101
openai-a
2,690
844
0
2026-10-30 21:19
extract
cmp-6101
openai-a
2,674
811
0
2026-10-30 21:19
extract
cmp-6101
openai-a
2,652
255
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,705
1,362
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,762
1,706
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,680
1,726
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,633
880
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,676
1,434
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,675
1,962
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,698
993
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,692
2,797
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,690
1,171
0
2026-10-30 21:19
extract
cmp-6101
openai-b
2,655
351
0
2026-10-30 21:20
extract
cmp-6101
openai-b
2,652
949
0
2026-10-30 21:20
extract
cmp-6101
openai-b
2,674
2,050
0
2026-10-30 21:20
extract
cmp-6101
google-a
3,012
3,695
0
2026-10-30 21:20
extract
cmp-6101
google-a
3,085
4,948
0
2026-10-30 21:20
extract
cmp-6101
google-a
2,994
4,977
0
2026-10-30 21:20
extract
cmp-6101
google-a
2,949
3,117
0
2026-10-30 21:21
extract
cmp-6101
google-a
2,981
4,065
0
2026-10-30 21:21
extract
cmp-6101
google-a
3,004
5,976
0
2026-10-30 21:21
extract
cmp-6101
google-a
2,983
9,777
0
2026-10-30 21:21
extract
cmp-6101
google-a
3,004
4,238
0
2026-10-30 21:21
extract
cmp-6101
google-a
2,990
4,811
0
2026-10-30 21:22
extract
cmp-6101
google-a
2,962
3,274
0
2026-10-30 21:22
extract
cmp-6101
google-a
2,998
4,837
0
2026-10-30 21:22
extract
cmp-6101
google-a
2,954
3,090
0
2026-10-30 21:22
extract
cmp-6101
google-b
3,085
3,541
0
2026-10-30 21:23
extract
cmp-6101
google-b
3,012
9,486
0
2026-10-30 21:23
extract
cmp-6101
google-b
2,949
10,518
0
2026-10-30 21:24
extract
cmp-6101
google-b
2,994
21,652
0
2026-10-30 21:24
extract
cmp-6101
google-b
2,983
8,481
0
2026-10-30 21:24
extract
cmp-6101
google-b
3,004
901
0
2026-10-30 21:24
extract
cmp-6101
google-b
2,990
3,181
0
2026-10-30 21:25
extract
cmp-6101
google-b
2,981
9,692
0
2026-10-30 21:25
extract
cmp-6101
google-b
2,962
2,956
0
2026-10-30 21:25
extract
cmp-6101
google-b
2,998
9,838
0
2026-10-30 21:25
extract
cmp-6101
google-b
2,954
2,376
0
2026-10-30 21:25
extract
cmp-6101
google-b
3,004
18,616
0
2026-10-30 21:26
judge
2026
google-b
912
6,921
0
2026-10-30 21:26
assist
before-2026
claude-haiku
6,171
1,325
0
2026-10-30 21:26
assist
before-2026
claude-haiku
6,107
2,149
0
2026-10-30 21:26
assist
before-2026
claude-haiku
6,114
700
1
2026-10-30 21:26
assist
before-2026
claude-haiku
6,161
903
0
Repair
2026-10-30 21:26
assist
before-2026
claude-haiku
6,163
1,566
0
2026-10-30 21:27
assist
before-2026
claude-haiku
6,108
2,253
0
2026-10-30 21:27
assist
before-2026
claude-haiku
6,108
1,999
0
2026-10-30 21:27
assist
before-2026
claude-haiku
6,149
1,871
0
2026-10-30 21:27
assist
before-2026
claude-haiku
6,112
706
0
2026-10-30 21:27
assist
before-2026
claude-haiku
6,147
988
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,158
876
0
2026-10-30 21:27
assist
before-2026
claude-haiku
6,124
2,403
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,107
850
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,101
1,091
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,167
1,114
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,080
708
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,094
667
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,165
1,381
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,131
1,396
0
2026-10-30 21:27
assist
after-2026
claude-haiku
6,125
759
1
2026-10-30 21:27
assist
after-2026
claude-haiku
6,171
1,577
0
Repair
2026-10-30 21:27
assist
after-2026
claude-haiku
6,107
825
1
2026-10-30 21:27
assist
after-2026
claude-haiku
6,155
639
0
Repair
2026-10-30 21:27
extract
sample-9001
claude-sonnet
9,075
924
0
2026-10-30 21:27
extract
sample-9001
claude-sonnet
9,133
1,258
0
2026-10-30 21:27
extract
sample-9001
claude-sonnet
9,098
1,752
0
The code
The full source is on GitHub with its tests and CI, and these six files are the best place to start.