Financial services · Credit-decisioning assistant
A credit assistant that can answer a regulator's questions.
A lender operating across several regulated markets built an assistant to help underwriters work faster: summarising applications, surfacing policy, drafting rationale. It was never meant to decide. The lender needed evidence that it stayed that way, and that its reasoning could stand up to a regulator.
Sector
Financial services (lending)
System
Underwriter-facing assistant over applications, policy and bureau data
Engagement
AI Assurance Assessment and AI Red Team
Timeline
Eight-week engagement; continuous assurance since
The situation
Decision support, in name.
The assistant sat beside the underwriter. It read the application and the bureau file, pulled the relevant policy, and drafted a recommendation with its reasoning. Underwriters kept the final call. Internally the system was described as decision support, and the controls were designed around that description.
The risk function asked the harder question: in practice, how often does the underwriter simply accept the draft? And if the draft is wrong, or quietly biased, or can be influenced by something in the application, who would know? They engaged us to measure the system's behaviour, not its intent.
What we found
The draft read as reasoned. The reasoning could be steered.
Application text could shape the recommendation
Free-text fields in the application - the purpose of the loan, an explanatory note - could lead the assistant to emphasise favourable factors and omit adverse ones. The draft read as reasoned; the reasoning was steered.
Factual grounding drifted under pressure
On a held-out set of applications, the assistant's rationale cited policy clauses that did not apply and, in a measured share of cases, stated a bureau figure that was not in the file.
Consistency varied with phrasing
Materially identical applications, described differently, produced different recommendations more often than the lender's tolerance allowed.
The audit trail recorded the answer, not the inputs
Logs captured the draft recommendation but not the retrieved policy, the prompt version or the model version behind it. A regulator asking why a particular outcome occurred could not have been answered.
What changed
A system the risk function can interrogate.
The assistant kept its role and lost its blind spots. Every draft is now grounded, consistent within an agreed tolerance and fully traceable - and the evidence behind that went into the lender's regulatory engagement.
- Application free text isolated from the reasoning context; only structured, validated fields inform the draft
- Groundedness checks on every rationale - cited policy and figures verified against the file before the draft is shown
- A consistency benchmark the lender runs on every model, prompt or policy change, with an agreed tolerance
- Full decision lineage logged: inputs, retrieved policy, prompt and model versions, draft and final decision
- Underwriter acceptance rate monitored as an operational signal and reviewed by the risk function
- Evidence pack prepared for the lender's regulatory engagement, built from the assessment's test results
Ongoing assurance
Treated, and tested, as a regulated system.
Every policy update, model change and prompt revision runs through the groundedness and consistency benchmarks, and the results go to the risk committee alongside the acceptance-rate data. We maintain the benchmark set as lending policy changes and re-run the adversarial suite each quarter.
When the lender expanded into a new market this year, we extended the test set to the new policy before launch.
How continuous assurance worksMore case studies
All case studiesBegin with evidence