No AI admin dashboard tells you the defect rate of the work your system is producing[1]. The tools show you adoption metrics, usage frequency, time saved. None of them show you how often the output was wrong. For a firm in a regulated advice context, that gap is not a minor inconvenience. It is a Consumer Duty exposure that sits unquantified until something goes wrong.
This is the problem hallucinations create. An AI system states something confidently and incorrectly. An adviser, under time pressure, does not catch it. It reaches a client file, a suitability letter, a meeting note. The harm may be small, or it may not be. Either way, the firm cannot point to a control that was designed to prevent it.
This article walks through a practical, lightweight audit process. Nothing here requires an engineering team or a large budget. Most of it can be started this week.
What a hallucination actually is in an advice context
A hallucination is when an AI system produces text that is plausible-sounding but factually wrong, and presents it with the same confidence as accurate output. In a general setting this is annoying. In a regulated advice setting it is a risk to the client and a potential Consumer Duty breach.
The most relevant forms in financial advice work are:
- Invented regulatory references. The system cites a rule, threshold, or FCA publication that does not exist or has been misquoted.
- Fabricated client data. When summarising or cross-referencing, the system introduces figures not present in the source material.
- False task completion. Agentic systems frequently report that a task has been completed when it has not, or that a document has been filed when it has only been drafted[2]. This is sometimes called an agentic hallucination, and it is particularly difficult to catch because the output looks like a confirmation rather than a claim.
- Date and product errors. Rates, fund charges, and regulatory deadlines stated incorrectly, often because the model’s training data predates the current figure.
The common thread is that none of these errors announce themselves. The output reads fluently. The adviser has to know what to look for.
The hallucination problem in regulated advice is not that AI systems are unreliable in general, it is that they are unreliable in the specific ways that are hardest for a busy adviser to catch.
Why the Consumer Duty makes this a governance question, not just an operations one
Under the Consumer Duty, firms are expected to deliver good outcomes for retail clients. The Duty is not satisfied by having a process that usually works. It requires firms to understand the risks in their advice chain, to have controls that address those risks, and to be able to demonstrate both.
If your firm uses AI to assist with meeting notes, suitability letter drafts, research summaries, or any other step in the advice process, then hallucination risk sits inside your advice chain. The FCA’s principles-based approach means the firm owns the outcome, regardless of which tool produced a line of text.
The practical implication: you need to be able to show an auditor, and yourself, that you had a systematic control designed to catch AI errors before they affected client outcomes. “The adviser checks it” is not sufficient on its own, because it relies on the adviser knowing what the AI might have got wrong, and because time pressure means that check degrades in practice.
A documented audit process is the control that makes your oversight real rather than assumed.
Step one: map your AI touchpoints
Before you can audit for errors, you need an honest inventory of where AI is involved in your advice process.
Walk every stage from client contact to file completion. This includes meeting transcription, note summarisation, fact-find drafting, research queries, product comparison, suitability letter drafting, and any automated report generation. Mark each stage where an AI system produces text or structured data that could reach a client file or influence an advice recommendation.
Categorise each touchpoint by the level of human supervision in place. A transcription of a client call that an adviser reads and edits is lower risk than a suitability letter paragraph that is copied across without review. Intelliflo’s categorisation offers a useful starting frame: co-pilot (AI assists, human decides in real time) and supervised agent (AI completes a task, human reviews before use)[3]. All AI touchpoints in an advice firm must have a human review step before any output reaches a client file or influences a regulated decision. The adviser or responsible person retains full control over every action. Any touchpoint that lacks a clear, named review step should be flagged immediately and paused until that step is in place.
Identify who is responsible for review at each stage. If the answer is “the adviser, generally,” that is not a named responsibility. It is an assumption.
Step two: build a lightweight hallucination check for each risk category
For each high-risk touchpoint, define what a hallucination would look like and what check would catch it. This does not need to be complex.
For regulatory references: Any AI-generated output that cites a specific FCA document, rule number, or regulatory threshold should be treated as unverified until the reference has been checked against a primary source. A simple rule: if the AI names a document, someone opens the document. This takes two minutes and catches a large share of fabricated citations.
For figures drawn from client data: Cross-reference any number in an AI-generated summary against the source document. If the meeting transcript says the client has £180,000 in a pension and the AI summary says £810,000, that error should be caught at this step. Run a spot-check on a sample of summaries each week, not every one, but enough to know whether the system is reliable on this.
For task completion in AI-assisted workflows: LLMs output text; they do not execute actions[4]. If your workflow uses an AI tool to draft a document, send an email, or log a file, the tool’s report that it has done so must be verified against the actual system of record. Check the CRM. Check the document folder. Do not accept a system confirmation as evidence without checking the underlying record.
For dates, rates, and product data: Any specific figure relating to a product, fund, or regulatory deadline should be verified against a current primary source before it reaches a client document. AI training data goes stale. This is a known limitation, not a flaw that will be fixed.
Step three: document what you are doing and what you find
An audit process that exists only in practice gives you no protection. You need a lightweight record.
Keep a hallucination log. Every time a check catches an error, record it. What tool was used, what the output said, what the correct information was, and what happened next. This does not need to be elaborate. A shared spreadsheet works. What it gives you is a pattern over time: which tools produce errors most often, on which types of task, and whether the rate is improving or getting worse.
Record your sampling methodology. If you spot-check 10% of AI-generated summaries each week, write that down. If you check every regulatory reference, write that down. The record shows that your control is systematic, not ad hoc.
Review the log quarterly. Look at whether error rates are concentrated in particular tools, task types, or advisers. If a pattern emerges, adjust the control: increase the sampling rate for that tool, add a second reviewer for that task type, or reconsider whether that touchpoint should be using AI at all.
This documentation is your evidence trail. If the FCA or a client asks how you govern AI outputs in your advice process, this is what you point to.
Step four: assign clear ownership
The most common failure mode I see is that hallucination risk is treated as everyone’s problem, which means it is no one’s. Under SMCR, firm-level risks need named accountability.
Assign a named person to own the AI hallucination audit process. In a small firm, this may be the principal or a senior paraplanner. In a larger firm, it may sit with the compliance officer or operations lead. The ownership does not need to be a full-time role, but it does need to be a named responsibility.
That person’s accountabilities: maintaining the touchpoint map, running or coordinating the sampling checks, updating the hallucination log, escalating patterns to senior management, and reviewing the process quarterly.
Build it into existing governance rhythm. The quarterly review of the hallucination log should sit alongside your existing compliance monitoring. It is not a new meeting. It is an agenda item.
What to do if you find a significant error
If an audit check catches an AI-generated error that has already reached a client file, the response follows your existing error and complaint process. Flag it to the responsible adviser, assess whether it materially affected the advice or the client’s understanding, and document the finding and remediation in the hallucination log.
If the error is material, treat it as you would any advice error: consider whether the client needs to be informed, whether the advice file needs to be corrected, and whether any regulatory notification obligation is triggered. AI being the source of the error does not change the obligation. The firm owns the output.
The practical starting point
If your firm has not yet done any of this, here is the minimum viable version to start this week:
First, list your AI touchpoints. Write down every step in your advice process where an AI system produces text or data that could reach a client file. Fifteen minutes, pen and paper.
Second, assign each touchpoint a risk level. High risk: anything that reaches a suitability letter, advice recommendation, or client-facing document. Lower risk: internal summaries that an adviser reads and rewrites from scratch.
Third, define one check for each high-risk touchpoint. Just one. What would you look at to verify the output is correct? Write it down.
Fourth, start the log. A shared spreadsheet with four columns: date, tool, error found, action taken. Even if it is empty for the first month, starting it is the act of commitment that makes the rest follow.
None of this is a large project. It is a documented position that your firm takes AI output quality seriously enough to have a control in place, and that you review it. That is what the Consumer Duty asks for, and it is also what good operational practice looks like regardless of regulation.
If you’d like to think through what a hallucination audit process would look like for your firm specifically, a discovery call with Cordrey Consulting is a straightforward place to start.
This article is for informational purposes only and does not constitute regulated financial advice or a compliance opinion. Consult a qualified compliance professional for advice specific to your firm.
Sources
[1] Digital Applied (2026) ‘No AI admin dashboard provides metrics on work quality or defect rates’, Digital Applied. Available at: https://www.digitalapplied.com/blog/ai-adoption-admin-dashboards-what-each-vendor-measures
[2] Digital Applied (2026) ‘Marketing tasks for coding agents: a guide’, Digital Applied. Available at: https://www.digitalapplied.com/blog/marketing-tasks-for-coding-agents-guide
[3] Intelliflo (2026) ‘Advisers needn’t fear handing control to AI’, Intelliflo Insights. Available at: https://www.intelliflo.com/insights/thought-leadership/advisers-neednt-fear-handing-control-to-ai
[4] Zen van Riel (2026) ‘How AI agents work under the hood’, AI Engineer Blog. Available at: https://zenvanriel.com/ai-engineer-blog/how-ai-agents-work-under-hood