Checking AI outputs in advice documents: a practical method
Why AI drafts go wrong, what the FCA and the Information Commission say about accuracy and accountability, and a seven-step method for checking AI outputs before any advice document reaches a client.
On this page
- Why AI outputs go wrong
- What the regulators say about accuracy and accountability
- Data protection
- Conduct
- A seven-step method for checking AI outputs
- Traceability is what makes checking quick
- Who checks, and how often
- Frequently asked questions
- What is an AI hallucination in an advice document?
- Do we need to check every AI-drafted document?
- If a tool shows the source of every figure, can we skip the check?
An AI draft of an annual review can look finished within minutes. The headline reads well, the figures sit in tidy tables and the tone is right. That is exactly why checking AI outputs matters: a fluent document is easy to trust, and a wrong number inside a fluent document is easy to miss. Before anything drafted by AI reaches a client, someone in the firm has to be satisfied that it is accurate, complete and about the right person.
This article sets out why AI outputs go wrong, what the regulators say about accuracy and accountability, and a practical checking method an adviser or paraplanner can use on every document.
Why AI outputs go wrong
Large language models write by predicting text. They are very good at it, but they can produce statements the source does not support. The Information Commission (formerly the ICO) uses the common term for this. In its generative AI call for evidence it says: "Developers should also assess and communicate the risk and impact of so-called 'hallucinations', ie incorrect and unexpected outputs."
In advice documents, the errors that matter most fall into a handful of types:
- Wrong figures. A value read from the wrong column, a contribution counted as growth, or a charge carried over from the previous year.
- Unsupported statements. A sentence about the client's objectives or circumstances that appears nowhere in the meeting or the file.
- Omissions. A scanned page or a second platform report that was never read, so the document rests on part of the evidence.
- Wrong client. Details from one client's papers appearing in another client's document when several packs are handled together.
- Overstatement. Wording that is more certain, or more positive, than the evidence supports.
People make every one of these mistakes when they are rushed. The difference with AI is volume: it can produce a lot of documents quickly, so the checking step has to be designed to keep pace rather than left to good intentions.
What the regulators say about accuracy and accountability
Data protection
The ICO's guidance on accuracy and statistical accuracy says: "Data protection's accuracy principle applies to all personal data, whether it is information about an individual used as an input to an AI system, or an output of the system." It adds that the principle "does not mean that an AI system needs to be 100% statistically accurate".
The practical reading for an advice firm: the tool does not have to be perfect, but the personal data in a finished client document has to be right, and the firm needs a process that makes it so.
Conduct
The FCA has not written AI-specific rules. Its AI Update says, at paragraph 3.45: "Firms that use AI as part of their business operations remain responsible for ensuring compliance with our rules, including in relation to consumer protection." At paragraph 3.40 it notes that SM&CR "emphasises senior management accountability and is relevant to the safe and responsible use of AI".
The suitability rules make a similar point about automated tools. Under COBS 9A.2.23R, a firm using automated tools "remains responsible for the suitability assessment" (COBS 9A.2). The Consumer Duty supplies the outcome test: PRIN 2A.2.8R says "A firm must avoid causing foreseeable harm to retail customers" (PRIN 2A.2), and PRIN 2A.5.3R(2) requires firms to communicate "in a way which is clear, fair and not misleading" (PRIN 2A.5).
None of these passages prescribes a checking method. That is for the firm to design. For most firms, a named person reviewing every client-facing document is the most direct way to meet the responsibility the firm already carries.
A seven-step method for checking AI outputs
This method works for annual reviews, discovery documents, attitude to risk reports and suitability drafts alike. It assumes the reviewer has the source documents open alongside the draft.
- Check the inputs first. Confirm the right client, the right documents and the right period were used. If the tool reports a file it could not read, resolve that before reading a word of the draft.
- Trace every figure to its source. Each value, contribution, withdrawal and charge should lead back to a named report and page. Open the page. Do not judge a figure by whether it looks plausible.
- Test the arithmetic. The opening value, plus money paid in, minus withdrawals, plus or minus investment return, should reconcile to the closing value. If it does not, find out why before going further.
- Check every statement about the client. Objectives, family circumstances, health, plans and attitudes should all come from the meeting or the file. Correct or delete anything you cannot find.
- Read for tone and certainty. Remove promises, predictions and anything more confident than the evidence. Check that risks and disadvantages are as visible as benefits.
- Add the judgement. The recommendation, the reasons for it and the adviser's view are the adviser's to write or confirm. A draft can set them out; it cannot own them.
- Record the approval. Keep the approved version, who approved it and when. SYSC 9.1.1R requires "orderly records" of the firm's business (SYSC 9.1), and the approved document is the one that counts.
Traceability is what makes checking quick
A check is only as fast as the time it takes to find the source. If a reviewer has to hunt through a fifty-page valuation for every number, the check gets shortened, and that is when errors slip through. So when you assess any AI tool, look at how it supports the reviewer:
| Error type | What the reviewer checks | What a well-designed tool provides |
|---|---|---|
| Wrong figure | The figure against the source page | A report and page reference for every figure |
| Omission | That every document was read | A clear warning when a file could not be read |
| Unsupported statement | The claim against the meeting or file | The quoted passage or the client's own words |
| Wrong client | Names, plans and policy details | Checks that stop client packs being mixed |
| Judgement presented as fact | Who wrote the recommendation | Adviser-only sections clearly flagged |
We built our own apps around this list because we use them in a working planning firm. Strata traces every figure back to the report and page it came from, separates investment return from money paid in, reports any file it could not read, guards against mixing up clients when several report packs are staged together, and clearly flags the adviser-only, judgement-based sections. Scrutineer quotes the passage behind each finding and flags any review that could only read part of a folder. Asclto checks every figure against the meeting transcript, and Freeboard quotes the client's own words for each conclusion. None of that removes the review. It makes the review fast enough to do properly every time.
Who checks, and how often
A workable pattern for most firms has two layers:
- The author checks every document in full. Whoever prepares the document, adviser or paraplanner, works through the seven steps before it goes anywhere.
- A second person samples. Compliance or a senior colleague re-checks a proportion of AI-drafted documents against their sources, weighted towards new tools, new staff and complex cases.
Keep a simple log of corrections by type: figures, statements, omissions, tone. After a few months it tells you where the real risk sits. If most corrections are figures from one platform's statements, that is a fix for the inputs. If most are tone, it is a fix for the house style. A log like this is also good evidence for your monitoring under the Consumer Duty, which we cover in Consumer Duty evidence for advice firms. For the wider picture of where AI helps and where it does not, see our plain guide to AI for financial advisers.
Frequently asked questions
What is an AI hallucination in an advice document?
It is an output that is incorrect or unexpected, such as a figure, a fact about the client or a statement of intent that the source documents do not support. The ICO describes hallucinations as "incorrect and unexpected outputs".
Do we need to check every AI-drafted document?
The FCA says firms that use AI remain responsible for complying with its rules, including on consumer protection. We recommend a full check of every client-facing document by the person who prepared it, with a second-line sample on top.
If a tool shows the source of every figure, can we skip the check?
No. Traceability makes the check quicker and more reliable; it does not replace it. The reviewer still opens the source, confirms the figure and approves the document.
To see figure-by-figure traceability on a real annual review, look at Strata or book a demo: twenty minutes on Microsoft Teams with Adam Booth, run live on a demo firm with its own clients, meetings and files, with nothing for you to prepare.
