This is part two of a two-part series on data quality. Part one was laying the foundation: what is data quality, and how should we think about it? And part two is the fun stuff: data quality and AI, with a focus on finance.
Introduction
The finance industry — my industry! — does not use AI enough. Outside of a few pockets of early adopters, and despite the proliferation of AI tools, we lag software, law, sales and marketing, and even education and healthcare.
This essay argues that the missing ingredient for finance is data quality. It shows how a large number of “AI failures” can be traced to bad data quality, either when sending data to AI, or evaluating data that comes from it. And it provides solutions: best practices and mental models to ensure that AI works for you, and works well.
The essay does all of this through a finance lens, centering the very specific requirements and failure modes of our industry — not just Wall Street finance, but Main Street finance as well.
Data quality, AI and finance — good looks, brains and money — what more does one need to succeed? Read on!
Who This Essay Is For
This essay is written for non-technical finance professionals who use AI and/or want to use AI more, especially those who work outside of Wall Street and institutional capital markets.
More specifically:
If you’re a finance manager or director, a CFO or VP of finance, a wealth advisor or RIA, an analyst or underwriter, an investor, a finance ops lead, an FP&A specialist, a bookkeeper or controller, a family office allocator, or any role along those lines;
If your work involves complex, detail-oriented, knowledge-intensive tasks where accuracy is paramount;
If you’ve ever prompted an LLM, used an app that uses AI, tried automating tasks or stitching together workflows, or are just curious and excited about AI;
then this essay is for you.
This essay is also for anyone, in any field, who uses, builds with, or builds on top of AI for professional tasks. Founders and operators, data people, executives and investors, function leaders, informed laypersons, strangers on the internet: the principles in this essay generalize perfectly to your work.
This essay is not written for specialist AI researchers and builders. Training and improving foundation models is a massive and insanely rapidly-evolving field with its own nuanced and distinctive body of work on data quality.
Let’s jump in!
Data, Three Ways
It’s helpful to distinguish three ways in which LLMs interact with data quality. There’s data quality for input to AI; data quality for output from AI, and AI as an evaluator of data quality. Let’s look at each of these in turn, starting with input.
Quality is Everywhere …
“Input” is broader than you think. Your prompts are input. Any files or resources you share are input. Connectors are input. Any app you use, that uses AI, is input to AI. Your history is input. Your skills are input. It’s all input, and it’s all data.
This point is often overlooked. We’re used to thinking of data as fields in a database, or documents in a folder. LLMs have a wider view, and treat any and every type of content as data input – facts, figures, codes, writing, images, audio, websites, traces, logs. If it’s bits on a disk, it’s data.
And if it’s data, it’s subject to quality. Any one of these inputs can be high-quality or low-quality, and the difference matters a great deal.
… And Must Be Protected
Almost all historical “quality engineering” has focused on the narrow view of data, as structured fields or documents1. This approach hasn’t kept pace with the way LLMs use data. We need to widen our view.
Garbage in, garbage out remains as true as ever. The problem is that LLM outputs are horribly plausible2. Detecting garbage is easy; detecting plausible untruths is much harder. Which means we have to be even more careful about what we send in.
It’s not just garbage (semantic junk) you have to watch out for, it’s lazy or shoddy or noisy inputs. LLMs excel at taking those, and polishing them, and making assertions with unfounded confidence. Careless in, convincing out3.
The Careful Feeding of Models
Here are a few things you should do to avoid the CICO pattern. Almost all of these can be reduced to a simple philosophy: maintain closer control of fewer but better data inputs.
Curate ruthlessly. Resist the temptation to throw everything into context, just because you can. More is almost always worse.
This is counter-intuitive and perhaps the single simplest, highest ROI move you can make. LLMs can handle vast amounts of input, but that doesn’t mean their answers necessarily get better with more input. Keep it short and relevant.
Provide constant grounding. Define model-independent sources of truth, and reference them across your entire workflow, not just at the beginning or end. Do NOT let your LLM write to your source of truth.
Watch for hidden handoffs. When chaining tools, or building workflows, or using apps, watch for seams where data moves from one AI-powered application to another, unknown and unseen by you. These are where quality degrades.
Imagine you’re a wealth advisor, doing a portfolio review with a client, and you use a note-taking app. Your actual data chain is microphone → speech-to-text → diarization → compaction → your prompt → the model. You think your input to the model is clean, but any one of these stages can introduce LLM errors, losses, or hallucinations. And this is just the transcript!
Stay current. Models are easily confused by version conflicts and stale data. This is especially noticeable in RAG chunks or file uploads, but it shows up everywhere.
Anyone who has ever received an attachment titled 2026-budget-v17-final-James-v2-FINAL.xlsx feels this pain. But at least human beings can recognize and solve the problem. LLMs just get confused.
Inspect the raw material. Never ever lose the habit of looking at the rawest possible versions of your inputs – files, transcripts, tables.
My friend C tells the story of getting invited to a Zoom call with Yumi. But she has no idea who Yumi is. Turns out, it was supposed to be a meeting between “you, me, and [somebody else]”.
This is funny, until Yumi gets an AI-generated email with full company financials attached …
Beware sycophancy. Prompt quality is part of input data quality, and it’s all too easy for a model to “lead” the user into low-quality prompts that merely maximize engagement and rewards.
Avoid context rot. Even with careful curation, long sessions and complicated projects can lead to confusion, distraction, attention dilution, repetition and poisoning (persistent errors). You can mitigate this with fresh context windows, concise summaries, and single-task focus.
You’re iterating on a complex underwriting model in a marathon chat. Around message 10, you ask “what if we assume 8% vacancy?”. It’s meant to be a temporary assumption, but by message 30, the LLM starts treating this as the actual rate. This poisons all subsequent iterations. Long sessions degrade!
Interestingly, these are all quality actions that don’t really have a great analogue in the pre-LLM world. Quality keeps evolving!
Data Quality for Outputs from AI
Now let’s look at LLM outputs. Just like every input to an LLM can be considered data, so can every output4. We’ve seen how to evaluate (and improve) data quality on the input side; how do we do the same on the output side?
Ask for grounding. The mirror image of providing grounding. Ask for grounding (facts, references, citations, logic) at every stage, not just the final output.
Show your work. LLMs tend to skip steps; over-extrapolate; generalize; hand-wave; make up plausible data. (This is as true for numerical output as it is for written output). Asking models to show their work is a different and equally valuable form of grounding.
Promote dissent. Pay special attention to outliers, contrary opinions, and revised priors. Explicitly ask for steelman and devil’s advocate cases.
“Are you sure? Check your work. Prove that. That doesn’t sound right. Where did you see that?” It’s astonishing how effective these questions are. Or perhaps not; current models are not very good at self-evaluation.
For that matter, neither are humans. The best analogy here is that models are junior analysts, and you are the MD sending an email at 11pm: “please fix”, with no further details given. It works!
Recognize signatures. Train yourself to spot places where AI tends to err. Many people can recognize bad AI writing (“The honest answer — it’s not writing; it’s expression”); similar patterns exist in finance, analysis, and data.
Watch out for these AI tells in particular:
Spurious precision
Confusion between inputs, assumptions and outputs
Internal inconsistency and calculation errors
Rationalization of suspiciously “clean” results
Scale and unit errors
No sense of materiality or proportion
Climbing the Quality Ladder
Notice that most of the above practices operate at the granular (unit) and aggregate (corpus) levels of data quality. But there are some things you should do at the fitness-for-purpose and business-outcome levels as well:
Beware fluency. LLMs are hyper-fluent generalists. They cosplay competence. Don’t mistake their fluency for correctness or expertise. Stay skeptical.
Imagine you ask an AI to write the feasibility memo for a new project. The result is gorgeous: confident prose, masses of detail, perfect editorial formatting, professional look and feel. But it rests on assumptions that are completely bogus. As a piece of analysis, it’s worthless.
A human reader might they see the polish (evidence of effort) and assume it’s backed by quality (also evidence of effort). That’s a dangerous assumption to make! AI output messes with our intuition.
An easy way to mitigate this is to separate analysis and presentation. Ask for simple, just-the-facts output first; defer any suggestions by the LLM to make it polished or presentable.
Watch for reward-hacking. Humans are notoriously poor at knowing what’s good for them. It’s easier for an LLM to fool a human into thinking something is good, than it is to actually produce something good.
You ask an LLM to write a regular monthly financial summary to share with the board. For consistency, you persist the chat. The LLM learns that you tend to go with versions that put you, and the company, in a good light. As a result, over time, your board updates become panglossian treacle.
Know what you don’t know. Estimate confidence and uncertainty, and explicitly match these to your goals and budget. Require your models to do the same.
You share your financial model and ask an LLM for a runway forecast. It gives you a very specific date. Okay; you’re experienced enough to know that this is bad. But what if you had asked for a more subtle, opaque, or complicated calculation: would you still recognize the danger of a point estimate?
Humans mistake polish for competence. We also mistake confidence for competence. And LLMs will cheerfully provide confident wrong answers.
Beware flattening. LLM outputs, especially at the end of a long process, tend to converge to a mushy middle: homogenized, averaged-out, general-purpose analysis or artefacts. Don’t accept this; constantly steer them away from generalizations and towards specifics.
You ask the LLM for risks to the forecast. It responds, “the macro could worsen, customers could churn, competitors could emerge, talent could leave”.
This is … not helpful. It could describe any company on earth. But what you really want is you-specific. Perhaps there’s a large customer you worry about, or two key execs who don’t get along. That’s what you really want to surface; those are the actual, actionable harbingers of churn and attrition.
Don’t anchor. If the output of a model, workflow, app or tool just repeats what you already know, what good is it?
Loop in the humans. Candidly, humans aren’t great at detecting errors. But at least their weaknesses are imperfectly correlated with AI weaknesses, so you get a diversification benefit.
Notice that many of these practices are about overruling intuitions (some would say biases) drawn from working with other human beings — fluency-expertise, agreeableness, self-confidence, alignment, generalization and confirmation.
Case Study: Board Financials
Much of the above is abstract. Let’s tie it together with a case study.
Imagine you’re a CFO, putting together the quarterly financials pack for your board. You’re going to pull together a bunch of inputs, run a few calculations, draft some outputs, clarify and iterate, and finally publish. The steps are mostly known, and easy to describe, but they’re operationally complex and full of details that you must get right.
You enlist AI to help. Unfortunately:
Your PDF parser skips a line (hidden handoffs)
Your revenue sheet uses an outdated contract (version conflict)
You’re missing an FX rate and the LLM invents it (grounding)
The LLM says “Great! I have everything I need. Would you like me to write the commentary?” (prompt leading)
You iterate several times and results start getting worse (context rot)
You use the wrong transcript because it was mislabeled (inspection gap)
Some of the calculations are just wrong (showing the work)
The LLM carries over data from the previous board pack (anchoring)
The commentary is dense and jargon-filled and seems convincing (fluency)
But on a deeper read, the conclusions are obvious (no dissent)
And say little that is specific to your firm or finances (flattening)
All of these are failures of the LLM. But more fundamentally, these are failures of data quality. And they can be fixed if you fix data quality.
An Optimistic Interlude
You might read all these examples and conclude that LLMs are hopeless at tasks involving complexity, nuance, or quality. Not so!
LLMs are smart, enthusiastic, energetic generalists. They’re amazing at managing large amounts of information, at building complex artefacts, at breadth across a dizzying range of competences.
For tasks on which they’ve been tuned — like software engineering — using LLMs feels like magic. Not the marketing-copy version of magic, or the stage-trick version of magic: the real thing. Take it from me, LLMs are incredible.
But LLMs are just tools. And like all tools, they’re not perfect; they can be used incorrectly; they can get things wrong.
The point of this essay is to unlock their use. Learn how to use these spectacular tools to their fullest potential. Steer, don’t fear!
Marking Your Own Homework: AI as Evaluator
AI is such a great, general-purpose tool. Why not use it to evaluate its own output? This is the third way that AI interacts with data quality.
The pros are obvious: scale, speed, cost, consistency, stamina, coverage. LLMs can handle more material, faster and cheaper than any human; they know so much, never get tired, never get bored.
The cons are equally obvious: self-certification and untethering. Self-certification is when models audit themselves. LLMs are not very good at this; as mentioned before, they don’t know what they don’t know, and when confronted with contrary evidence, they tend to either double down or oscillate wildly. This not exactly a behaviour that inspires confidence5.
Untethering is a different failure mode, where a closed cognitive loop (input → output → evaluate → more input → more output) leads to models drifting away from reality, detached from their grounding. We’ll talk more about this below.
And of course AI judges have their own biases, blind spots, dependencies, confident errors, and so on. These are bad in themselves; they are also gameable by the evaluatee. So you have to ask, who evaluates the evaluators, and that just leads to infinite regress...
“You see a lot, doctor. But are you strong enough to point that high-powered perception at yourself?”
— Clarice Starling, to Hannibal Lecter, at their first meeting.
(The Silence of the Lambs, 1991)
Contamination
There’s a theme running through all the above: contamination.
We’ve talked a lot about the importance of grounding: of connecting your model — both inputs and outputs — to an authoritative external source of truth. But what if that external authority is itself the product of an LLM?6
LLMs are great at producing large quantities of output. LLMs are great at processing large quantities of input. The temptation is to connect the two: to pipe one AI app’s output to another AI app’s input. From there, it’s a small step to having LLMs write to your internal sources of truth. Contamination.
If your LLM — or an app built on AI, which means most apps these days — writes directly to your general ledger, your CRM, your customer metrics, your operating handbook, you should be worried.
Errors, once created, persist and propagate. Downstream AI apps read bad data, assume it’s correct, and write even more bad data back into your files. Nuance and detail are lost; falsehoods and inventions become enshrined as truth; and your models build ever-more elaborate superstructures on ever-more shaky foundations. Soon, your organization is subtly detached from reality; your context has been poisoned7.
The way to avoid this is informational discipline. Isolate your primary sources. If you have genuinely proprietary data, quarantine it now! Flag LLM outputs, keep them separate, be explicit and conscious of when and how you join them. Supervise and inspect. Chain rarely. None of this is new; it’s the same set of practices we’ve been talking about all along. But contamination makes these practices a must-have, not just a nice-to-have.
In Conclusion
This brings us to the end of another Pivotal essay.
Articles about AI are often too vague to be useful, or too specific to last beyond the next model update. In this essay, I’ve tried to find a middle ground, with concrete examples and suggestions but also general principles that stand the test of time. I hope you found it helpful!
Toronto, August 2026
Never miss a post!
Pivotal is a free newsletter of long-form essays on finance, AI, data, startups and tech. I write infrequently and in depth; I like to think my pieces are worth writing, and worth reading.
Subscribe to Pivotal and share it with your friends and colleagues: especially those who work in Wall Street and Main Street finance. Encourage what you want to see more of in the world.
The honourable exception is quality at the frontier labs; out of scope for this essay.
By construction. Two forces compound here: architecture and incentives. LLM architectures produce what is most likely to statistically resemble a good answer, but that doesn’t make the answer correct. And LLM incentives (reward functions) boost answers that humans like and trust and use, which again doesn’t make the answer correct.
Or should that be half-baked in, well-done out? Yes, I spent way too much time cooking up that pun.
It’s all data, young man, data all the way down.
“You’re right. I messed up.” – an epitaph for Claude.
This has already happened for the internet. The web was once an incredible source of diverse, human-generated, high-quality-in-aggregate information to train models with. But now, a significant fraction of online content is LLM-generated.
And that fraction is increasing. It’s a textbook example of Gresham’s Law: “bad money drives out good”. As machine content proliferates, human writing migrates to closed or highly filtered channels. The web has been poisoned.
And it’s a one-way street. Even if AI production ceased tomorrow, it’d be impossible to recover the pre-AI distribution of accurate, trustable, human-sourced, independent content. The degradation is insidious, and often unrecoverable. Data quality is a ratchet!
(For those of you keeping score at home, data quality is an elephant, a ladder, and now a ratchet. Oh, and it’s also a lever: a thing you can act upon to create large-scale change.)
Fantastic article.
super insightful!