Thirty minutes auditing the wrong party
The most sophisticated tool in the room was arguing with me, and I answered it with the least sophisticated one — a desk calculator, the kind with a solar strip and oversized keys — to prove myself wrong.
By then I had re-run the model three times, once the long way, once on paper. Every pass gave me the same answer. Every pass disagreed with the machine by $15 million in total cost of sales. And every pass sent me back to audit the only party in the room I thought I could inspect: myself.
Some background. I write software for a living and I'm learning finance the slow way, which for me means building research models by hand in Google Spreadsheets — unfashionable, but I trust the process because I built every cell of it.
AI came in as an assistant at first: summarize this, find me that, nothing load-bearing. Then, working through Apple's last earnings report, I promoted it, and moved the model's calculations out of the sheet and into the agent's instructions.
Eventually I gave up and asked it how it had arrived at its number.
It told me my numbers were right and its own were wrong. Then it explained. I had given it a margin percentage, rounded to one decimal place. It used that rounded percentage in one cell of its check table and the original unrounded figure in the next, and it never registered that the two described the same quantity.
Its own summary:
"I mixed the rounded input with the unrounded figure inside the same table. Your sheet is internally consistent; mine wasn't."
What actually broke
The label matters here, so I want to be precise. Nothing was invented. No arithmetic step was carried out incorrectly. Every individual operation in that table was defensible on its own terms. What went wrong is that a single quantity had two provenances inside one artifact, and nothing in the process forced them to agree.
Any engineer reading this already has a name for it. Two sources of truth for the same value, no invariant holding them together. I would flag that in a code review without having to think about it. I didn't catch it in a spreadsheet because I had handed the invariant to the model along with the arithmetic, and I hadn't noticed I'd done it.
It is also the least detectable class of error there is. When a step is wrong, you find it by checking steps. When every step is right and the total is still wrong, checking steps returns nothing — which is an accurate description of what my thirty minutes produced.
Why I checked myself first
Two things were running, and neither of them was a considered judgment.
The first was a category error. I had filed the task under deterministic computation: five or six operations on numbers I had supplied. Engineers don't check calculators. I wasn't deferring to the model's intelligence, I was deferring to a determinism I assumed was there and wasn't.
The second was domain authority. I am competent at engineering and a student of finance. In the domain where you have no standing, the confident party wins by default. What makes this one uncomfortable is that the disputed step was arithmetic — the single part of the exercise I was fully qualified to adjudicate.
Does it matter?
After it walked me through the error, the model added a line: "Does it matter? No."
On its own terms, that's correct. Fifteen million dollars against Apple's cost base is rounding dust. If the question is whether the stock is cheap, the number does not move.
But it graded the wrong thing. It measured the magnitude of the output. The defect was in the integrity of the method, and integrity has no magnitude. A method that mixes provenances produces a $15 million error on a company this size, and an error of some other size on the next one.
There's a second cost that's harder to price. I keep this process by hand because I trust it, and that trust is the entire point — it's why the output means anything to me. And someone learning a subject cannot afford immaterial errors, because he has no reliable way to tell which errors are immaterial. That judgment is precisely the thing he is trying to acquire. "This doesn't matter" is a claim only an expert can evaluate, offered to someone who isn't one yet.
What I changed
Less than you'd expect, and I'd rather say so than pretend otherwise.
The agent still runs the model. I didn't pull the calculations back into the sheet, because the leverage is real and one wrong total doesn't undo it. What changed is a habit — I check its table against the sheet before I accept it — and one line in the instructions: "When you give me a figure, tell me whether it's sourced from a filing, derived from guidance, or illustrative. If it's from memory rather than a document, say so."
That's a partial fix, and I know exactly how partial. Both figures in that table were legitimately sourced. Under the new rule the model labels each of them correctly and can still use them interchangeably, because nothing in the rule says these two describe the same quantity and must agree. What provenance labels buy me is visibility: I'd have seen two labels hanging off one number, and thirty minutes would have been five. They don't make the inconsistency impossible.
I don't have the rest of the fix yet. What I have is a sharper sense of where the seam runs. Computation delegates cleanly. The invariant doesn't — least of all to a system that grades its own errors by how large they happened to come out.