← Back to Field Notes

· 2 min read

Chief AI Antagonist: Why Gold Isn't Enough for AI

95% accuracy doesn't cut it for closing your books. Before an LLM can answer questions about your business, you need a deterministic metrics layer underneath it.

In an era of agentic everything, I often find myself assuming the title "Chief AI Antagonist." I've nayed several "talk to your data" natural-language query engines. It's not that I don't want to offer this—I would like nothing more than to build cool AI features on top of our data, internal and external, answering simple questions like "What was our revenue last month?"

What I can't have is that question yielding a different answer than "What was our TOTAL revenue last month?"—or giving the same answer to "What was our gross revenue?" and "What was our net revenue?"

We've all gotten used to the medallion architecture: bronze, silver, and gold layers. But that misses a key component required to build a reliable harness. We need another layer. People talk about platinum queries, but we need a platinum layer—a consumption layer, a semantic layer, a metrics layer. Call it whatever you want, but you need to define—deterministically—the things the agent answers, and then let the LLM pick and choose what to do with the puzzle pieces.

Proper data architecture is becoming table stakes, but adding this layer infused with business logic—driving consistency not just across reports, but across semantic terms—is non-negotiable. Everyone wants to talk to an agent and make dashboards and reports obsolete, yet we still need traceability, SQL inspection, and verifiability baked in. 95% doesn't cut it for closing your books.

Building out this metrics layer is the unlock not just for AI, but for true insights. Once you've modeled the ontology of where your business data comes from, you can pull the levers that actually drive change—creating ML models that surface real insight across a population and translate into clear signals for individual accounts. That is what drives customer value.