There's still no free lunch in data analytics

Every time someone in your company needs an answer from data, someone pays a human cost: the time, judgment, and design work to make the information findable, trustworthy, and usable. This has been true since the first analyst opened a SQL editor. Dashboards, data catalogs, semantic layers: these are all different bets on where to concentrate that effort. AI agents are the latest bet. They shift where the cost lands, but they don't remove it.

In this post, we'll map how this cost is distributed across the systems data teams already use, and then look at what AI agents actually change about the equation.

The human economy of retrieval

This cost can be paid in different ways. A classic framework for thinking about it is schema-on-write vs. schema-on-read. As Hrishi Olickel elegantly put it, think of schema-on-write as putting clothes straight into the wardrobe, sorted by function, and schema-on-read as sorting through a messy pile when you need something. The choice decides whether you pay upfront or at retrieval.

This illustrates that any system distributes its cost between upfront effort and usage effort. For analytics, we can break this down into three dimensions:

• Indexing cost: the upfront effort to prepare, structure, and organize information so it can be found later. Designing schemas, writing metric definitions, building dashboards, documenting business logic. This includes the part nobody budgets for: getting four teams to agree on what "active customer" means.

• Querying cost: the effort at the moment of retrieval. Browsing a wiki, writing SQL, crafting a prompt, or Slacking the data team.

• Reliability: the trustworthiness of what comes back. A number can be technically correct (the SQL ran) but semantically wrong (it used the wrong definition of "active customer").

Different systems distribute these three costs differently.

Let's make this concrete. A CFO wants to know how many active enterprise customers renewed last quarter. Before anyone can answer, someone needs to decide which table holds customer data, what "active" means (last login? contract status? usage threshold?), what counts as "enterprise" (revenue band? employee count? plan tier?), and what date range defines "last quarter" in your fiscal calendar. Every system below handles this CFO's question differently. The question is who does the thinking, and when.

How data systems have always distributed cost

Before AI agents entered the picture, every data team already lived with some combination of the following systems. Each makes a different bet on where to concentrate human effort.

In practice, these systems aren't mutually exclusive. Most data teams run several in parallel (dashboards for the board deck, self-serve for exploration, Slack for everything the dashboards don't cover) and layered on top of each other (a semantic layer feeding a BI tool feeding a dashboard). We'll discuss them separately to isolate the trade-offs, but a real setup is usually a combination.

Slack the data team

The CFO's question goes to a senior data engineer, who drops what they're doing, explores the warehouse, applies tribal knowledge about what "active customer" means in this context, and delivers an answer.

• Indexing cost: minimal. The warehouse schema exists, the tables are there, but nobody has invested in making the answer to this specific question findable. No metric definition, no documented business logic for "active enterprise customer."

• Querying cost: close to zero for the CFO. But the cost doesn't disappear. It lands entirely on the data team. Every question pays the full retrieval price from scratch. Nothing is recorded. When the same question comes back next quarter from a different stakeholder, the data engineer does the same work again.

• Reliability: high, if you get the right person. The answer quality depends on which engineer picks up the question and whether they're applying fresh logic or muscle memory from last time. But it doesn't scale.

One senior data manager we spoke to described a weekly rotation where one person is dedicated to answering ad-hoc questions, consuming about 20% of a team of five or six people. Permanently.

The total cost of this system is high. If you're a head of data, you already know where it lives: in your team's time, in the senior engineer who can't finish their actual project because they're answering the same revenue question for the third time this month. It persists because it's the path of least political resistance: no project to fund, no stakeholder alignment required, no process to design. Just ask.

Write the SQL yourself

An analyst opens a query editor, explores the warehouse schema, figures out which tables are relevant, applies whatever business logic they know, and produces the answer to the CFO's question themselves.

• Indexing cost: minimal. Same as above: the warehouse exists, but nobody has curated the path from "CFO's question" to "the right query."

• Querying cost: high. The analyst needs to know the schema, understand the data model, know which of the four "users" tables is the right one, and apply the correct business logic. Every query is a mini-research project.

• Reliability: depends entirely on the analyst. A senior analyst who's been at the company for three years will get the right renewal count. A new hire will produce something that runs but may use the wrong definition of "enterprise," and nobody will catch it because the SQL looks fine.

This is the most transparent system. The cost is visibly paid by one person at query time. It also shows most clearly why indexing has value: every hour spent writing SQL from scratch is an hour that could have been avoided if someone had already defined what "active enterprise customer" means and where to find it.

Pre-built dashboards

The opposite bet. A data team invests heavily upfront: defining metrics, building views, validating logic, getting stakeholder sign-off. The CFO opens a dashboard and the renewal number is there.

• Indexing cost: high. Someone needs to define the metrics, model the data, build the views, validate the logic with stakeholders, and get sign-off that "active enterprise customer" means what everyone thinks it means. This is weeks of work before the first chart renders.

• Querying cost: near zero.

• Reliability: high, for the questions the dashboard was designed to answer.

For stable, high-frequency reporting (the monthly board deck, the weekly sales pipeline), dashboards are the right answer. The indexing cost was paid long ago and the answers are trusted. Dashboards also win organizationally because executives want consistency: one number, same source, no debate.

The limitation is scope. The CFO's next question, "but what about renewals by region, excluding partner deals?", hits a wall. At that point, the CFO either goes back to Slack or builds something themselves.

Self-service BI

Self-service tools shift some querying cost to the business user: drag-and-drop interfaces, explore modes, natural language features. The CFO (or more likely, their RevOps lead) can try to build the view themselves.

• Indexing cost: high, but different from dashboards. Instead of building finished views, the data team builds the underlying models, curated explore layers, and documentation that let business users compose their own answers. The scope is broader (you're enabling a range of questions, not answering a specific one), which means more modeling work, not less.

• Querying cost: moderate. Lower than raw SQL, higher than a pre-built dashboard.

• Reliability: depends on how much curation went into the indexing. The best implementations control which fields and joins are available, so the business user has freedom within a pre-validated scope. Without that curation, a RevOps lead building their own view may not know that the "revenue" column in one table excludes partner deals, or that the "customers" table has a legacy duplicate. The tool just moves the reliability risk from the data team to someone less equipped to catch it.

Self-service works well within bounded trust zones. Outside those zones, people stop trusting the numbers. It wins politically because business teams want autonomy, and nobody wants to wait three sprints for a dashboard update.

Spreadsheets

When the dashboard doesn't cover the question and the data team can't turn it around fast enough, the business user opens a spreadsheet.

A GTM strategy lead we spoke to described the split: "80 to 85% of the data points we need are already structured. Maybe 10-15% is true ad-hoc: 'wait, our win rate dropped against a competitor, why is that?'"

That 10-15% is where spreadsheets live.

• Indexing cost: low. Paid by one person (the person building the sheet).

• Querying cost: high, even for that person. They're working from a data dump or a CSV export, rebuilding logic from scratch, cross-referencing with other sources. And impossible for everyone else. Nobody else can read their spreadsheet.

• Reliability: near zero. No governance, no versioning, silently wrong the moment the source data changes.

If the CFO's renewal question isn't covered by the dashboard and the data team is busy, someone will answer it in a spreadsheet. Spreadsheets win because the business can't wait. The cost doesn't disappear. It moves to a place with no quality control.

Data catalog

A data catalog is a bet on documentation. Invest upfront in describing your tables, columns, metrics, and definitions, so that anyone querying the data can find what they need.

• Indexing cost: high.

• Querying cost: moderate. The user still needs to search, read, and interpret.

• Reliability: high when the catalog is fresh. Degrades fast.

The problem is that "high when fresh" has a short half-life. The business changes, the data model evolves, and the documentation doesn't keep up. Six months after launch, half the entries are stale. Eighteen months later, nobody trusts them enough to check.

The concept is sound. The failure mode is that documentation is detached from the moment work happens. Someone learns that "active customer" now excludes trial accounts, but that knowledge gets applied in a query, not captured in the catalog. Nobody is rewarded for catalog upkeep, and the feedback loop from usage to updates doesn't exist.

Code-defined semantic layers

This is the code-first approach to indexing. Metric definitions live in version-controlled files, go through pull request review, and are enforced programmatically. The CFO's "active enterprise customer" renewal count is defined once in code and served via API. dbt's metrics layer is the most prominent example. LookML serves a similar function within the Looker ecosystem, and in many stacks the two coexist: dbt handles the transformation logic while LookML defines the explore layer on top.

• Indexing cost: the highest of any system here, and not primarily because it requires engineering skill. The real cost is the semantic modeling: deciding what "active customer" means across every context, how it maps to physical tables, which edge cases to handle, which to ignore. This is the work the CFO's question depends on, made explicit and codified rather than left in someone's head.

• Querying cost: low. Metrics are accessible via API or through the BI tool's interface.

• Reliability: high, within the defined scope.

The strategic value of this system goes beyond convenience. A code-defined semantic layer is where the company settles disputes about meaning: what counts as a customer, what counts as revenue, what "active" means. It's institutional arbitration, codified.

Two trade-offs. First, business users can't contribute definitions directly. If a RevOps lead knows that "active customer" should exclude trial accounts in EMEA but not in North America, that knowledge needs to be translated into code by someone on the data team, reviewed, merged, and deployed. That friction is partly by design: code review and version control are what make the definitions trustworthy. But it also means extending scope always requires the same engineering cycle, regardless of how simple the business logic change is. Second, the definitions represent the data team's understanding of business logic at the time they were written, which may lag behind what the business actually means today.

Where the cost lands

Deciding where to pay

No system is universally right. The choice depends on what you're optimizing for.

Rigidity vs. flexibility. If you know the questions in advance and they'll be asked repeatedly, pay the indexing cost upfront. Dashboards and dbt definitions are the right bet for stable, high-frequency reporting. If the questions are unpredictable, exploratory, or evolving, heavy upfront indexing either doesn't cover them or becomes stale before they're asked.

Deterministic answers vs. probabilistic discovery. Some retrievals need to be exactly right: the board deck, the ARR calculation, anything a decision runs on. Others are about exploration and hypothesis generation, where an approximate answer is useful and a wrong one is cheap to discard. Invest in reliability where the cost of a wrong answer is high. Accept approximation where speed and breadth matter more.

Who bears the querying cost. "Slack the data team" is free for the business user and expensive for the data engineer. Dashboards are free for everyone but locked in scope. Self-service shifts the cost to the business user, along with the reliability risk. Who can absorb the effort, and who can't afford the errors?

In practice, this means routing by stakes and frequency. Recurring high-stakes numbers (ARR, board metrics, pipeline) belong in governed dashboards or a semantic layer. Bounded exploration ("show me renewals by region") belongs in self-serve with curated models. Ambiguous one-offs where being wrong is cheap ("did churn spike after that pricing change?") are fine for analysts or agents.

Remember: "pay upfront" means "pay to maintain." Every system with high indexing cost commits you to ongoing maintenance. A dashboard nobody updates is a liability. A catalog nobody maintains is worse than no catalog. The indexing cost is never truly one-time. If you can't sustain the maintenance, the upfront investment decays into a false sense of reliability.

What AI agents change

On the querying side: a step change

The first thing most teams think about with AI agents for data is conversational analytics. A business user asks a question in natural language, and the agent generates SQL, builds a chart, or surfaces the right dashboard. The CFO doesn't need to learn a BI tool or wait for the data team. Querying cost drops dramatically.

Given the right context, this works. Text-to-SQL handles the CFO's renewal question. Agent-built visualizations start replacing self-service BI for exploratory work. Where data catalogs or semantic layers already exist, agents also improve discoverability: instead of searching and browsing, the user just asks, and the agent finds the relevant metric or table.

The querying cost reduction is real. But it depends entirely on the quality of the context the agent has access to.

On the indexing side: the requirements go up

Agents don't just consume existing context. They turn previously tolerable ambiguity into system-level failure.

When a human analyst hits an ambiguous definition, they bridge it with judgment: they ask a colleague, check the context, make a call. That's slow, but it's self-correcting. An agent doesn't bridge ambiguity. It scales it. If "active customer" is ambiguous, the analyst gets one wrong answer. The agent gets 500 wrong answers before anyone notices, all plausible, all consistent with each other, all based on the same stale definition. That's not just more indexing demand. It's a multiplier on unresolved semantic debt.

This means more comprehensive metric coverage in your semantic layer, more business context layered on top of your LookML or dbt models, documented links between business concepts and physical data. Everything that currently lives in people's heads needs to be explicit and machine-readable for the agent to use it.

Other AI agents can assist with this work: extracting entities from documentation, generating descriptions, consolidating context from Notion pages and Slack threads and code comments. This reduces the initial effort. But the indexing still needs human judgment. Agents can accelerate the mechanics of writing it down; they can't decide what's canonical.

The reliability question

Even with good context, the agent inherits the limitations of its sources. If the dbt definitions are incomplete, the agent works with incomplete context. If the catalog is stale, the agent reasons on stale information.

In practice: the agent returns the CFO's renewal count, but it's off by three customers because it included a revenue stream that was reclassified in January. The number looks plausible. Nobody questions it. It's not a hallucination in the classic sense. It's a semantic error: the agent used a definition that was correct three months ago.

How do you detect this? You can review dbt PRs for definition changes. You can diff metric outputs week over week. But there's no obvious mechanism for knowing when the context feeding your agent has drifted enough to produce unreliable answers.

As one data lead put it: "We automated context collection. DAGs pull queries, YAML, Notion. There's not a lot of maintenance, but there's also zero evaluation. If things drift, we have no idea."

What if agents get good enough to gather all the context they need at query time? The cost doesn't disappear. It shifts to reliability (did the agent fetch the latest definition of this metric?) and back to querying (the user now needs to reverse-engineer how the agent assembled its context to trust the answer). Either way, it becomes a verification and maintenance problem.

Agents redistribute cost, they don't remove it

Agents compress the querying cost, increase the indexing requirements, and introduce new reliability risks. The economics shift, but they don't simplify.

The opportunity is that agents can also help distribute the indexing work more broadly: assisting with documentation, surfacing gaps in coverage, flagging stale definitions. The cost doesn't have to concentrate in one team doing one big project. It can be spread across people and tools, captured continuously rather than built once.

The real bottleneck is semantic coordination

Someone always pays the indexing cost, the querying cost, and the reliability cost.

The approaches that look free hide the cost in places that don't show up on a dashboard: engineer hours, answer latency, decisions made on wrong numbers. The approaches that look expensive can be the most efficient, if the indexing is distributed broadly and maintained continuously rather than concentrated in one team at one moment.

Agents give you a new option for distributing that cost. The querying cost drops. The reliability question shifts. But the indexing (the work of deciding what your data means) remains human work. The question is how you organize it.

Concentrating all the indexing in a single project, by a single team, at a single point in time, captures a snapshot of business logic that starts decaying immediately. The business moves on. The definitions don't.

The alternative is continuous indexing: small contributions from many people over time, captured at the moment the knowledge surfaces, persisted so it compounds rather than evaporates. Not a six-month semantic layer project. Not an agent reinventing the context every query. Something that turns every retrieval into a little bit of durable indexing.

What does that look like in practice? When the RevOps lead corrects a renewal number because the definition changed in January, that correction gets captured as a definition update, not buried in a Slack thread. When the data engineer answers the CFO's question and applies a filter nobody documented, that filter becomes part of the institutional knowledge, not part of an ephemeral conversation. The indexing happens as a byproduct of the work people are already doing. The data team governs the semantic capture (what definitions are canonical, what rules apply, what quality bar is required) but they're not the only source of it. Business users, analysts, even agents contribute knowledge. The data team curates and validates. That's a different role than "answer every question yourself."

This raises its own hard questions. Which semantics must be centrally governed? What can business users contribute safely? What should agents propose but never decide on their own? Those are organizational design problems, not technology problems, and they deserve their own treatment.

But the underlying economics are clear. The bottleneck in analytics has always been semantic coordination: getting people to agree on what the data means, and keeping that agreement current. Agents make that bottleneck more economically visible, not by creating it but by scaling the cost of leaving it unresolved. The goal is not eliminating human work. It's making sure that scarce expert judgment goes into reusable decision logic rather than repeated answer production.

Written by Matthieu Blandineau