Verifying product metrics in due diligence: the deck versus the database

Verifying product metrics is the part of technical due diligence that investors most often do not know they can ask for. The deck says monthly active users, net retention, documents processed, automation rate. Those numbers came from somewhere, usually a database and an event pipeline the audit team already has access to. Re-running the number is a day of work, and the gap between the slide and the query is one of the most useful findings an investor receives.

This is a different exercise from checking a model’s accuracy claim, which we treat in how we verify AI performance claims. Here the question is how many customers use the product, how often, how much, and whether the definitions behind those words survive a second look.

Ask for the query behind each headline metric

We begin with the deck rather than the database. For each metric on the growth slides we ask one question: show us how this number is produced. Sometimes the answer is a dashboard with a saved query. Sometimes it is a spreadsheet the head of product updates by hand on the first of the month, with a column nobody can explain. The form of the answer is the first finding: a company that can point at a query for every metric runs on instrumentation, and one that assembles its numbers by hand is reporting a belief with decimals.

We also ask when each definition last changed; a retention figure redefined two quarters ago without a restatement is a chart with a hidden step.

Re-run it on a read replica

With the query in hand, we run it ourselves, on a read replica or an export. The point is reproduction rather than suspicion. Most of the time the number matches, and we write that down, because a verified metric is worth more to a committee than an unverified one.

When it does not match, the cause is usually one of a small set: test and internal accounts counted as customers, a date range that stops before a bad month, a join that double counts users with two logins, a filter added for a board meeting and never removed. These are ordinary mistakes, and they move the number in a direction that is rarely random.

What counts as active

The definition is where most of the gap lives. "Active user" can mean logged in once this month, performed the core action, or had an API key that sent a heartbeat. "Customer" can mean paying, trialing, or signed a letter of intent. For AI products, "automation rate" can mean the share of tasks the system completed, the share it attempted, or the share of a pilot customer’s tasks on the best week.

We rewrite each headline metric with its definition spelled out and recompute it under the strict version, the one that will appear in the next investor’s diligence. Where the deck’s definition is defensible we say so. Where it is generous we show both numbers side by side and leave the pricing to the deal team.

Tracking plan hygiene and the billing reconciliation

Two checks catch what the queries miss. The first is the event pipeline, where we check whether a tracking plan exists, whether events are named consistently across web, mobile and API, and what share arrive with missing properties. A pipeline with no plan produces usage metrics that drift every time an engineer renames an event, and that drift gets presented as growth or churn depending on its direction.

The second is billing. Revenue is the one metric with an external witness, so we reconcile usage against the invoicing system, checking that the active users appear in the billing records, that their tiers match what they are charged, and that the usage curve bends where the revenue curve bends. For AI products this also feeds the cost side: usage per customer is the denominator of the margin question, and a usage figure that cannot be reconciled leaves the margin unverifiable too.

The spreadsheet nobody can reproduce

In almost every audit there is one metric that lives in a spreadsheet, usually the most flattering one. A founder built it for a fundraise from three exports taken on different days, and has since moved on or cannot remember the steps. We ask for it to be reproduced from source in front of us; if it cannot be, the metric comes out of our report, and we tell the committee why. A data room that holds the dashboards and their queries, the kind we describe in what belongs in a technical data room, makes this short; its absence is a finding about how the company runs.

The output is a one-page table: each headline metric, the deck’s number, the number we reproduced, the definition used, and what explains the difference. Most gaps are small and explainable, and the investor gets numbers they can repeat to their own committee without a caveat. When the gaps are large, the finding says more about the company than about the metric, and we put it to the CTO in the session that changes the most findings. The check belongs inside the audit we run for funds rather than in a separate analytics workstream, because the people who read the code are the ones who can read the query.