“The best time to figure out how to attribute ROI to tech initiatives is before you need to.”
– Jeff Roberts, CEO, Innovation Vista
Attribution · Not a New Problem, But Harder Than Ever
Technology leaders have been wrestling with ROI attribution – whether for efficiencies or revenue – since the first ERP implementation. But the current wave of AI investments has made the problem significantly more acute for two reasons. First, AI tools tend to augment human work rather than replace discrete processes; they make people better rather than eliminating steps. Second, AI initiatives are often layered on top of multiple simultaneous changes, making isolation nearly impossible after the fact.
The result is that most companies either never attempt rigorous attribution, or they attempt it retroactively and produce numbers that satisfy no one. Finance does not trust them. The board politely ignores them. And the next investment cycle starts with the same vague promises and the same inability to prove what worked.
There are, however, several approaches that serious organizations use to get closer to real answers. None of them is perfect. Each involves tradeoffs between rigor, cost, and practicality. Understanding those tradeoffs is the first step toward picking an approach that actually fits your organization.
Approach 1 · Before-and-After Comparison
This is the most common method because it is the most intuitive. You measure performance before the technology goes live, measure it again afterward, and attribute the difference to the initiative.
The appeal is obvious: it is simple, requires minimal advance planning, and produces a single number that executives can put on a slide. If your sales cycle averaged 47 days before the AI tool and 36 days after, you have a story to tell.
The problem is equally obvious. Organizations are not controlled environments. Between the “before” snapshot and the “after” snapshot, dozens of variables change. New hires ramp up. Market conditions shift. Competitors adjust pricing. Seasonality plays its role. Attributing the entire delta to a single technology initiative requires you to either ignore all of those factors or hand-wave them away; and sophisticated boards will not let you do either.
Before-and-after works best for initiatives with dramatic, unambiguous impact on a single metric. If you automate a manual process that previously took 40 hours per week and now takes two, the math speaks for itself. For anything subtler, this approach tends to produce arguments rather than answers.
Approach 2 · A/B Cohort Analysis
A more rigorous variant is to create two groups: one that uses the new technology and one that does not. You compare performance between the groups over the same time period, controlling for as many variables as possible.
This is the gold standard in terms of methodological rigor. It is essentially what pharmaceutical companies do with clinical trials; the control group eliminates the noise of external factors because both groups experience the same market conditions, the same seasonality, and the same competitive dynamics. The only variable is the technology itself.
The tradeoff is practical rather than theoretical. Most mid-market companies cannot or will not withhold a promising tool from half their team for the sake of measurement purity. Sales leaders will resist it. High performers will demand access. And if the tool genuinely works, you are deliberately handicapping half your revenue engine during the study period. There are also ethical dimensions in some contexts; withholding a tool that improves customer outcomes to prove a point about ROI feels wrong, and it should.
Where this approach shines is during phased rollouts. If you are deploying a new tool across regions or business units sequentially, you have a natural experiment. The groups that go live first become the test cohort; the groups still waiting become the control. This requires no artificial withholding, just disciplined measurement during the window before everyone is live.
Approach 3 · Multivariate Regression Analysis
For organizations with strong data teams, regression analysis offers a way to mathematically isolate the contribution of a technology initiative from other variables. You build a model that accounts for all the factors that influence the outcome (headcount changes, marketing spend, seasonal patterns, pricing shifts) and then measure how much of the remaining variance correlates with the technology adoption.
This approach has the highest theoretical ceiling for accuracy. Done well, it can produce genuinely defensible numbers that stand up to board-level scrutiny. It also scales; once you build the model, you can reuse the framework for future initiatives.
The downsides are significant. It requires clean, comprehensive data across all the variables you want to control for; and most mid-market companies do not have that. It requires statistical expertise to build and validate the model. And perhaps most importantly, the results are only as good as the variables you include. If a major factor is not in the model, the attribution will be wrong, and you may not know it is wrong. Garbage in, gospel out; a dangerous combination when the numbers carry the false confidence of mathematical precision.
Approach 4 · Practitioner Self-Report with Calibration
This approach asks the people closest to the work how much the technology helped. You survey sales reps, customer service agents, or operations staff and ask them to estimate what percentage of their improvement they attribute to the new tool versus other factors.
It sounds unscientific, and in its raw form it is. People overestimate the impact of things they like and underestimate the impact of things they find annoying. Recency bias, social desirability, and simple innumeracy all pollute the data.
But calibrated self-report can be surprisingly useful. The key word is “calibrated”. You set anchors: ask practitioners to estimate the impact on specific tasks rather than global performance. You triangulate across multiple respondents. You compare their estimates against whatever quantitative data you do have and adjust for known biases. And you present the results as ranges rather than false-precision point estimates.
The advantage is that this method captures something the quantitative approaches miss entirely: the qualitative texture of how technology changes work. A regression model might tell you that the AI tool contributed 6% to revenue growth. A calibrated practitioner survey might tell you that the tool’s real value was eliminating two hours of daily CRM data entry, which freed reps to make four more calls per day; and that insight is often more actionable than the number itself.
Approach 5 · Proxy Metric Mapping
Rather than trying to measure the technology’s impact on the ultimate outcome (revenue, profit, retention), you identify intermediate metrics that the technology directly influences and track those instead. Then you use established relationships between those proxy metrics and business outcomes to estimate impact.
For example, if your AI tool reduces average response time to customer inquiries from four hours to 45 minutes, and you have historical data showing that faster response times correlate with 15% higher close rates, you can build an attribution chain without needing to isolate the AI’s contribution to revenue directly.
This works well when the causal chain between the technology and the proxy metric is short and clear. It breaks down when the chain is long, the relationships are weak, or the proxy metric is influenced by too many other factors. It also requires that you actually have the historical correlation data to build the chain; and building that baseline takes time and forethought.
The Best Answer · Negotiate the Measurement Before You Deploy
Each of these approaches has a place. The right choice depends on the size of the investment, the decision it needs to inform, the data you have available, and the organizational appetite for rigor.
But the CFO in our opening vignette identified the deeper truth. The biggest mistake companies make is not choosing the wrong method; it is failing to choose any method until someone asks for the numbers. By then, you have no baseline. You have no control group. You have no proxy metrics. You are left with before-and-after comparison and a prayer.
The organizations that consistently make smart technology investments are the ones that build measurement into the deployment plan from the start. They define what success looks like before the tool goes live. They establish baselines. They identify which approach they will use to attribute impact and set up the data collection to support it.
It takes a few extra hours of planning. It saves months of retroactive guesswork. And it turns the inevitable board question from a moment of anxiety into a moment of credibility.
