Case study · Strategy & vision
Proving Lead Qualification Before Building It
I replaced a lagging returns metric with a pre-sale quality score and validated lead qualification manually, laying the foundation that helped halve returns in 2025.
The context
Lead quality was measured by the share of leads that partners returned. Teams repeatedly ran A/B tests to reduce returns, but the results remained inconclusive. We wanted to qualify leads before selling them, yet qualifying every lead automatically was judged too complex to build.
The challenge
I needed a metric that reflected match quality before a lead was sold, and evidence that lead qualification created enough value to justify development.
How I approached it
My research showed that partners did not return only poor-quality leads. They also used returns to signal which types of leads they did not want, and returns could arrive up to two weeks after a sale. That made the metric a lagging indicator and skewed experiment results.
I proposed a Lead Quality Score for every consumer request before matching, built from automated checks, manual qualification by phone, signals from consumers and partners, and the use of quality as a matching criterion. Unlike returns, it could serve as the primary metric in A/B tests.
Rather than waiting for an automated solution, we began qualifying leads by phone. I set up a BI dashboard tracking which leads had been called, their quality and the revenue they generated, so we had evidence from a working process before committing engineering capacity.
What it delivered
The data demonstrated the value of qualification and reframed the requirement from qualifying all leads to qualifying as many leads as possible, which made the solution buildable and its impact measurable. The Lead Quality Score became the foundation for lead qualification, which helped halve returns in 2025 without losing gross margin.
Looking back
I learned that the existing metric was obscuring the problem rather than clarifying it. Returns arrived late and reflected partner preferences as well as poor quality, so repeated experiments could not give us a reliable answer. I would challenge such proxy metrics earlier and look sooner for a measure that exists before the outcome we want to influence.
I also learned to prove value through a workable manual process before asking engineering to automate it. Phone qualification and the dashboard gave us evidence, changed “all leads” into “as many as possible,” and made the solution buildable. The principle I carry forward is to validate value and measurement first, then invest in scale.