A New Interface Needs New Success Metrics

Several product categories are questioning the interaction that made them successful. Dating products are looking beyond swiping towards AI matching and in-person meetings. Commerce platforms continue to invest in live shopping instead of treating the product page as the only place where conversion happens. AI companies are making chat more widely available while also exploring dedicated hardware.

The obvious product response is to debate the new interface. Should matching happen without a swipe? Should shopping feel more like entertainment? Does an AI assistant need its own device?

I think that discussion starts one level too low.

The more important question is whether the company is willing to change how it defines success. If the interface changes but the metrics stay the same, the team will usually rebuild the old product underneath a new interaction.

Take a dating product moving away from swiping. If the team is still judged on profiles viewed, daily sessions and actions per session, it has a problem. A genuinely useful match may reduce all three. An in-person event might create value that barely appears in the existing funnel. The new experience will then be pushed to generate more frequent engagement, because that is what the dashboard rewards. Before long, swipe-free matching starts behaving remarkably like swiping.

The same applies to live shopping. If every decision is evaluated using the conversion logic of a conventional product page, the team will optimise the live experience into a less efficient checkout flow. That misses why someone might join in the first place: demonstration, discovery, entertainment or trust. These are not excuses to ignore revenue. They are reasons to be more precise about the path to revenue.

I have seen versions of this inside product organizations. A company announces a strategic shift, teams design new experiences, and the existing scorecard remains untouched. Leadership then asks why early conversion is lower, why usage is less predictable or why the new proposition attracts a different type of customer. The roadmap may have changed, but the operating system has not.

This is also why noisy dashboards are more than an analytics problem. They allow every stakeholder to find a metric that supports the old behaviour. One person points to engagement, another to short-term revenue, and another to adoption. Nobody has to make the uncomfortable choice about what the new product should actually improve.

Before building a new primary interaction, I want a team to state the new value exchange in plain language. What can the customer now achieve that was difficult before? What behaviour would indicate that this happened? Which existing metric might get worse if the new model works?

That last question is especially useful. If nobody can name a metric that may decline, the proposed change is probably incremental, regardless of how novel the interface looks.

The answers should shape the experiment. A restaurant beta can reveal that customers value something other than the feature the team expected. Post-launch testing can expose behaviour that a controlled test missed. Small process experiments can also show whether the team is learning or merely producing more activity. But those methods only help when people are allowed to question the original measurement model.

Product sense is often described as an ability to recognise what customers will value. I would add a less glamorous part: recognising when the company’s current metrics will prevent it from delivering that value.

A new interface is easy to demonstrate. A new success model is harder because it changes priorities, targets and sometimes the commercial story. That is precisely why product and growth leaders should deal with it first. Otherwise, we are not creating a new product. We are giving the old funnel a new screen.