Tip
Archie Abrams talks about metric traps at Shopify: “When you have teams naturally break up the world into different funnel stages, it gets very seductive to look at my part of the funnel and what’s my conversion rate through that part of the funnel. But in practice, it’s actually almost always easier to just make it harder to do the thing right before your step in the funnel to increase your conversion rate. Instead of I’m trying to convert a bunch of people, I just want more people to get activated. The easiest way to increase my signup-to-activated rate is just make it harder to sign up.”
Turns out AI product team incentives work the same way.
You’re running quarterly business reviews with your AI product teams. The onboarding team proudly presents: “Trial-to-paid conversion up from 12% to 18%—crushing it!” The activation team shows: “Signup-to-activated up from 45% to 62%—best quarter ever!” Everyone’s hitting their numbers. Bonuses all around.
But when you look at the top-line business metrics, total activated users is flat. Total paying customers barely moved. How is every team hitting their conversion rate goals while the business isn’t growing?
Younger leaders celebrate the conversion rate wins at face value. They haven’t run enough orgs to recognize the game being played. You’ve seen this movie before. In 2016, your enterprise sales team hit 85% close rate on qualified leads—incredible! Except they’d quietly made lead qualification so strict that they were only pitching sure things. Total deals closed went down.
You dig into the onboarding team’s “win.” They made the free trial harder to get—added a credit card requirement, added verification steps. Fewer signups, but higher intent people, so conversion rate went up. Great for their metric, terrible for total customers.
You dig into the activation team’s “win.” They made signup harder—added more required fields, more verification. Fewer people signed up, but those who did were more likely to activate. Great for their conversion rate metric, terrible for top-of-funnel volume.
Both teams were implicitly optimizing locally by constricting the funnel stage right before them. It’s the rational response when teams are measured on conversion rates instead of absolute throughput. You kill their conversion rate metrics and replace them with absolute numbers: “How many activated users did you add this quarter?” not “What’s your conversion rate?” Suddenly the incentives flip—now teams want to make it easier to get to their stage, not harder.
This judgment—recognizing when teams are gaming local metrics at the expense of system-wide goals—comes from watching enough organizations optimize themselves into stagnation. Junior leaders celebrate improving conversion rates. You’ve learned that conversion rates are dangerous metrics because they’re easily gamed by constricting upstream flow. That wisdom comes from seeing teams optimize locally and destroy globally repeatedly.
Context
Archie Abrams is VP of Product and Head of Growth at Shopify, leading over 600 people. At Shopify, teams don’t optimize for conversion rates—they optimize for absolute numbers of users reaching each milestone.
Archie learned that when teams are measured on conversion rates, they implicitly make upstream steps harder to game their local metric. For experienced executives structuring AI product team incentives, this pattern recognition is critical—you’ve watched enough teams game local metrics to know conversion rates drive perverse incentives.
That comes from seeing the optimization-into-stagnation pattern play out repeatedly.