🎬 TL;DR: WATCH THE VIDEO (click above) for the full breakdown with real examples from top products and surprising AI research. Only takes 3.6 minutes at 2x speed. ⏩
Researchers showed 16,000 people poems and asked them to guess: human or AI? People were awful at this. They thought AI poems were written by humans more often than actual human poems were. And they rated the AI poems as higher quality, too.
Why was it so hard for people to evaluate AI quality? They had never read an AI poem before, so how would they know how to identify one when asked? They had no reference points for AI poetry. They were flying blind.
Users face the same challenge when it comes to your AI feature. AI is new. People are just getting familiar with what it is and how it behaves. They’ve likely never experienced AI before in the way you’re offering it to them.
And so just like the AI poetry, people won’t have the reference points to truly understand how valuable your product or feature is until they use it. Behavioral scientists call this the "experience gap": we can't judge something we've never encountered before.
This “AI Experience Gap” has implications for your pricing strategy and how you try to get users to engage with the feature.
Why that’s a problem: The AI experience gap
In this teardown, I look at two different email products: Shortwave and Superhuman. Both use AI to help you manage your email. They have the same type of features, but they price them differently. This pricing difference changes how people engage with the features.
Let’s just look at “AI email search.” First, AI email search is very different from normal email search. AI lets you use natural language to find that pesky email from 3 years ago from the guy whose last name you can’t remember. This is not possible in normal email search.
Superhuman: Locks their AI email search behind their business tier. It’s only available in the middle tier. This means users need to upgrade to experience it. There’s no way to trial it before upgrading. I need to cross my fingers and take a leap of faith. That’s incredibly hard to do. Not only because there’s no moment when I’m forced to reckon with how terrible my life is without AI email search, but also because normal email search is just so mediocre. If I think it’s anything like the normal email search, my WTP is likely low.
Shortwave: Lets you experience AI search in their basic tier. And the feature really wowed me when I tried it. I couldn’t stop talking about how impressive it was. Shortwave’s AI search understood my natural language query and found exactly what I needed. Suddenly I had a reference point for what AI search could do that regular search couldn't. Experience gap bridged!
Note: Shortwave could still optimize their pricing further. In the teardown (pro tip: watch it ;) I deep dive on their specific pricing model. They cap AI search to “50 threads”. This will likely backfire. Why? A pricing strategy that enables a “free trial” should:
NOT give you a worse product by downgrading the experience. 50 threads is worse then their higher tier of 75 threads. The goal of a pricing strategy is to wow someone enough to drive WTP, not have them think the feature is subpar.
Anchor you on a metric you understand. Slack limits my history to 90 days. Days is a unit of measure I understand and value. If you gave me 20 days or 120 days, I understand how this differs from 90 days. I don’t understand “50 threads”. In the video teardown, I also cover how Slack attempts to sell their email search and the pros and cons of their approach.
Mental model formation: Why this isn't just about email search
The bigger issue? We're in this moment where people don't have mental models for AI capabilities yet. With traditional features, users understood what "better search" or "faster processing" meant because they could compare to existing experiences.
AI is different. The difference between AI and non-AI features is often qualitative, not incremental. Your job as a product builder is helping users develop mental models for what great AI feels like.
How to bridge the gap
The way forward is actually pretty simple: give users the good feature first, let them experience its full power, and only ask them to pay for continued access after they've bridged the experience gap.
This works because people need to actually try something before they can judge whether it's worth paying for. For AI features, that means hands-on experience, not trying to imagine capabilities from marketing copy.
3 insights for driving AI feature adoption from the teardown:
💡 People struggle to evaluate AI value until after they experience it—so don't hide your best stuff behind paywalls
💡 Bridge the experience gap first, monetize second—let people fall in love, then ask them to pay. Obviously the feature has to WOW someone to make this strategy possible.
💡 Avoid usage-based metrics that could confuse to non-users—"50 threads" means nothing to users who don't know what threads they're missing.
Working on AI feature adoption? I'd love to share some thoughts. Email me at kristen@irrationallabs.com.
🎬 This is just a sneak peek! WATCH THE VIDEO (click below) for the full teardown with actionable examples. Only takes 3.6 minutes at 2x speed. ⏩
Have a friend who would enjoy these teardowns? Click the button below to refer them (& earn some great rewards).👇
📧 Questions about product adoption? Shoot me an email: kristen@irrationallabs.com.
Want to increase conversion, retention, engagement? Reach out to Irrational Labs.
We design products that change behavior, using behavioral science. Check out our case studies to see it in action.
