A product manager on your team has an idea at 9am. By 9:20, they have a working prototype: screens, flows, interactions all built with Lovable or v0 without touching a design tool or filing a ticket. This change in capability and pace is mind-blowing, and we're not immune to it at Great Question. As an AI-native company, product managers, designers, marketers, the CEO are all prototyping with AI.
Lenny Rachitsky's large-scale AI productivity survey found that 19.8% of product managers are already using AI to create mockups and prototypes, with another 44.4% wanting to start. That +24.6 percentage point swing makes prototyping the single most-wanted AI use case for product managers.
Now here's the uncomfortable part: user research has the largest AI demand gap of any PM task. Only 4.7% say AI-powered research is their primary use case today, but nearly a third want it to be. That's a 27.2 percentage point gap that dwarfs every other category.
Teams can build testable prototypes in minutes. They still test them in weeks. That gap is the real limitation of AI usability testing right now.
To understand why testing is now the bottleneck, you have to see what's happening on the prototyping side. It's moving faster than most testing workflows were designed to handle.
Lenny's guide to AI prototyping for product managers walks through the current generation of tools: Lovable, v0, Replit, Bolt, that turn text prompts, hand-drawn sketches, or Figma designs into interactive, multi-screen prototypes in minutes. Not wireframes. Working front-ends with real interactions, sometimes backed by a database. A product manager can go from idea to something shareable in under 10 minutes without touching code or filing a design request.
And this isn't just individual product managers experimenting. Lenny's follow-up on getting entire teams prototyping with AI shows how companies like Shopify, Ramp, Duolingo, and Intercom have made AI prototyping a team-wide practice, not a side project. Shopify gives employees access to tools like Claude, Cursor, and Copilot and encourages them to contribute to a shared library of prompts and agents. Duolingo went from 100 courses in 12 years to 150 courses in 12 months, partly by letting teams prototype and iterate faster with AI. Ramp built AI user personas that give PMs instant feedback on any spec.
The result: prototyping has gone from a specialized design activity to something anyone on a product team can do before lunch. It's a genuinely exciting time to be a product manager.
The math used to work differently. Building a prototype took days or weeks. Testing it took days or weeks. Everything was slow, so no single step felt like the problem.
AI collapsed the building side. But the testing side still looks like 2021. You write a test plan. You figure out who to test with. You recruit participants (if you use a panel, you wait hours; if you want your actual customers, you wait days or weeks pulling lists, sending emails, juggling calendars). You run sessions. You watch recordings. You tag and code transcripts. You write up findings. You present to stakeholders.
The prototype took 20 minutes. The test could realistically take weeks.
And because prototyping is getting faster, teams are generating more things to test. The backlog isn't shrinking, it's growing. That's the bottleneck nobody planned for.
This isn't just a "tools need to catch up" problem. There's a structural reason prototyping leaped forward and testing didn't, and understanding it changes how you approach AI usability testing.
Prototyping is a generative task. You describe what you want, AI creates it. The output is concrete: screens, code, interactions, and you can evaluate quality immediately: does it look right? does it work? The feedback loop is tight and visual. That's exactly the kind of task LLMs are built for.
Research is an interpretive task. You're not generating answers, you're trying to understand behavior, motivation, context. Things that are messy, contradictory, and resist clean categorization. When a participant says "this is fine" while hesitating for six seconds and then navigating to a completely different screen, the insight isn't in the words. It's in the gap between what they said and what they did. That's a fundamentally different cognitive challenge than "turn this description into a React component."
This is why AI can sometimes be known for strengthening biases, and amplifying stereotypes in research contexts. AI processes language, not necessarily meaning, although I'm sure it will only get better and better on that front too. But for now, it can tell you what people said. It struggles with why they said it, whether they meant it, and what they would have said if you'd asked a different question.
67% of teams say they'd trust AI-generated tests — but only with human review. That's a rational response to the difference between generating artifacts and interpreting human behavior.
The most hyped promise in AI usability testing right now: tools that simulate how users would respond to your interface without involving real participants. AI personas, synthetic users, automated heuristic evaluations that scale to hundreds of screens in minutes.
The appeal is obvious. If you could skip recruitment entirely, the testing gap disappears overnight. Some teams are experimenting with this already. Ramp built AI user personas that give PMs instant feedback on any spec, and it's a useful exercise for pressure-testing assumptions before committing to a real study. Syntethic users have their time and place.
But there's an important difference between "useful for generating hypotheses" and "reliable for product decisions." LLMs produce the most statistically plausible response to any prompt. That's almost never the response that changes your product direction. The insights that actually matter, a power user who's built workarounds you didn't know existed, a customer segment that uses your product in a way you never designed for, come from the unpredictable collision between real people and real interfaces.
Smashing Magazine's analysis found meaningful overlap between synthetic and human feedback on surface-level usability issues. But the surprises, the findings that shift roadmaps only came from real participants. Synthetic testing finds what you'd expect. Human testing finds what you wouldn't.
The teams getting the most out of synthetic testing treat it like an internal review step, not a full replacement for real usability testing. Run your prototype through AI personas first, fix the obvious issues, then test with real people. You'll waste fewer sessions on findability problems and spend more time surfacing the deeper stuff.
So if the interpretive side of research resists full automation, where does AI genuinely help? In the operational layer that surrounds the interpretive work: the hours of logistics, processing, and coordination that eat up 70-80% of a researcher's time.
The analysis bottleneck is mostly solved. Automated transcription, sentiment tagging, theme clustering across sessions, highlight reel generation. A researcher who used to spend a week coding transcripts now spends an afternoon reviewing AI-generated themes and correcting the ones it got wrong. When one researcher can process five studies in the time it used to take to process one, the capacity problem changes shape entirely.
Structured sessions scale without quality loss. For unmoderated testing: task-based sessions where participants complete missions at their own pace: AI moderation works. It asks follow-up questions when someone pauses, probes on interesting responses, adapts based on what's happening. Teams can run 50 sessions in the time it used to take to run 10. The participants are still real. The moderator is automated. And the insights per session stay high because the follow-ups are contextual, not scripted.
Cross-study synthesis makes continuous discovery real. A usability issue from this week's prototype test connects to a theme from last month's interview series and a pain point from the quarterly survey. Teresa Torres' continuous discovery framework argues teams should test assumptions weekly. AI-powered research repositories make that realistic by keeping insights connected instead of scattered across a dozen tools.
But here's the part most conversations about AI usability testing miss: none of these improvements matter much if you're testing with the wrong people.
Every AI improvement above: faster analysis, scaled moderation, cross-study synthesis, multiplies the value of each session. But value multiplied by zero is still zero. And a session with someone who has no context for your product is close to zero for anything beyond basic findability checks.
This is the piece most AI testing tools skipped. They bolted intelligence onto the analysis side and left the hardest operational problem untouched: finding the right people, getting them scheduled, and getting them into a session fast enough to match the speed of AI prototyping.
For surface-level usability: is the button findable, does the flow make sense to a new user, a research panel of paid participants works fine. But the questions that actually shape products are contextual: does this redesign break workflows power users depend on? Will this change reduce support tickets or create new ones? Do enterprise customers with complex permissions setups experience this flow differently than a single-user account?
Those questions need people who carry context. Your actual customers.
And recruiting from your own customer base has historically been the slowest, most manual part of the entire research lifecycle. Pulling CRM lists, coordinating with customer success to avoid contacting accounts mid-renewal, sending emails, juggling calendars, managing incentives, tracking who've participated recently so you don't burn out the same handful of willing volunteers.
When teams connect their research CRM to customer data and can segment by usage, plan type, support history, or custom properties, recruitment goes from a multi-day coordination project to something that takes minutes. That's the piece that actually matches the speed of AI prototyping. Not faster analysis — faster access to the people whose feedback changes decisions.
There's a deeper pattern worth naming. Lenny's survey revealed something striking: AI is helping PMs produce but not think. The top AI use cases are all production tasks: writing PRDs, creating prototypes, drafting emails. Strategic and discovery work sits near the bottom. But I don't necessarily think that's because we're thinking less, high-quality PRDs are helping teams think better and in more structured ways.
Usability testing lives right at this fault line. The production side of testing: scheduling, transcribing, tagging, reporting, is exactly where AI excels. The thinking side, choosing who to test with, deciding what to test, interpreting what the results mean for your roadmap. That's where the biggest challenges still lie for product teams. Knowing what to build is a universal pain point.
The teams getting this right are using AI's speed advantage to make room for more thinking, not less. Faster transcription means more time interpreting. Automated tagging means more time debating what the patterns mean. Scaled moderation means more sessions with the right people, not just more sessions.
If your team is generating prototypes faster than you can test them: which, given the data, is increasingly likely, here's a practical path:
Automate analysis first. If your researchers are still manually coding transcripts, fix that before anything else. It's the single biggest time save and the lowest-risk way to introduce AI into your research workflow.
Run one study with mixed participants. Half panel, half real customers. Compare the feedback quality. The panel participants will tell you the button is hard to find. Your customers will tell you the button does the wrong thing. That comparison makes the case for investing in your participant infrastructure better than any internal pitch.
Then fix your plumbing. The average research team uses 7-12 tools across the research lifecycle. That fragmentation is invisible when you run studies quarterly. When you're testing weekly to keep up with AI-generated prototypes, it becomes the bottleneck: context switches, data trapped in silos, insights that disappear into shared drives.
The teams closing the gap between AI prototyping speed and testing speed aren't doing it with a single tool or one clever hack. They're rethinking what research infrastructure looks like when the rest of the product development cycle moves in minutes. The AI part is the easy part. The research operations part is what separates teams that test fast from teams that learn fast.
No. AI accelerates the work around testing: transcription, tagging, synthesis, even moderation of structured sessions. But it can't replace what happens when a real person encounters your product with their own context, habits, and workarounds. The research community is clear on this: AI augments researchers, it doesn't replace participants.
AI usability testing uses artificial intelligence to assist with real tests: AI prototyping tools, automating transcription, identifying themes, moderating structured sessions with real participants. Synthetic testing uses AI to simulate user responses entirely, without real people involved.
With AI prototyping tools, you can have a testable prototype in 15-20 minutes. With automated recruitment from your own customer base and AI-moderated sessions, you can go from prototype to usability data within a day or two, compared to the 2-4 weeks most teams still experience. The speed depends more on your participant infrastructure than on any AI feature.
Focus less on individual tool features and more on whether the tool connects to your existing research workflow. Here at Great Question, we use a range of AI prototyping tools. The biggest waste of time isn't picking the wrong tool, it's adding another disconnected point solution to a stack that's already fragmented. Look for tools that handle recruitment, testing, analysis, and synthesis in one place, especially if they let you recruit from your own customer base rather than only offering access to a panel.
Tania Clarke is a B2B SaaS product marketer focused on using customer research and market insight to shape positioning, messaging, and go-to-market strategy.
.png)
