Research
/
2024–2026
Synthetic User Research Panel
A tool that lets teams test their ads on AI versions of real small business owners, and get feedback in hours instead of weeks.
Project details
Company
Meta
Role
Research lead, end to end
Methods
In-depth interviews (n=100), comparative validation against a live human panel
Year
2024–2026
Overview
The Problem: Meta’s teams make ads for small businesses faster than research can test them. Testing with real people takes weeks, so most ads shipped without anyone checking if they worked.
The Method: I interviewed 100 small business advertisers and turned each one into an AI digital twin. Teams upload an ad, and the personas react to it the way the real people would.
The Core Insight: Building each digital twin from one real person, not an average, is what made it agree with real humans.
The Impact: Matched a live human panel's key themes 80% of the time, and became the small business layer of Meta’s central synthetic research platform.
Who’s on the panel?
100 small business advertiser digital twins. 1:1 from real people.
Most AI personas are made-up “average” customers, which is why people don’t trust them. I went the other way: every persona is one real person I recruited and interviewed, with their own words, frustrations, and context. When a digital twin reacts to an ad, you can trace it back to what that person actually said.
Can you trust it?
80% alignment, proven before anyone relied on it.
A fast tool is dangerous if it's wrong, so I set the bar before launch. I showed the same ads to the AI panel and to a live panel of real people, compared which ads each preferred and why, and refined the twins round after round until their key themes matched the humans' 80% of the time. Only then did it go to other teams.
How does a team use it?
Three steps, no recruiting.
Pick the people you want to reach, upload an ad, and read the results: overall themes plus every twin's own reaction. Testing an ad went from a weeks-long research project to an afternoon.
What did the interviews reveal?
Who runs the ads mattered more than industry or location.
The existing framework grouped advertisers by industry and location. The interviews showed the bigger difference was who actually runs the ads, so I organized the panel around that.
What about people who don’t advertise?
Then I built a second layer: synthetic personas.
The panel only covered people who already advertise, but 95% of Meta’s small business audience doesn’t advertise yet. So I started with them, building four synthetic personas of non-advertisers from 9 of Meta’s own research studies. Creative and marketing teams can brief against them like talking to the customer directly.
How do teams use a persona?
Brief it, then test it.
Each synthetic persona shows who they are, which messages land and which fail, and the Meta study behind every claim. Teams write the brief against a persona, make the design, then test it on that persona’s own digital twins from the panel.
Where does it fall short?
I was upfront about its limits.
A fast tool can start to feel like the truth, so I made clear where it falls short.
It misses feel. It caught colors and faces, but not whether a face felt warm. Where feel matters, pair it with live interviews.
Digital twins drift. I’d re-interview 10% of the panel each quarter.
People can over-trust it, so every answer links back to its real source quote.
What changed?
Outcome
Adopted by 24 stakeholders across Meta for 28 studies, shaping product and creative roadmaps
Selected as the small business layer of Meta’s central synthetic research platform
3 of 4 synthetic personas adopted by small business product marketing
Validated at 80% theme alignment against a live human panel before scaling
Next steps when I left: personas for advertisers, and a panel of 100 non-advertiser digital twins for creative testing, so every audience would have both.
Next project
