01 · The Validation Problem

What the Industry Is Not Measuring

Two questions. Most platforms only answer one.

The synthetic audience and behavioral simulation market has moved from research curiosity to contested commercial territory in under two years. From well-funded U.S. startups like Aaru to specialized European platforms like Lakmoos, vendors increasingly position validation and predictive accuracy as primary differentiation. In practice, these claims rest on a single question: how closely does the simulated response match a known human response on a held-out dataset?

That is a meaningful metric — and an important first step. It also happens to be the easier question. It demonstrates that simulation can reproduce patterns observed in existing human data.

“Replication alone is not enough. The real test is whether behavioral signals translate into real-world outcomes.”

Does a higher simulation sentiment score translate to a better-performing ad in the real world — against real audiences, on real platforms? To our knowledge, Predikta is the first behavioral AI simulation platform to move beyond response replication, having conducted a blind backtest against independently sourced historical campaign data from a third-party agency, measuring whether higher simulation scores correlate with actual Click-Through Rates (CTR) and Cost Per Click (CPC) outcomes.

A simulation score that predicts human responses but cannot predict which ad drives more clicks is an interesting research result. Whether it can support real marketing decisions is a different question. The Predikta × AdSpark backtesting study was designed to close that gap — and its findings carry implications not only for Predikta, but for the standard of evidence the industry should now be required to meet.

02 · Predikta’s Evidence Architecture

Two Layers, Two Different Questions

Most platforms have one layer of evidence. Predikta built two.

Most platforms in this behavioral simulation space rely on internal benchmarking against their own data. Predikta has deliberately built two distinct and complementary validation layers, each answering a question the other cannot.

The significance of the two-layer structure is not just in the numbers — it is the logical completeness. Layer 1 establishes that the behavioral simulation model is accurate. Layer 2 establishes that behavioral accuracy has predictive value in the real market. Both questions must be answered before a marketing team can have justified confidence in using simulation scores to make pre-launch decisions. Predikta is the only platform in this competitive roundup that has answered both.

03 · The Backtesting Study

Design, Results, and Implications

Three design choices defined its credibility.

The study was designed specifically to avoid the methodological shortcuts that make internal validation exercises easy to dismiss.

Across 20,000 ad pairs, Predikta correctly identified the stronger-performing ad approximately 70% of the time for conversion ads, based entirely on simulated audience responses. In a setting where random guessing yields 50% accuracy, a 20-percentage-point advantage achieved before any ad was deployed is not a marginal result. It suggests that the gap between simulation and market reality is smaller than the industry has assumed.

Ads in the top 20% of Predikta scores delivered 3.4 times the CTR of lower-scoring ads at a cost of 2.6 times less per click. At typical campaign spending levels, those differences are commercially significant before a single peso is allocated to a live campaign.

The study also identified where the advantage is strongest: Awareness and Conversion campaigns — the two objective types where creative quality has the most direct influence on outcomes. That is precisely the layer Predikta’s psychographic simulation is designed to model.

“Testing Predikta against real-world campaign outcomes has been a long-standing priority for our team. Moving beyond replication of survey responses to evaluating Predikta’s output against observed campaign performance represents a meaningful development step.”

— Sophia Lanuzo, Lead R&D

04 · Research Frontier

Academic Work Shaping the Science

The backtesting study results do not stand alone. The broader academic literature on psychographically grounded agent simulation is converging on findings that reinforce precisely what the AdSpark study demonstrates in commercial terms.

Our Foundational Paper (arXiv:2505.22125). Our arXiv paper established the behavioral architecture that the backtesting study later tested in a real commercial setting. The paper instantiated agents from survey data covering personality, values, beliefs, and socio-political attitudes. Critically, it showed that agents could maintain stable, psychographically anchored outputs regardless of how scenarios are framed — the behavioral property that makes real-world commercial prediction possible.

LLM Agents Grounded in Self-Reports (arXiv:2411.10109). Using 1,052 Americans, this study showed that agents built from qualitative interview and structured survey data reached 82–86% agreement on held-out social attitude items, compared with 74% for demographics-only agents. This independently supports Predikta’s design choice: grounding agents in rich psychographic data is measurably more effective than demographic profiling alone.

AgentSociety (arXiv:2502.08691). A large-scale social simulation framework that uses LLM-driven agents integrating Maslow’s hierarchy of needs, different emotional states, motivations, and cognitive processes. This signals where the frontier is heading: motivational agent architectures that model dynamic needs and affective states beyond static profile-based simulation.

Polarization Simulation (Springer, 2025). A 100-agent study grounded in psychometric and demographic data from Serbian social media users demonstrates that psychographically grounded agents can produce coherent ideological clustering and differentiated behavior. This provides adjacent support for the stability property Predikta’s paper documents.

Open challenge. A 2025 meta-review notes LLMs “struggle significantly in scenarios that more accurately reflect real-world conditions” — particularly around emotional nuance, cultural context, and group dynamics. Predikta’s Philippines-specific, survey-grounded dataset directly responds to the cultural-context critique. The AdSpark backtesting results are evidence that the response is working.

05 · Product Landscape

Who Is Doing What

The field is converging on the same question — but not the same answers.

The synthetic audience and behavioral simulation space is increasingly converging around a single question: not whether synthetic respondents can be generated, but how their outputs are validated. To our knowledge, no other platform has published a comparable validation architecture in the Philippines or broader Southeast Asia — one that integrates local survey grounding, population-scale modeling, and externally verified commercial outcomes.

06 · Competitive Validation Matrix

A Standard the Field Must Now Meet

Five dimensions that matter when deciding whether simulation scores can be trusted for real media budget decisions.

Predikta is the only platform in this comparison with publicly documented evidence across all five dimensions. Other platforms may provide validation claims, partner integrations, or market-specific capabilities — but publicly available evidence is generally partial, unpublished, or not directly tied to campaign-level advertising outcomes.

Key findings

Four numbers from the paper

Abstract

From the paper

01 · The question

Reading sentiment after the fact is not enough

Most sentiment analysis labels what people have already said: posts, reviews, survey answers. That shows how people felt, after the fact. It cannot show how a particular group will react to a policy, a message or an event they have not seen yet, and it misses the psychological reasons behind a reaction.

The study asks whether AI agents grounded in real psychological data can simulate those reactions before the fact, and explain them.

02 · How the study worked

One agent per respondent, three stages

Each AI agent was built from one real respondent in a nationally representative survey of 2,485 Filipinos, sampled across all 17 administrative regions. The 150-item survey covered sociodemographics and validated measures of personality, values, beliefs, and social and political attitudes. All simulations ran on Llama 3.1 70B, an open-weight language model.

Answers were scored with Quadratic Weighted Accuracy (QWA): identical answers score 100%, and larger disagreements are penalized more heavily than small ones.

03 · What it found

Agents answered, and felt, like the people behind them

They replicated real survey answers. With narrative (contextualized) profiles, agents matched their human counterparts’ survey answers with 92% alignment on average. Narrative profiles beat label-based ones (Wilcoxon signed-rank test, p < 0.0001; Cohen’s d = 0.70): a 76% chance that a randomly chosen agent with a narrative profile outperforms one with a label-based profile.

They simulated sentiment on real issues. Against ground-truth sentiment, agents reached 81–86% accuracy across the five issues. Narrative profiles scored higher in four of the five, with the largest gain on political dynasties.

Average accuracy (QWA) over five runs per issue. Source: Table 3, arXiv 2505.22125.

Results were stable. Each issue was simulated five times. Scores barely moved between runs: standard deviations of ±0.17% to ±0.51% with narrative profiles.

Framing did not throw the agents off. Positive versus negative framing made no statistically meaningful difference to accuracy (paired t-test, p = 0.9676; Cohen’s d = 0.02). Alignment stayed between 77% and 88% across issues and framings.

04 · Why it matters

The science under Predikta

This paper is the scientific layer of Predikta’s evidence: agents grounded in Filipino survey data can reproduce how real people answer, and how they feel about real issues. Whether those signals pick winning ads was tested separately, in a blind backtest of 20,000 ad pairs with AdSpark. Read how that test worked.