A recent research paper, released on October 9, outlines a new method enabling large language models (LLMs) to replicate human consumer behavior with notable accuracy. This innovation could potentially transform the market research industry, which is valued in the billions. The technique aims to generate synthetic consumer profiles capable of delivering not only realistic product ratings but also the qualitative reasoning behind those ratings at an unprecedented scale.
Historically, efforts to utilize AI for market research encountered issues, particularly in that LLMs often produced unrealistic or poorly distributed numerical ratings when tasked with rating products on a scale. In response to this limitation, the authors, led by Benjamin F. Maier, introduce the “semantic similarity rating” (SSR) method, which prompts the LLM to generate detailed textual opinions regarding products. These textual responses are then quantitatively assessed by measuring their semantic similarity against predefined reference statements.
In experiments utilizing a substantial dataset from a major personal care company—which included 57 product surveys and 9,300 human responses—the SSR method achieved 90% consistency with human test-retest reliability. The distribution of ratings generated by the AI closely mirrored that of human respondents, suggesting that the method enables scalable consumer research simulations while maintaining traditional survey metrics.
This research emerges amid growing concerns over the reliability of traditional online survey panels, which face contamination from AI-generated responses. A 2024 report from the Stanford Graduate School of Business noted that some individuals used chatbots to craft artificial answers, leading to data quality issues. The SSR method offers a constructive alternative by creating high-fidelity synthetic data in a controlled manner.
The findings also highlight the effectiveness of text embeddings, which are crucial for accurately measuring consumer intent. While the SSR model’s success indicates potential for further application, enterprises must ensure that the methodologies are robust and meaningful when mapping textual data to numerical ratings. The implications for businesses could be substantial, particularly for those operating in fast-paced consumer goods markets, as this method could deliver insights more quickly and cost-effectively than traditional approaches. However, the method’s performance on more complex purchasing scenarios remains to be evaluated, and it primarily focuses on population-level behavior rather than individual consumer choices.
Source: https://venturebeat.com/ai/this-new-ai-technique-creates-digital-twin-consumers-and-it-could-kill-the