AI Agent Simulations of People // real individuals
Effectiveness of interviews in predicting human behavior and attitudes
The promise of human behavioral simulation — general-purpose computational agents that replicate human behavior across domains — could enable broad applications in policymaking and social science. We present a novel agent architecture that simulates the attitudes and behaviors of 1,052 real individuals — applying large language models to qualitative interviews about their lives, then measuring how well these agents replicate the attitudes and behaviors of the individuals that they represent. The generative agents replicate participants’ responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and outcomes in experimental replications. Our architecture reduces accuracy biases across racial and ideological groups compared to agents given demographic descriptions. This work provides a foundation for new tools that can help investigate individual and collective behavior.
In a recent study, researchers explored the effectiveness of interviews in predicting human behavior and attitudes. The study involved creating generative agents based on interview data and testing their predictive performance on various tasks. The results showed that interview-based agents outperformed agents created using demographic information or persona descriptions, highlighting the rich and comprehensive nature of interview data.
The study’s findings have significant implications for fields such as social science and machine learning. By leveraging interview data, researchers can create more accurate and nuanced models of human behavior, which can inform policy decisions, social interventions, and product development. Moreover, the study’s approach can be applied to a wide range of domains, from economics and politics to psychology and education.
One of the most striking findings of the study was the efficiency of interviews in identifying valuable insights. Even short interviews were found to contain sufficient richness to outperform agents informed solely by surveys and experiments. This suggests that interviews can be a valuable tool for researchers and practitioners looking to gain a deeper understanding of human behavior and attitudes.
The study also explored the issue of bias in predictive modeling, specifically in the context of demographic attributes. The researchers found that creating individualized models based on interviews reduced performance gaps across demographic groups, compared to relying solely on demographic attributes or personas. This highlights the importance of considering the complexities of human behavior and attitudes in predictive modeling.
To facilitate further research and innovation, the researchers plan to provide controlled access to the agent bank, a dataset of generative agents created using interview data. The agent bank will be made available to researchers through a two-pronged access system, with open access to aggregated responses on fixed tasks and restricted access to individualized responses on open tasks. This approach aims to balance the benefits of accessing the agent bank with the need to protect participant privacy and safety.
