Synthetic Surveys

Explore a decision, quickly and affordably. 

Synthetic Surveys turns an operational or policy decision into a structured survey, simulates responses, and generates a report to inform decision-making. Useful for exploring ideas or comparing options when you don't have the time or resources for conducting real surveys; i.e. "Which flyer design is better for my event?" 

This is an experimental tool. Its responses are generated by Large Language Models but the methodology is grounded in emerging research. The app is not a substitute for conducting surveys with real human subjects, nor a replacement for community engagement. 

Yet governments and businesses have long used statistical methods and models to inform decision-making. In some psychology and marketing studies, LLMs acting as simulated participants reproduced about three-quarters of the main findings from experiments involving real people (Cui et al., 2024)(Yeykelis et al., 2024).

Download and run

Synthetic Surveys runs in your computer’s terminal and uses an online AI provider to generate responses.

1. Install uv using the installation instructions.
2. Open a terminal and paste:

uv tool install synthetic-surveys
synthetic-surveys

The first command installs Synthetic Surveys; the second launches it. Thereafter, run:

synthetic-surveys

On your first launch, follow the prompts to connect your OpenRouter(https://openrouter.ai/) account. AI usage is billed to this provider account.

To update, run:

uv tool upgrade synthetic-surveys

Research Base

• Aher, G. V., Arriaga, R. I., & Kalai, A. T. (2023). Using large language models to simulate multiple humans and replicate human subject studies. Helped motivate the app by reproducing findings from classic behavioral experiments with simulated participants.
• Argyle et al. (2023). Out of One, Many: Using Language Models to Simulate Human Samples. Informed the use of demographic profiles to condition simulated survey responses.
• Bisbee et al. (2024). Synthetic Replacements for Human Survey Data? The Perils of Large Language Models. Motivated drafting alternate wording for priority questions to help mitigate bias towards particular wording.
• Cui, Z., Li, N., & Zhou, H. (2024). Can AI replace human subjects? A large-scale replication of psychological experiments with LLMs. [Preprint]. Helped motivate the app by demonstrating replication of findings from psychology experiments.
• Cummins (2025). The threat of analytic flexibility in using large language models to simulate human data: A call to attention. Highlights how question structure can influence simulated responses.
• Madden (2025). Evaluating the Use of Large Language Models as Synthetic Social Agents in Social Science Research. Informed giving simulated respondents fresh context, vs letting them "see" each others' responses. 
• Maier et al. (2025). LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings. Informed using semantic similarity scoring in Pro mode.
• Moshagen, M., Thielmann, I., Hilbig, B. E., & Zettler, I. (2019). Meta-Analytic Investigations of the HEXACO Personality Inventory(-Revised): Reliability Generalization, Self–Observer Agreement, Intercorrelations, and Relations to Demographic Variables. Provides pooled human-sample correlations used to construct HEXACO personality profiles.
• Pew Research Center (n.d.). Writing Survey Questions. Informed randomly assigning forward or reversed Likert scales across respondents.
• Rupprecht, Ahnert, and Strohmaier (2025). Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses. Motivated alternating Likert answer order to address bias towards selecting last answer. 
• Saynova et al. (2025). Identifying Non-Replicable Social Science Studies with Language Models. Informed consideration of prompting strategies.
• Thielmann et al. (2020). The HEXACO–100 Across 16 Languages: A Large-Scale Test of Measurement Invariance. Provides observed human personality-score distributions used to sample HEXACO profiles.
• Valenzuela, Winter, and Rivera (2025). Using large language models for survey research in communication: opportunities and challenges. Informed survey design, avoiding unclear, leading, double-barreled, and overlapping questions.
• Xie et al. (2026). Evaluating the statistical realism of LLM-generated social science data. Informed checking response variation and the relationships between how simulations answer particular questions.
• Yeykelis, L., Pichai, K., Cummings, J. J., & Reeves, B. (2024). Using large language models to create AI personas for replication and prediction of media effects: An empirical test of 133 published experimental research findings. [Preprint]. Helped motivate creating the app by demonstrating replication of marketing findings using simulated participants.

Potential Limitations

• Gao et al. (2025). Take caution in using LLMs as human surrogates. Documents limitations of using language models as substitutes for human participants.
• Peng et al. (2026). Digital twins are funhouse mirrors: Five systematic distortions. Documents persistent stereotyping and limited individual variation even in agents grounded in human response histories.
• Rossi, Harrison, and Shklovski (2024). The Problems of LLM-generated Data in Social Science Research. Explains how generated responses can reproduce historical patterns and values from training data.