AI for Social Science
Research Methods

Generative AI creates a measurement problem for survey research. When respondents can outsource summarization, explanation, or fact retrieval, open-ended answers may no longer reflect their own reasoning. We introduce an auditable toolkit that combines response-process paradata with prompt-specific semantic benchmarks to detect AI use in open-ended responses. We validate the approach in a randomized survey experiment in which participants complete a summary task under either blocked AI access or observed access to an embedded AI tool, and by administering the same instrument to a synthetic AI agent. AI-assisted and automated responses leave distinct behavioral and textual traces: they show reduced drafting and revision, greater similarity to AI benchmark output, and lower distinctiveness relative to peer responses. These findings show that, in the age of generative AI, authorship must be measured rather than assumed.

We introduce GenAI-Powered Inference (GPI), a statistical framework for both causal and predictive inference using unstructured data, including text, image, and video. GPI leverages open-source Generative AI models—such as large language models and diffusion models—not only to generate unstructured data at scale but also to extract low-dimensional representations that capture their underlying structure. Applying machine learning to these representations, GPI enables estimation of causal and predictive effects while quantifying associated estimation uncertainty. Unlike existing approaches to representation learning, GPI does not require fine-tuning of generative models, making it computationally efficient and broadly accessible.

Observational studies of visual media effects are vulnerable to confounding bias due to the complexity of visual data, the difficulty in variable measurement, and the uncertainty of selection. We propose an inference framework that leverages multimodal LLMs and machine learning-based inference techniques to (1) identify potential confounders through reasoned visual assessment, (2) measure semantically complex features that are difficult to capture with conventional methods, and (3) apply double-Lasso for covariate selection and hierarchical OLS regression for effect estimation. We demonstrate this framework by revisiting the relationship between political personalization and social media engagement using 59,020 Instagram posts from U.S. politicians.

An emerging body of research has demonstrated the persuasive power of conversational AI or "chatbots," examining their effect on attitudes towards policy, belief in conspiracy theories, and agreement with propaganda statements. While important, this literature has overlooked a crucial distinction: the difference between what people believe and what they say. Discourse and attitudes are connected but distinct phenomena—what people are willing to say is furthermore a social phenomenon, contextually and relationally specific.

In this project, we use three experimental studies to compare the effect of conversations with a chatbot on individuals' discourse as compared with their attitudes. In the first experiment, we found larger effects on what people are willing to say than their self-expressed attitudes, especially after interactions with a chatbot instructed to use facts. In the second experiment we expanded the number of conversation topics and disentangled the mechanisms by which fact-based conversations change expression. In a third planned experiment we extend these findings by measuring the effect of chatbot conversations on dialogs with real individuals.

Who speaks in local democracy, and how much? Despite growing scholarly interest in political participation beyond voting, we lack scalable tools to measure citizen voice in local government. We develop an automated audio analysis pipeline that classifies speakers in city council meetings as elected officials or members of the public—directly from audio, using neural speaker diarization combined with multimodal LLM few-shot classification.

We introduce two methodological innovations: a temporal exemplar selection with iterative bootstrapping strategy, and a multi-agent deliberation framework. An AND ensemble combining the baseline and bootstrapped classifiers achieves 93.2% accuracy (κ = 0.79), citizen precision of 0.82, and citizen F1 of 0.83—substantially outperforming any single method. Our pipeline targets 1,600+ meetings from seven U.S. rent control cities and their neighbors (2017–2023).