The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective

In plain words

Researchers increasingly ask a language model to play the part of a customer, for example to predict what people would buy at different prices. The usual way to set this up copies a human experiment: each simulated customer sees one scenario and is not told that the scenario is part of a test. The paper shows that this habit breaks for language models. When the price changes, the model also quietly changes other details it invents about the shopper and the store, such as what they paid last time or what a rival product costs. The price effect then mixes with those invented details.

Adding those details to the prompt as explicit facts does not fix it cleanly, because the model then pays too much attention to them. The authors' fix is to tell the model how the experiment is designed: that the price is randomly drawn from a stated range and that the customer does not know this. The paper reports that this consistently improves agreement with a real human survey, for every model tested, and that it still helps after fine-tuning and when irrelevant extra data is mixed in.

TL;DR

• Blinded LLM simulations of experiments treat the treatment (price) as correlated with unspecified variables, which breaks the unconfoundedness assumption §1, §2.
• Controlling for covariates in the prompt can reduce that confounding but causes focalism, so error first falls and then rises as covariates are added §3, app. E.
• Unblinded prompts, which state the randomization design, lower error against a human survey across GPT models, after fine-tuning, and with an irrelevant dataset mixed in §5.

Problem

Experiments with human subjects randomise treatments so that unseen variables stay balanced. A language model simulating subjects generates those variables on the fly from the prompt, so changing the treatment in the prompt can change them too §2, §5. The authors say this breaks unconfoundedness and produces implausible demand curves; in their pricing study the simulated curve is inverted-U, while the human curve slopes downward §2, fig. 2. The usual remedy, adding covariates to the prompt, can trade confounding for ecological validity §3.

The paper's scope is one demand-estimation setting (40 consumer packaged-goods products, one human survey) and one main model, gpt-4o-mini-2024-07-18, for the covariate analysis §2, §3. Model-level claims in §5 use other GPT models, listed only in a figure §5, fig. 6.

Method

The paper has two parts: a demand-estimation study that exposes the confounding, and an experimental fix, unblinding, that is tested on several models and in fine-tuning.

Study design. For each of 40 products the regular price is taken from Walmart.com around April 2024, and the price is swept from 0% to 200% of that price in 20% steps, giving 11 points §2, app. A. For each prompt and each product-price cell, the model is called independently 50 times at temperature 1, and the purchase probability is the share of purchase answers §2.

Diagnosing confounding. The model is asked to fill in unspecified variables (last price paid, competing price, expiry days, and others) given the randomized price. The resulting correlations with price are reported §2, fig. 1, app. B, app. C.

Human benchmark. A pre-registered survey gave purchase-intention questions for the same 40 products, with random prices. Each respondent answered all 40 questions §2.

Covariate control. Two approaches are tested. First, demographics are generated as 500 personas per product and used as covariates §3, app. D, fig. 3. Second, a sequence of twelve stages adds behavioural and psychological covariates from a public panel of consumers §3, app. E. Stage 1 has 14 demographic variables, and stage 12 has 30 variables in total app. E.

Unblinding (the proposed fix). The blinded system prompt tells the model it is a customer filling in a blank §5. The unblinded system prompt additionally says that the price is randomly and uniformly drawn between a minimum and a maximum, and that the customer sees only one price and does not know about the randomization §5. The two system prompts are the only versions given in the text; the user-message templates for these runs are not shown.

Fine-tuning test. Fine-tuning uses the human survey data in a 2 × 2 design, where the prompt type (blinded or unblinded) is chosen separately for the fine-tuning and evaluation stages §5. Evaluation is leave-one-category-group-out over four groups: beverages; refrigerated goods; snacks and bakery; household and pet supplies §5. A second round adds 10,000 observations from an observational Amazon browsing dataset, which record purchases only, to the fine-tuning data, using a separate prompt §5, app. G.

Theory. The authors define an ambiguous prompting strategy as one that maps two questions with different correct answers to the same prompt §4. They argue that no single model can answer both questions correctly under such a mapping §4, app. F.1. They also write the simulated data-generating process as mixing an unobserved-factor term that is correlated with price (observational) with one that is not (experimental) app. F.2.

# Simulate purchase probability for one product across the price grid  [§2]
# Input:  prompt template, product, regular price
# Output: one purchase probability per relative price

def simulate_demand(template, product, regular_price):
    probabilities = []
    # Relative prices 0%, 20%, ..., 200% of the regular price  [§2, app. A]
    for relative_price in [0.0, 0.2, 0.4, 0.6, 0.8, 1.0,
                           1.2, 1.4, 1.6, 1.8, 2.0]:
        price = relative_price * regular_price
        prompt = fill(template, product, price)
        # 50 independent draws at temperature 1  [§2]
        answers = [ask_model(prompt, temperature=1.0)
                   for draw in range(50)]
        probabilities.append(share_of_purchase(answers))
    return probabilities

# Mean absolute error of a model against the human survey  [§5, fig. 6]
# Input:  model, system prompt, (product, price, human_probability) cells
# Output: mean absolute error over the cells

def evaluate_mae(model, system_prompt, cells):
    errors = []
    for product, price, human_probability in cells:
        # Average over 50 draws at temperature 1  [§5]
        answers = [ask_model(model, system_prompt, product, price,
                             temperature=1.0)
                   for draw in range(50)]
        model_probability = share_of_purchase(answers)
        errors.append(abs(model_probability - human_probability))
    return mean(errors)

Hyperparameters

Results

Author-reported. Setup: demand estimation for 40 consumer packaged-goods products against a pre-registered Prolific survey (n = 991 after exclusions) §2; fine-tuning on the survey with leave-one-category-group-out evaluation and MAE on purchase probabilities §5.

Demand curves (out-of-box, blinded, averaged over products). Human curve is downward sloping; GPT curve is inverted-U, and stays so when the price-0 point is removed §2, fig. 2.

Covariate study (MAE of purchase probability, app. E table, stage numbers as listed):

Fine-tuning on the survey (MAE against human survey; rows are fine-tuning prompt):

Fine-tuning on survey plus Amazon data:

Unblinding across models. The text says unblinded prompts gave lower MAE than blinded prompts for every model tested, with relative improvements from about 1% to over 60% [§5, fig. 6]. The per-model values are only in the figure.

Headline claims. Fine-tuning with unblinded prompts cut MAE by about 15% against the best blinded model (0.134 to 0.113) §5, tab. 1; the table values give about 15.7%. With the Amazon data added, the blinded fine-tuned model degraded (0.134 to 0.233 under blinded evaluation) and unblinded fine-tuning stayed near its earlier error §5, tab. 1, tab. 2.

Paper vs code

No official code. The paper and its arXiv listing contain no repository link, and the TeX source has no code or data URL [raw source: main_v2_arxiv.tex]. A GitHub repository search for "unblinding llm simulation", "blinded unblinded llm survey" and the author name "toubia" returned no repository owned by the authors.

Pitfalls

• Survey respondents vs API calls. Each human answered 40 product questions in one session, while the model received an independent call per question §2, footnote. The paper does not model this clustering; a reader should expect the human benchmark's uncertainty to be understated.
• Survey price range is not stated. The paper says human prices were "randomly drawn for each product" §2, but the range is not given. The simulation sweep is 0–200% of the regular price §2, so the two may not cover the same support.
• Target noise is not reported. The MAE targets are human purchase probabilities per (product, price) cell §5. The number of respondents per cell is not reported, so the noise in the target is unknown.
• Unblinding prompt placeholders. The unblinded system prompt contains {min_price} and {max_price}, and the values used are not stated §5. Implementers must choose them. Setting them to the sweep range (0–200%) is an assumption, not the paper's stated choice.
• Only system prompts are given. The blinded and unblinded system prompts are shown in §5, but the user-message templates used with them are not. The unblinded effect may depend on the user text as well.
• Persona temperatures differ from the rest. Personas are generated at temperature 1, but purchase decisions for personas are elicited at temperature 0 app. D, while every other purchase cell uses temperature 1 with 50 draws §2, §5. Reproducing one protocol with the other temperature changes the output distribution.
• Covariate pattern is not a clean threshold. The text says the error improves at first and then degrades beyond a threshold §3. The table shows the lowest MAE at stage 2 (0.0971), a rise through stage 11, and a fall at stage 12 (0.1929 below 0.2025) app. E. Stage 8 is also below stage 7. Do not implement the non-monotone shape as a single turning point.
• One model for the covariate study. The covariate, persona and confounding diagnostics use gpt-4o-mini only §2, §3. The claim that the issue is general across LLMs rests on the theory §4, app. F.1 and on the unblinding figure §5, fig. 6; the model list is not in the text.
• Theorem assumes a deterministic mapping. The impossibility argument treats the model as a function of the prompt app. F.1. The experiments sample at temperature 1 §2, §5, and the paper does not discuss how sampling changes the argument.
• Fine-tuning recipe is missing. Epochs, learning rate, batch size, file format and the way survey responses become training examples are not stated §5, app. G. Results cannot be reproduced from the text alone.
• Amazon data is not described beyond its use. The added dataset is observational and records purchases only §5. The paper gives no filtering rules beyond sampling 10,000 observations app. G.
• Reported improvements are relative. "Up to over 60%" and "about 1%" are relative MAE changes read from a figure §5, fig. 6, not table values.

Lineage

• Builds on: the consumer panel used for the covariate study is from toubia2025twin §2, §3 (cited, no card here).
• Compared against: Argyle et al. ("silicon samples") and blinded between-prompt designs, which this paper treats as the standard it critiques §1, §3.
• Superseded by: not known from content on this wiki.