Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues

In plain words

Negotiating well is hard for software. A good negotiator has to follow what has already been said, guess what the other side really wants, and reply in a way that is both sensible and strategic. This paper checks how well large language models manage those separate skills. It uses public datasets of negotiation chats, in which two people split a set of items (food, water and firewood; or balls, hats and books; or trade artifacts; or a job offer) and each person has their own point values for them.

From these chats the authors write 35 test questions, for example "how many items of each kind did you end up with?", "which item is the other side's top priority?" or "is the other person happy with the deal?". Each question is put to the models as a prompt, and the answers are compared with what the human players actually did or reported. Several language models are compared with a trivial majority-answer guess and with a small model that was fine-tuned on each task.

The main finding is that GPT-4 is the strongest model overall, but it is still weak at judging how people feel about a deal and at writing replies that are natural and strategically smart. The paper is best read as a map of which parts of negotiation current models handle and which they do not.

TL;DR

• 35 zero-shot tasks from four negotiation datasets (CRA, DND, CA, JI), split by time stage, objectivity and task type §2.2.
• Six LLMs plus a Majority baseline and a per-task fine-tuned Flan-T5 base §3.1 §3.2.
• GPT-4 is best on most tasks; subjective satisfaction and likeness predictions are weak; replies are close to human on coherence but not on strategy §4 §5.
• Results are author-reported, zero-shot, with one prompt template per task §2.3.

Problem

Negotiation research needs models that can understand a dialogue, track the outcome, infer the partner's priorities and generate strategic replies. Earlier LLM negotiation agents were built end to end, so their individual abilities were not measured separately §1 §6. Without a per-skill evaluation, it is hard to tell whether a model fails at reading the context, at modelling the partner, or at choosing a move §1.

The evaluation assumes the Multi-Issue Bargaining Task setting: two players, a fixed number of items per issue, private point values that stay fixed through the dialogue, and a score equal to the inner product of the values and the items the player receives §2.1. Results depend on this framing and on English dialogues [Broader Impact and Ethical Considerations] [Limitations].

Method

The framework has three steps: design tasks from human-human dialogue datasets, build a prompt per task, and evaluate models on those prompts §2.

Datasets. Four datasets were chosen for a well-defined structure, diversity of scenario and available metadata §2.1 tab. 1.

• CRA: a trading scenario over three artifacts, 119 dialogues.
• DND: a scenario over three kinds of generic item, 6,251 dialogues.
• CA: campsite neighbours sharing three supply types, 1,030 dialogues.
• JI: a recruiter and a job candidate bargaining over five job terms, 2,639 dialogues.

These are the dataset sizes and scenarios in tab. 1. Test instances are sampled per task and capped at 200 tab. 1 caption §2.1.

Task axes. Each of the 35 tasks is tagged on three axes §2.2:

• Objectivity: objective tasks have one correct answer; subjective tasks ask about outcome satisfaction and partner likeness as reported in post-game surveys. Response generation is also treated as subjective and is judged by humans §2.2.1.
• Time stage: Start (no dialogue; only the scenario, item counts and the agent's own priorities), During (the dialogue so far), End (the whole dialogue) §2.2.2.
• Task type: Comprehension, Annotation (dialogue acts and strategies), Partner Modeling (inferring the partner's priorities or feelings), Generation §2.2.3.

Every task is written from the point of view of an agent negotiating for itself, so the partner's private preferences are never shown in the prompt §2.2.

Prompt template. The zero-shot prompt has five parts: a task description, the issue counts and the agent's values, the dialogue or utterance (depending on the stage and task), extra information (dialogue-act or strategy labels, for annotation tasks only), and the question §2.3. The full task list and worked prompts are in app. A and app. D. The question asks for a machine-readable answer, usually in tags, and some tasks ask for step-by-step thinking app. D.

Prompting variants (explored on subsets). Chain-of-thought, two randomly sampled few-shot examples for annotation tasks, two prior utterances for annotation tasks, and a sweep over the number of dialogue utterances shown to the model §5.

# Zero-shot task prompt: five parts  [§2.3] [app. D]
# Input:  task (stage, description, question), instance (one dataset row)
# Output: one prompt string

def build_prompt(task, instance):
    # 1. Task description, written for the agent's own side  [§2.2]
    description = task.description
    # 2. Item counts and the agent's own point values only  [§2.2]
    counts_and_values = (instance.counts, instance.own_values)
    # 3. Dialogue slice allowed by the time stage  [§2.2.2]
    context = instance.dialogue_until(task.stage)
    # 4. Dialogue acts or strategy labels, annotation tasks only  [§2.3]
    extra = instance.labels if task.is_annotation else None
    # 5. The question with its answer format  [§2.3] [app. D]
    question = task.question
    return join(description, counts_and_values, context, extra, question)

# Zero-shot evaluation of one task  [§2.1] [§3.2] [§3.3]
# Input:  model, task, dataset instances
# Output: one score for the task

def evaluate_task(model, task, instances):
    # 1. Keep at most 200 test instances for this task  [§2.1]
    sampled = take(instances, 200)
    outputs = {}
    for instance in sampled:
        # 2. One zero-shot prompt per instance  [§2.3]
        prompt = build_prompt(task, instance)
        # 3. Greedy-style decoding; GPT calls use temperature 0
        #    [code: models/open_ai.py#L112]
        outputs[instance.id] = model.generate(prompt)
    # 4. Score with the task's metric (accuracy, macro-F1, ...)  [§3.3]
    return score(task, outputs, sampled)

Baseline. A Majority baseline returns the most frequent training answer. Flan-T5 base is fine-tuned separately on each task, with the long task description replaced by a short instruction app. B. Start-stage tasks are left out of both baselines because a rule-based parser solves them exactly app. B. Fine-tuning used AdamW, a linear schedule, up to 5 epochs, and checkpoint selection on the dev split §3.1 app. B.

Hyperparameters

Results

Author-reported, zero-shot, on each task's test set. Setup: four datasets, six LLMs plus two baselines, tab. 1 and §3.

Overview by task type fig. 3 §4:

• Comprehension, Start: GPT-4 averages 81% accuracy; GPT-3.5 and Mistral-7B are above 70%. A rule-based parser reaches 100% on these tasks, so they test little §4.1.
• Comprehension, End: most models sit close to the Majority baseline. GPT-4 averages 65.3% accuracy fig. 3 §4.1.
• Comprehension, subjective: all models score poorly. GPT-4's satisfaction predictions correlate about 0.3 with human ratings §4.1.
• Annotation: GPT-3.5 (62.4%) and GPT-4 (52.5%) average above the fine-tuned Flan-T5 on macro-F1, but Flan-T5 beats GPT-4 on most individual tasks §4.2 fig. 3.
• Partner modeling, objective: GPT-4 is best; smaller open models mostly match the Majority baseline §4.3 fig. 3.
• Partner modeling, subjective: Flan-T5 reaches 50.5% accuracy; GPT-4 reaches the highest PCC, 0.39. Its confusion matrix shows it often predicts neutral or slight liking where partners show strong liking §4.3 fig. 4.

Response generation tab. 2 §4.4. Human coherence and strategy are the reference. GPT-4 is close to the human coherence score on DND and CA, but not on strategy except on CA, where it approaches human level. Coherence is consistently higher than strategy. BLEU and ROUGE favour the fine-tuned Flan-T5, which the paper attributes to dataset-specific training §4.4.

tab. 2. BLEU, ROUGE-L and BERTScore use the full 200-instance test set; coherence and strategy use 50 instances rated by humans. * marks a significant gap below the human reference (paired bootstrap) tab. 2 caption.

Prediction balance tab. 3 §5.1. On well-balanced tasks GPT-4 has the smallest gap between accuracy and macro-F1 and the lowest KL divergence. On imbalanced dialogue-act and strategy tasks every model loses macro-F1 relative to accuracy; GPT-4 has the smallest gap among them.

tab. 3 (averages over the selected tasks app. A). On the imbalanced group GPT-4 has the smallest accuracy-F1 gap, while GPT-3.5 has the lowest KL-D.

Prompting variants (GPT-4 unless noted) §5:

• Chain-of-thought gives nearly 100% accuracy on the multi-hop arithmetic tasks tested (maximum possible points and actual total points) §5 fig. 5.
• Two few-shot examples improve two of four annotation tasks §5.
• Two prior utterances improve one annotation task and slightly hurt the other three §5.
• Showing more utterances lowers accuracy on the agent's own priorities and raises accuracy on the partner's priorities; the paper attributes this to recency bias §5 fig. 6.

Full task-wise results are in app. A. The table below copies them. A blank cell means the model was not run, or failed the 80% valid-response check; the paper says Majority and Flan-T5 were not run on Start-stage tasks app. A.

app. A. The paper's table also omits the two generation tasks, which are reported in tab. 2. Note: the paper's PCC and BERTScore are not computed by the released code (see Paper vs code).

Paper vs code

Code: the repository linked from the abstract, DSincerity/SysEval-NegoLLMs, pinned at 637436096c51aacafd210b5f19f1cbd889ed643a, Apache-2.0. The README names the first author as the contact. The repo has the LLM runner, the Flan-T5 baseline and the scoring code; the appendix prompts were not compared with tasks/*.py line by line.

• Learning rate differs. Paper: 5e-5 §3.1. Shipped script and argument default: 5e-4 finetune.task.sh arguments.py.
• Batch size differs. Paper: 8 §3.1. Shipped script: 3 finetune.task.sh. Argument default: 16 arguments.py. Evaluation batch size default: 16 arguments.py.
• Epochs agree. Paper: up to 5 §3.1. Script: 5 finetune.task.sh.
• AdamW weight decay is not applied. The paper names AdamW §3.1. The code defines a weight-decay argument with default 0.01 arguments.py, but the training arguments passed to the trainer do not include it main.py. The optimizer and scheduler are not named in code.
• Checkpoint selection is on loss. Paper: best-performing checkpoint §3.1. Code: best checkpoint by evaluation loss main.py arguments.py, not by the task metric.
• PCC and BERTScore are not computed. The paper reports PCC for subjective tasks §3.3 and BERTScore with bert-base-uncased §3.3. The released code has no PCC or BERTScore call (repository search). The regression entry lists the two end_deal_total tasks TASKTYPE.json, which are scored with accuracy eval_metric.py. Subjective tasks therefore have no PCC in the released scorer.
• Macro-F1 covers all labels seen. Scoring averages per-label F1 over the union of gold and predicted labels eval_metric.py. This matches "macro-F1" in the paper, but a label predicted and never gold still counts. Scoring applies F1 only to strategy tasks and to the JI and DND dialogue-act tasks eval_metric.py. The CRA dialogue-act task is not in that list; I did not trace which scorer it gets.
• Flan-T5 inputs are trimmed differently. Paper: long task descriptions removed and replaced by a short instruction such as "predict highest priority" app. B. Code: a regex removes the description and tag text, and deletes the value block for end_deal_specifics tasks prompt_modifier.py. The short instruction string does not appear in the repository.
• GPT runs are pinned to a model string and temperature 0. Paper: no model snapshot or temperature given §3.2. Code: gpt-4-1106-preview openai_gpt4.sh, temperature 0 open_ai.py open_ai.py.
• Failed or over-length GPT prompts are stored as a placeholder. Before each call the code stores the literal string "looks good." as the output, and only overwrites it if the call succeeds open_ai.py. A length or API failure therefore leaves that string as the stored output, and the scorer receives it as the prediction.
• Over-length open-model prompts are dropped. For the open models, an input over the token limit is skipped, so no output is recorded for it hf_model.py. The paper's 80% valid-response rule is not in the released code.
• Decoding limits for open models are not in the paper. Flan-T5 gets 100 new tokens; Mistral, Vicuna and Wizard get 200 hf_model.py. Vicuna and Wizard prompts are wrapped as "User: ... Assistant:" hf_model.py. Sampling settings are not set in the call, so the library defaults apply hf_model.py.
• Few-shot examples are sampled without a seed in that function. Two other instances are drawn with np.random.choice open_ai.py. The paper says "randomly sampled" §5.
• Task names differ. Paper: prefixes sta_, dur_, end_ app. A. Code: mid_ for the During stage, and a _w suffix for some job-interview tasks openai_gpt4.sh. Look up tasks with the code names.
• Chain-of-thought and partial-dialogue runs use the flags use_cot and num_utts_partial_dial on the listed tasks openai_gpt4_additional_exp.sh. The paper gives no sweep values; the script uses 2, 4, 6 and 8.

Pitfalls

• Start-stage tasks are nearly solved by a parser. The paper says a rule-based parser reaches 100% on them §4.1 app. B. Scores on them measure reading of the prompt, not negotiation skill.
• Subjective scores are weak evidence. PCC is reported in the paper but not computed by the released code, so the subjective numbers cannot be reproduced from the repo as shipped. Satisfaction and likeness results are the least reliable part of the paper §4.1 §4.3.
• Human judgements include the authors. Five expert annotators rated generation, and the authors were among them [Broader Impact and Ethical Considerations]. The inter-rater ICC is 0.82 on 50 examples §4.4. The sample is small.
• Missing cells are not zeros. A blank cell can mean "not run" or "failed the validity check". Compare models only on tasks where both have a value app. A.
• Results are tied to prompts and model snapshots. The paper uses one zero-shot template per task and says prompt engineering was not explored in depth [Limitations]. The code pins gpt-4-1106-preview.
• Chain-of-thought gains are measured on few tasks. The paper reports near-perfect accuracy with chain-of-thought on arithmetic tasks, but tested only a few tasks §5.
• The majority baseline is a real competitor. On end_deal_total_ca the majority answer (0.142) beats GPT-4 (0.083) and the open models app. A.
• Few-shot and prior-utterance gains are mixed. Each helped on some tasks and hurt on others, so a single setting should not be assumed to help §5.
• English only, four datasets, one bargaining framework. Results may not carry to other languages, cultures or negotiation structures (fixed-value bargaining, no preference change, no fractional items) [Broader Impact and Ethical Considerations] [Limitations].

Lineage

• Builds on: the four datasets CRA, DND, CA and JI, each cited in §2.1 and app. A; the Multi-Issue Bargaining Task framework §2.1; the Flan-T5 model §3.1.
• Compared against: GPT-3.5 and GPT-4 §3.2; Mistral-7B, Wizard-13B, Vicuna-13B and Vicuna-33B §3.2; a Majority baseline and a per-task Flan-T5 fine-tune §3.1.
• Related negotiation agents: end-to-end self-play RL §6; modular RL policies §6; LLM agents for balloon-price bargaining and multi-agent games §6. The paper positions itself as a per-skill evaluation, not an agent §1 §6.
• Superseded by: not known from content on this wiki.