---
kind: concept
slug: human-ai-collaboration
title: Human-AI collaboration
summary: >-
  Splitting a task between a person and an AI system, each doing the part it suits, with both improving over time. Covers the Hybrid Intelligence framing, a survey of collaboration patterns and build-time guidance, an AI-drafted writing workflow and a self-report scale for checking AI output. All evidence is author-reported; no card tests whether the combination beats either part alone.
tags: [human-ai-collaboration, hybrid-intelligence, human-in-the-loop, ai-assisted-writing, critical-thinking]
papers: ["2105.00691", "2212.08104", "2403.04931", "2512.12413"]
related: [rlhf, human-evaluation, human-in-the-loop, active-learning, automated-fact-checking, psychometric-scale-validation]
status: draft
updated_at: 2026-10-24
---

## Overview

Human-AI collaboration is a way of dividing work in which a person and an AI
system each take the part they are better at, and the two improve each other
over time. Four cards cover it from different angles. A position paper names
the arrangement Hybrid Intelligence and sets out its two directions
[arxiv:2105.00691 §4]. It contrasts human strengths, such as intuition, common
sense and handling unlabelled cases, with machine strengths, such as pattern
recognition, consistency and speed [arxiv:2105.00691 §3]. A survey of
collaboration with large foundation models sorts the field into build-time
human guidance and five run-time patterns [arxiv:2403.04931 §2–§3]. A narrative
review shows one concrete workflow, in which a language model drafts the text
and the authors then rewrite it [arxiv:2212.08104 p. 2]. A last paper looks at
the human side and offers a survey scale for how people check AI output
[arxiv:2512.12413 §2–§7].

None of the four cards builds a combined person-and-model system and measures
it against either part alone. Hybrid Intelligence's claim of superior results is
a definition and a prediction, not an observed outcome [arxiv:2105.00691 §1]
[arxiv:2105.00691 §4]. The survey and the review report no experiments of their
own [arxiv:2403.04931 §1] [arxiv:2212.08104 §1–§9]. Trust, governance and
digital work are named as open research directions, not answered
[arxiv:2105.00691 §6]. The survey adds open challenges on scalable human
guidance and accountability [arxiv:2403.04931 §6]. For learning from human
judgements as a training signal, see [RLHF](/concepts/rlhf); for people
reviewing a system's outputs, see [Human evaluation](/concepts/human-evaluation);
for labelling the most useful samples, see [Active learning](/concepts/active-learning);
for the measurement side, see
[Psychometric scale validation](/concepts/psychometric-scale-validation).

## Variants

The Hybrid Intelligence paper describes two directions that differ in which
side teaches. The survey splits the field by timing: human guidance while the
model is built, and collaboration patterns once it is in use. The writing
workflow and the self-report scale cover narrower ground.

### AI in the loop of human intelligence

The AI takes over tasks that machines can solve alone, or it gives a person
predictions to decide with. The examples are forecasting, financial markets and
automated choice of hyperparameters for new models [arxiv:2105.00691 §3.1]. The
AI can also produce an output that a person then evaluates, such as a model
that reads CT scans to support a physician's diagnosis [arxiv:2105.00691 §3.1].
Card: [Hybrid Intelligence](/papers/2105.00691).

### Human intelligence in the loop of AI

People teach the system. The tasks named are setting or tuning hyperparameters,
training and debugging models, and interpreting unsupervised results such as
clusters [arxiv:2105.00691 §3.2]. The paper says that interactive learning often
needs many labelled examples from people, and that human input matters most
when data is scarce [arxiv:2105.00691 §4]. Card: [Hybrid Intelligence](/papers/2105.00691).

### Roles in augmentation

A figure in the paper separates two arrangements. In augmented human
intelligence, the AI teaches and the person learns. In augmented machine
intelligence, the person teaches and the AI learns [arxiv:2105.00691 fig. 2].
Card: [Hybrid Intelligence](/papers/2105.00691).

### Build-time human guidance

The survey places human input in three steps of model development. People
curate and label data, and active learning sends uncertain samples to annotators
[arxiv:2403.04931 §2.1] [fig. 5]. Human preferences shape the objective through
RLHF and DPO [arxiv:2403.04931 §2.2]. Human evaluation and alignment metrics
judge the result, and their findings feed back into data curation
[arxiv:2403.04931 §2.3] [fig. 4]. The survey gives no stopping rule for this
loop and no threshold for the active-learning round [arxiv:2403.04931 §2–§2.1].
Card: [A Survey on Human-AI Collaboration with Large Foundation Models](/papers/2403.04931).

### Run-time collaboration patterns

Once a model is deployed, the survey names five ways to split roles
[arxiv:2403.04931 §3.2] [tab. 2]. Each pattern has a named risk and a
safeguard. The survey proposes the safeguards but does not test them.

| Pattern | Human role | Risk named | Suggested safeguard |
|---|---|---|---|
| Human steers AI | Gives instructions, corrections or preferences | Feedback may be biased, inconsistent or narrow | Diverse feedback, trained evaluators, clear rubrics |
| AI augments human | Makes the final decision; AI supports | Over-trust, or uncertainty goes unnoticed | Show confidence, explain outputs, allow override |
| Mixed-initiative | Co-designs or reviews with the AI | Roles, authorship or next steps become unclear | Turn-taking, visible reasoning, revision history |
| Dynamic delegation | Steps in as expert or supervisor when handed control | Handoff comes too late or too often, or the person is unprepared | Escalation rules, readiness checks, shared situational awareness |
| Supervisory orchestration | Sets goals, reviews plans, intervenes | Errors build up silently; tool use and responsibility are opaque | Plan review, approval checkpoints, execution traces, audit logs |

Card: [A Survey on Human-AI Collaboration with Large Foundation Models](/papers/2403.04931).

### AI drafts, human revises

The review was written with ChatGPT, a GPT-3.5 model released on 30 November
2022 [arxiv:2212.08104 p. 2]. The procedure was to prompt each section, have the
model draft it, then correct and enrich the text, remove repetitions and replace
the references the model suggested. The authors repeated these revisions until
they were satisfied [arxiv:2212.08104 p. 2]. They report that 4.3% of the final
manuscript was identical to the preliminary text, 13.3% differed only in minor
ways, 16.3% kept a related meaning, and 6% of the draft's references were
correct [arxiv:2212.08104 p. 2]. The card notes that how the similarity figures
were computed is not stated. The authors also call nearly all of the draft's
references clearly incorrect [arxiv:2212.08104 §10]. Card:
[The Role of AI in Drug Discovery](/papers/2212.08104).

The same review names joint work between AI researchers and pharmaceutical
companies as a route to more accurate and efficient trials, with one example
[arxiv:2212.08104 §6]. The source cited for that example is a press release, so
it is weak evidence.

### Human checking of AI output

The last card measures the human side. Its scale has 13 self-report items,
rated on a 5-point scale, in three subscales: verification (5 items), motivation
to understand AI (4) and reflection on responsible AI (4) [arxiv:2512.12413 tab. 1].
In a Prolific sample of 376, the total scale had Cronbach's α of .88
[arxiv:2512.12413 §4.1.2], and a first-order confirmatory factor analysis gave
RMSEA .06 and CFI 1.00 [arxiv:2512.12413 §4.2.1]. Test–retest correlations were
.70 over one week (n = 92) and .65 over two weeks (n = 122)
[arxiv:2512.12413 §6.2]. For criterion validity, a higher total score went with
more correct veracity judgements on a 12-statement fact-checking task that used
a GPT-4o chatbot, with R² = .06 [arxiv:2512.12413 §7.2]. The same paper found
no reliable link with the Cognitive Reflection Test [arxiv:2512.12413 §4.2.2].
Card: [Understanding Critical Thinking in Generative AI Use](/papers/2512.12413).

## When to use which

The cards give no latency, memory or model-access constraints. The rows below
rest on constraints the papers themselves name, such as data scarcity, the need
for expert review, deployment risk and whether a survey is feasible.

| Situation | Use | Why |
|---|---|---|
| The machine can solve the task alone | AI in the loop of human intelligence | Paper: AI automates tasks machines can solve alone [arxiv:2105.00691 §3.1] |
| A person must act on or check a prediction | AI in the loop, as decision support | Paper: AI gives predictions for decisions and outputs for evaluation [arxiv:2105.00691 §3.1] |
| Little labelled data, but people can label or judge | Human intelligence in the loop of AI | Paper: human input is especially valuable when little data is available [arxiv:2105.00691 §4] |
| Labelling budget is limited | Active learning in build-time guidance | Survey: low-confidence samples go to annotators. Threshold not stated [arxiv:2403.04931 §2.1] |
| Model should follow human preferences | RLHF or DPO in build-time guidance | Survey: preferences shape the objective [arxiv:2403.04931 §2.2]. See the [RLHF](/concepts/rlhf) concept |
| Person must keep the final decision | AI augments human | Survey's safeguards: show confidence, allow override. Not tested [arxiv:2403.04931 tab. 2] |
| Model should hand over control when unsure | Dynamic delegation | Survey: needs escalation rules and readiness checks. Not tested [arxiv:2403.04931 tab. 2] |
| Agent runs multi-step tasks with tools | Supervisory orchestration | Survey: plan review, approval checkpoints and audit logs [arxiv:2403.04931 tab. 2] |
| Roles, authorship or next steps are shared | Mixed-initiative | Survey: turn-taking and revision history [arxiv:2403.04931 tab. 2] |
| Human expertise must be passed on to novices | Human learning from AI outputs | Paper: people can learn from AI and pass expert knowledge on [arxiv:2105.00691 §5] |
| Expert text is needed and a model drafts it | AI drafts, human revises | Draft was mostly rewritten: 4.3% identical, 6% of references correct [arxiv:2212.08104 p. 2]. Check every claim and citation by hand |
| Measure how well people check AI output, in a survey | Self-report scale | Reliable and stable in online samples [arxiv:2512.12413 §4.1.2] [§6.2]. It captures self-described habits, and its link to accuracy is small (R² = .06) [arxiv:2512.12413 §7.2] |

Whether a combined system outperforms its best part is not tested in any of the
four cards. The Hybrid Intelligence card gives no metric or threshold for
"superior results", so any such test has to be designed separately
[arxiv:2105.00691 §4]. The survey names measures of trust, workload and
collaboration quality as needed, but gives none [arxiv:2403.04931 §3.3] [§6].

## Implementation sketch

The paper has no algorithm. The sketch below restates its two loops as a
single round. It does not say how tasks are split or how outputs are combined,
and the paper does not either.

```pseudocode
# One round of a human-AI split, following the paper's two loops
# Conceptual: the paper gives no algorithm and no interfaces  [§4]
def hybrid_round(task, human, ai):
    # Split the task by which side suits each part  [§4]
    # The paper does not say how the split is made
    human_part, ai_part = allocate(task, human, ai)
    # Person's judgement on the part they suit  [§3]
    human_out = human.decide(human_part)
    # Model's prediction on the part it suits  [§3.1]
    ai_out = ai.predict(ai_part)
    # Combining the two outputs is left open  [§4]
    result = combine(human_out, ai_out)
    # Learning runs both ways: the model learns from people  [§3.2]
    # and people learn from the model  [§3.1]
    human, ai = exchange_feedback(human, ai, result)
    return result, human, ai
```

## Contradictions

None known between the cards. The Hybrid Intelligence card predicts that a
combined system beats either part [arxiv:2105.00691 §4]. The survey says that
interfaces, role allocation, training and evaluation decide how a team
performs, and that model performance alone does not [arxiv:2403.04931 §3]. That
is consistent with the prediction, but neither is tested. The other two cards
do not measure combined performance, so they neither support nor contradict
the prediction.

## Papers

- [Hybrid Intelligence](/papers/2105.00691): position paper defining humans and AI combining complementary strengths and learning from each other; no experiments.
- [A Survey on Human-AI Collaboration with Large Foundation Models](/papers/2403.04931): survey of build-time guidance, five run-time patterns, ethics and applications; quoted figures come from cited studies.
- [The Role of AI in Drug Discovery](/papers/2212.08104): narrative review drafted with ChatGPT and revised by the human authors; reports the drafting procedure and similarity figures.
- [Understanding Critical Thinking in Generative AI Use](/papers/2512.12413): develops and validates a 13-item self-report scale of critical thinking in AI use, with criterion evidence from one fact-checking task.