Human-AI collaboration

Overview

Human-AI collaboration is a way of dividing work in which a person and an AI system each take the part they are better at, and the two improve each other over time. Four cards cover it from different angles. A position paper names the arrangement Hybrid Intelligence and sets out its two directions 2105.00691 §4. It contrasts human strengths, such as intuition, common sense and handling unlabelled cases, with machine strengths, such as pattern recognition, consistency and speed 2105.00691 §3. A survey of collaboration with large foundation models sorts the field into build-time human guidance and five run-time patterns 2403.04931 §2–§3. A narrative review shows one concrete workflow, in which a language model drafts the text and the authors then rewrite it 2212.08104 p. 2. A last paper looks at the human side and offers a survey scale for how people check AI output 2512.12413 §2–§7.

None of the four cards builds a combined person-and-model system and measures it against either part alone. Hybrid Intelligence's claim of superior results is a definition and a prediction, not an observed outcome 2105.00691 §1 2105.00691 §4. The survey and the review report no experiments of their own 2403.04931 §1 2212.08104 §1–§9. Trust, governance and digital work are named as open research directions, not answered 2105.00691 §6. The survey adds open challenges on scalable human guidance and accountability 2403.04931 §6. For learning from human judgements as a training signal, see RLHF; for people reviewing a system's outputs, see Human evaluation; for labelling the most useful samples, see Active learning; for the measurement side, see Psychometric scale validation.

Variants

The Hybrid Intelligence paper describes two directions that differ in which side teaches. The survey splits the field by timing: human guidance while the model is built, and collaboration patterns once it is in use. The writing workflow and the self-report scale cover narrower ground.

AI in the loop of human intelligence

The AI takes over tasks that machines can solve alone, or it gives a person predictions to decide with. The examples are forecasting, financial markets and automated choice of hyperparameters for new models 2105.00691 §3.1. The AI can also produce an output that a person then evaluates, such as a model that reads CT scans to support a physician's diagnosis 2105.00691 §3.1. Card: Hybrid Intelligence.

Human intelligence in the loop of AI

People teach the system. The tasks named are setting or tuning hyperparameters, training and debugging models, and interpreting unsupervised results such as clusters 2105.00691 §3.2. The paper says that interactive learning often needs many labelled examples from people, and that human input matters most when data is scarce 2105.00691 §4. Card: Hybrid Intelligence.

Roles in augmentation

A figure in the paper separates two arrangements. In augmented human intelligence, the AI teaches and the person learns. In augmented machine intelligence, the person teaches and the AI learns 2105.00691 fig. 2. Card: Hybrid Intelligence.

Build-time human guidance

The survey places human input in three steps of model development. People curate and label data, and active learning sends uncertain samples to annotators 2403.04931 §2.1 fig. 5. Human preferences shape the objective through RLHF and DPO 2403.04931 §2.2. Human evaluation and alignment metrics judge the result, and their findings feed back into data curation 2403.04931 §2.3 fig. 4. The survey gives no stopping rule for this loop and no threshold for the active-learning round 2403.04931 §2–§2.1. Card: A Survey on Human-AI Collaboration with Large Foundation Models.

Run-time collaboration patterns

Once a model is deployed, the survey names five ways to split roles 2403.04931 §3.2 tab. 2. Each pattern has a named risk and a safeguard. The survey proposes the safeguards but does not test them.

Card: A Survey on Human-AI Collaboration with Large Foundation Models.

AI drafts, human revises

The review was written with ChatGPT, a GPT-3.5 model released on 30 November 2022 2212.08104 p. 2. The procedure was to prompt each section, have the model draft it, then correct and enrich the text, remove repetitions and replace the references the model suggested. The authors repeated these revisions until they were satisfied 2212.08104 p. 2. They report that 4.3% of the final manuscript was identical to the preliminary text, 13.3% differed only in minor ways, 16.3% kept a related meaning, and 6% of the draft's references were correct 2212.08104 p. 2. The card notes that how the similarity figures were computed is not stated. The authors also call nearly all of the draft's references clearly incorrect 2212.08104 §10. Card: The Role of AI in Drug Discovery.

The same review names joint work between AI researchers and pharmaceutical companies as a route to more accurate and efficient trials, with one example 2212.08104 §6. The source cited for that example is a press release, so it is weak evidence.

Human checking of AI output

The last card measures the human side. Its scale has 13 self-report items, rated on a 5-point scale, in three subscales: verification (5 items), motivation to understand AI (4) and reflection on responsible AI (4) 2512.12413 tab. 1. In a Prolific sample of 376, the total scale had Cronbach's α of .88 2512.12413 §4.1.2, and a first-order confirmatory factor analysis gave RMSEA .06 and CFI 1.00 2512.12413 §4.2.1. Test–retest correlations were .70 over one week (n = 92) and .65 over two weeks (n = 122) 2512.12413 §6.2. For criterion validity, a higher total score went with more correct veracity judgements on a 12-statement fact-checking task that used a GPT-4o chatbot, with R² = .06 2512.12413 §7.2. The same paper found no reliable link with the Cognitive Reflection Test 2512.12413 §4.2.2. Card: Understanding Critical Thinking in Generative AI Use.

When to use which

The cards give no latency, memory or model-access constraints. The rows below rest on constraints the papers themselves name, such as data scarcity, the need for expert review, deployment risk and whether a survey is feasible.

Whether a combined system outperforms its best part is not tested in any of the four cards. The Hybrid Intelligence card gives no metric or threshold for "superior results", so any such test has to be designed separately 2105.00691 §4. The survey names measures of trust, workload and collaboration quality as needed, but gives none 2403.04931 §3.3 §6.

Implementation sketch

The paper has no algorithm. The sketch below restates its two loops as a single round. It does not say how tasks are split or how outputs are combined, and the paper does not either.

# One round of a human-AI split, following the paper's two loops
# Conceptual: the paper gives no algorithm and no interfaces  [§4]
def hybrid_round(task, human, ai):
    # Split the task by which side suits each part  [§4]
    # The paper does not say how the split is made
    human_part, ai_part = allocate(task, human, ai)
    # Person's judgement on the part they suit  [§3]
    human_out = human.decide(human_part)
    # Model's prediction on the part it suits  [§3.1]
    ai_out = ai.predict(ai_part)
    # Combining the two outputs is left open  [§4]
    result = combine(human_out, ai_out)
    # Learning runs both ways: the model learns from people  [§3.2]
    # and people learn from the model  [§3.1]
    human, ai = exchange_feedback(human, ai, result)
    return result, human, ai

Contradictions

None known between the cards. The Hybrid Intelligence card predicts that a combined system beats either part 2105.00691 §4. The survey says that interfaces, role allocation, training and evaluation decide how a team performs, and that model performance alone does not 2403.04931 §3. That is consistent with the prediction, but neither is tested. The other two cards do not measure combined performance, so they neither support nor contradict the prediction.

Papers

• Hybrid Intelligence: position paper defining humans and AI combining complementary strengths and learning from each other; no experiments.
• A Survey on Human-AI Collaboration with Large Foundation Models: survey of build-time guidance, five run-time patterns, ethics and applications; quoted figures come from cited studies.
• The Role of AI in Drug Discovery: narrative review drafted with ChatGPT and revised by the human authors; reports the drafting procedure and similarity figures.
• Understanding Critical Thinking in Generative AI Use: develops and validates a 13-item self-report scale of critical thinking in AI use, with criterion evidence from one fact-checking task.