---
kind: concept
slug: instruction-tuning
title: Instruction tuning
summary: >-
  Fine-tuning a pretrained model on instruction-style pairs so it follows requests. Forty-six carded papers vary the data and method: curated pairs without RLHF (LIMA), prompted task mixtures (T0, Flan), model-generated or crowd-written dialogue (Self-Instruct, CAMEL, OpenAssistant), domain and LoRA tuning, and image, video, speech and driving inputs. Results are author-run.
tags: [alignment, supervised-fine-tuning, data-quality, domain-adaptation, multi-task, data-mixture, distillation, parameter-efficient, synthetic-data, multimodal, code-generation, video, speech, tool-use, autonomous-driving, summarization, scaling]
papers: ["2305.11206", "2303.14070", "2304.01097", "2307.14334", "2301.13688", "2301.12726", "2311.00176", "2305.16326", "2110.08207", "2202.03052", "2212.13138", "2303.17760", "2304.07327", "2304.08485", "2305.10403", "2308.12950", "2403.08295", "2304.10592", "2306.05424", "2305.06500", "2306.15195", "2305.11000", "2306.13549", "2305.16355", "2312.09245", "2310.06825", "2310.03744", "2311.05437", "2312.00784", "2311.17005", "2301.13848", "2305.09617", "2210.11416", "2309.16609", "2305.15334", "2306.14795", "2307.06435", "2308.08747", "2401.04088", "2406.12793", "2212.10560", "2304.14178", "2305.03726", "2312.13286", "2311.15826", "2303.03846"]
related: [dataset-curation, rlhf, llm-as-a-judge, llm-evaluation, visual-instruction-tuning, multi-task-learning, multimodal-language-model, retrieval-augmented-generation, hallucination-mitigation, knowledge-distillation, chain-of-thought-prompting, domain-adaptation, few-shot-learning, language-model-pretraining, parameter-efficient-fine-tuning, low-rank-adaptation, prompt-engineering, zero-shot-learning, self-consistency, soft-prompt-tuning, llm-agent, multi-agent-systems, vision-language-pretraining, code-completion, positional-encoding, constrained-decoding, text-summarization, human-evaluation, tool-use, visual-prompting, visual-grounding, multimodal-fusion, discrete-speech-units, vision-transformer, benchmark-validity, neural-scaling-laws, text-to-image-generation, in-context-learning, emergent-abilities]
status: draft
updated_at: 2026-11-01
---

## Overview

Instruction tuning continues training a pretrained model on pairs of a
request and a desired response, so that the model follows requests in the
prompt rather than only continuing text. The core loss is the usual
next-token loss, applied to the response [arxiv:2305.11206 §2]. What varies
across the carded papers is the data (a few curated pairs, large prompted
dataset mixtures, or model-written dialogue), what is trained (all weights,
an adapter, or only a projection between frozen encoders and a frozen
language model), and whether a preference stage such as RLHF or DPO follows
[arxiv:2310.06825 §4] [arxiv:2401.04088 §4].

Forty-six carded papers cover this topic. Almost every number below is
author-reported, from the authors' own benchmarks, and often from one run.

## Variants

### Small curated sets

Tuning on about 1,000 hand-picked pairs can reach strong open-ended quality.
LIMA tunes LLaMA 65B on 1,000 pairs with a plain supervised loss and no
RLHF, and reports it equal or preferred to GPT-4 in 43% of comparisons
[arxiv:2305.11206 §1]. The checkpoint was chosen by hand on a 50-example
dev set, and the ablations use 7B models [arxiv:2305.11206 §5]. Card:
[LIMA](/papers/2305.11206).

OpenAssistant collects volunteer-written conversation trees, with 161,443
messages in 35 languages, and tunes Pythia, Falcon and LLaMA models with
supervised tuning and PPO-based RLHF [arxiv:2304.07327 tab. 1]. The
data's licence is called permissive but no licence is named [arxiv:2304.07327 app. L].
Card: [OpenAssistant Conversations](/papers/2304.07327).

Instruction tuning is not the only lever. A benchmarking study over ten LLMs
finds that instruction tuning, more than size, decides zero-shot
news-summary quality [arxiv:2301.13848 §3.2]. Card:
[Benchmarking LLMs for News Summarization](/papers/2301.13848).

### Prompted multitask mixtures

T0 rewrites 62 datasets with several prompts each and tunes T5+LM 11B
(3B also trained). It matches or exceeds GPT-3 on 9 of 11 held-out datasets
[arxiv:2110.08207 §6.1]. Its medians span all prompts, while GPT-3 was
given one prompt per dataset [arxiv:2110.08207 §6.1]. Card:
[T0](/papers/2110.08207).

Flan-PaLM widens the mixture to 1,836 tasks plus nine human-written
chain-of-thought sets. At 540B it reaches 75.2% five-shot MMLU with CoT and
self-consistency [arxiv:2210.11416 tab. 4]. Its checkpoint was chosen on the
held-out tasks, so those numbers are slightly optimistic [arxiv:2210.11416 §2.2].
Card: [Flan-PaLM](/papers/2210.11416).

Tuning also changes how a model weighs in-context examples against its prior
knowledge. In a flipped-label test, PaLM drops as low as 31% when every
exemplar label is flipped, while Flan-PaLM does not fall below chance
[arxiv:2303.03846 §5] [arxiv:2303.03846 fig. 7]. The same study reports that
instruction tuning improves learning from exemplars but strengthens priors
more [arxiv:2303.03846 §5]. Card: [Larger language models do in-context
learning differently](/papers/2303.03846).

The Flan Collection study varies the design choices on T5-XL. Mixing
zero- and few-shot templates is part of the recipe, but removing few-shot
templates can raise zero-shot CoT [arxiv:2301.13688 tab. 1]. Card:
[The Flan Collection](/papers/2301.13688).

Distillation can also produce reasoning-tuned small models. Chain-of-thought
solutions from a large teacher are distilled into T5 and FlanT5 models of
250M to 11B parameters; FlanT5-XXL reaches 27.1% on GSM8K against 16.1%
before, but every specialised model scores 0.0 on BBH-CoT [arxiv:2301.12726 tab. 1].
Card: [Specializing Smaller LMs to Multi-Step Reasoning](/papers/2301.12726).

Frontier-scale and open general models use the same recipe with more
stages. PaLM 2 reports Flan-style tuning, with BIG-Bench Hard at 78.1 against
65.2 for PaLM with CoT [arxiv:2305.10403 tab. 6], but its model sizes and
training settings are not published [arxiv:2305.10403]. Card:
[PaLM 2](/papers/2305.10403). Mistral 7B, Gemma, Qwen, Code Llama and
ChatGLM also ship chat variants tuned with SFT and sometimes RLHF
[arxiv:2310.06825] [arxiv:2403.08295] [arxiv:2309.16609] [arxiv:2308.12950]
[arxiv:2406.12793]. A survey of LLMs tabulates tuning settings for many of
these models, copied from their papers [arxiv:2307.06435 §4]. Cards: [Mistral 7B](/papers/2310.06825),
[Gemma](/papers/2403.08295), [Qwen](/papers/2309.16609),
[Code Llama](/papers/2308.12950), [ChatGLM](/papers/2406.12793). Mixtral is
instruction-tuned by SFT and then DPO [arxiv:2401.04088 §4]; its instruct
model reaches MT-Bench 8.30 [arxiv:2401.04088 tab. 3]. Card:
[Mixtral of Experts](/papers/2401.04088).

### Synthetic and model-written data

Self-Instruct avoids human writers. Starting from 175 hand-written seed
tasks, GPT-3 is prompted to write new instructions, then inputs and outputs.
Its filters use ROUGE-L and heuristics. The result is about 52K instructions
and 82K instances [arxiv:2212.10560 §2] [arxiv:2212.10560 tab. 1]. Fine-tuning
GPT-3 on this data raises zero-shot SuperNI ROUGE-L from 6.8 to 39.9, against
40.8 for InstructGPT-001 [arxiv:2212.10560 tab. 3]. One author judged only 54%
of a 200-instruction sample fully valid [arxiv:2212.10560 tab. 2]. Card:
[Self-Instruct](/papers/2212.10560).

CAMEL generates 25,000 role-play conversations with gpt-3.5-turbo agents,
then tunes LLaMA-7B. Human votes prefer CAMEL over gpt-3.5-turbo in 76.3% of
cases on AI Society [arxiv:2303.17760 tab. 1]. Card: [CAMEL](/papers/2303.17760).

Gorilla tunes LLaMA-7B on about 1,645 hub APIs, each with ten GPT-4-written
requests [arxiv:2305.15334 §3.1]. It beats zero-shot GPT-4 on all three hub
benchmarks, and its authors report fewer hallucinated calls [arxiv:2305.15334 tab. 1].
Card: [Gorilla](/papers/2305.15334).

Tuning on generated data can also erode general knowledge; see the
forgetting row in [When to use which](#when-to-use-which).

### Domain adaptation

Medical models include ChatDoctor, which tunes LLaMA-7B on about 100k
dialogues [arxiv:2303.14070 tab. 1]; DoctorGLM, which uses LoRA on
ChatGLM-6B [arxiv:2304.01097 §3.3]; Med-PaLM 2, which tunes PaLM 2 on
MultiMedQA and reports 86.5 on MedQA [arxiv:2305.09617 §4.1]; and Med-PaLM M,
which tunes PaLM-E on 14 medical tasks [arxiv:2307.14334 tab. 2]. Med-PaLM
(the Flan-PaLM variant) only trains a 100-token soft prompt on 40 exemplars
[arxiv:2212.13138 app. A.1]. Cards: [ChatDoctor](/papers/2303.14070),
[DoctorGLM](/papers/2304.01097), [Med-PaLM 2](/papers/2305.09617),
[Med-PaLM M](/papers/2307.14334), [MultiMedQA](/papers/2212.13138).

GeoChat applies the LLaVA-1.5 recipe to remote sensing. It trains LoRA on
Vicuna-v1.5 7B with about 318k image-instruction pairs made from detection,
scene and VQA datasets [arxiv:2311.15826 tab. 1]. Vicuna writes the questions
and answers from text descriptions, not from the images [arxiv:2311.15826 §4].
Zero-shot scene classification reaches 84.43% on UCMerced and 72.03% on AID
[arxiv:2311.15826 tab. 5]. Card: [GeoChat](/papers/2311.15826).

A benchmark study compares LoRA-tuned LLaMA 2 13B and PMC LLaMA 13B with
zero- and few-shot GPT-3.5 and GPT-4. The tuned baseline wins on 10 of 12
datasets, but its literature baselines were not re-run [arxiv:2305.16326 p. 6].
Card: [Biomedical NLP benchmark](/papers/2305.16326).

ChipNeMo applies domain-adaptive pretraining, with about 24B tokens, and then
supervised tuning on 1.4K domain samples. Its 70B model scores 76.6 on an
internal design-knowledge set against 58.4 for GPT-4 [arxiv:2311.00176 app. Auto Evaluation Results].
Continued pretraining on a chat-aligned model weakens its alignment
[arxiv:2311.00176 §2.3]. Card: [ChipNeMo](/papers/2311.00176).

### Parameter-efficient tuning

LoRA is used in several cards. DoctorGLM applies rank 8 with alpha 16 to the
query and value projections, on one A100 for about 8 hours [arxiv:2304.01097 §3.3].
PandaGPT trains only a projection and LoRA on attention, on 160k examples
[arxiv:2305.16355 §3]. SpeechGPT's third stage uses LoRA [arxiv:2305.11000 §5.1].
Card: [PandaGPT](/papers/2305.16355), [SpeechGPT](/papers/2305.11000).
Med-PaLM is the extreme case, training one 100-token prompt
[arxiv:2212.13138 app. A.1].

### Multimodal instruction tuning

Most multimodal cards keep the vision encoder and the language model frozen
and train a connector first. LLaVA trains a linear projection in stage one
and the projection with the language model in stage two, on 158K
GPT-4-written visual instructions [arxiv:2304.08485 app. C]. Its successor,
LLaVA-1.5, uses a two-layer MLP and reports author-run state-of-the-art
results on 11 benchmarks [arxiv:2310.03744]. MiniGPT-4 has only a linear layer
in its first stage; its second stage on about 3,500 curated descriptions
cuts the failure rate on detailed descriptions from 35% to 2%
[arxiv:2304.10592 tab. 3]. Cards: [LLaVA](/papers/2304.08485),
[LLaVA-1.5](/papers/2310.03744), [MiniGPT-4](/papers/2304.10592).

mPLUG-Owl also keeps the language model frozen in its first stage. It
trains the vision encoder and an abstractor on image-caption pairs, then
trains LoRA adapters on the language model with text-only and image-text
instructions [arxiv:2304.14178 §3.2]. On the authors' 82-question OwlEval
set, 66 answers are rated A or B against 54 for MiniGPT-4, by manual rating
[arxiv:2304.14178 §4.2]. Card: [mPLUG-Owl](/papers/2304.14178).

Other designs change what is trained or what is in the prompt. InstructBLIP
trains only the Q-Former, which also reads the instruction, and reports a
15.0% average relative gain over BLIP-2 with FlanT5-XL [arxiv:2305.06500 §3.1].
Shikra writes coordinates as
text, and reaches 87.01 REC on RefCOCO val for the 7B model [arxiv:2306.15195 tab. 3].
Visual prompts such as boxes and masks are blended into the image in
ViP-LLaVA, which reports 87.91% on Visual7W at 13B [arxiv:2312.00784 §5.2].
LLaVA-Plus trains on tool-use dialogues and scores 69.5 against 57.1 for LLaVA
on In-the-Wild [arxiv:2311.05437 tab. 4]. Cards: [InstructBLIP](/papers/2305.06500),
[Shikra](/papers/2306.15195), [ViP-LLaVA](/papers/2312.00784), [LLaVA-Plus](/papers/2311.05437).

Some cards train on several images or on interleaved sequences. Otter adds
other image-instruction-response examples to each training item, drawn from
MIMIC-IT, about 3M pairs [arxiv:2305.03726 §4] [arxiv:2305.03726 tab. 1]. Only
the perceiver resampler and the gated cross-attention layers train, about
1.4B parameters [arxiv:2305.03726 §3.1]. Card: [Otter](/papers/2305.03726).
Emu2 is a 37B model that places images and text in one sequence, with 64
embeddings per image [arxiv:2312.13286 §2.1]. Emu2-Chat is tuned on
academic-task and multimodal chat data, with 16x16 pooling per image,
8,000 steps and a global batch of 768 [arxiv:2312.13286 §2.3.1] [app. C.2].
It scores 84.9 on VQAv2 and 48.5 on MM-Vet, author-reported [arxiv:2312.13286 tab. 2].
Card: [Emu2](/papers/2312.13286). The card records that the released weights
are listed as non-commercial, so check that before any commercial use.

A survey of multimodal LLMs covers these training stages and evaluation
choices, but runs no experiments of its own [arxiv:2306.13549]. The OFA model
tunes one encoder-decoder on vision and language tasks, with VQA test-std 82.0
[arxiv:2202.03052 tab. 2]. Cards: [OFA](/papers/2202.03052),
[survey of multimodal LLMs](/papers/2306.13549).

### Video, speech and driving

Video-ChatGPT tunes Vicuna-7B on 100K video-instruction pairs and scores
2.40 on a GPT-judged benchmark [arxiv:2306.05424 tab. 1]. MVBench introduces
a 20-task temporal benchmark, and its VideoChat2 model averages 51.1 against
35.5 for VideoChat [arxiv:2311.17005 tab. 2]. Cards: [Video-ChatGPT](/papers/2306.05424),
[MVBench](/papers/2311.17005).

SpeechGPT converts speech to discrete units and tunes a LLaMA backbone on
about 9M speech-text pairs. Its evaluation is qualitative, with no baselines
[arxiv:2305.11000 tab. 1]. Card: [SpeechGPT](/papers/2305.11000).

DriveMLM tunes LLaMA-7B to output driving decisions with an explanation,
and reaches driving score 76.1 against 71.4 for Apollo on CARLA Town05 Long
[arxiv:2312.09245 tab. 4]. Card: [DriveMLM](/papers/2312.09245).

### Motion and other domains

MotionGPT adds motion codes to the vocabulary of Flan-T5 and instruction-tunes
on text-to-motion, motion-to-text, prediction and in-between prompts
[arxiv:2306.14795 §3.3]. Card: [MotionGPT](/papers/2306.14795).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Little data, strong base model, no preference data | Small curated set, plain SFT | 1,000 pairs were enough for open-ended quality in one author-run study [arxiv:2305.11206 §1] |
| Need broad zero-shot task coverage | Prompted multitask mixture | T0 and Flan report gains on held-out tasks [arxiv:2110.08207 §6.1] [arxiv:2210.11416 tab. 4] |
| Compute-limited adaptation of a large model | LoRA or a prompt | DoctorGLM used one A100 for about 8 h [arxiv:2304.01097 §3.3]; Med-PaLM trained one prompt [arxiv:2212.13138 app. A.1] |
| Adapt a vision or speech model to an LLM | Frozen encoder and LLM, train a connector | Training fits in hours on 8 A100s in the LLaVA-style cards [arxiv:2304.08485 app. C] [arxiv:2306.05424 §5.1] |
| Need the model to keep old general knowledge | Mix in general data, test forgetting | Sequential tuning lowered general scores in every model tested; mixing 10k Alpaca samples helped one split [arxiv:2308.08747 §5.4] |
| Domain with lots of unlabelled text | Domain pretraining, then tuning | ChipNeMo's gains rely on about 24B tokens [arxiv:2311.00176 §3.5] |
| Specialised imagery such as remote sensing | Generate pairs from labelled data, then LoRA-tune a general VLM | GeoChat reports 84.43% zero-shot on UCMerced [arxiv:2311.15826 tab. 5]; its training list includes RSVQA-LRBEN [tab. 1] |
| Multi-image or video context, few trainable weights | Train a resampler and gated cross-attention on a frozen base | With in-context training, Otter's 2-shot MME-Cognition is 1430.3 against 1310.7 without it [arxiv:2305.03726 tab. 10] |
| No human-written data, and generated data is acceptable | Bootstrap from a few seed tasks with the model itself | GPT-3 tuned on generated data reaches SuperNI ROUGE-L 39.9 from 6.8, but only 54% of a sampled set was fully valid [arxiv:2212.10560 tab. 3] [tab. 2] |
| Model must override its priors from in-context labels | Compare the untuned and tuned checkpoints on flipped labels before choosing | In PaLM vs Flan-PaLM, the tuned model followed its priors more at 100% flipped labels [arxiv:2303.03846 §5] [fig. 7] |
| Tool calls from a retriever at test time | Train with the retrieved document, or not | A model trained with documents scores 0 zero-shot, so a retriever-free deployment needs the other model [arxiv:2305.15334 tab. 2] |

Constraints the cards give: data licence, teacher-model terms (several
pipelines rely on GPT-4 outputs), and compute. Cards that name hardware
range from 4 A100s to 128 A100s, and TPU runs use up to 512 chips. The Flan-PaLM card puts 540B finetuning
at 0.2% of pretraining compute [arxiv:2210.11416 §2.2].

## Implementation sketch

Common core, in Python-flavoured pseudocode. The loss is computed on the
response tokens only, and each variant changes which parameters receive
gradients and where the data comes from.

```pseudocode
# Instruction-tuning loop, common core  [arxiv:2305.11206 §2]
# Input:  pairs (instruction, response); model; trainable_params
# Output: tuned model

def instruction_tune(model, pairs, trainable_params, epochs=3):
    # Only the chosen parameters get gradients: all weights, LoRA
    # adapters or a connector [arxiv:2304.01097 §3.3] [arxiv:2304.08485 app. C]
    optimizer = make_optimizer(trainable_params)
    for _ in range(epochs):
        for instruction, response in pairs:
            # Concatenate prompt and answer into one token sequence
            tokens = tokenize(instruction + response)
            # Mask the prompt so only the response is scored
            labels = mask_prompt(tokens, prompt_length(instruction))
            loss = model(tokens, labels=labels).loss
            loss.backward()
            optimizer.step()
            optimizer.zero_grad()
    return model
```

For multimodal models, freeze the encoder and the language model in stage
one, train only a connector, then unfreeze the language model in stage two
[arxiv:2304.08485 app. C].

## Contradictions

- **Size of the tuning set.** LIMA reports strong results from 1,000 pairs
  [arxiv:2305.11206 §1], while Flan-PaLM reports gains from a far larger
  mixture [arxiv:2210.11416 tab. 4]. The settings differ (chat quality versus
  benchmark accuracy), so the two results are not directly comparable.
- **Mistral 7B.** The abstract says the model beats Llama 2 13B on all
  benchmarks, but the body's table shows one exception, NQ, at 29.0% against
  28.8% [arxiv:2310.06825 tab. 2].
- **Med-PaLM 2.** The 86.5 MedQA score comes from a MedQA-only variant;
  the unified model reports 85.4 [arxiv:2305.09617 §3.2] [arxiv:2305.09617 §4.1].
- **Judge dependence.** Video-ChatGPT scores its own model at 2.40 with a
  GPT-3.5 judge and 2.49 with a Vicuna judge [arxiv:2306.05424 tab. 4].
- **Paper and code.** ChatDoctor's README and paper give different learning
  rates, epochs and batch sizes [arxiv:2303.14070]. CAMEL's repository
  prompts and termination rules differ from the paper [arxiv:2303.17760].
  Code Llama's paper and its repository disagree on which sizes support
  infilling [arxiv:2308.12950].
- **mPLUG-Owl's first stage.** The text gives learning rate 1e-4 and 2,000
  warm-up steps [arxiv:2304.14178 §4.1], while the appendix table gives 2e-4
  and 375 steps [arxiv:2304.14178 app. A]. The abstract says LoRA also trains
  the abstractor, but §3.2 freezes it [arxiv:2304.14178 §1] [arxiv:2304.14178 §3.2].
- **GeoChat's LoRA targets.** The paper applies LoRA to W_q and W_v only
  [arxiv:2311.15826 §5.1]. The released code applies it to more layers; see
  the [GeoChat card](/papers/2311.15826).

## Papers

- [LIMA](/papers/2305.11206): 1,000 curated pairs for LLaMA 65B.
- [OpenAssistant Conversations](/papers/2304.07327): volunteer-written dialogue with RLHF.
- [Benchmarking LLMs for News Summarization](/papers/2301.13848): evaluation of instruction tuning for summarization.
- [T0](/papers/2110.08207): prompted multitask training for zero-shot transfer.
- [Flan-PaLM](/papers/2210.11416): scaling instruction tuning to 1,836 tasks.
- [The Flan Collection](/papers/2301.13688): design choices for Flan-style mixtures.
- [Specializing Smaller LMs to Multi-Step Reasoning](/papers/2301.12726): distilled reasoning for small models.
- [ChatDoctor](/papers/2303.14070): medical dialogue tuning of LLaMA-7B.
- [DoctorGLM](/papers/2304.01097): LoRA medical dialogue tuning of ChatGLM-6B.
- [Med-PaLM 2](/papers/2305.09617): instruction-tuned PaLM 2 for medicine.
- [Med-PaLM M](/papers/2307.14334): one multimodal model for medical tasks.
- [MultiMedQA](/papers/2212.13138): soft-prompt tuning of Flan-PaLM for medicine.
- [Biomedical NLP benchmark](/papers/2305.16326): LoRA tuning against zero- and few-shot LLMs.
- [ChipNeMo](/papers/2311.00176): domain adaptation for chip design.
- [OFA](/papers/2202.03052): one sequence-to-sequence model for vision and language.
- [LLaVA](/papers/2304.08485): visual instruction tuning with GPT-4-written data.
- [MiniGPT-4](/papers/2304.10592): a projection between a frozen vision encoder and Vicuna.
- [InstructBLIP](/papers/2305.06500): instruction-aware Q-Former tuning.
- [LLaVA-1.5](/papers/2310.03744): baselines for visual instruction tuning.
- [Shikra](/papers/2306.15195): spatial referring through text coordinates.
- [SpeechGPT](/papers/2305.11000): speech units with a LLaMA backbone.
- [PandaGPT](/papers/2305.16355): multimodal tuning through a shared embedding space.
- [Video-ChatGPT](/papers/2306.05424): video instruction tuning.
- [survey of multimodal LLMs](/papers/2306.13549): survey of multimodal instruction tuning.
- [ViP-LLaVA](/papers/2312.00784): visual prompts for region-level instructions.
- [LLaVA-Plus](/papers/2311.05437): tool-use dialogue for multimodal models.
- [MVBench](/papers/2311.17005): temporal video benchmark and VideoChat2.
- [DriveMLM](/papers/2312.09245): instruction-tuned driving decisions.
- [CAMEL](/papers/2303.17760): synthetic role-play conversations.
- [Gorilla](/papers/2305.15334): tuning for API calls.
- [MotionGPT](/papers/2306.14795): motion as tokens for instruction tuning.
- [A Comprehensive Overview of Large Language Models](/papers/2307.06435): survey of LLM tuning and architectures.
- [Catastrophic forgetting under continual tuning](/papers/2308.08747): forgetting of general knowledge.
- [Mixtral of Experts](/papers/2401.04088): SFT and DPO tuning of a sparse mixture model.
- [Gemma](/papers/2403.08295): instruction tuning with SFT and RLHF.
- [ChatGLM](/papers/2406.12793): alignment and tool use in the GLM-4 family.
- [PaLM 2](/papers/2305.10403): technical report with instruction tuning.
- [Code Llama](/papers/2308.12950): instruction tuning for code.
- [Qwen](/papers/2309.16609): base and chat models with SFT and RLHF.
- [Mistral 7B](/papers/2310.06825): instruction tuning of a 7B model.
- [Self-Instruct](/papers/2212.10560): model-written instructions from 175 seed tasks.
- [mPLUG-Owl](/papers/2304.14178): modular image input to LLaMA-7B with LoRA tuning.
- [Otter](/papers/2305.03726): multi-image in-context instruction tuning.
- [Emu2](/papers/2312.13286): 37B interleaved model with chat and generation tuning.
- [GeoChat](/papers/2311.15826): instruction tuning for remote-sensing dialogue.
- [Larger language models do in-context learning differently](/papers/2303.03846): flipped and unrelated labels, with PaLM vs Flan-PaLM.