Dataset curation

Overview

Curation decides what a model or benchmark is trained or scored on: where examples come from, which ones are kept, how they are labelled, and what is written down about the result. The 55 cards on this page repeat the same steps: choose sources, filter by rules and by models, remove duplicates, label with human checks, split held-out data carefully, and document what was kept and what was dropped 2104.08758 §1 2112.06409 §1.

The survey that maps this area describes the work as data-centric. It reports no experiments of its own, so its value is the taxonomy and the names of the techniques 2112.06409 §2–§5.

Numbers below are author-reported unless a card says otherwise. Check the linked card before relying on one.

Licence and reuse. Many releases here are not free for commercial use, even when the paper is CC BY. Examples: the DialogSum data repository declares non-commercial terms 2105.06762 §2; the Omnidata starter set is research-only 2110.04994 §4.2; ARKitScenes data and code fall under a non-commercial licence 2111.08897 §1. The MovieNet paper is CC BY 4.0, but that licence covers its text only. Its films, trailers and labels are reached through a user agreement that the project page describes, and the paper says the data will be published in line with regulations 2007.10937 §1 (card). SMHD is shared under a data-use agreement 1806.05258 §3.1. CINIC-10 states no licence for its dataset, and its ImageNet images come from a source with separate terms 1810.03505 §2 §3. DeepWeeds states CC BY 4.0 for its images and annotations 1810.05726 § Data Availability. Objaverse-XL keeps each object's own licence, and the dataset as a whole is ODC-By 1.0 2307.05663 app. C. Its paper says savable Polycam objects are CC BY 4.0 2307.05663 §3.1, but the repository says Polycam data is for academic, non-commercial use on request README.md. Treat Polycam-derived objects as unresolved. The NLLB training bitext comes from many public corpora under their own terms 2207.04672 §8.1.2. Check each card's code_license and the dataset's own terms. WebFace260M and WebFace42M are marked "Public" in the paper's dataset table, but its text grants access only to applicants who sign a licence for research use 2103.04098 tab. 1 §6. The dynamic hate-speech dataset's README states CC BY 4.0, but its repository has no LICENSE file README.md. HLE asks users not to train on its questions 2501.14249 §1. Self-Instruct's released generated data has no licence stated in its paper, and it was produced with the GPT-3 API, whose terms the paper does not discuss 2212.10560 §3 app. A.2. A dataset-analysis toolkit depends on a public sparse autoencoder and a Llama 3.3 reader model; its card records no licence for either 2512.10092 §4.

Variants

Web text for language-model pretraining

These papers all start from web crawls or large document collections. The main decisions are the language filter, the rule filters, and the dedup method.

• C4 applies rule filters to one April 2019 Common Crawl snapshot and keeps English text, with a word blocklist. The blocklist removed 42% of documents classed as African American English against 6.2% for White-aligned English 2104.08758 §5.3. The authors advise against blocklists 2104.08758 §6. Card: Documenting Large Webtext Corpora.
• The Pile combines 22 weighted English sources with per-source processing and documentation 2101.00027 §2. Near-duplicate removal covers only OpenWebText2 and Pile-CC 2101.00027 app. D. Its 1.3B models beat CC-100 and raw Common Crawl on Pile components 2101.00027 §3, §4. Card: The Pile.
• OSCAR 22.01 assigns one language per document rather than per line, trims short lines at the head and tail, and adds annotations such as "noisy" or "tiny" instead of deleting text 2201.06642 §3 §4 §5. The authors say they cannot yet state its quality 2201.06642 §1. Card: Cleaner Document-Oriented Multilingual Crawled Corpus.
• OPT deduplicates about 180B tokens from several sources with MinHash 2205.01068 §2.3. Card: OPT.
• PaLM 2 uses a more multilingual, deduplicated mix, with de-identification and quality filtering. It also adds toxicity control tokens. Its mixture weights and sizes are not stated 2305.10403 §3. Card: PaLM 2.
• BLOOM trains on the ROOTS corpus of 46 natural and 13 programming languages 2211.05100 §1 §3.1. Card: BLOOM.
• Pythia trains twice on the Pile, once as released and once deduplicated, in the same data order, and saves checkpoints for analysis 2304.01373 §2.2. Card: Pythia.
• CamemBERT uses French web text from OSCAR. A 4 GB sample performed about as well as the full 138 GB, and web text beat Wikipedia of the same size 1911.03894 §6. Card: CamemBERT.
• GottBERT compares German OSCAR with a filtered version that is 121 GB against 145 GB. Filtering made no clear difference, and the best filtered/unfiltered pairs differ by at most 0.85 points 2012.02110 tab. 3. Card: GottBERT.
• Minerva keeps web pages whose raw HTML contains math markup, so that TeX and AsciiMath survive, and drops pages with MathML only 2206.14858 §2.1 app. B.2. Card: Solving Quantitative Reasoning Problems.
• Phi-3 combines heavily filtered web data with LLM-written synthetic reasoning data 2404.14219 §2. Card: Phi-3 Technical Report.
• phi-1 uses about 6B tokens of filtered Python code, selected by a classifier trained on about 100k GPT-4 ratings, plus synthetic textbooks 2306.11644 §2 §2.1. At 350M parameters, HumanEval rose from 12.19% on unfiltered code to 17.68% on filtered code and 20.12% with the textbooks added 2306.11644 §2.1. Card: Textbooks Are All You Need.
• E5 pretrains on CCPairs, about 270M text pairs from forums, encyclopedias, papers and web pages, kept by a consistency filter 2212.03533 §3. Card: Text Embeddings by Weakly-Supervised Contrastive Pre-training.
• Word-similarity sets for six Indian languages start from two English lists, RG-65 and WordSim-353. Three native speakers translate the pairs, and eight native speakers rate each pair from 0 to 10, averaged 2609.34138 §4.1 §4.2. Inter-annotator agreement is reported as Fleiss' kappa per language, from 0.756 (Tamil) to 0.887 (Urdu) 2609.34138 tab. 1. The paper does not say how the 97 to 163 pairs per language were picked from the base lists 2609.34138 §4. Card: Word Similarity Datasets for Indian Languages.
• Carbon accounting for Noor counts the data stage as a cost: the 210 TB of raw dumps were downloaded and discarded, and about 2 TB of curated data was kept 2610.00223 §5.1.1. Card: Carbon Footprint of Noor.

Parallel text for translation

• NLLB-200 builds training data for 202 languages from three sources: public parallel corpora, bitext mined with LASER3 sentence encoders, and backtranslation from two generators 2207.04672 §8.1.2 §8.1.3. Mined pairs are dropped below a LASER score of 1.06 or above a length ratio of 9.0. Pairs also fail on language identification and toxicity, and duplicates are removed after normalisation 2207.04672 §8.1.4. The final set covers about 18B sentence pairs in 2440 directions 2207.04672 §8.1.6. The authors report that filtering improved bilingual models and reduced hallucinated toxicity 2207.04672 tab. 27. Most of the 40,602 benchmark directions have no training data 2207.04672 §8.3.2. Card: No Language Left Behind.

Image-text pairs and vision data

• LAION-400M keeps Common Crawl alt-text pairs whose CLIP similarity is at least 0.3. It then removes duplicates with a Bloom filter over URL and alt-text. The illegal-content filter has no stated method or threshold 2111.02114 §2.1.1. Card: LAION-400M.

• LAION-5B uses the same idea at larger scale, with a CLIP similarity threshold of 0.28 for English and 0.26 for other languages. It tags NSFW and watermark content and keeps it, rather than deleting it 2210.08402 §3.1. Card: LAION-5B.

• CC12M relaxes CC3M's unimodal filters for about four times the data. Caption precision falls to 76.6% from 90.3% on a 100-example check, and the gain shows up after fine-tuning 2102.08981 §2.2 §2.3 tab. 3. Card: Conceptual 12M.

• BLIP (CapFilt) trains a captioner to write new captions for web images, then a filter removes mismatched texts, both original and synthetic. The paper describes one bootstrapping round 2201.12086 §3.3. Card: BLIP.

• DataComp fixes the training recipe and lets participants choose the pairs. Its best baseline intersects an image-similarity filter with the top 30% by CLIP score 2304.14108 §4.1. With ViT-L/14 at the largest scale, ImageNet zero-shot accuracy goes from 0.723 unfiltered to 0.792 filtered 2304.14108 tab. 3. Card: DataComp.

• Places gathers candidates from web image search, has workers confirm labels, and uses a trained classifier to find more images for rare classes 1610.02055 §2.2. Card: Places.

• VGGFace2 downloads about 1,400 images per name. A six-stage pipeline then detects faces, drops faces that a per-identity classifier scores below 0.5, removes near-duplicates, and has annotators clean doubtful cases 1710.08092 §4 §4.4 tab. 2. Card: VGGFace2.

• Omnidata renders labelled views from 3D scans, with camera and point sampling that can be steered. Its starter set has 14,601,449 images 2110.04994 §3.1 §4.2. Card: Omnidata.

• ARKitScenes records homes with a tablet's LiDAR and aligns a laser-scanner depth map to each video by rendering scans from the camera pose 2111.08897 §3.2. Card: ARKitScenes.

• RDD2022 collects road photos from six countries. Boxes are hand-drawn in Pascal VOC XML, and only the training split is labelled 2209.08538 p. 10 p. 11. Card: RDD2022.

• PubTables-1M merges header cells that the source markup over-splits, then drops tables whose text does not match the page 2110.00061 §3 alg. 1. Card: PubTables-1M.

• xView annotates satellite chips in stages: worker review, supervisory checks, and expert review against a gold set 1802.07856 §3.2 §3.3. Card: xView.

• ChartQA pairs human-written questions with machine-generated ones on real charts 2203.10244 §1 tab. 1. Card: ChartQA.

• DeepWeeds photographs eight weed species and a Negatives class in the field, with experts labelling whether each image contains a target species. Negatives are not stratified across folds, and five stratified 60/20/20 folds are released 1810.05726 § Dataset Collection § Deep Learning. Balance is approximate: Charters Towers has 434 positives and 605 negatives 1810.05726 tab. 1. The paper says an image may show several species, but does not say how such images were labelled, and the released label file has one class per row 1810.05726 § Deep Learning. Card: DeepWeeds.

• CINIC-10 fills CIFAR-10's classes with downsampled ImageNet images: 60,000 CIFAR-10 images plus 210,000 ImageNet images, drawn at 21,000 per class and split equally 1810.03505 §2. The repository README says the ImageNet part was not vetted, and the test set includes CIFAR-10 training images 1810.03505 §2. Card: CINIC-10 is not ImageNet or CIFAR-10.

• Objaverse-XL collects about 10.2M 3D objects from five web sources. It removes files with identical SHA-256 content hashes, which the authors say removes about 23M files, and does not remove near duplicates 2307.05663 §3.1 app. C. An object is removed as NSFW when at least 3 of its 12 renders score above 0.9 on a CLIP classifier. This removes 815 objects 2307.05663 §3.3. A 2-layer MLP on CLIP features flags bad renders, trained on 1.2K manual labels, at a threshold of 0.5 2307.05663 §3.3. The official repository does not include the classifiers or the deduplication code README.md. Card: Objaverse-XL.

• MovieNet gathers about 1,100 films with trailers, photos, scripts, synopses and subtitles. Scripts are aligned to the film timeline by matching dialogue to subtitles under a monotone constraint, solved by dynamic programming 2007.10937 §3.1 app. A. The character boxes come from four steps: keyframe selection, a detector trained on those labels, detector proposals cleaned by hand, then identity labels. Only the top ten credited cast members are named; everyone else falls into one other class 2007.10937 §3.2. Tags are kept only at a minimum frequency of 25, which leaves 80 action and 90 place classes 2007.10937 §3.2. The paper's counts disagree in places: trailers are 60K in the text and 68K in the genre table 2007.10937 §3.1 tab. 3. The public repository holds the toolbox, not the alignment or benchmark code . Card: MovieNet.

Face data and self-training cleaning

• WebFace260M and WebFace42M start from about 4M celebrity names, and an image search gives up to 200 images per name, in tiers whose ranking the paper does not state 2103.04098 §2. The CAST pipeline then cleans the faces in three rounds. Each round clusters each name's faces, keeps the largest cluster, and merges or drops folders by the cosine similarity of their centres. A new student then becomes the next teacher 2103.04098 §3 fig. 4. Identities that overlap the benchmarks or the paper's own test set are removed 2103.04098 §3. The result is 2M identities and 42M faces 2103.04098 tab. 2. The paper reports that WebFace42M with ArcFace reaches 97.70 on IJB-C, against 96.03 for MS1MV2 2103.04098 tab. 5. Not every benchmark improves: on MegaFace rank-1, the WebFace4M model scores below MS1MV2 2103.04098 tab. 5. The cleaning steps are not fully specified. DBSCAN's parameters and the centre definition are not stated 2103.04098 §3. Card: WebFace260M.

Speech and speaker-held-out data

• Speech Commands collects one-second word clips from volunteers, filters by file size, loudness and one worker's transcription, and assigns splits by hashing file names so they stay stable across releases 1804.03209 §4 (card TL;DR). Card: Speech Commands.
• FLEURS is n-way parallel across 102 languages. Other workers check the recordings, and the splits hold speakers out so a model cannot recognise a voice it has already heard 2205.12446 §2.1 §4.2. Card: FLEURS.
• Lyft L5 assigns each vehicle's logs to one split only 2006.14480 §3.1. Card: One Thousand and One Hours.
• Jordanian Arabic speech from public media has about 97,000 recordings, 113 h in total, of which 52 h are transcribed 2609.24410 §6.4. Audio is split at silences, resampled to 16 kHz, and kept only between 1 s and 20 s. Initial transcripts come from a commercial speech API and are corrected by hand 2609.24410 §6.2 §6.3. The paper gives 37,000 labelled records in one place 2609.24410 §6.4.2 and 52 h elsewhere, and states no confidence filter for pseudo-labels 2609.24410 §7.3. Over five noisy-student rounds, word error rate on the authors' evaluation set fell from 0.568 to 0.515 2609.24410 tab. 4. Each round is chosen on the same score it reports, and the paper gives no dataset licence or data link 2609.24410 §9 § Data availability. Card: Jordanian speech-to-text.

Robot demonstrations

• DROID records 76k successful teleoperated Franka demonstrations across 564 scenes, on one shared hardware and recording setup 2403.12945 §3.1 §3.2. About 16k unsuccessful episodes are released but not counted in the 76k 2403.12945 §3.2. A scene is counted as new only after a substantial change to the workspace. The paper merges 2,080 raw scene IDs into 564 with a heuristic it calls conservative 2403.12945 app. D. Its co-training batches are half the task's own demonstrations and half successful DROID episodes 2403.12945 §5.1. The experimental subset is the first 40K successful episodes that had language labels at training time, so a later rebuild gives a different set 2403.12945 app. F.2. Card: DROID.

Human-written, crowd-written and conversation data

• StrategyQA has crowd workers write questions from Wikipedia terms. Five pretrained solvers reject a question when at least four answer it; three fine-tuned solvers reject it when all three succeed. Decompositions are written before evidence is matched, to avoid bias 2101.02235 §3.1 §3.2. Card: Did Aristotle Use a Laptop?.
• 2WikiMultiHopQA generates questions from templates and filters them so each needs two steps. Each sample carries its evidence triples 2011.01060 §2.1 §2.2. Card: 2WikiMultiHopQA.
• DialogSum cleans and normalises dialogues from several sources. Each summary is checked by a second annotator, and the authors re-check a 10% sample of each batch 2105.06762 §2.1 §2.2 §2.3. Card: DialogSum.
• ToxicChat labels real first-turn prompts with a human-AI pipeline. Prompts scored below a moderation threshold are labelled non-toxic without human review 2310.17389 §2.2 app. D. Card: ToxicChat.
• SMHD finds diagnosed users with high-precision self-report patterns and a keyword within 40 characters, then matches controls on post count and subreddit overlap 1806.05258 §3.1 §3.2. Card: SMHD.
• Kuzushiji crops characters from 35 printed books. Class counts are imbalanced and some classes are multi-modal 1812.01718 §2. Card: Deep Learning for Classical Japanese Literature.
• OpenAssistant collects volunteer conversation trees, with quality labels and rankings of replies. Moderation uses peer labels, points and human moderators 2304.07327 §2 §3 §4. Card: OpenAssistant Conversations.
• LIMA fine-tunes on 1,000 prompt-response pairs. Sources are filtered Stack Exchange answers, wikiHow articles, writing prompts and written examples 2305.11206 §2.1 §2.2. Card: LIMA.
• ASDiv gives each problem a type and grade level, and scores corpus variety with a BLEU-based lexical diversity measure 2106.15772 §3 tab. 2. Card: A Diverse Corpus for Math Word Problem Solvers.
• Small-model RL on maths curates 39,659 questions from two sources by dropping trivial and noisy items with small models 2503.16219 §2. Card: Reinforcement Learning for Reasoning in Small LLMs.
• Humanity's Last Exam drops questions that frontier models already answer, and keeps a private set to check for tuning on the public one 2501.14249 §3.1 §3.2 app. B.1. Card: Humanity's Last Exam.

Dynamic, model-in-the-loop collection

• Dynamically generated hate-speech data runs four rounds. Annotators write posts that fool the current target model, and later rounds mix in label-flipping perturbations of earlier posts 2012.15761 §5. Each round is split 80/10/10, and about half of each test set comes from annotators unseen in training 2012.15761 §5. The in-loop error rate falls from 54.7% in round 1 to 27.7% in round 4 2012.15761 tab. 3, and HateCheck accuracy rises from 60% to 95% (author-reported) 2012.15761 §6.3. The authors say later rounds are harder, so the falling rate understates the change 2012.15761 §6.1. Pitfalls: 18 perturbations do not flip the label, which the authors found after publication 2012.15761 §5. The v0.2.3 release removes duplicates, all of them in round 1 README.md. Card: Learning from the Worst.

Model-generated instruction data

• Self-Instruct starts from 175 hand-written seed tasks and prompts a base GPT-3 davinci to write new instructions, then inputs and outputs. The result is about 52K instructions and 82K instances 2212.10560 §2 tab. 1. New instructions are kept only if their ROUGE-L against every existing task is below 0.7, and text filters remove malformed items 2212.10560 §2.2. Fine-tuning GPT-3 on the result raises zero-shot SuperNI ROUGE-L from 6.8 to 39.9, against 40.8 for InstructGPT-001 2212.10560 tab. 3. Only 54% of 200 sampled instructions had all fields judged valid, by one author 2212.10560 tab. 2. Card: Self-Instruct.

Measuring and documenting the data

• Learning curves, not single scores. The Transformer training notes argue for reporting full BLEU learning curves against wall-clock time, not one best score 1804.00247 §2.1. Card: Training Tips for the Transformer.
• Documentation. C4 is documented from three views: where text comes from, what it says, and what was removed. The authors recommend reporting domains, checking benchmark contamination, measuring social bias, and reporting excluded content 2104.08758 §1 §6. Benchmark examples occur verbatim in C4 at rates from 1.87% to 24.88% of generation targets 2104.08758 §4.2 tab. 2.
• Data documentation for the Pile covers bias, profanity and consent issues 2101.00027 §6.
• Fairness benchmark datasets. A survey of 15 tabular datasets admits a dataset only if it is public, sits in a known repository and was used in at least three fairness papers 2110.00530 §2.1. For each one it learns a Bayesian network to show links from the protected attribute to the label, then scores one logistic-regression fit on a single 70/30 split 2110.00530 §2.2 §4.1. The network is read as a screen for possible bias, not as a causal result 2110.00530 §2.2. Several zero fairness gaps come from constant predictions: Credit card clients and Diabetes never predict the positive class, and OULAD always does 2110.00530 tab. 15. Card: A survey on datasets for fairness-aware machine learning.
• Licence fields. LAION-400M does not state its dataset licence, but keeps per-image Creative Commons tags in the metadata 2111.02114 §2.
• Auditing a corpus by its concepts. A sparse autoencoder on one reader model's activations gives each document a vector of named concept latents, max-pooled over tokens. The authors use these vectors to diff two datasets, find correlated concepts, cluster by chosen properties and retrieve documents by property 2512.10092 §3 §4. Author-reported: the SAE diff costs 2 to 8 times fewer tokens than LLM hypothesis baselines, but only about 2 times for a single two-dataset comparison 2512.10092 §4.1 app. D.4. Its results depend on the SAE's labels, and texts must fit a 2048-token limit 2512.10092 §6 §4 fn. 1. Card: Interpretable Embeddings with Sparse Autoencoders.
• Reading samples from web-mined corpora. A manual audit of 205 language-specific subsets from five public corpora (CCAligned, ParaCrawl v7.1, WikiMatrix, OSCAR and mC4) reads up to 100 lines per language, with one rater per line 2103.12028 §4.1. Macro C (correct) shares range from 23.74% (WikiMatrix) to 87.21% (OSCAR), and 15 subsets contain no correct sentence, 87 are under 50% correct 2103.12028 tab. 3 §4.2 §7. The authors also found 83 nonstandard language codes overall, and at least 48 of the affected corpora are entirely spurious 2103.12028 §5 §7. Their recommendation is to read about 100 sentences and check the codes before use 2103.12028 §7. Card: Quality at a Glance.
• Describing rare cases. A position paper on automated driving argues that corner cases need a machine-readable description, so datasets can be checked for how many they contain. It builds on existing scenario languages and gives three conceptual examples: glare, unknown objects and occlusion. It defines no schema, labelling procedure or severity metric, and reports no experiments 2109.09607 §1 §3 §4 §5. Card: Description of Corner Cases in Automated Driving.

When to use which

Choose by constraint, not by which paper is best known.

Implementation sketch

The steps below combine the cards. They are not one paper's algorithm. Where a card gives a threshold or a rule, the step names it.

# Curation pipeline assembled from the cards (not one paper's algorithm)
# Input:  sources (each with domain, date, licence), scorer, gold set
# Output: kept records, a held-out split, and a log of what was removed

def curate(sources, scorer, threshold, gold):
    # 1. Keep provenance with every record  [arxiv:2104.08758 §6]
    records = [with_provenance(r) for s in sources for r in s.fetch()]
    log = {"rule": [], "model": [], "duplicate": []}
    # 2. Rule filters: count what they remove  [arxiv:2201.06642 §5]
    kept = []
    for r in records:
        if rule_fails(r):
            log["rule"].append(r)
        else:
            kept.append(r)
    # 3. Model filter with a stated threshold  [arxiv:2210.08402 §3.1]
    kept = [r for r in kept if scorer(r) >= threshold]
    # 4. Deduplicate; say whether matching is exact or near
    #    [arxiv:2101.00027 app. D]
    kept, log["duplicate"] = dedup(kept)
    # 5. Label by people, and accept a batch only if it passes the gold set
    #    [arxiv:1802.07856 §3.3]
    batches = label(kept)
    accepted = [b for b in batches if passes_qc(b, gold)]
    # 6. Split by group (speaker, vehicle, source), not by row
    #    [arxiv:2205.12446 §2.1]
    train, test = group_split(accepted)
    # 7. Publish the counts and the removed-content log with the data
    #    [arxiv:2104.08758 §6]
    return train, test, log

Contradictions

• Keep noisy pairs or filter them. CC12M keeps a looser pool and reports lower caption precision (76.6% against 90.3%), yet its fine-tuned results improve 2102.08981 §2.3 tab. 3. BLIP removes mismatched web text and its synthetic captions with a learned filter 2201.12086 §3.3. DataComp reports that filtering beats raw size at fixed compute 2304.14108 §4.1. The papers use different models and tasks, and the wiki does not resolve which holds in general.
• Size or curation for instruction data. LIMA argues that about 1,000 curated examples are enough to teach style, and that quality and variety matter more than count 2305.11206 §5. OpenAssistant releases about 161,443 messages and reports gains from tuning on them 2304.07327 tab. 1. The two use different base models and evaluations.
• Whether filtering helps a language model at all. GottBERT reports no clear effect from its filters 2012.02110 tab. 3. The phi-1 ablation reports a gain from a classifier-based code filter at 350M parameters 2306.11644 §2.1. These are different data and scales.
• Whether automatic filters fix quality. On CCAligned, a Transformer language-ID filter applied to the corpora below 50% correct raised median precision from 13.8% to 43.9%, but cut recall by 77.5% 2103.12028 §4.3. The audit's conclusion is that automatic LangID filters are not a fix 2103.12028 §7. DataComp reports that filtering beats raw size at fixed compute 2304.14108 §4.1. The filters, languages and metrics differ, so neither result settles the question.

Papers

• Places: An Image Database for Deep Scene Understanding: scene-centric photo collection built from web search with human confirmation.
• VGGFace2: face dataset built by a staged filtering pipeline for pose and age variety.
• xView: satellite object-detection dataset with a three-stage quality check.
• Training Tips for the Transformer Model: evaluation by learning curves rather than single scores.
• Speech Commands: one-second word clips with filtering and hash-based splits.
• SMHD: Reddit diagnosis dataset with matched control users.
• Deep Learning for Classical Japanese Literature: character-image datasets with imbalanced classes and domain transfer.
• CamemBERT: French pretraining, with data-size and data-source ablations.
• One Thousand and One Hours: self-driving motion dataset from one route.
• 2WikiMultiHopQA: template-built multi-hop QA with evidence triples.
• GottBERT: German pretraining on OSCAR, with and without filtering.
• The Pile: 22-source English corpus with documentation and partial dedup.
• Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection: four rounds of model-in-the-loop hate-speech data with label-flipping perturbations.
• Did Aristotle Use a Laptop?: implicit-reasoning QA with model-in-the-loop filtering.
• Conceptual 12M: relaxed-filter web image-caption pairs.
• WebFace260M: web face dataset cleaned by CAST self-training, with the FRUITS time-limited test.
• Quality at a Glance: manual audit of 205 web-crawled multilingual subsets.
• Documenting Large Webtext Corpora: documentation and audit of C4.
• DialogSum: spoken dialogue summarisation dataset with annotation checks.
• A Diverse Corpus for Math Word Problem Solvers: ASDiv, with a lexical diversity measure.
• PubTables-1M: table dataset with canonicalised markup labels.
• Omnidata: steerable renderer for multi-task vision data from 3D scans.
• LAION-400M: CLIP-filtered 400M image-text pairs.
• ARKitScenes: mobile LiDAR RGB-D indoor dataset.
• MovieNet: film dataset with trailers, scripts and manual character, scene and action labels, plus nine tasks.
• A survey on datasets for fairness-aware machine learning: 15 tabular fairness datasets with per-dataset Bayesian-network and fairness scores.
• End-to-end Jordanian dialect speech-to-text self-supervised learning framework: Jordanian Arabic speech from public media, with noisy-student pseudo-labelling.
• Data Collection and Quality Challenges in Deep Learning: survey of data-centric collection, cleaning and fairness methods.
• Cleaner Document-Oriented Multilingual Crawled Corpus: OSCAR 22.01 with document-level language labels.
• BLIP: bootstrapped captions and filtering for image-text pretraining.
• ChartQA: chart QA benchmark with human and generated questions.
• OPT: open decoder-only LMs with a deduplicated web and book mix.
• FLEURS: 102-language parallel speech benchmark with held-out speakers.
• Minerva: maths-heavy continued pretraining with LaTeX kept intact.
• RDD2022: multi-country road damage image dataset.
• LAION-5B: 5.85B CLIP-filtered image-text pairs with safety tags.
• Text Embeddings by Weakly-Supervised Contrastive Pre-training: E5, trained on consistency-filtered text pairs.
• Pythia: training suite on the Pile and a deduplicated Pile.
• OpenAssistant Conversations: volunteer conversation trees with rankings.
• DataComp: benchmark for choosing image-text training data.
• PaLM 2 Technical Report: multilingual, deduplicated pretraining mixture with toxicity control.
• LIMA: 1,000 curated examples for instruction tuning.
• Self-Instruct: model-written instruction and instance data from 175 seed tasks, with ROUGE-L and heuristic filters.
• Textbooks Are All You Need: filtered code plus synthetic textbooks for a small code model.
• ToxicChat: real user prompts with a human-AI labelling pipeline.
• Word Similarity Datasets for Indian Languages: Annotation and Baseline Systems: six translated and human-rated word-similarity sets, with baselines for three languages.
• CINIC-10 is not ImageNet or CIFAR-10: CIFAR-sized benchmark with added ImageNet images and equal splits.
• DeepWeeds: field-collected weed photos with a Negatives class and five folds.
• Phi-3 Technical Report: small model from filtered web and synthetic data.
• Humanity's Last Exam: expert-written benchmark filtered against frontier models.
• Reinforcement Learning for Reasoning in Small LLMs: curated maths problem set for small-model RL.
• A Holistic Assessment of the Carbon Footprint of Noor: full project accounting, including data storage and processing.
• BLOOM: 176B multilingual model trained on the ROOTS corpus.
• Description of Corner Cases in Automated Driving: position paper on machine-readable corner-case descriptions.
• No Language Left Behind: 202-language translation data from mined and backtranslated bitext.
• Objaverse-XL: 10.2M web-crawled 3D objects with hash dedup and render filters.
• DROID: in-the-wild robot demonstrations with co-training and calibration.
• Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit: SAE concept latents for dataset diffing, correlations, clustering and retrieval.