wikillm

Paper · 2026-10-15 · cs.CL, cs.AI

ESG-Kor: A Korean Dataset for ESG-related Information Extraction and Practical Use Cases

ESG-Kor: 118,946 Korean sentences from company sustainability reports, labelled E, S, G, R or I by three annotators against ESG rating-agency rules. Fine-tuned KoBERT, DistilKoBERT, KB-ALBERT and KLUE-RoBERTa reach 85.87% (four-class) and 86.67% (binary) accuracy, best for KLUE-RoBERTa-large (author-reported). Two use cases: SME sentences and MSCI key-issue retrieval.

Text classificationLanguage model pretrainingDataset curationdatasettext-classificationinformation-extractionkoreanesg

arXivPDFCodeMarkdown10 min read

AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Companies publish long sustainability reports that describe their environmental, social and governance (ESG) activities, mostly as plain text with no standard format. This paper builds a labelled Korean dataset so that a language model can find the ESG-related sentences in such reports and sort them into categories. The dataset is about 119 thousand sentences, each marked by people who followed the rules that ESG rating agencies publish.

The authors fine-tune four Korean BERT-style models on the labelled sentences and show that they sort sentences well. They then test two uses: the same kind of classifier applied to reports from small and medium companies, and a search that pulls out the sentences in one company's report that match ESG key issues.

The dataset is the main contribution. The model results show it is usable, but they are not a strong benchmark comparison, and the paper's own descriptions of its setup leave several details open (see Pitfalls).

TL;DR

  • Dataset: 118,946 manually labelled Korean sentences from sustainability reports of 17 companies in seven sectors, with five labels E, S, G, R, I §3.1 tab. 1 tab. 3.
  • Four Korean BERT-style models fine-tuned for classification reach 84.19 to 85.87% four-class and 85.20 to 86.67% binary accuracy; KLUE-RoBERTa-large is best (author-reported) tab. 4.
  • Use cases: 86.72% accuracy on 403 SME sentences, and a human score of 4.67 out of 5 for retrieved key-issue sentences (author-reported, 10 raters).

Problem

ESG information is written as free text in reports that have no standard format, so finding the relevant sentences by hand is slow §1. The authors say that pre-trained language models for specialised domains need domain datasets, that most such datasets are English, and that Korean ESG data did not exist at this scale §1 §2. The paper does not measure the cost of manual extraction; it gives only the labelling effort (about 8 to 9 months for the dataset) §3.3.

Method

Data collection §3.1. Sustainability reports (173 volumes, PDF) were collected from Korean companies, 2002 to 2021 §3.1. Table 1 lists 17 companies in seven sectors, spanning manufacturing, finance and IT tab. 1. The text mentions "53 sectors" as the source pool, but only these seven sectors appear in the dataset §3.1.

Preprocessing §3.2. Only the text was taken from the PDFs (tables, images dropped). Sentences were split with the Korean sentence splitter KSS. Numbers and special characters were normalised, duplicates removed, and spelling checked with py-hanspell. Badly split, very long or very short sentences were fixed or deleted by hand §3.2.

Labelling §3.3. Five labels: E (environmental), S (social), G (governance), R (ESG-related but not clearly one of E, S or G, or mixing them), I (not ESG) §3.3. The labelling rules were taken from ESG rating providers (MSCI, Refinitiv) and Korean agencies (KCGS) §3.3. Three annotators labelled each sentence in two stages: clear sentences first, then disputed ones discussed. A sentence any annotator disagreed on after discussion was deleted, which removed fewer than 0.1% of sentences §3.3.

Dataset statistics §3.4. Counts by class and sector are in tab. 3: E 19,789; S 45,782; G 8,819; R 5,704; I 38,852; total 118,946. Token lengths (counted by a tokenizer the paper does not name) have maximum 233, minimum 3, mean 42, standard deviation 20 §3.4. Length distribution is in fig. 3; class distribution by sector is in fig. 2.

Classification model §4.1. A BERT-style encoder produces a vector C for the [CLS] token. A classification layer W and softmax give class probabilities eq. 1. Training minimises cross-entropy eq. 2.

Pseudocode
# Fine-tune a pre-trained encoder on ESG-Kor  [§4.1] [eq. 1] [eq. 2]
# Input:  sentences (list of text), labels (class index per sentence)
# Output: fine-tuned encoder and classification layer

def fine_tune(sentences, labels, encoder, head):
    # Split 70 / 15 / 15 into train, validation and test  [§4.1]
    train, valid, test = split_70_15_15(sentences, labels)
    for batch in batches(train, size=64):  # batch size 64  [§4.1]
        # Tokenize, truncating to 256 tokens  [§4.1]
        tokens = tokenize(batch.text, max_length=256)
        # Take the [CLS] vector C from the encoder  [§4.1]
        cls_vector = encoder(tokens)[CLS]
        # Class probabilities as softmax(C W^T)  [eq. 1]
        probs = softmax(head(cls_vector))
        # Cross-entropy against the true class  [eq. 2]
        loss = -log(probs[batch.label])
        # AdamW update, epsilon 1e-08  [§4.1]
        update(encoder, head, loss)
    return encoder, head

Task definitions §4.2. Four-class: E, S, G and I; R is excluded. Binary: E, S, G and R merged into "relevant" against I. The paper calls the four-class task "multiclass" in places (§4.2, §6). Sector-specific runs were not done for IT, which is included only in the overall evaluation §4.2.

Use case 1: SME classification §5.1. Sentences from the 2022 reports of three SMEs (403 sentences, labelled with the same process) were classified for ESG relevance by a model trained on ESG-Kor (binary) §5.1 tab. 5.

Use case 2: ESG key-issue extraction §5.2. The paper uses MSCI's key issues: 10 main issues and 33 detailed issues across E, S and G §5.2 app. A.2.

Pseudocode
# Retrieve sentences for each ESG key issue from one report  [§5.2]
# Input:  report sentences, key issues (main issue -> detailed issues)
# Output: up to 10 sentences per main issue

def extract_key_issues(sentences, key_issues, classify, embed):
    # Keep sentences classified as ESG-related; labels used: not stated [§5.2]
    related = [s for s in sentences if classify(s) in ESG_LABELS]
    sentence_vecs = [embed(s) for s in related]
    results = {}
    for main_issue in key_issues:
        # Most similar sentence for each detailed issue, by cosine  [§5.2]
        chosen = [
            best_match(sentence_vecs, embed(detail))
            for detail in main_issue.details
        ]
        # Fill up to 10 with the next most similar to the main issue  [§5.2]
        results[main_issue.name] = fill_top_k(
            chosen, sentence_vecs, embed(main_issue), k=10
        )
    return results

Hyperparameters

NameValueWhereNotes
Batch size64§4.1
Maximum length256 tokens§4.1Longest sentence is 233 tokens §3.4; tokenizer for that count not stated
OptimizerAdamW§4.1weight decay, betas, scheduler, warm-up: not stated
Adam epsilon1e-08§4.1
Learning rate, KoBERT1e-04§4.1
Learning rate, DistilKoBERT5e-04§4.1
Learning rate, KB-ALBERT5e-05§4.1
Learning rate, KLUE-RoBERTa5e-05§4.1Text names one "KLUE-RoBERTa"; the large variant's rate is not stated separately
Split70% train / 15% validation / 15% test§4.1stratification and seed: not stated
Epochsnot statedearly stopping and checkpoint choice: not stated
Losscross-entropy over classeseq. 2classification-layer bias and dropout: not stated
Hardware4 × NVIDIA GeForce RTX 3090§4.1
Sentence encoder (use case 2)KoSimCSE, RoBERTa backbone§5.2checkpoint and pooling: not stated
Retrieved sentences (use case 2)10 per main issue§5.2

Model sizes and elapsed times, from tab. A.1:

ModelParameters4-class time (s)2-class time (s)
KoBERT92M28122944
DistilKoBERT28M809839
KB-ALBERT9M14091424
KLUE-RoBERTa111M31333286
KLUE-RoBERTa-large337M94929965

Results

Author-reported. Setup: the ESG-Kor split above, classification accuracy in percent, four-class (E, S, G, I) and binary (relevant vs I) tasks. Columns are the sector subsets (Auto = automotive, Air = aviation, Elec = electronics, Fin = finance, Heavy = heavy industry, Steel, IT omitted from sector runs) and Full, the whole dataset tab. 4 §4.2. Per-sector columns are as printed; the paper does not say whether the per-sector numbers come from separate models or from one model's per-sector test subsets (see Pitfalls).

TaskModelAutoAirElecFinHeavySteelFull
Four-classKoBERT84.1691.6282.6887.0390.6085.8484.70
Four-classDistilKoBERT83.5390.7581.9886.3390.6485.3384.19
Four-classKB-ALBERT83.8891.1582.2886.5791.1784.6984.59
Four-classKLUE-RoBERTa-base84.3592.1682.9787.8391.4386.1485.27
Four-classKLUE-RoBERTa-large84.5992.6483.8288.2891.5687.3085.87
BinaryKoBERT84.7091.4284.5788.7190.9585.8886.11
BinaryDistilKoBERT84.3690.5684.2487.5392.6784.7385.20
BinaryKB-ALBERT84.2090.3884.0887.8392.0883.8485.69
BinaryKLUE-RoBERTa-base85.2992.0785.4888.6093.1685.2386.30
BinaryKLUE-RoBERTa-large85.9692.5186.1288.7893.5486.1086.67

Use case results, author-reported:

SettingDataResultSource
SME binary classification403 sentences (254 relevant, 149 irrelevant)86.72% accuracytab. 5 §5.1
Key-issue retrieval, human rating10 raters, Hyundai Construction report, 1 to 5 scalemean 4.67; per-issue 4.4 to 4.9tab. 6 §5.2

Paper vs code

The paper links https://github.com/nowzer0/ESG-Kor §1. The README names the Hanyang University group as creator, so the repository is treated as the authors' README.md. At the pinned commit it holds a README, an AFL-3.0 LICENSE and two archives, ESG_Kor/Multiclass.zip and ESG_Kor/Binary.zip, of roughly 8 MB each (directory listing via the GitHub API). No model, training or configuration code is in the tree, so the hyperparameters above cannot be checked against code. The archives were not opened while compiling this card, so their contents (labels, splits, any scripts) are unverified.

Discrepancies between the paper and the README:

  • The README describes the multi-class task as E, S or G only, and the binary task as ESG relevance README.md. The paper's four-class task includes I and excludes R §4.2 tab. 4. The README's wording does not settle which label set the archives use.

Pitfalls

  • Label set. The four-class task (Table 4) is E, S, G and I with R removed; §4.2 describes "multiclass" as E, S and G only. The paper does not say whether R sentences are removed from both train and test, or only from the label space. Assume removal from both and state it in your code.
  • Binary grouping. Relevant is E, S, G and R together, against I §4.2.
  • Split details. The 70/15/15 split is not said to be stratified, seeded or repeated. The paper does not say which split Table 4 uses; the test split is the natural reading but is not stated. Each cell is one run; no variance is reported.
  • KLUE-RoBERTa-large learning rate. §4.1 gives one rate, 5e-05, for "KLUE-RoBERTa"; Table 4 reports base and large separately. Using 5e-05 for the large model is an inference, not something the paper states.
  • Training schedule. Epochs, warm-up, scheduler, weight decay, dropout, early stopping and checkpoint selection are not stated. Results cannot be reproduced exactly from the card.
  • Sequence length. Training uses maximum length 256 §4.1; the 233-token maximum §3.4 fits only if the same tokenizer is used. The paper's argument that 512 is a common limit §3.4 does not change the 256 used for training.
  • Claim vs table, KB-ALBERT. §4.2 argues that KB-ALBERT, with fewer parameters, is at least comparable to KoBERT and DistilKoBERT. In Table 4's ESG-Kor column KoBERT is higher than KB-ALBERT in both tasks (84.70 vs 84.59 four-class; 86.11 vs 85.69 binary). The claim holds only against DistilKoBERT. The paper's explanation (KB-ALBERT is pre-trained on financial text) is the authors' interpretation and is not tested here §4.2.
  • Claim vs table, parameter count. §4.2 says models with more parameters performed better. The numbers in tab. A.1 and tab. 4 do not follow a single trend: KB-ALBERT (9M) is above DistilKoBERT (28M) in both tasks. Treat the statement as loose.
  • SME accuracy is not a whole fraction. 86.72% of 403 sentences is 349.5 correct, which is not possible. 349/403 is 86.60% and 350/403 is 86.85%. The paper does not explain this (averaging over runs would explain it, but is not stated) tab. 5 §5.1.
  • Sector experiments. §4.2 says "sector-specific experiments". It is not stated whether a separate model was trained per sector or one model was evaluated on each sector's test subset. This changes what the per-sector columns mean.
  • Sector count. §3.1 mentions 53 sectors as the pool; tab. 1 lists seven sectors and 17 companies. The 53-sector figure is not explained.
  • Label quality. Three annotators, rules from third-party agencies, and disputed sentences dropped, but no inter-annotator agreement is reported §3.3. The paper itself calls the R class and borderline sentences the hard cases §3.3; no measurement of label reliability is given.
  • Key-issue selection is ambiguous. §5.2 says the top sentences were chosen for each detailed key issue, then combined by main issue into 10 sentences, with at least one top sentence per detailed issue. The text does not say how the at-least-one rule and the cap of 10 interact, or whether the 10 are per main issue; the pseudocode above follows one reading.
  • Use-case evidence is thin. Key-issue retrieval rests on one report (Hyundai Construction, 2010 to 2020) and ten raters, with no baseline retriever §5.2. The SME test set is 403 sentences from three reports §5.1.
  • ESG relevance in use case 2. The filter keeps "ESG-related" sentences, but which ESG-Kor labels count as related (E, S, G only, or also R) is not stated §5.2.
  • Source text and licence. The sentences come from companies' sustainability reports, which are third-party copyrighted works. The paper gives no licence for the dataset text. The repository's AFL-3.0 licence is for the repository, which holds the data archives. Do not reproduce dataset sentences in your own material without checking that licence. This card reproduces none of them.
  • Missing from the paper. No comparison with LLMs (acknowledged as a limitation § Limitation), no ablation of the labelling stages, and no baseline for the two use cases.

Lineage

  • Builds on: BERT-style encoders with the Korean models KoBERT, DistilKoBERT, KB-ALBERT and KLUE-RoBERTa §4.1; KoSimCSE as the sentence encoder for use case 2 §5.2; MSCI's ESG key issues, used as the retrieval targets app. A.2.
  • Compared against: the four Korean models on the same dataset only tab. 4. No external dataset or baseline is used.
  • Superseded by: not stated.

Source and license

Adapted from “ESG-Kor: A Korean Dataset for ESG-related Information Extraction and Practical Use Cases” by Jaeyoung Lee, Geonyeong Son, Misuk Kim (arXiv:2610.21368), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/nowzer0/ESG-Kor at commit 41d6f0efe71a (AFL-3.0). Any quoted code is shown under that license.