A Unified Deep Learning Architecture for Abuse Detection

In plain words

Online platforms have to catch many kinds of abusive posts: hate speech, sexism, racism, bullying, sarcasm. Most detectors are built for one of these and tuned to one platform, so they miss the others. This paper asks whether a single neural network can handle all of them.

The model reads two kinds of input about each tweet. One is the words of the tweet, turned into vectors that a pretrained word table supplies. The other is facts about the tweet and its author, such as counts of mentions, emotion scores, follower numbers and account age. Each input gets its own branch, and the two branches are joined near the end to make one decision.

Training two branches inside one network is tricky, because one branch can learn faster and drown out the other. The authors' fix is to let the branches take turns: on each batch, the whole network sees the data, but only one branch is allowed to change. The authors report that this turn-taking works best on all four datasets they tried, and that the same settings worked without retuning.

TL;DR

A single two-branch Keras model combines a GRU over pretrained word vectors with a dense network over Twitter metadata, trained with alternating updates between the branches § Deep Learning Architectures [§ Training the Combined Network]. On four Twitter abuse datasets, the authors report AUC between 0.92 and 0.98 under 10-fold cross-validation, above the baselines they compare against tab. 2. The paper's sections are unnumbered in the source, so the citations use section titles. There is no official code.

Problem

Abusive behaviour on Twitter takes several forms, and detectors are typically built for one type and one platform § Introduction. Text-only classifiers are the common approach, but the authors argue that text alone is not enough, and that per-task feature engineering is costly § Introduction § Deep Learning Architectures. They therefore want one architecture that takes text and metadata together, with no retuning per task § Introduction § Summary.

Method

The model has three parts: a text branch, a metadata branch and a classifier over their concatenated outputs fig. 2 [§ Combining the Two Classification Paths]. Each branch can also be trained alone, as a standalone classifier fig. 1 § Deep Learning Architectures.

Text branch § Text Classification Network:

1. Tweets are cut or left-padded with zeros to 30 words. The length is the 95th percentile of tweet length in the corpus. Words that occur only once are removed.
2. Each word is mapped to a 200-dimensional GloVe vector. Words missing from GloVe get random vectors, which the embedding layer then learns.
3. A GRU with 128 units reads the sequence. Recurrent dropout is 0.5. Attention is switched off for tweets.
4. The text branch outputs 128 activations, one per RNN unit.

Metadata branch § Metadata Network:

1. The raw features are numeric. Categorical features are enumerated or one-hot encoded, and the features are listed in § Dataset and [§ Feature Extraction].
2. A batch-normalisation layer standardises the batch.
3. Five dense layers of sizes 512, 245, 128, 64 and 32 form a bottleneck, with tanh activations. A sixth dense layer brings the output to 128 units, the same width as the text branch.

Combined classifier § Combining the Two Classification Paths:

Concatenate the 128-unit text output and the 128-unit metadata output into 256 features, then apply a softmax dense layer with one unit per class fig. 2.

Training § Training the Combined Network § Experimental Setup:

The paper compares four ways to train the combined model. Naive training updates all weights at once. Transfer learning pretrains each branch, freezes them, and trains a new joining layer. Fine-tuning does the same but leaves the branches trainable. The interleaved method is the one the paper proposes and reports as best on every dataset § Experimental Results tab. 2. Its procedure:

# Interleaved training  [§ Training the Combined Network]
# Input:  text (N, 30) word ids; metadata (N, M); labels (N, C) one-hot
# Output: the trained combined network

def train_interleaved(text, metadata, labels, max_epochs):
    # Two identical combined networks, A and B  [fig. 2]
    model_a = build_combined_network()
    model_b = build_combined_network()
    # Freeze the text path of A and the metadata path of B
    # [§ Training the Combined Network]
    freeze(model_a.text_path)
    freeze(model_b.metadata_path)
    for epoch_number in range(max_epochs):
        for batch_number, (text_batch, meta_batch, y) in enumerate(
            minibatches(text, metadata, labels, batch_size=512)
        ):
            # Even (batch + epoch) uses A; odd uses B
            # [§ Training the Combined Network]
            if (batch_number + epoch_number) % 2 == 0:
                active, other = model_a, model_b
            else:
                active, other = model_b, model_a
            # Full forward pass; only the unfrozen path gets gradients
            # [§ Training the Combined Network]
            train_on_batch(active, [text_batch, meta_batch], y)
            # Copy the updated weights so both models match
            # [§ Training the Combined Network]
            other.set_weights(active.get_weights())
    return model_a

Key loss: categorical cross-entropy over the whole network, optimised with Adam § Experimental Setup. There is no numbered equation in the source.

Hyperparameters

Results

Author-reported. Setup: 10-fold cross-validation; four Twitter datasets tab. 1; the same architecture and settings on each; baselines are a TF-IDF Naive Bayes model and each dataset's published method [§ Experimental Setup] § Experimental Results.

Datasets tab. 1, as printed:

WV = word vectors, TF = tweet-based, UF = user-based, NF = network-based metadata. Counts are as printed; see Pitfalls for text-versus-table differences.

Table 2 tab. 2, as printed (AUC, Acc., Prec., Rec., F1):

Metadata importance on WebSci, AUC only tab. 3:

Author-stated headline claims: the interleaved model is best on all four datasets § Classification performance; the abstract gives a 21 to 45% AUC improvement, which the body does not recompute as one figure § Abstract. On WebSci the paper itself says the comparison with Chatzakou et al. is not direct, because their results are user-level and this paper's are tweet-level § Experimental Results. On the toxic gaming dataset, the same model reaches accuracy 0.93 and AUC 0.89, which the authors say beats the easiest task in the original work § Generalizing to Other Platforms.

Paper vs code

No official code.

The arXiv abstract page lists no code link; its comments field holds only keywords. The source's commented-out line says an implementation is available at a link "redacted for double blind" § Introduction, commented in source. Searches: the arXiv abstract page; GitHub repository search for "abuse detection", "hate speech twitter metadata", "abuse detection unified architecture" and "hate speech founta" (no matching author repositories); a GitHub user search for "founta" (the matching account shows unrelated repositories). No project page was found.

Pitfalls

• Dataset counts disagree. The text gives 9,484 WebSci tweets before spam removal § Dataset, while Table 1 lists 6,091 tab. 1. The sarcasm text says about 60k tweets kept § Dataset, while Table 1 lists 61,075 tweets and 60,255 users. Use the table counts and mark the discrepancy.
• Hate dataset metadata is contradictory. The dataset section says no metadata is used beyond word vectors § Dataset. Table 1 marks tweet-based metadata as available for this dataset tab. 1, and the discussion says the classifier uses "the raw text and the tweet-based metadata" [§ Experimental Results]. Table 2's metadata-only row for this dataset (AUC 0.75) implies tweet-based features were used.
• Layer size "245". The metadata network is given as 512, 245, 128, 64, 32 § Metadata Network. The value 245 is probably a typo, but the paper does not say. Do not silently change it to 256; either flag it or test both.
• Recurrent dropout semantics. The paper says p=0.5 means each neuron is "available for activation with probability 0.5" [§ Text Classification Network]. Many frameworks define dropout as the probability of dropping a unit. Check which convention the implementation uses. The paper gives no code for this.
• Which path updates on which batch. The text says even batches update only the text path and odd batches only the metadata path [§ Training the Combined Network]. The procedure says model A (text path frozen) is used when (batch + epoch) is even, which would update only the metadata path. The two statements conflict. The pseudocode above follows the procedure (model A, frozen text path, on even parity) and the Keras-style freeze; verify against any released code before relying on it.
• Concatenation point. The 128 text activations are described as "one for each RNN unit" § Combining the Two Classification Paths. The paper does not say whether these are the last hidden state, a pooled sequence, or another reduction.
• Tanh scope and metadata details. The tanh activation is mentioned in the metadata paragraph, but whether it applies to all dense layers, and whether dropout or other regularisation applies to the metadata branch, is not stated. The definitions of sentiment, emotion and offensiveness scores are not given § Feature Extraction.
• Early stopping is underspecified. The validation-split size and which model's loss is monitored are not stated. Patience is 10 epochs [§ Experimental Setup].
• Learning rate and batch size for non-interleaved modes are not stated. Only the interleaved batch size (512) is given [§ Training the Combined Network].
• Architecture is not fully task-agnostic. The gaming experiment switches attention on § Generalizing to Other Platforms, and the paper concedes that per-task tuning could improve results § Summary. "No retuning" applies to the four Twitter datasets with the attention setting unchanged.
• Metadata features depend on external scorers. Sentiment, emotion and offensiveness scores are used but their source is not named [§ Feature Extraction]. Replicating them exactly is not possible from the paper alone.
• Cost. The authors trained on three GPUs § Experimental Setup. Interleaved training runs the whole network on every batch, but updates only one path, so each batch costs a full forward pass plus one weight copy [§ Training the Combined Network]. The paper gives no timing figures.
• No variance reported. Scores are given without standard deviations across folds tab. 2.
• Not directly comparable baselines. The WebSci comparison with Chatzakou et al. mixes user-level and tweet-level results § Experimental Results. The sarcasm experiments use the original imbalanced data, because the baseline's imbalanced results are the reference § Dataset.
• Data availability. The gaming dataset was obtained from its authors § Generalizing to Other Platforms. No public release is stated.

Lineage

• Builds on: interleaved training from Hidasi et al. 2016, which the paper cites for parallel recurrent networks in recommender settings [§ Training the Combined Network]. No arXiv id is given in the source. GloVe word vectors § Text Classification Network. Metadata features from Chatzakou et al. 2017 § Feature Extraction.
• Compared against: Naive Bayes with TF-IDF; Chatzakou et al. 2017; Waseem and Hovy 2016; Davidson et al. 2017; Rajadesingan et al. 2015 tab. 2.
• Superseded by: not known from content in this wiki.