Deep Transfer Network with Joint Distribution Adaptation: A New Intelligent Fault Diagnosis Framework for Industry Application

In plain words

Machines such as wind turbines and gearboxes are monitored with sensors, and a model learns to tell healthy operation from a specific fault. A model trained under one set of conditions, such as one load or one fault size, often does badly when the machine runs under different conditions, because the sensor signals look different. Collecting enough labelled faults for every condition is usually expensive.

This paper trains a neural network on labelled data from one condition and then adapts it to new, unlabelled data from another. The adaptation pushes the features of the two data sets closer together. It does this for the overall feature distribution and, using the network's own predicted labels, for each fault class separately. The authors report that matching each class separately works better than matching the overall distribution alone. The results are on three fault data sets and are the authors' own.

TL;DR

• A method for transfer learning in fault diagnosis: a CNN is pre-trained on labelled source data, then fine-tuned with unlabelled target data and a domain-adaptation penalty §III-B alg. 1.
• The penalty is joint distribution adaptation (JDA): a maximum mean discrepancy (MMD) between the marginal feature distributions, plus class-wise MMDs computed on target pseudo-labels §III-A eq. 10.
• Ten transfer tasks on wind turbine, bearing and gearbox data. DTN with JDA averages 97.5%, the plain CNN 80.0%, DTN with marginal-only adaptation (MDA) 87.4% tab. III.
• All numbers are author-reported. The method has no official code.

Problem

Fault diagnosis models are usually trained and tested on data from the same distribution §I. The authors argue this rarely holds in industry, where operating conditions, fault severities and fault types change, so a model often has to be retrained from scratch §I. Their goal is to reuse a model learned from rich labelled source data for a new but similar target task that has few or no labels §I §III-A.

The framing assumes that the two domains differ in their marginal distributions and in their conditional distributions, P(Y|X). The authors argue that matching only the marginals can leave the class structure misaligned §III-A fig. 2. They assume the source and target label spaces are the same (Y_s = Y_t) §III-A. The paper does not give a test for when this assumption holds.

Method

Notation follows the paper: source data (X_s, Y_s), unlabelled target data X_t, and a CNN with l layers §III-A. Features are taken from the last hidden fully connected layer. The mapping phi(x) = h_{l-1}(x) is the output of the first l-1 layers §III-B.

1. Base CNN §II-A eq. 1 eq. 2.

1. A CNN applies convolutions, ReLU, pooling and fully connected layers. The convolutional layer output is given by the expression in eq. 1.
2. The base network is trained on the labelled source data with cross-entropy loss §III-B eq. 11. The paper writes the generic training objective in eq. 2 but does not state a specific optimiser for this step §III-B.

2. Marginal and conditional discrepancy §III-A eq. 3 to eq. 9.

1. The distance between two data sets is measured with MMD, computed in a reproducing kernel Hilbert space with a nonlinear feature map eq. 3.
2. Marginal distribution adaptation (MDA) minimises the MMD between the source and target marginals of the feature map eq. 4 eq. 6.
3. The conditional distribution cannot be computed directly, because target labels are unknown eq. 7 eq. 8. The paper approximates it with pseudo-labels produced by the pre-trained model §III-A.
4. For each class c, the conditional MMD compares source samples with true label c against target samples with pseudo-label c eq. 9.

3. Joint distribution adaptation (JDA) penalty eq. 10.

1. The JDA term D_H(J_s, J_t) is the marginal MMD plus the sum of class-wise MMDs over all C classes eq. 10. The paper writes the terms without weights, so each has weight 1.
2. The full training loss is the cross-entropy loss plus lambda times the JDA term eq. 11. lambda is a non-negative regularisation weight §III-B.
3. Gradients with respect to the network weights are given in eq. 12 to eq. 15.

4. Training procedure alg. 1.

# DTN with JDA  [alg. 1]
# Input:  labelled source (x_s, y_s); unlabelled target x_t; CNN; lambda
# Output: adapted CNN and predicted target labels

def train_dtn_jda(source, target, network, lam):
    # Train the base CNN on labelled source data  [alg. 1 line 2]
    network = fit_cross_entropy(network, source)
    # Initial pseudo-labels for target samples  [alg. 1 line 3]
    target_labels = predict(network, target)
    while not converged(target_labels):
        # Features from the last hidden layer  [§III-B]
        source_feats = hidden_features(network, source.x)
        target_feats = hidden_features(network, target.x)
        # Marginal term plus class-wise terms  [alg. 1 line 6] [eq. 10]
        jda = mmd(source_feats, target_feats)
        for c in classes:
            jda += mmd(source_feats[source.y == c],
                       target_feats[target_labels == c])
        # Update weights on cross-entropy plus lam times JDA  [eq. 11]
        network = sgd_step(network, cross_entropy + lam * jda)
        # Re-predict target pseudo-labels  [alg. 1 line 8]
        target_labels = predict(network, target)
    return network, target_labels

The paper stops the loop when the pseudo-labels no longer change or when the training has converged alg. 1 line 9. It does not define the convergence test. Training uses mini-batch SGD with backpropagation §III-B.

5. Classifiers for the baselines §III-D.

• Shallow baselines (SVM, RF, TJM, TCA, JDA) use 29 statistical features per signal §III-D. TJM, TCA and JDA then train an SVM or RF on the adapted features §IV-A.
• EMD uses the energy of the first five intrinsic mode functions as features §III-D.
• The "DTN with MDA" baseline removes the class-wise term from the objective §III-D.

Hyperparameters

Results

Author-reported. Setup: three mechanical fault data sets, ten transfer tasks in total §IV-A. Accuracy is on the target test samples. Each task has a 4000-sample test set tab. I tab. II.

Data and tasks.

• Wind turbine, ten machine conditions (labels 0 to 9), six tasks A to F, with source and target defined by load tab. I §IV-A.
• Bearing (Case Western Reserve University, public), four conditions (labels 0 to 3), two tasks G to H, differing by fault diameter (0.18 mm and 0.53 mm) tab. II §IV-A.
• Gearbox, three conditions (labels 0 to 2), two tasks I to J, with fault types that differ between sets tab. II §IV-A.

Main result tab. III. Accuracy (%) per task, best row in bold in the paper.

Other reported results.

• DTN with JDA is best on all ten tasks in tab. III and is the paper's headline result §IV-B.
• The authors describe the improvement over the baseline CNN as 17.5% §IV-B, which equals the difference of the two averages in tab. III.
• Task H→G: DTN with MDA reaches 63.7%, below the 70% of the CNN without transfer. DTN with JDA reaches 100% §IV-B fig. 10 tab. III.
• The authors argue that JDA lets the loss converge more smoothly than MDA. In fig. 8 (JDA) the transfer loss falls steadily; in fig. 9 (MDA) it spikes around iteration 300 and stays noisy, with test accuracy near 87% §IV-D.
• Visualisations with t-SNE fig. 5 to fig. 7 show that same-class samples from source and target lie closer together after DTN with JDA.

Paper vs code

No official code.

The paper names no code repository, and its arXiv metadata has no code link. A GitHub repository search was run for "joint distribution adaptation fault", "deep transfer network fault diagnosis" and "DTN JDA transfer". It found four repositories by other owners, with no license, and none by the authors. They were not used. No discrepancies can be checked.

Pitfalls

• Overclaim in the text. §IV-B says DTN with JDA is "almost over 95%" on all tasks §IV-B. tab. III lists 94.3% for F→E, below 95.
• Undefined per-class MMD. Eq. 9 needs both source and target samples for each class. The paper does not say what happens when no target sample has a given pseudo-label, or when the division by the class counts is zero §III-A eq. 9.
• Pseudo-label errors feed the penalty. The class-wise term depends on pseudo-labels that start from the base CNN. The authors note that the initial labels contain mistakes and that iteration can improve them, but they give no error analysis §III-A. The authors note that in H→G the pre-trained network sends samples to the wrong class, which hurts MDA §IV-B fig. 10.
• Weights of the JDA terms. eq. 10 sums the marginal and class-wise terms with weight 1 each. Only the overall lambda in eq. 11 scales them. The paper does not discuss different relative weights.
• Choice of lambda. The text says lambda was "set by searching" a grid §IV-A. It does not say whether this used labelled target data, a validation split or the reported test set. Treat the reported numbers as optimistic until this is clear.
• Baseline tuning. The SVM penalty and kernel parameter are fixed at 8 "empirically" §IV-A, while the DTN lambda is searched. The comparison is not tuned evenly across methods.
• Training-loss spikes. The paper attributes the MDA spike to a conflict between the cross-entropy gradient and the regularisation gradient §IV-D, but gives no measurement. The gradient formulas are in eq. 12 to eq. 15.
• Data availability. Only the bearing data set is public, from Case Western Reserve University §IV-A [ref. 41]. The wind turbine data come from the authors' own platform, and the gearbox data from their own test rig §IV-A. The paper does not say whether these are released. Reproducing the wind turbine and gearbox tasks is not possible from the paper alone.
• Signal preprocessing. The two accelerometer signals are combined into one sample §IV-A, but segment length, normalisation and windowing are not stated.
• Sample-count mismatch. When source and target sizes differ, the paper says re-sampling "can be applied" but does not say which method was used §III-C.

Lineage

• Builds on: maximum mean discrepancy for distribution matching §II-C eq. 3; and JDA of Long et al. (2013) [ref. 33], which the paper extends from shallow features to deep networks §III-A.
• Compared against: SVM, RF, EMD, CNN, TJM [ref. 31], TCA [ref. 32] and JDA [ref. 33] tab. III. The MDA-only variant is the paper's own ablation.
• Superseded by: not stated.