Impact of canny edge detection preprocessing on performance of machine learning models for Parkinson's disease classification

In plain words

Parkinson's disease can make handwriting shaky, so researchers study drawings of spirals made by patients and healthy people. This paper asks whether a basic machine-learning classifier can tell the two groups apart from simple numbers computed from the drawing images.

The authors prepare the images in two ways. One version is the plain drawing. The other first finds the outlines of the strokes with an edge detector, then sharpens the ridges. They also create extra training images by rotating, shifting, flipping and brightening the originals. Nine common classifiers are trained on each version, and the study records accuracy, how much memory each trained model takes, and how long a prediction takes.

The main message is that the edge-based preparation usually makes the classifiers worse. The extra images help some models a lot. Some models stay small and fast on every dataset, while others grow as the data grows. The training set is very small, and all results come from the authors' own experiments.

TL;DR

• Nine classifiers are compared on spiral drawings from the Parkinson's Image dataset, on four dataset variants p. 4 p. 8.
• Canny edge detection followed by Hessian filtering degrades most models; the augmented originals give the highest AUCs, around 0.97 for RF, XGBoost and KNN fig. 13.
• Random Forest keeps a fixed model size of about 61 KB across datasets; SVM and KNN grow to over 100 KB on the augmented data fig. 40.
• All results are author-reported, use 10-fold cross-validation and have no held-out test set p. 8.

Problem

Early Parkinson's detection from handwriting needs cheap, explainable models, but the authors say prior studies often report high accuracy with opaque feature selection p. 4. They also note that resource-constrained or edge devices need small memory and fast predictions p. 2 p. 21. This paper tests whether Canny edge preprocessing and augmentation change accuracy, model size and prediction time for classic classifiers, on a spiral-only subset of the Parkinson's Image dataset p. 4 p. 8. Its own stated limitation is that spiral drawings do not capture all PD symptoms p. 34.

Method

1. Data. Only the spiral part of the Parkinson's Image dataset is used. It has 102 spiral images, 72 for training (36 per class) and 30 for testing (15 per class) p. 7 p. 8. The paper does not evaluate on the test set; it uses cross-validation on the training images instead p. 8.

2. Four datasets. DS0 is the original 72 training images p. 8. DS1 is DS0 after Canny edge detection and Hessian filtering p. 8 fig. 5. DS2 is an augmented version of DS0, and DS3 is an augmented version of DS1 p. 8 alg. 1. Augmentation makes 71 copies per original image, 5112 in total, 2556 per class alg. 1 p. 8. The paper does not say whether the originals are kept in the augmented sets.

3. Canny edge detection and Hessian filtering (DS1, DS3). Canny runs Gaussian smoothing, Sobel gradients, non-maximum suppression, double thresholding and hysteresis p. 5 p. 6 eq. 5 eq. 6 eq. 7 eq. 8 fig. 2 fig. 3. The Hessian filter is then applied to the Canny output p. 7 p. 8. For each pixel it builds second derivatives eq. 10 eq. 11 eq. 12, forms the 2×2 Hessian eq. 9, and takes the eigenvalue used as edge intensity eq. 13 p. 7.

4. Augmentation (DS2, DS3). Each training image is rotated by up to 360 degrees, shifted by a 0.01 range in width and height, flipped horizontally and vertically, and brightness-jittered over the default range; each output is resized to 128 × 128 p. 8 alg. 1.

5. Features. Each image is summarised by eleven statistics: mean, standard deviation, skewness, kurtosis, energy, power, median, variance, minimum, maximum and RMS, indexed 0 to 10 in that order p. 9.

6. Feature selection. Recursive feature elimination (RFE) ranks features per model; features with rank 2 or lower are kept alg. 3 p. 10. Algorithm 4 describes a permutation-importance ranking as an alternative alg. 4 p. 11. The shared feature set per dataset is then chosen by reading correlation matrices, with no stated rule fig. 7 fig. 8 fig. 23 fig. 24 p. 13 p. 22. The chosen sets are DS0 [1, 2, 3], DS1 [0, 1, 4, 9, 10], DS2 [1, 2, 3, 4, 6] and DS3 [1, 2, 3, 4, 6] p. 14 p. 22.

7. Classifiers. Nine are trained with default settings from the Python library: logistic regression (LR), decision tree (DT), random forest (RF), gradient boosting (GB), XGBoost (XGB), naive Bayes (NB), k-nearest neighbours (KNN), support vector machine (SVM) and AdaBoost (AdB) p. 4 p. 11 p. 12.

8. Evaluation. Stratified 10-fold cross-validation, repeated ten times, is used for each model p. 11 p. 31. Metrics are accuracy eq. 14, AUC from the ROC curve eq. 15 fig. 12 fig. 13 fig. 28 fig. 29, F1 eq. 19, and a confusion matrix per dataset fig. 16 fig. 30. Learning curves fig. 14 fig. 15 fig. 31 fig. 32, bias–variance curves fig. 17 fig. 18 fig. 33 fig. 34, model size fig. 19 fig. 20 fig. 35 fig. 36 and prediction time fig. 21 fig. 22 fig. 37 fig. 38 are also reported.

9. Statistics. Accuracies from each model are compared across datasets with Shapiro–Wilk normality tests and Mann–Whitney U tests on N = 100 observations per model (df = 98) p. 31 p. 32 fig. 44. Cliff's delta is reported as an effect size eq. 21 fig. 45 p. 33.

Pseudocode for step 3 (Canny, then Hessian), following the paper's text and printed parameters:

# Canny edges, then a Hessian ridge measure  [p. 5-8] [fig. 5]
# Input:  grey image (H, W), intensities 0-255
# Output: edge-intensity map (H, W)

def canny_then_hessian(image):
    # Gaussian smoothing; sigma = 2.8 was chosen for DS1 and DS3  [fig. 5]
    smoothed = gaussian_smooth(image, sigma)
    # Sobel gradient magnitude and angle of the smoothed image  [eq. 6-8]
    magnitude, angle = sobel_gradient(smoothed)
    # Thin the edges: suppress a pixel that is below the sum of its two
    # neighbours along the quantised edge normal, as printed  [p. 6] [fig. 3]
    thin = non_maximum_suppression(magnitude, angle)
    # Double threshold with threshold1 = threshold2 = 51, then hysteresis
    # [fig. 5] [p. 5]
    edges = hysteresis(thin, low=51, high=51)
    # Second derivatives of the edge image  [eq. 10-12]
    ixx, iyy, ixy = second_derivatives(edges)
    # Mean curvature and the root term of the 2x2 eigenvalue formula
    # [eq. 13]
    mean_curvature = (ixx + iyy) / 2
    root = sqrt(mean_curvature**2 - (ixx * iyy - ixy**2))
    # Edge intensity is the minimum-curvature eigenvalue  [p. 7] [eq. 13]
    ridge = mean_curvature - root
    return ridge

Pseudocode for step 5 (features) and step 6 (ranking), from p. 9 and alg. 3:

# Eleven statistics per image  [p. 9]
# Output: feature vector (11,); index order 0 to 10 as listed in the text
def image_statistics(values):
    return [
        mean(values),
        std(values),
        skewness(values),
        kurtosis(values),
        energy(values),
        power(values),
        median(values),
        variance(values),
        minimum(values),
        maximum(values),
        rms(values),
    ]


# Rank features per model with recursive feature elimination  [alg. 3]
# Input:  features (N, 11), labels (N,), models (list)
# Output: per model, the indices of kept features
def rank_features(features, labels, models):
    kept = {}
    for model in models:
        # Estimator, step size and fold count are not stated  [alg. 3]
        ranking = rfe_ranking(model, features, labels)
        # Lower rank means more important; keep rank 2 or lower  [p. 10]
        kept[model] = [i for i, r in enumerate(ranking) if r <= 2]
    return kept

Pseudocode for step 8 (evaluation), from p. 11 and p. 33:

# Cross-validated evaluation of one model on one feature set  [p. 11]
# Input:  features (N, k), labels (N,)
# Output: one accuracy and one F1 per fold and repeat

def evaluate(model, features, labels):
    scores = []
    # Ten repeats of stratified 10-fold CV, per [p. 11] and [p. 31]
    for repeat in range(10):
        folds = stratified_kfold(labels, n_splits=10, shuffle=True)
        for train_idx, test_idx in folds:
            model.fit(features[train_idx], labels[train_idx])
            predicted = model.predict(features[test_idx])
            # Accuracy  [eq. 14];  F1  [eq. 19]
            scores.append(accuracy_and_f1(predicted, labels[test_idx]))
    return scores

Hyperparameters

Results

Author-reported. Setup: 10-fold cross-validation on the training images of the Parkinson's Image dataset (spiral subset), nine classifiers, default settings p. 8 p. 11. No external baseline is compared numerically.

AUC on each dataset, read from the ROC legends fig. 12 fig. 28 fig. 13 fig. 29:

Accuracy. The paper gives per-model accuracy as bar charts fig. 11 fig. 27 and as box plots over the 100 observations fig. 42 fig. 43. The pooled values in the first two columns below were computed by this card by summing the diagonal and dividing by the total in the confusion matrices of fig. 16 (DS0: 72 samples; DS2: 5112 samples). The paper does not state these numbers. The last two columns are the medians read from fig. 42 and fig. 43.

The text reports a DT accuracy of 91.7% on DS0 and an average of about 86% on DS2 p. 16. The DS1 and DS3 accuracies are in fig. 30 and fig. 27 and are not tabulated here.

Model size (memory footprint), in KB, from the summary table fig. 40:

Prediction time, in milliseconds, from the summary table fig. 41:

Statistical comparison. Accuracies are not normally distributed according to Shapiro–Wilk p. 31. In the DS2-versus-DS3 Mann–Whitney comparison fig. 44, only two of the cells have p ≥ 0.05: LR on DS2 against SVM on DS3 (p = 0.54) and NB on DS2 against XGB on DS3 (p = 0.11). All other cells are below 0.05 fig. 44 p. 32. The Cliff's delta matrix is in fig. 45.

Paper vs code

No official code. The paper links no repository, and its data-availability statement names only a Kaggle dataset p. 35. Searches: GitHub repository search with the terms "parkinson canny hessian", "Parkinson spiral canny Szczuko" and "parkinson spiral hessian" returned no results; a GitHub user search for "Sameer Bhat Gdansk" returned none. No project page is linked.

Pitfalls

• Eq. 16 is not the true positive rate. As printed, TPR = TP / (TP + FP), which is precision. The text defines TPR over all positives, which implies TP / (TP + FN). Use TP / (TP + FN) when re-implementing eq. 16 p. 12.
• Eq. 18 is a single-operating-point formula. AUC = (1 + TPR − FPR) / 2 holds only for one threshold. The paper reports AUCs from full ROC curves and does not say which value is in each legend eq. 18 fig. 12 fig. 13.
• Cliff's delta sign convention contradicts itself. Its definition says positive values mean Model 1 (DS2) outperforms Model 2 (DS3) p. 33. A later paragraph says a value of 1 means "identical performance" and calls 0 the case of equal accuracy p. 34. Read the sign as "Model 1 higher" and check against fig. 45 before relying on it eq. 21.
• Feature selection and augmentation leak into evaluation. RFE is run on the whole training set before cross-validation alg. 3, and the feature sets are chosen by looking at correlations on the same data p. 13 p. 22. This makes the reported CV accuracies optimistic. The paper does not say that augmented copies of one original image are kept in the same fold; if they are not, copies of one image can appear in both the training and validation folds alg. 1 p. 8.
• Text and figures disagree. The conclusion calls SVM one of the models that "excelled on augmented datasets" p. 35, yet the DS2 median accuracy for SVM is 0.50 fig. 42, while the confusion matrices give a pooled 0.86 fig. 16. The text gives KNN about 92% on DS2 p. 16, matching the pooled figure, while the box plot median is 0.84 fig. 42. The paper does not explain either gap.
• Canny thresholds are equal. With threshold1 = threshold2 = 51, the hysteresis step has no weak-edge range, so it reduces to one threshold. The paper describes hysteresis as a separate step p. 5 fig. 5. Whether this is intended is not stated.
• Non-maximum suppression as printed. The rule suppresses a pixel when it is below the sum of its two neighbours p. 6. Standard Canny implementations compare with each neighbour; this card follows the printed text. The angle ranges printed in p. 6 include the typo "22.5° to 22.5°"; the ranges in fig. 3 are the ones to use.
• Angle formula is ambiguous. The gradient angle is written as arctan of a ratio eq. 4 eq. 8. The paper does not say whether atan2 is used, so the quadrant is not defined.
• Hessian stencil. Eqs. 10 and 11 use central differences in each direction, but Eq. 12 is a one-sided 2 × 2 stencil with +1 offsets in both x and y eq. 12 p. 7. Do not assume they come from one formula. The eigenvalue expression eq. 13 is standard. Its radicand equals ((Ixx − Iyy) / 2)² + Ixy², which cannot be negative for a symmetric Hessian. The paper does not say whether negative edge intensities are clipped or thresholded.
• Which eigenvalue. The text says the edge intensity uses the minimum-curvature eigenvalue p. 7; this is λ2 with the minus sign eq. 13. The paper does not state a ridge threshold or any normalisation after the filter.
• Where the eleven statistics come from. The text says the statistics are computed on the images p. 9, Algorithm 3 says "extract HoG features" alg. 3, and the preprocessing flow passes "HoG images" to the feature stage p. 8. The card cannot establish which input the statistics use, so an implementer should ask the authors or treat it as unresolved.
• Gradient calculation in Eq. 7. The magnitude and angle are written in terms of φ_p and φ_q, which are the Gaussian, not the derivatives of f_s. The intended partial derivatives are likely ∂f_s/∂p and ∂f_s/∂q eq. 7 eq. 8 p. 5.
• Memory and time are not defined. The paper does not say how model size was measured (serialised file, in-memory object, or something else), nor the batch size, number of prediction calls, or warm-up used for prediction time fig. 19 fig. 21 p. 19 p. 20. The unit KB is as printed.
• Bias–variance method not stated. The decomposition (bootstrap, repeated splits, or other) is not described, so the bias, variance and MSE values in fig. 17, fig. 18, fig. 33, fig. 34 and fig. 39 cannot be reproduced from the text.
• Calibration is not applied. All nine models are used uncalibrated with default settings p. 14 p. 22. The authors note calibration could improve accuracy and leave it for future work p. 14 p. 34.
• Data and software details. The image mode is written as "CMGY format" p. 8. The Python version is printed as "3.0" p. 7. The Canny library behind the threshold parameters is not named. The dataset is from a public Kaggle source, and its licence was not checked by this card.

Lineage

• Uses data from: the Parkinson's Image dataset of Zham et al. (2017), reference 49 in the paper p. 4 p. 8. No arXiv card for this source.
• Builds on: the Canny edge detector p. 5 and Hessian ridge filtering p. 7, both cited to earlier work; no arXiv card.
• Compared against: the paper's own nine classifiers on four datasets; no external baseline numbers are given.
• Superseded by: not known from content in this wiki.