--- kind: concept slug: random-convolutional-kernels title: Random convolutional kernels summary: >- Time series classifiers that convolve each series with many short kernels, pool every response with simple statistics and fit a linear model on the features. MultiRocket uses fixed kernels and is author-reported close to HIVE-COTE 2.0 at a fraction of its training time on 109 UCR datasets. Arsenal uses random kernels in an ensemble and handles multivariate series. tags: [time-series, classification, feature-extraction, convolution] papers: ["2102.00457", "2104.07551"] related: [convolutional-neural-network, ensembles, ensemble-learning, time-series-classification] status: draft updated_at: 2026-11-01 --- ## Overview The idea is to turn a time series into a fixed-length feature vector without training any convolution filters. The series is convolved with a bank of short kernels at several dilations. Each response is summarised by pooling operators, such as the proportion of positive values. A linear classifier is then fitted on the pooled numbers [arxiv:2102.00457 §3]. The name fits two forms of this idea. MultiRocket uses a fixed set of kernels: 84 of them, with dilations set by a formula [arxiv:2102.00457 §3.2]. Its only random choice is which training series supplies the bias values [arxiv:2102.00457 §3.2]. Arsenal, a module of HIVE-COTE 2.0, draws its kernels at random: the weights, biases, dilations and padding are all sampled [arxiv:2104.07551 §3.3, alg. 4]. The invariant the variants share: the feature extractor is built without using the labels, from the series length and either a fixed recipe or random draws. Only the final classifier is fitted to the labels [arxiv:2102.00457 §3.5] [arxiv:2104.07551 §3.3]. The MultiRocket authors argue that accuracy comes from diverse representations of the same series. HIVE-COTE 2.0 gets that diversity by combining four different modules [arxiv:2102.00457 §3.1] [arxiv:2104.07551 §3]. Arsenal gets it inside one family, from 25 members with 2,000 random kernels each [arxiv:2104.07551 tab. 2]. The page draws on two cards. ROCKET, the origin of the Arsenal design, and MiniRocket are named in these cards, but neither has a card here. This page uses ROCKET only through the HIVE-COTE 2.0 result tables. ## Variants The building blocks below are the ones the MultiRocket card describes. Each is a design choice that other variants in this family can change. The Arsenal section that follows changes the kernels and the classifier. - **Kernels.** 84 fixed kernels of length 9. Each has weight 2 at three taps and −1 at the other six. The 84 kernels are all ways to choose the three taps [arxiv:2102.00457 §3.2]. - **Dilations.** Each kernel runs at a set of dilations. Their exponents are spread evenly from 0 to a maximum that depends on the input length [arxiv:2102.00457 §3.2]. The paper's typeset maximum differs from the code's formula (see Contradictions). - **Biases.** For each kernel and dilation, the convolution runs on one random training series. Quantiles of that output become one bias each [arxiv:2102.00457 §3.2]. - **Padding.** About half of the kernel and dilation combinations are padded, so the output keeps the full length. The rest are trimmed at the edges [arxiv:2102.00457 §3.2]. - **Pooling operators.** Four are used per response and bias: PPV, the proportion of positive values; MPV, the mean of the positive values; MIPV, the mean index of the positive values; and LSPV, the longest stretch of positive values [arxiv:2102.00457 §3.4, alg. 1]. - **Input representations.** The base series and its first-order difference, each with its own dilations and biases [arxiv:2102.00457 §3.1, §3.3]. ### MultiRocket (default, 50k features) Keeps the pipeline above and uses both representations with all four operators. That gives 6,216 kernel/dilation/bias combinations per representation, times 2 representations and 4 operators, for 49,728 features [arxiv:2102.00457 §3.5, §1]. The default classifier is a ridge regression. The paper says logistic regression is an option for larger datasets but gives no details [arxiv:2102.00457 §3.5]. Card: [MultiRocket](/papers/2102.00457). Author-reported results on 109 univariate UCR datasets, 30 resamples each [arxiv:2102.00457 §4.1]: - Significantly more accurate than MiniRocket. Against MiniRocket with 10k features, MultiRocket (50k) wins on 76 datasets, loses on 29 and ties on 4 [arxiv:2102.00457 app. D]. - Not significantly less accurate than HIVE-COTE 2.0, TS-CHIEF or InceptionTime. HIVE-COTE 2.0 is still ahead on 59 datasets [arxiv:2102.00457 §4.1, fig. 6]. Training time, total single-thread figures from the paper's table [arxiv:2102.00457 tab. 2]: | Method | Total train time | |---|---| | MiniRocket, 10k features | 2.44 min | | MultiRocket, 10k features | 4.38 min | | MiniRocket, 50k features | 5.25 min | | MultiRocket, 50k features | 15.77 min | | HIVE-COTE 2.0 | 340.21 h | On 32 threads, the 50k version takes about 5 minutes for all 109 datasets, against about 2 minutes for MiniRocket with 10k features [arxiv:2102.00457 §1, §4.1]. ### MultiRocket, 10k features A smaller variant with the same operators. The paper uses it as the setting matched to MiniRocket's default feature count [arxiv:2102.00457 §4, app. D]. Its author-reported wins over MiniRocket (10k) are 70 datasets [arxiv:2102.00457 app. D]. Its total single-thread time is 4.38 minutes [arxiv:2102.00457 tab. 2]. Card: [MultiRocket](/papers/2102.00457). ### Arsenal: random kernels in a ridge ensemble Arsenal is one of the four modules of HIVE-COTE 2.0. It replaces the single ROCKET ridge classifier with 25 ROCKET-style members, each built on 2,000 random kernels [arxiv:2104.07551 §3.3, alg. 4, tab. 2]. Each kernel has: - a length of 7, 9 or 11; - weights drawn from N(0, 1) and then mean-centred; - a bias drawn from U(−1, 1); - an exponentially spaced dilation, and a random padding flag [arxiv:2104.07551 §3.3]. Each kernel gives two features per series: the maximum response, and the proportion of positive responses [arxiv:2104.07551 alg. 4]. Each member is a ridge classifier with its regularisation chosen by built-in cross-validation [arxiv:2104.07551 §3.3, alg. 4]. For multivariate data, a kernel draws a random number of dimensions [arxiv:2104.07551 §3.3]. Card: [HIVE-COTE 2.0](/papers/2104.07551). Arsenal is reported inside the ensemble, not alone. The card gives no standalone accuracy for it. What it does give: - Time for one resample of the 112 UCR problems, single core: 27.91 h for Arsenal, against 2.85 h for the ROCKET baseline and 340.21 h for HIVE-COTE 2.0 [arxiv:2104.07551 tab. 4]. - Against the ROCKET baseline, HIVE-COTE 2.0 has a mean accuracy difference of −2.49% on UCR, and wins on 97 of 112 datasets [arxiv:2104.07551 tab. 3]. The baseline is a single ROCKET ridge classifier with 10,000 kernels [arxiv:2104.07551 tab. 2]. - On the 26 UEA multivariate problems, HIVE-COTE 2.0 beats ROCKET with a mean difference of −2.52%, and wins on 19 datasets [arxiv:2104.07551 tab. 5]. These rows measure the full ensemble, not Arsenal alone. ## When to use which | Situation | Use | Why | |---|---|---| | Univariate series, batch training on CPU, accuracy matters most among fast methods | MultiRocket, 50k features | Author-reported accuracy close to HIVE-COTE 2.0, at a fraction of its single-thread time (15.77 min vs 340.21 h) [arxiv:2102.00457 §4.1, tab. 2] | | Tight training budget, or a need to keep the feature count near MiniRocket's | MultiRocket, 10k features | Same operators at 4.38 min single-thread, still ahead of MiniRocket (10k) on most datasets in the paper [arxiv:2102.00457 tab. 2, app. D] | | Random kernels are wanted, or the series are multivariate | Arsenal, as a module of HIVE-COTE 2.0 | Random kernels with per-kernel dimension sampling for multivariate data [arxiv:2104.07551 §3.3]. Arsenal's standalone accuracy is not reported in the card | | Accuracy is the priority and a very long training budget is acceptable | HIVE-COTE 2.0 | Author-reported as significantly more accurate than the ROCKET baseline on UCR and UEA, at about 340 single-core hours on UCR [arxiv:2104.07551 tab. 3, tab. 4, tab. 5] | | Memory-constrained feature storage | Expect the 50k version to be large | Float32 features at 49,728 columns come to about 199 kB per series, derived from the feature count [arxiv:2102.00457 §3.5] | | Multivariate series with MultiRocket | Not established | The MultiRocket paper covers univariate series only [arxiv:2102.00457 §5]. The code has a multivariate module that the paper does not describe [code: multirocket/multirocket_multivariate.py] | | Large datasets where the default ridge classifier is slow | Try logistic regression | The paper names it as an option without details [arxiv:2102.00457 §3.5]. The code uses Adam with learning rate 1e-4, up to 200 epochs [code: multirocket/logistic_regression.py#L14-L22] | | You need a learned network, fine-tuning or GPU training | Not this family | The kernels and biases are fixed or drawn at random, and only the classifier is fitted [arxiv:2102.00457 §3.2, §3.5] [arxiv:2104.07551 §3.3] | | Shipping a product with reference code | Implement from the description | The reference repositories are GPL-3.0, a copyleft licence [card metadata: code_license] | ## Implementation sketch The common core, in Python-flavoured pseudocode. The steps follow MultiRocket; variants differ in the parts marked in the notes below. ```pseudocode # Fit the dilations and biases for one representation [arxiv:2102.00457 §3.2] # Inputs: train (N, L); features per kernel (74 by default) def fit_representation(train, per_kernel): length = train.shape[1] # Dilation exponents spread evenly up to a length-based maximum # [arxiv:2102.00457 §3.2] [code: multirocket/multirocket.py#L105] max_exp = log2((length - 1) / 8) dilations = unique(floor(2 ** linspace(0, max_exp, 32))) biases = [] for dilation in dilations: for kernel in range(84): # Bias source: one random training series [arxiv:2102.00457 §3.2] series = train[random_index(len(train))] response = convolve_full(series, kernel, dilation) # Quantiles of the response become the biases biases += quantile(response, quantile_levels(per_kernel)) return dilations, biases # Feature vector for one series [arxiv:2102.00457 §3.1, §3.3, §3.4] def features(series, params_base, params_diff): # First-order difference, one value shorter [arxiv:2102.00457 §3.1] diffed = series[1:] - series[:-1] # Pool each representation over its kernels, dilations and biases return concat([pool_all(series, params_base), pool_all(diffed, params_diff)]) # Four pooling operators for one response and one bias [arxiv:2102.00457 §3.4] def pool_four(response, bias): # A value is positive when it is above the bias shifted = response - bias positive = shifted > 0 ppv = mean(positive) # Mean of the positive values; 0 when there are none mpv = mean(shifted[positive]) if any(positive) else 0 # Mean index of the positive values; -1 when there are none mipv = mean(where(positive)) if any(positive) else -1 # Longest run of positive values [arxiv:2102.00457 alg. 1] lspv = longest_run(positive) return [ppv, mpv, mipv, lspv] # Classifier: ridge regression on the feature matrix, with the default # settings in the paper's text [arxiv:2102.00457 §3.5] ``` Notes on where implementations differ, all from the card's comparison of paper and code [arxiv:2102.00457 §3.2, §3.4] [code: multirocket/multirocket.py#L244]: - **Threshold sign.** The code counts a value as positive when it is above the bias. The paper's text reads as the opposite sign [arxiv:2102.00457 §3.4.1]. - **MPV offset.** The code adds the bias back into MPV. The paper defines MPV as the mean of the positive values, so with the code's threshold the two differ by twice the bias [arxiv:2102.00457 §3.4.2, alg. 1]. - **Longest run.** Algorithm 1 and the code measure gaps between non-positive positions. That gives one more than the run length for a run between two non-positive values, which differs from the equation in §3.4.4 [arxiv:2102.00457 §3.4.4, alg. 1]. - **Trimmed responses.** Unpadded combinations are 8 × dilation values shorter than the series. PPV divides by that shorter length, so its scale varies across combinations [arxiv:2102.00457 §3.2]. - **Randomness.** The paper's only random choice is the bias series [arxiv:2102.00457 §3.2]. The code draws it inside a Numba-compiled function, so seeding NumPy outside may not make runs repeatable. This is a reading of the code, not a tested result [code: multirocket/multirocket.py#L60]. To get Arsenal from the sketch, replace the 84 fixed kernels with random draws as listed in its section, and keep two features per kernel: the maximum and the proportion of positive responses [arxiv:2104.07551 alg. 4]. The paper's text describes a majority vote over members, but the code weights each member by its cross-validated ridge score raised to the power 4 (see Contradictions). ## Contradictions Across cards, the two papers report different things rather than contradicting each other on the same quantity: - **Timings across papers.** MultiRocket reports 15.77 min for 109 UCR datasets [arxiv:2102.00457 tab. 2]. HIVE-COTE 2.0 reports 2.85 h for ROCKET, and 27.91 h for Arsenal, on 112 UCR datasets [arxiv:2104.07551 tab. 4]. The dataset sets, implementations and hardware differ, so these numbers should not be divided into a speed-up. - **Dataset sets across papers.** MultiRocket uses 109 univariate UCR datasets. HIVE-COTE 2.0 uses 112 UCR problems [arxiv:2102.00457 §4.1] [arxiv:2104.07551 §4]. Accuracy claims from the two cards are not measured on the same set. - **Arsenal vote.** The HIVE-COTE 2.0 paper says new cases are classified by majority vote [arxiv:2104.07551 §3.3]. The code weights each member by its ridge cross-validation score raised to the power 4 [code: src/main/java/tsml/classifiers/kernel_based/Arsenal.java#L206] [code: src/main/java/tsml/classifiers/kernel_based/Arsenal.java#L338-L339]. The paper gives no formula for the probabilities [arxiv:2104.07551 §3.3]. - **Arsenal normalisation.** The paper does not mention normalising the input. The code defaults `normalise` to true [code: src/main/java/tsml/classifiers/kernel_based/Arsenal.java#L52]. - **Arsenal padding.** The algorithm writes the padded length as m + 2p, while the text describes padding as a yes/no flag. The padding amount is not stated [arxiv:2104.07551 alg. 4, §3.3]. Within the MultiRocket paper, these sources disagree: - **Dilation maximum.** The paper typesets the maximum exponent with a different denominator than the code's log2((L − 1) / 8). The paper's form gives a much smaller maximum dilation [arxiv:2102.00457 §3.2] [code: multirocket/multirocket.py#L105]. - **Speed.** Section 4.2 gives about 4× for the 50k version over MiniRocket (50k) on one thread [arxiv:2102.00457 §4.2]. The totals in the table give about 3× (15.77 / 5.25) [arxiv:2102.00457 tab. 2]. Section 1 gives about 10× over MiniRocket (10k) [arxiv:2102.00457 §1], while the table gives about 6.5× (15.77 / 2.44). The timings measure different things, so do not mix them. - **Dataset count.** Most of the paper uses 109 datasets. The table caption and the HIVE-COTE 2.0 timings say 112 [arxiv:2102.00457 §1, §4] [arxiv:2102.00457 tab. 2, §4.2]. - **Per-dataset values.** The accuracies quoted for two single datasets do not say which method each value belongs to [arxiv:2102.00457 §4.1]. ## Papers - [MultiRocket](/papers/2102.00457), arXiv:2102.00457: adds the first-difference input and the PPV, MPV, MIPV and LSPV operators to the MiniRocket pipeline, with a 50k-feature default. - [HIVE-COTE 2.0](/papers/2104.07551), arXiv:2104.07551: a meta ensemble whose Arsenal module is 25 ROCKET-style ridge members built on random kernels; reports the ensemble's results, not Arsenal's alone.