A Practitioner's Guide to Kolmogorov-Arnold Networks

In plain words

A standard neural network puts a fixed squashing function, such as ReLU, at each neuron and learns weights on the connections between neurons. A Kolmogorov-Arnold Network (KAN) reverses this. It puts a learnable one-variable curve on every connection and only adds the results together at each node. The name comes from a mid-twentieth-century theorem that continuous functions of many variables can be built from sums of one-variable functions. KANs take inspiration from that idea but do not implement the theorem exactly.

This review collects the many KAN variants published since 2024, grouped mostly by the type of curve each one uses. Its central claim is that the choice of curve, called the basis, matters most. It compares KANs with ordinary neural networks, physics-informed networks and kernel methods, and it ends with a checklist for choosing a configuration. Its overall message is that KANs are not a clear winner over well-tuned MLPs, and that results depend strongly on the problem.

TL;DR

• A KAN places a trainable univariate function on every edge and sums at each node. The review treats it as a structured alternative to MLPs, not as an implementation of Kolmogorov's theorem §2.
• The basis family (B-spline, Chebyshev, Jacobi, Fourier, Gaussian, wavelet, Sinc, ReLU-power and others) is the main design axis §5, tab. 5.
• Reported comparisons are mixed. KANs often match or beat MLPs and PINNs on accuracy or convergence speed, but usually train more slowly per iteration tab. 1, §11.
• The review's "Choose-Your-KAN" guide and its decision algorithm give a default (cubic B-spline on a uniform grid) and a basis choice by solution structure §10, alg. 1.
• Use it to decide whether a KAN is worth trying and which basis to start from. It does not give settled training recipes: learning rates, budgets and most regularization weights are not stated.

Problem

Fixed activations in MLPs limit adaptability. Networks are hard to interpret and to attribute causally. Accurate MLPs often need many parameters. MLPs also show spectral bias: they learn low frequencies before high ones, which slows convergence on oscillatory or sharp solutions §1.

The review asks when a KAN should be preferred over an MLP or a PINN. Its answer is that the evidence is inconsistent. Fairness-focused studies reach conflicting conclusions, and the best basis changes with the PDE being solved [§1, "MLPs vs. KANs"]. The literature it covers spans regression, function approximation and PDE solving tab. 1. It also notes that comparisons often match parameter counts but ignore basis-induced bias and normalization or grid constraints §11.

Method

The review is not a method paper, so this section sets out the components it describes, in the order it presents them.

Architecture. A KAN layer maps n_l inputs to n_{l+1} outputs through one univariate function for each input-output pair. Each output is the sum of its edge functions. Layers are composed to make a deep KAN §2. The original KAN uses a trainable B-spline on each edge §2, §5.1.

Forward pass of a deep KAN §2:

# KAN forward pass: stacked layers of learnable edge functions  [§2]
# Input:  x (n_0,)
# Output: y (n_L,)

def kan_forward(x, layers):
    # Layer l holds one univariate function per edge (i -> j)  [§2]
    for edge_functions in layers:
        # Each output node sums its incoming edge functions; there is
        # no separate nonlinearity at the node  [§2]
        x = [sum(phi(x[i]) for i, phi in enumerate(row))
             for row in edge_functions]
    return x

Each edge function is a fixed-knot B-spline with trainable coefficients §5.1:

# Univariate edge function as a fixed-knot B-spline  [§5.1]
# Input:  x (scalar), coeffs (N,), knots (N + k + 1,), degree k
# Output: scalar

def edge_function(x, coeffs, knots, degree):
    # Basis values B_n^(k)(x) from the Cox-de Boor recursion  [§5.1]
    basis = bspline_basis(x, knots, degree)  # (N,)
    # The trainable coefficients c_n weight the basis  [§5.1]
    return dot(coeffs, basis)

Relation to Kolmogorov superposition theory (KST). The review states that KANs are inspired by KST but are not exact realisations of it. KST has universal, fixed inner functions and a single outer function per target. KAN edge functions are learned, use smooth B-splines, and are not constrained by the separation conditions of the theorem. Deep KANs also distribute nonlinearity over several layers §2. In the compressed modern form of KST that the review cites, the inner function is universal and the outer function depends on the target §2.

Relation to MLPs. Under a ReLU^k activation, an MLP of width W and depth L can be re-expressed exactly as a KAN of the same width, with at most twice the depth, a grid of size G = 2 and B-spline bases of order k [§4, theorem MLP ⇒ KAN, attributed to Wang]. The review also states the converse §4. An MLP maps activate-then-mix, whereas a KAN maps activate-then-mix on each input coordinate §4.

Relation to kernel methods. Per the review, a shallow KAN on one-dimensional inputs is a classical kernel method in all but the way its coefficients are fitted §1, §3.

Basis families. Table 5 lists the bases the review covers. Each row is a univariate function phi with its support, its grid requirement and its normalization. Reference keys are omitted here; see tab. 5 for the full table.

Decision procedure (the review's Choose-Your-KAN guide). The review gives a seven-step recipe, summarised in its decision algorithm. The algorithm is reproduced here in condensed form, with its branches and their order alg. 1.

# Choose MLP or KAN, then a basis  [alg. 1] [§10]
# Input:  many_samples (bool), has_prior (bool), structure (str)
# Output: model class and basis name

def choose_model(many_samples, has_prior, structure):
    # Step 1: MLP baseline when data are large and no prior exists  [alg. 1]
    model = "KAN"
    if many_samples and not has_prior:
        model = "MLP"
    # Step 2: basis from solution structure; branches tested in order  [alg. 1]
    basis = None  # alg. 1 defines no default branch
    if structure == "smooth or boundary-sensitive":
        basis = "B-spline (cubic, extended grid)"
    elif structure == "periodic or globally oscillatory":
        basis = "FourierKAN or KAF"
    elif structure == "polynomial-like or spectrally smooth":
        basis = "Chebyshev or Jacobi, tanh normalization"
    elif structure == "shocks, discontinuities, boundary layers":
        basis = "SincKAN, rKAN, DKAN or HRKAN"
    elif structure == "multiscale or spatially heterogeneous":
        basis = "Wavelet KAN or FBKAN"
    elif structure == "high-dimensional or separable":
        basis = "KKAN, SPIKAN or KAF (RFF)"
    return model, basis

The seven steps of the guide §10:

1. Start with a cubic B-spline KAN on a uniform grid. For PDEs, enable grid extension and post-training refinement §10 Step 1.
2. Match the basis to the structure of the target: Chebyshev with tanh normalization for globally smooth or oscillatory functions; Jacobi for controllable smoothness; SincKAN, rKAN or DKAN for discontinuities; FourierKAN or KAF for periodic structure; wavelets for local or multi-scale features §10 Step 2.
3. If speed matters, use ReLU^m or Gaussian RBF bases, JAX jit/XLA or fused CUDA kernels, and orthogonal polynomials or RBFs with l1-plus-entropy pruning §10 Step 3.
4. Stabilize with tanh normalization for Chebyshev and Jacobi, min-max or sigmoid/tanh scaling for RBF and wavelet bases, and extended or free-knot grids for splines. Optional 1-Lipschitz layers may be inserted between the summation and the linear maps §10 Step 4.
5. For PINNs, use the weighted loss L = w_u L_data + w_f L_phys + w_b L_bc. Add augmented Lagrangian or residual attention, residual-adaptive sampling, or coarse-to-fine grid updates only when needed §10 Step 5.
6. Warm up with Adam or RAdam, then refine with (L-)BFGS. Use mixed precision and gradient clipping for stiff PDEs §10 Step 6.
7. Increase capacity locally: local refinement with free knots, subdomain partition-of-unity decomposition (FBKAN), or selective basis enrichment [§10 Step 7].

Accuracy techniques named in the review include physics-constrained losses, adaptive sampling and grids, domain decomposition, function decomposition, hybrid and ensemble models, sequence and attention hybrids, handling of discontinuities and sharp gradients, and staged optimization §6, §10.

Efficiency techniques. The review lists three sources of speed-up: replacing iterative B-spline evaluation with non-iterative ReLU-power activations; CUDA matrix kernels; and model restructuring, such as Random Fourier Feature compression or custom backward kernels to reduce memory stalls. JAX implementations with jit/XLA are reported to give substantial end-to-end speedups §7.1.

Regularization. The review groups methods into sparsity, Lipschitz control, weight decay and implicit or dropout-style approaches §8. The L1 sparsity penalty and entropy balancing are written as follows §8:

\|\varphi_{ij}\|_1 = \tfrac{1}{N_p} \sum_{s=1}^{N_p} |\varphi_{ij}(x^{(s)})|, \qquad \|\Phi\|_1 = \sum_{i,j} \|\varphi_{ij}\|_1

S(\Phi) = -\sum_{i,j} \frac{\|\varphi_{ij}\|_1}{\|\Phi\|_1} \log\Big(\frac{\|\varphi_{ij}\|_1}{\|\Phi\|_1}\Big)

\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{pred}} + \lambda \big( \mu_1 \sum_\ell \|\Phi_\ell\|_1 + \mu_2 \sum_\ell S(\Phi_\ell) \big)

Other approaches in §8: a Bayesian alternative (Gaussian-process and spike-and-slab priors) §8; LipKAN, which inserts 1-Lipschitz layers and uses an l_{1.5} penalty §8; L2 weight decay with temporal and mutual-information terms §8; DropKAN, which masks outputs after the spline evaluation rather than the inputs §8; and the implicit bounding from nested tanh in Chebyshev KANs §8.

Convergence and scaling. Theory and empirics are summarized in §9:

• Under the conditions in the review, a deep spline KAN attains a Sobolev approximation rate of order L^{-2s/d}, which doubles the classical parameter rate §9.1.
• For KAN regression, spline-based KANs reach the minimax rate n^{-2r/(2r+1)} for Sobolev-r functions, with knot count scaling as n^{1/(2r+1)} §9.1.
• Empirically, error follows a power law l ∝ N^{-alpha} with a task-dependent alpha > 0, and refining the spline grid helps more than widening layers §9.1.
• Flatter NTK spectra, reported for KANs and learnable-basis PINNs, indicate weaker spectral bias. Broader spectra can also raise curvature and worsen conditioning §9.2.
• Under gradient flow with G = D^T D positive definite, the loss decays exponentially at a rate set by the smallest eigenvalue of G; the same holds in expectation for SGD §9.3.

Hyperparameters

The review gives few concrete values. Most rows below are qualitative, and the rest are stated as "not stated".

Results

Author-reported, but this is a review. It runs no experiments of its own; its figures are illustrative activations. The table below is aggregated from literature results, which the authors describe as qualitative trends that mix metrics such as relative L2 and RMSE. Setup is per source paper and not restated here tab. 1.

Table 1 records mixed outcomes, and the review says it reflects prevailing tendencies rather than definitive results for every case §1.

Paper vs code

No official code. The paper links github.com/AmirNoori68/kan-review as a structured reference for KAN research §1, abstract. The paper describes it that way. I did not open the repository, and reference repositories of a survey are not treated as method code here. The review's repository table (Table 2) lists other authors' and third-party projects, and none of them is used here. No discrepancies found, since there is no code to compare against.

Pitfalls

• Mixed evidence. Table 1 mixes metrics and setups and says so tab. 1. Do not rank KAN and MLP from it. The review's own conclusion is that KANs can match or beat MLPs under some bases and problems, and fail under others §11.
• Algorithm and text disagree. The guide's text and alg. 1 are not identical. Alg. 1 has a branch for high-dimensional or separable problems (KKAN, SPIKAN, KAF) that the text's step 2 does not mention §10, alg. 1. The text's step 2 gives Jacobi for controllable smoothness, but alg. 1 lumps Jacobi with Chebyshev. Alg. 1 adds self-scaled quasi-Newton methods to the refinement stage alg. 1 Step 5. The text adds Lipschitz layers and mixed precision, which alg. 1 does not contain §10 Steps 4, 6.
• Branch order and gaps. Alg. 1 is an if/elif chain with no default branch and no tie-break. A problem that is both smooth and periodic matches the first branch that applies. The review does not say how to decide. Implement the branch order as written, and treat the tie-break as an open choice alg. 1.
• Basis forms are idealized. Table 5 shows the Chebyshev and Jacobi rows with tanh inputs, but normalization is a separate choice that the review treats as a design decision §10 Step 4, tab. 5. Implement the normalization as its own step.
• Theory has conditions. The Sobolev rate applies to grid size G = 2 and a fixed width W_0(d), with a constant that does not depend on depth L §9.1. The prose states the rate in parameters P, while the corollary states it in depth L §9.1; check which one a claim refers to. The review also says some of these gains may come from optimization dynamics rather than from asymptotic expressivity §9.1. The gradient-flow bound needs G positive definite; the learning-rate condition is not stated in the review §9.3.
• Regularization signs and weights. The objective adds mu_2 times S(Phi) as written, but the review does not state the sign convention, the weights lambda, mu_1, mu_2, or lambda_1 §8. Check the sign against the original KAN paper before implementing (see Lineage).
• First-layer collapse with large Gaussian width. For Gaussian KANs, a large eps makes the first-layer feature matrix nearly rank one, so distinct inputs become indistinguishable §8. Keep eps in a well-conditioned range; the review gives no numeric range.
• Training cost. Many cited PINN comparisons report KANs as slower to train per iteration tab. 1. Efficiency techniques reduce this §7.1, but the review gives no speed numbers for them.
• Plain Chebyshev KANs can be unstable. The review's case study says targeted refinements (tanh normalization, linear heads, tanh-based grids, hybrid optimizers, trainable degree) are needed §11.
• KST does not guarantee KAN behaviour. The review says KANs are not exact realisations of KST: the inner functions are learned and the separation constraints are absent §2. Do not argue KAN properties from KST proofs.

Lineage

• Builds on: Kolmogorov superposition theory and its refinements by Lorentz and Sprecher §2, tab. 3, and the KAN architecture of Liu et al., which is summarized on this site at /papers/2404.19756 §2, §4.
• Compared against: MLPs and PINNs tab. 1; DeepONet and Fourier neural operators as counterparts to KAN-based operator learners tab. 4.
• Superseded by: not stated in the review.