To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents

In plain words

A tool-using language model often calls a tool when it should first ask the user for a missing detail. The paper's claim is that this is not only a matter of weak evidence. Even when the model's internal signals for "call" and for "ask" are equally strong, it still leans toward calling. The authors call this an intrinsic bias.

To look inside the model, they use a sparse autoencoder, which splits the model's internal signals into many separate features that can be read one at a time. They find a small group of features that track the call and ask decisions, measure how far the model leans toward calling, and then subtract that lean with a computed nudge to the internal signals while the model generates text.

In their tests this reduces unneeded calls and raises the rate of correct "ask first" answers, at a small cost in correct calls. It does not help every model: one of the six models gets worse, and another is not affected.

TL;DR

• Six models from three families call tools too often on When2Call: call accuracy is high, no-call accuracy is much lower, and overall accuracy is 55% to 70% §1.
• The paper's hypothesis (IBH) is that the call/no-call decision has a fixed call offset that does not depend on the activations, so the model still prefers to call at activation parity §4.1.
• AMCS subtracts that offset with a steering vector along SAE decoder directions; it raises no-call accuracy by 4 to 17 points on five of six models §5.2, tab. 1.
• It does not work uniformly across models: on Gemma-3-1B it lowers overall accuracy, and on Gemma-3-4B it has almost no effect §5.2, n. 2.
• All numbers are author-reported on When2Call and a 100-example ACEBench subset §6.1.

Problem

Models answer well when a tool call is required, but they call tools too often when none is needed. On When2Call, call accuracy stays high while no-call accuracy is much lower, which leaves overall accuracy at 55% to 70% for the six models studied §1 fig. 1. The paper treats the wrong calls as more than a benchmark artifact: it asks what internal mechanism produces them §2.

The analysis assumes white-box access to residual-stream activations at one layer per model, and an SAE trained on that layer §3.1. Only single-turn decisions are studied app. A.

Method

The method has three stages. Each stage uses the same SAE for one model.

Stage 1. Find the decision features §3.

1. Train one TopK SAE per target model on the residual stream at one hook layer §3.1 tab. 5. The dictionary has M = 8d features and keeps K = floor(d / 32) active per token §3.1 eq. 1 eq. 2.
2. Train in two stages: first on OpenWebText2 (about 50M tokens), then on the When2Call training split (about 10M tokens) §3.1.
3. Build the discovery set from the When2Call evaluation split. The target model answers every context, and an LLM judge classifies each answer into one of four types §3.2 app. B. Answers judged tool_call form the call set D+; answers judged request_for_info form the no-call set D- §3.2 app. C. direct_answer and cannot_answer answers are dropped app. C.
4. Read each feature at the action boundary, the last prompt token before the first generated token §3.2.
5. Score each feature with the mean activation gap eq. 3 and the directional AUROC eq. 4. Keep the features that are in the top R on both scores. The result is the call basis C; swapping the two sets gives the no-call basis N §3.2.
6. Check the bases with a 5-fold cross-validated logistic probe on the selected features. The top few features (K ≤ 5 typically) reach the residual-stream upper bound, while random features stay near chance §3.3 eq. 5 fig. 3.

# TopK sparse autoencoder on one residual layer  [§3.1] [eq. 1] [eq. 2]
# Input:  h (d,), residual activation at the action boundary
# Output: z (M,) sparse code, h_hat (d,) reconstruction
# Sizes:  M = 8 * d,  K = floor(d / 32)

def sae_forward(h, W_enc, W_dec, b_pre, K):
    # Subtract the learned pre-bias before encoding  [eq. 1]
    centred = h - b_pre                       # (d,)
    pre = W_enc @ centred                     # (M,)
    # Keep the K largest pre-activations and zero the rest  [eq. 1]
    top = indices_of_k_largest(pre, K)        # (K,)
    z = zeros_like(pre)                       # (M,)
    # The code also applies ReLU  [code: sae/sae_model.py#L66-L71]
    z[top] = relu(pre[top])
    # Decode with the unit-norm decoder columns  [eq. 2]
    h_hat = W_dec @ z + b_pre                 # (d,)
    return z, h_hat

# Rank SAE features for one decision  [§3.2] [eq. 3] [eq. 4]
# target:    codes of answers judged tool_call   (D+)
# reference: codes of answers judged request_for_info (D-)
# R:         ranking cutoff (swept in the paper)

def rank_features(target_codes, reference_codes, R):
    # Mean activation gap per feature  [eq. 3]
    gap = mean(target_codes, axis=0) - mean(reference_codes, axis=0)
    # Directional AUROC per feature  [eq. 4]
    auroc = directional_auroc(target_codes, reference_codes)
    # Keep features that are in the top R on both scores  [§3.2]
    top_gap = top_indices(gap, R)
    top_auroc = top_indices(auroc, R)
    return intersection(top_gap, top_auroc)   # call basis C
    # Swap target and reference to obtain N  [§3.2]

Stage 2. Test whether activation parity removes the bias §4.

7. Summarise each example by a signed margin: the mean call-basis activation minus the mean no-call-basis activation eq. 6 §4.1. Positive margins favour a call.
8. Compare two accounts. The activation-only account says the decision depends only on the margin, so it is neutral at m = 0 eq. 7. IBH adds a call offset beta_0 > 0 eq. 8, which moves the neutral point to m* = -beta_0 / beta < 0 §4.1.
9. Fit the logistic model in eq. 8 on cached activations and the observed decisions to estimate beta and beta_0 §4.3. Every model in the main study has a negative neutral point m* §4.3 fig. 6. The paper reads the response-level and input-level plots in its figures as consistent with IBH §4.2 fig. 4 fig. 5.

# Signed activation margin and the offset hypothesis  [§4.1] [eq. 6]
# z:  SAE code at the action boundary
# C, N: call and no-call feature index sets

def margin(z, C, N):
    # Mean call activation minus mean no-call activation  [eq. 6]
    return mean(z[C]) - mean(z[N])

# Intrinsic Bias Hypothesis: logistic with a call offset  [eq. 8]
def p_call_ibh(m, beta, beta_0):
    # Expected signs under IBH: beta > 0 and beta_0 > 0  [eq. 8]
    return sigmoid(beta * m + beta_0)

# Neutral point under IBH  [§4.1]
m_star = -beta_0 / beta     # negative when beta_0 > 0

Stage 3. Cancel the offset with AMCS §5.

10. Restrict the bases to the top r features on each side (r = 5, 10, ..., 30 in the sweep) §5.2. Recompute the margin on that restricted basis and fit eq. 9 on a calibration split, using observed decisions only §5.1 app. F.
11. Keep the candidate only when beta_r > 0 app. F alg. 1.
12. Compute the shift delta_r = -beta_0r / beta_r eq. 10. This removes the offset: the calibrated logit then equals beta_r times the margin §5.1.
13. Weight each feature by its share of the calibration gap on its side eq. 11. The weights only allocate the shift; the magnitude comes from delta_r §5.1.
14. Build the steering vector from the call and no-call decoder columns eq. 12. Use alpha in [0, 1] to split the shift between the two sides §5.1.
15. At inference, add the vector to the hook layer output at every token of autoregressive generation. This costs one addition per token and needs no extra model calls §5.1 app. F.

# Fit the diagnostic logistic and the closed-form shift  [§5.1] [eq. 9] [eq. 10]
# margins:   signed margin on the restricted basis, one per example
# decisions: 1 if the model answered with a call, else 0

def fit_delta(margins, decisions):
    # Logistic fit P(call | m) = sigmoid(beta * m + beta_0)  [eq. 9]
    beta, beta_0 = logistic_fit(margins, decisions)
    # Closed-form shift that removes the offset  [eq. 10]
    delta = -beta_0 / beta
    return beta, beta_0, delta
    # Skip this candidate when beta <= 0  [alg. 1]

# Allocation weights from calibration gaps  [§5.1] [eq. 11]
# gaps: dict feature -> call-minus-no-call activation gap on the calibration set

def allocation_weights(gaps):
    # Normalise absolute gaps within one side so they sum to 1  [eq. 11]
    total = sum(abs(g) for g in gaps.values())
    return {j: abs(g) / total for j, g in gaps.items()}

# AMCS steering vector  [§5.1] [eq. 12] [alg. 1]
# delta_r < 0 on the call side; alpha = 0.8 in the main experiments  [§5.2]

def amcs_vector(decoder, C_r, N_r, w_C, w_N, delta_r, alpha, r):
    # Call-side term: lowers the margin, suppresses call features  [eq. 12]
    v_call = zeros(d)                         # (d,)
    for j in C_r:
        v_call += alpha * r * delta_r * w_C[j] * decoder[:, j]
    # No-call-side term: raises the margin, promotes no-call features  [eq. 12]
    v_nocall = zeros(d)                       # (d,)
    for j in N_r:
        v_nocall += (1 - alpha) * r * (-delta_r) * w_N[j] * decoder[:, j]
    return v_call + v_nocall                  # added to the hook layer output

Algorithm 1 alg. 1 gives the same procedure as numbered steps, with the calibration pass repeated separately for each budget r app. F.

Hyperparameters

Values from the paper unless marked "code". Items the paper does not state are marked "not stated".

Results

Author-reported. Setup: When2Call, six instruction-tuned models (Qwen3.5-4B and 9B, Gemma-3-1B and 4B, Ministral-3-3B and 8B), SAE and diagnosis layer as in §3.1 and tab. 5, AMCS with alpha = 0.8 and r averaged over {5, ..., 30}, and the baselines Prompt, Suppress and Promote §5.2 tab. 1. Accuracy in percent. Overall is as reported by the paper; per-budget numbers are not given in the main text.

Table 1. When2Call steering results tab. 1. Each cell is absolute accuracy (%). The paper also reports the change against Init as a subscript; those differences can be recomputed from this table.

Points from the text §5.2:

• Prompt raises no-call accuracy most but cuts tool-call accuracy by up to 56 points (Ministral3-8B) and drops overall accuracy below Init there.
• Suppress and Promote act on one side only. On Qwen3.5-4B each moves no-call accuracy by under 10 points.
• Tool-call accuracy under AMCS falls by at most about 5 points on five models; Gemma-3-1B drops by 8.84. AMCS gives the top overall score on Qwen3.5-4B, Qwen3.5-9B and Ministral3-3B.
• On Gemma-3-4B, AMCS is a null case; on Gemma-3-1B it is a failure case (overall falls 1.64 points) §5.2, n. 2.

Table 2. Frozen transfer to ACEBench tab. 2. Each cell is overall / call / no-call accuracy (%). 100 examples per model: 50 Special-Incomplete (gold: ask) and 50 Normal single-call examples (gold: call) app. G.1. The When2Call SAE, features and directions are held fixed. CAA uses a per-model coefficient chosen on When2Call; AMCS uses r = 25 and alpha = 0.8 for all four models §6.1.

Macro overall rises from 78.00 to 83.75 and no-call from 58.50 to 71.50, with call accuracy 97.50 to 96.00 §6.1. The paper calls the margins small (100 examples per model) and says AMCS is not best on every model.

Table 3. Diagnosis on Qwen3.5-4B across SAE retraining tab. 3. Each row refits the logistic model after rediscovering Top-25 features in that SAE. Raw m* is not comparable across rows.

Original-SAE diagnosis for all six models tab. 3 tab. 9. Top-25 features per side, raw mean-difference margin tab. 10 tab. 11.

The raw m* values differ by orders of magnitude across models because the margin is in each SAE's own activation units. The paper says these numbers are for comparison within a model only §4.3.

Robustness §6.3 tab. 9 tab. 10 tab. 11.

• SAE retraining (one factor changed: two hook layers, K/2, 2K, a second seed): IBH holds in 34 of 36 settings across the six models. The two misses are Ministral-3-3B at layer 24 and Ministral-3-8B at K/2, both with a slightly positive m*.
• Feature budget (N in {5, 10, 15, 20, 25, 30}): IBH holds in 35 of 36 settings. The one miss is Gemma-3-4B at N = 5.
• Margin construction at Top-25 (raw, standardised, difference over sum): IBH holds in all 18 settings.

Preservation of valid calls §6.2 tab. 7. On the 1,295 When2Call examples with a gold call, AMCS keeps 95.73% of the unsteered tool calls on the five models other than Gemma-3-1B. Of the flips to no-call, 97.3% fall in the lower-margin half. On calls that both responses keep, the joint score for function name and arguments moves by -0.27 points on average.

Gemma-3-1B is a failure case even at Top-6, so the paper says AMCS needs downstream checks per model rather than a general guarantee app. G.2.

Other settings app. H app. H.1 app. H.2 app. H.3.

• Models above 10B (Qwen3-14B, Gemma-3-12B-IT, not steered): positive slope, parity probability above 0.5, m* negative for both tab. 12.
• Pretrained base checkpoint (Qwen3.5-4B-Base): the offset is already present before the public post-training stage (m* = -0.691), though the comparison does not say which stage creates it tab. 13.
• Reasoning on (Qwen3.5-4B): no-call accuracy rises 23.07 points with call accuracy almost unchanged. The parity probability rises by 0.0395, but the change in m* crosses zero, so the paper does not claim reasoning moves the boundary tab. 14.
• Perplexity (When2Call test split): Qwen and Ministral stay near baseline for all tested alpha; Gemma-3 has extreme values under Suppress and at AMCS alpha = 1.0 on Gemma-3-1B tab. 15.

Paper vs code

Repository: https://github.com/SKURA502/agent-sae, MIT license, commit 2319c456e778fa25a25c5c55eb01e17afb2b67f0. The repository is linked from the paper's abstract, so it is treated as the official code. Code paths below are relative to the repository root. The code is in raw/code/2605.18882/.

Discrepancies the implementer should decide on:

• Feature ranking. Paper: the top-R intersection of the mean gap and the directional AUROC §3.2 eq. 3 eq. 4. Code: a threshold filter (mean_diff > 0 and AUROC > 0.6 in the CLI, AUROC > 0.50 in the shell script), then the top 30 by mean difference feature_discovery.py feature_discovery.py feature_discovery_when2call.sh. The rank cutoff and the intersection rule are not the same.
• Negative class. Paper: no-call set D- is only request_for_info answers, and the other two judged types are dropped §3.2 app. C. Code: the negative class is every label that is not the target label feature_discovery.py. Whether direct_answer and cannot_answer are in the input depends on the activation file, which was not checked.
• Feature weights. Paper: weights from the calibration-set gap between call and no-call decision responses §5.1 eq. 11. Code: absolute differences from a failure-versus-success attribution file, with the discovery mean difference as fallback eval_amcs_accuracy.py. These are different quantities.
• Delta fit. Paper: fit on the calibration split, restricted to examples whose observed decision is call or no-call app. F. Code: the mask first keeps examples whose ground truth is call or no-call, and then keeps predictions in that set sweep_amcs_topn.py sweep_amcs_topn.py. The code uses scikit-learn's LogisticRegression with its defaults; the paper does not state a regularisation sweep_amcs_topn.py. compute_delta returns delta without a beta > 0 check sweep_amcs_topn.py; the sweep loop was not read for one.
• Delta per budget. Paper: delta_r is refit for each budget r §5.1 eq. 10 app. F. Code: the single-run evaluator takes one hard-coded delta = -0.533, the raw Qwen3.5-4B value from the paper's Table 3 eval_amcs_accuracy.py eval_amcs.sh. Per-top-N delta is computed only in the sweep sweep_amcs_topn.py.
• Alpha. Paper: alpha = 0.8 in the main and transfer experiments §5.2 §6.1. Code: the evaluator and shell script default to 0.5 eval_amcs_accuracy.py eval_amcs.sh; the sweep runs alpha in {0.2, 0.6, 1.0} sweep_amcs_topn.sh. The value 0.8 does not appear in the code.
• Shell script flags. scripts/eval_amcs.sh passes --s-min-tc, --s-max-rfi and --z-min eval_amcs.sh. The Python evaluator defines none of these flags and calls strict parse_args eval_amcs_accuracy.py. As read, the script would fail with unrecognised arguments. It was not run.
• SAE loss. Paper: squared L2 reconstruction loss §3.1. Code: squared error divided by the squared norm of the input, per example sae_model.py.
• Stage 2 size and batch. Paper: about 10M tokens and 16,384 tokens per batch in both stages §3.1 app. D. Code: 5M tokens and 4,096 per batch in stage 2 train_sae.sh train_sae.sh.
• Decoder norm. Paper: decoder columns constrained to unit norm §3.1. Code: normalised at initialisation and then every 10 steps, not after every step sae_model.py train_sae.py train_sae.py.
• Sparse code. Paper eq. 1 has no ReLU §3.1. Code applies ReLU to the kept values sae_model.py.
• Margin reading in README. The README says the margin uses summed activations [README.md#L245]. The paper and the sweep code use means §4.1 sweep_amcs_topn.py.
• Suppress and Promote scales. Paper: 0.5 and 1.5 tab. 1. Shipped sweep defaults are 0.0, 0.4, 0.8 and 1.2, 1.6, 2.0 eval_suppress_accuracy.sh. The paper's exact values are not in the shipped defaults.
• Not in the repository. I searched for ACEBench and CAA and found no code for either. The transfer results (§6.1), the preserved-call judge (app. G.2) and the perplexity run (app. J) have no matching script that I checked.
• Not compared. analysis/steering.py (706 lines) and utils_validation/bias/tc_bias_logistic.py, which the paper uses for the Figure 6 fit, were not read.

Matches checked: the optimizer, learning rate, warm-up/stable/decay schedule, stage 1 batch and token count, M = 8d and K = floor(d/32) defaults, and the hook layers in the README table and training script (Qwen3.5-4B, Qwen3.5-9B, Ministral-3-3B, Ministral-3-8B, and Gemma-3-4B) agree with the paper train_sae.py train_sae.py README.md train_sae.sh train_sae.py.

Pitfalls

• Margin units. The offset delta = -beta_0 / beta is in the units of the margin it was fitted on. Apply it only to margins built the same way, from the same SAE and feature basis. The paper states that margins are not mean-centred, so the zero point is activation parity app. G.3; a centred margin moves the intercept.
• Raw m across models.* Raw m* values differ by orders of magnitude (from -0.27 to -496.9 in tab. 9 and 10). Compare them only within a model. The paper gives standardised numbers for the figure only §4.3.
• Calibration of r and alpha. Calibration repeats for each r, because each budget gives a different basis and bias estimate app. F. The paper's headline uses alpha = 0.8 and r averaged over 5 to 30 §5.2. Per-r results are not shown in the main text.
• Linear approximation. The margin shift equals delta only under a local linear approximation, because decoder directions need not be orthogonal. The paper verifies the realised shift instead of assuming it app. A app. F.
• Model dependence. Gemma-3 models have weak call/no-call separation on the basis, so AMCS is a model-conditional intervention. Gemma-3-1B is a failure case and Gemma-3-4B a null case §5.2, n. 2 app. J.
• Valid calls can be lost. On Gemma-3-1B, a reduced Top-6 intervention still flips 24.18% of valid calls tab. 7. The paper recommends downstream checks per model.
• Gains are small. ACEBench uses 100 examples per model. The paper says the margins are small and not significant across all models §6.1.
• Single-turn only. Multi-turn behaviour and long-horizon agents are not tested app. A.
• Reasoning. Reasoning raises no-call accuracy, but the paper does not claim it moves the neutral point app. H.3.
• Cost. Each model needs an SAE trained on about 60M tokens over two stages, and each steering run takes about 4 GPU-hours app. L. The 98 When2Call evaluation runs took about 392 GPU-hours in total app. L.
• Missing details. Not stated: the test split size used for steering (only "held-out"), the regularisation in the probe and the logistic fit, the four-way judge model, decoding for the main runs, sequence length for stage 1 in the paper, and weight decay for AdamW.

Lineage

• Builds on: the When2Call benchmark for the call/no-call labels §2 §3.2. The paper cites it without an arXiv id in its source, so no link is given here.
• Builds on: TopK sparse autoencoders §3.1. The paper cites the TopK and vanilla SAE works without arXiv ids in its source, so no link is given here.
• Compared against: Prompt, Suppress and Promote (own baselines) §5.2 tab. 1; CAA, Contrastive Activation Addition §6.1 app. G.1.
• Superseded by: not known from content in this wiki.