wikillm

Paper · 2026-10-31 · cs.CL, cs.AI

ShiftRAG: Bypassing the Textual Bottleneck via Decoupled Learning and Continuous Soft Tokens

One LLM with two LoRA modes: a retriever trained with a layer-gap contrastive loss and a soft orthogonality penalty, and a generator fed by soft knowledge tokens built from question-passage embeddings. Author-reported, Gemma2-9B: HotpotQA R@2 0.6598 vs 0.6318 for NV-Embed-v2; EM 0.5610 vs 0.5140.

Retrieval-augmented generationDense retrievalSoft prompt tuningKnowledge distillationretrieval-augmented-generationmulti-hop-qadense-retrievallorasoft-promptsknowledge-distillation

arXivPDFCodeMarkdown16 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Retrieval-augmented generation first looks up passages and then has a language model write an answer from them. The search step often knows more than the passages it returns: how strongly each passage matches the question, as seen inside the model's own hidden states. Handing the model plain text throws that knowledge away.

ShiftRAG keeps one language model but gives it two working modes, selected by small add-on weight sets. In search mode the model is trained to rank passages by how well their match with the question shows up in its deeper layers. In answer mode the model reads the passages as text and also gets a few learned "hint" vectors per passage, built from the search-mode view of the question and the passage. The two modes are trained separately and only meet at answer time.

TL;DR

ShiftRAG uses one LLM with two LoRA adapters, one for retrieval and one for answering. Retrieval is trained with a layer-gap contrastive loss plus a soft orthogonality penalty. The answering mode receives soft knowledge tokens derived from question-passage embeddings and relevance scores. Author-reported results with Gemma2-9B beat NV-Embed-v2 on retrieval and exact match on HotpotQA, 2WikiMultihopQA and MuSiQue. The official code has no LICENSE file, and several settings differ from the paper (see Paper vs code).

Problem

  • In decoder-only LLMs, token representations in deep layers converge into a narrow subspace (rank collapse). That blurs the distinction between useful evidence and distractors §1 fig. 1.
  • Contrastive adaptation for retrieval hurts autoregressive generation, while sharing one adapter for both tasks makes the two objectives compete. The paper reports negative gradient alignment on shared LoRA parameters in 84.2% of the first 500 updates §4.4 app. E.5.
  • Passing only passage text to the generator discards the retriever's question-passage alignment. The paper reports small correct/incorrect entropy gaps for baseline generators §1 fig. 1.
  • The method needs white-box weight access. Soft tokens are continuous and not readable as text §5.

Method

ShiftRAG starts from one pretrained decoder model and attaches two separate low-rank adapter sets, one per task §3. In encoder mode the causal mask is removed, so attention is bidirectional and every token sees the whole input §3 §3.1. In decoder mode the model returns to causal decoding §3. Inference has three stages: encode the question and retrieve the top-K passages by final-layer cosine similarity, translate each question-passage pair into soft knowledge tokens, then decode the answer from the question, the passage text and the soft tokens §3 fig. 2 §3.1.1.

Encoder mode (§3.1). Each layer's mean-pooled vector gives a gap score for a candidate layer relative to the final layer. The paper partitions the first half of the layers into five buckets and samples one candidate per bucket §3.1.1. The relevance score is the largest gap over the candidates eq. 1 eq. 2. The loss pushes the positive passage's score above the batch's other passages, with scores scaled by 1/τ eq. 3. A soft orthogonality term penalises high cosine similarity between token vectors of the same sequence in the last n layers §3.1.2 eq. 4 eq. 5. The total encoder loss adds it with weight λ_SOA eq. 6. A short masked-next-token (MNTP) warm-up, following LLM2Vec, adapts the encoder to bidirectional attention before the main objective §3.1.3 app. B.

Semantic signal translator (§3.2). For each retrieved passage k, SST concatenates the question and passage embeddings, encodes them into a hidden vector, and broadcasts it to M positions. A low-rank projection gives M offset vectors. Each passage's offsets are scaled by a relevance weight α_k computed from the top-K similarities. The passage's soft tokens are T_k = E + α_k ΔE_k + P, where E are learnable anchors and P learnable positions §3.2. The anchors start from pretrained latent vectors §3.2 app. B. The token sets of all K passages are concatenated §3.2.

Decoder mode (§3.3). LoRA_dec is trained with two terms eq. 9. The teacher is the same frozen backbone with the rationale of each passage in its context. The student sees the passages and the soft tokens in place of the rationales eq. 7. The distillation term sums the KL divergence from teacher to student over answer positions eq. 8. The next-token term is cross-entropy on the answer. Rationales are single sentences generated offline and used only for training app. H.1.

Pseudocode
# Inference  [§3] [fig. 2]
# Input: question, corpus of passages (encoded offline in encoder mode)
# Output: answer string
def answer(question, corpus_index):
    # Encode the question in encoder mode, final layer  [§3.1.1]
    e_q = encoder_embed(question)
    # Top-K passages by cosine similarity to the question  [§3.1.1]
    passages = top_k_cosine(e_q, corpus_index, k=K)
    # Soft knowledge tokens for every passage  [§3.2]
    soft_tokens = sst(e_q, embed(passages))     # (K * M, h)
    # Decode in decoder mode with text and soft tokens  [§3.3]
    return decoder_generate(question, passages, soft_tokens)
Pseudocode
# Encoder loss for one batch  [§3.1] [eq. 3] [eq. 6]
# Input: queries, positive passages, hard negatives
# Output: scalar loss
def encoder_loss(queries, positives, negatives):
    # Final-layer mean-pooled query vector  [§3.1.1] [eq. 1]
    e_q = mean_pool(hidden_states(queries)[-1])
    # Gap per candidate layer and relevance = max over candidates [eq. 1-2]
    scores = max_gap_scores(e_q, positives, negatives)   # (B, B + N)
    # Contrastive loss with scale 1/tau over the batch  [eq. 3]
    loss_lgc = cross_entropy(scores / tau, labels=arange(B))
    # Soft orthogonality on last n layers, queries and documents  [eq. 4-5]
    loss_soa_q = soa_over_last_layers(queries)
    loss_soa_d = soa_over_last_layers(positives + negatives)
    # Weighted sum of the two terms  [eq. 6]
    return loss_lgc + lambda_soa * (loss_soa_q + loss_soa_d) / 2
Pseudocode
# Soft orthogonal alignment for one layer  [§3.1.2] [eq. 4]
# H: (S, h) hidden states of the S tokens of one sequence
# Output: scalar penalty, normalised by S squared
def soa_layer(H):
    # Row-wise L2 normalisation  [§3.1.2]
    H_bar = normalise_rows(H)               # (S, h)
    # Token-level Gram matrix, a cosine similarity table  [§3.1.2]
    G = H_bar @ transpose(H_bar)            # (S, S)
    # Squared Frobenius distance to the identity, divided by S^2  [eq. 4]
    return squared_frobenius(G - identity(S)) / S**2
Pseudocode
# Semantic signal translator for K passages  [§3.2]
# e_q: (h,), e_docs: (K, h), anchors E and positions P: (M, h)
# Output: soft knowledge tokens (K * M, h)
def translate(e_q, e_docs, E, P, alpha):
    # Relevance weight per passage from the top-K similarities  [§3.2]
    weights = alpha(e_q, e_docs)            # (K,)
    tokens = []
    for k in range(K):
        # Hidden vector from question and passage  [§3.2]
        y_k = encode(concat(e_q, e_docs[k]))
        # Broadcast to M rows, then a low-rank projection  [§3.2]
        U_k = low_rank_project(broadcast(y_k, M))   # (M, r)
        # Offsets scaled by the passage's relevance weight  [§3.2]
        dE_k = offsets_from(U_k)                    # (M, h)
        # Soft knowledge tokens for passage k  [§3.2]
        tokens.append(E + weights[k] * dE_k + P)
    return concat(tokens)
Pseudocode
# Decoder training step  [§3.3] [eq. 7-9]
# Input: question, passages, rationales per passage, answer tokens
# Output: scalar loss
def decoder_loss(question, passages, rationales, answer, soft):
    # Student: LoRA_dec on, soft tokens in place of rationales  [eq. 7]
    student = decoder_logits(question, passages, soft, answer)
    # Teacher: same backbone, adapter off, rationales as context  [eq. 7]
    with no_grad():
        teacher = decoder_logits(question, passages, rationales, answer,
                                 adapter=False)
    # Next-token cross-entropy on the answer  [eq. 9]
    loss_ntp = cross_entropy(student, answer)
    # Sum over answer positions of KL(teacher || student)  [eq. 8]
    loss_kd = sum_over_positions(kl(teacher, student))
    # Weighted sum of the two terms  [eq. 9]
    return loss_ntp + lambda_kd * loss_kd

Hyperparameters

Values stated in the paper are cited to the paper. Values that exist only in the repository are marked "code only" and cited to the code. The paper gives no optimiser, learning rate, step count or decoding settings.

NameValueWhereNotes
BackbonesGemma2-9B (default), LLaMA3.1-8B, Mistral-7B, Qwen2.5-7B§4.1 app. AGemma2-9B used for main results
Encoder LoRA rankr = 16§4.1Mistral config uses r = 32 Mistral_7B.json
Decoder LoRA rankr = 64§4.1 app. Acode alpha 16 Gemma2_9B.json
Encoder dataE5 dataset, following LLM2Vec§4.1 app. Acode loads meaningful96/EGLLM_Emb train_emb.py
Decoder data10,000 examples: 2,000 / 6,000 / 500 / 1,500 for 1 / 2 / 3 / 4 hopsapp. A tab. 8hop-aware mix of NQ, HotpotQA, 2Wiki, MuSiQue
Global batch, encoder16app. Acode: 8 per device x 2 accumulation Gemma2_9B.json
Global batch, decoder8app. Acode: 8 per device, no accumulation Gemma2_9B.json
LGC temperature tau0.05app. Acode scale 20 = 1/tau Gemma2_9B.json
LGC layer buckets5 buckets over the first half (§3.1.1); "bucket size of four" app. A§3.1.1 app. F.6see Paper vs code
SOA layerslast n = 4§4.1 app. A
lambda_SOA1e-3app. A
Encoder warm-up (MNTP)3,000 steps total, first 1,000 use MNTPapp. Bcode: 1,000 + 2,000 Gemma2_9B.json
Encoder learning ratenot statedcode only: 2e-4; 5 epochs set, but the run stops at 2,000 main steps Gemma2_9B.json Gemma2_9B.json
Encoder recall metricR@K, K in {2, 5, 10}app. A
Soft tokens per passage M5§4.1 §4.8 fig. 63 to 7 gives stable EM §4.8
Passages per question Knot statedcode: up to 10 at inference and in decoder training eval_e2e.py Gemma2_9B.json
lambda_KD0.75app. Acode: lambda_KD 1.0, lambda_NTP 0.8 (0.75 for LLaMA) Gemma2_9B.json
SST low-rank size rnot statedcode only: 512 Gemma2_9B.json
Decoder learning ratenot statedcode only: 2e-5 Gemma2_9B.json
Decoder stepsnot statedcode only: 700 steps, overriding 3 epochs Gemma2_9B.json
Anchor initialisationabout one hour; steps not statedapp. Bcode only: lr 1e-4, 1,000 steps, batch 2 x 4 accumulation train_token.py train_token.py train_token.py train_token.py
Inference decodingnot statedcode only: greedy, max 64 new tokens eval_e2e.py
Hardware4 x RTX A6000 for training, 1 GPU for inferenceapp. A

Results

Author-reported. Setup: five benchmarks with 1,000 test questions each (HotpotQA, 2WikiMultihopQA, MuSiQue, NaturalQuestions, SimpleQA). Baselines that only retrieve use Gemma2-9B as their generator §4.2 app. A. Table 1 and Table 2 show Gemma2-9B rows; backbone-wise rows are in tab. 10 and tab. 11.

Retrieval, Gemma2-9B backbone (Recall@2, @5, @10) tab. 1

MethodHotpotQA R@2 / R@5 / R@102Wiki R@2 / R@5 / R@10MuSiQue R@2 / R@5 / R@10
LLM2Vec0.6130 / 0.7854 / 0.86120.5845 / 0.6451 / 0.68290.3598 / 0.5126 / 0.5977
NV-Embed-v20.6318 / 0.7982 / 0.87360.5956 / 0.6632 / 0.72020.3781 / 0.5290 / 0.6236
OneGen0.5792 / 0.7450 / 0.84020.5584 / 0.6077 / 0.63920.3320 / 0.4777 / 0.5658
GritLM0.5890 / 0.7487 / 0.80850.5558 / 0.6311 / 0.69770.3760 / 0.4990 / 0.5983
ShiftRAG0.6598 / 0.8214 / 0.88060.6080 / 0.7109 / 0.75300.4186 / 0.5634 / 0.6516

Question answering, Gemma2-9B generator (EM / F1) tab. 2

MethodHotpotQA EM / F12Wiki EM / F1MuSiQue EM / F1
LLM2Vec0.5030 / 0.61060.4400 / 0.49140.2430 / 0.3670
NV-Embed-v20.5140 / 0.62200.4580 / 0.50640.2480 / 0.3783
GritLM0.4430 / 0.52580.3020 / 0.36650.1870 / 0.2872
OneGen0.4670 / 0.58690.4220 / 0.47850.2200 / 0.3342
ShiftRAG0.5610 / 0.67900.5080 / 0.55280.3050 / 0.4129

Backbone-wise ShiftRAG (R@2 and EM only; full rows in tab. 10 and tab. 11)

BackboneHotpotQA R@2 / EM2Wiki R@2 / EMMuSiQue R@2 / EM
Mistral-7B0.6235 / 0.46700.5837 / 0.32500.4019 / 0.2180
LLaMA3.1-8B0.6685 / 0.46300.6025 / 0.42000.4338 / 0.2360
Qwen2.5-7B0.6505 / 0.48400.5837 / 0.42100.3753 / 0.2050
Gemma2-9B0.6598 / 0.56100.6080 / 0.50800.4186 / 0.3050

Encoder-loss ablation (Recall@2 / @5 / @10 on MuSiQue) tab. 3: full 0.4186 / 0.5634 / 0.6516; without SOA 0.4045 / 0.5511 / 0.6493; without LGC 0.4008 / 0.5457 / 0.6140; without both 0.3916 / 0.5382 / 0.6051. The paper says the LGC term carries most of the gain and SOA gives a small, consistent gain §4.3 tab. 3.

Decoder-component ablation (MuSiQue EM / F1) tab. 4: full 0.3050 / 0.4129; without SST 0.2450 / 0.3639; without LoRA_dec 0.2650 / 0.3883; without both 0.1970 / 0.2979 §4.3 tab. 4.

Training strategy tab. 5 and tab. 25 (HotpotQA R@2 / EM, training time): joint with shared adapter 0.4896 / 0.1300, 21 h; joint with separate adapters 0.5264 / 0.4730, 23 h; alternating with shared adapter 0.5166 / 0.2200, 20 h; alternating with separate adapters 0.5938 / 0.5220, 10 h; separate learning (ShiftRAG) 0.6598 / 0.5610, 10 h §4.4 app. E.5.

Textual evidence versus soft tokens (EM / F1 on MuSiQue) tab. 6: ShiftRAG 0.3050 / 0.4129; textual relevance scores in the prompt 0.2650 / 0.3814; no SST 0.2450 / 0.3639 §4.5.

Latency and memory, HotpotQA tab. 7: NV-Embed-v2 8.72 s and 38.82 GiB, EM 0.514; GritLM 2.42 s and 20.25 GiB, EM 0.443; ShiftRAG 2.79 s and 24.14 GiB, EM 0.561 (single GPU, author-measured) §4.7 tab. 7.

Training cost tab. 19: LLM2Vec 5 h 30 min on 96,000 examples; OneGen 14 h on 174,500; GritLM 48 h on 2,886,912 with full fine-tuning; ShiftRAG 10 h 10 min on 106,000 examples app. D.1.

Single-hop and factual QA: NaturalQuestions Recall@2 0.6290, EM 0.3350, best of the compared methods tab. 12 app. C.3. SimpleQA EM 0.6740, F1 0.8005 against GritLM 0.5860 / 0.7182 tab. 13 app. C.4.

Embedding and generation under Mistral-7B tab. 14: MTEB clustering 0.5110, STS 0.8414, summarisation 0.3759 (ShiftRAG) against GritLM 0.5082, 0.8303, 0.3565. MMLU 0.6195 and TyDiQA 0.5681 are higher than GritLM's 0.5768 and 0.5545; GSM8K 0.5536 is below GritLM's 0.5762 tab. 14 app. C.5.

Latent chain-of-thought baselines adapted to retrieval tab. 15: Sim-CoT EM 0.5360 / 0.4420 / 0.2070 (HotpotQA / 2Wiki / MuSiQue) against ShiftRAG 0.5610 / 0.5080 / 0.3050 app. C.6.

Paper vs code

Repository: https://github.com/meaningful96/ShiftRAG at commit 75c9277be8df28e8e880c7535cd0c96585ad6814. The paper links it from the abstract. The repository has no LICENSE file, so no code is quoted here; the findings below are described in our own words. The README reports the same headline numbers as the paper README.md.

Differences:

  • Candidate layers. The paper samples one layer from each of five buckets over the first half of the layers §3.1.1. The code uses fixed layers 1, 4 and 8, plus the top layer Gemma2_9B.json. The score is the maximum over those candidates HardNegativeNLLLoss.py. The paper's own description of bucket size is also inconsistent (four in app. A, two to six swept in app. F.6).
  • Memory bank. The code keeps a 256-entry bank of earlier passage vectors and tops each batch up to 16 passages Gemma2_9B.json utils_emb_train.py. The paper does not mention this.
  • Weighting of the SOA term. The paper weights the query term and the passage term equally eq. 6. The code averages three terms (query, positive, negative) with weight 1/3 each, so passages get two thirds of the penalty utils_emb_train.py.
  • Encoder adapters. The paper describes one encoder adapter, LoRA_enc, with rank 16 §4.1. The code trains a warm-up adapter (LoRA1), freezes it, and trains a second adapter on top train_emb.py. The saved encoder is a composition of the two, so its rank is 32 utils_emb_train.py. The Mistral config uses rank 32 as well Mistral_7B.json.
  • Warm-up data. The paper averages the MNTP loss over questions and passages app. B. The code applies it only to the positive passage text utils_emb_train.py.
  • Prompt and pooling. The code wraps queries and passages with fixed retrieval instructions ("Represent the query..." / "Represent the document...") utils_emb_train.py. Pooling covers all non-padding tokens, including the template, because the skip flag defaults to off decoupledLLM.py. The paper does not mention either.
  • Decoder training passages. The code always keeps the gold passages and fills up to ten with randomly sampled distractors utils_gen_train.py. At inference, the ten top-ranked retrieved passages are used eval_e2e.py eval_e2e.py. The paper does not say which passages were used for training, and it does not measure the effect of this mismatch.
  • Distillation weight and normalisation. The paper states lambda_KD = 0.75 app. A. The shipped configs use lambda_KD = 1.0 and lambda_NTP = 0.8, or 0.75 for LLaMA Gemma2_9B.json. The loss is the weighted sum utils_gen_train.py. The paper sums the KL over answer positions eq. 8. The code averages over positions utils_gen_train.py, and its next-token loss is also an average utils_gen_train.py.
  • Anchor initialisation. The paper initialises the anchors from pretrained latent vectors §3.2 app. B. The code trains five prompt-tuning virtual tokens with PEFT's text initialisation from the instruction string train_token.py. At each step it uses a random prefix of one to five of them utils_token_train.py. The exported file keeps the five vectors, which are copied into the anchors train_token.py SST.py.
  • Relevance weight. The paper calls alpha_k a normalised relevance weight over the top-K similarities §3.2. The code takes a softmax over the K raw question-passage cosines with temperature 1 SST.py utils_gen_train.py. An optional gate exists but is off in every shipped config Gemma2_9B.json.
  • SST offsets. The paper says the low-rank projection yields offsets but does not give the mapping. The code multiplies a per-passage coefficient matrix by a learned r x h basis SST.py, with r = 512 in the shipped configs Gemma2_9B.json.
  • Consistent items. Temperature 0.05 (scale 20) Gemma2_9B.json; lambda_SOA = 1e-3 and SOA over the last four layers Gemma2_9B.json; five soft tokens Gemma2_9B.json; decoder rank 64 Gemma2_9B.json; 1,000 MNTP steps plus 2,000 main steps Gemma2_9B.json; 1,000 test questions per dataset . Scoring at inference uses final-layer cosine similarity through a flat inner-product index over normalised vectors eval_e2e.py eval_e2e.py.
  • Repository table numbers. The README cites the dataset tables as paper Table 6 (training statistics) and Table 7 (test statistics). In the TeX they are Table 8 and Table 9 README.md tab. 8 tab. 9.

Pitfalls

  • Unstated optimiser and steps. The paper gives no learning rates, step counts or decoding settings for the decoder and anchor stages. The code uses one learning rate of 2e-5 for the decoder and SST together, with 700 steps Gemma2_9B.json Gemma2_9B.json. The optimiser is the default of the trainer, with no separate group for the SST layers utils_gen_train.py. Freshly initialised SST weights may therefore learn slowly. This follows from the code, not from a run.
  • Passage count K. The paper says "top-K" but never gives K for question answering. The code uses ten eval_e2e.py. A study of soft-token count varies only M §4.8 fig. 6.
  • Training and test passages differ. Decoder training always includes gold passages utils_gen_train.py, while inference passes the retriever's top ten. The reported results use the second setting at inference. The effect of the difference is not measured.
  • Near-uniform relevance weights (inferred). With temperature 1 and cosine values in a narrow band, the softmax over ten passages stays close to 0.1 each SST.py. The "without relevance weight" ablation sets every weight to 1 rather than to 1/K app. C.7 tab. 16. That ablation does not isolate the softmax weighting from the offset scaling.
  • Anchor training is uneven. Each step uses a random prefix of one to five anchors utils_token_train.py. Later anchors receive fewer updates. The paper says the stage takes under an hour app. B but gives no count of steps.
  • Sign and scale of the gap. The gap is final-layer cosine minus candidate cosine, so a positive value means the final layer matches the question better eq. 1. Each gap lies in [-2, 2], and the loss multiplies the score by 20, so the softmax temperature is set by that factor alone HardNegativeNLLLoss.py. The paper reports little sensitivity to tau across 0.001 to 0.1 app. F.5.
  • SOA scale. The penalty is normalised by S squared, and padding is masked out utils_emb_train.py. Its weight of 1e-3 is small. The ablation drop without SOA is 0.35% on MuSiQue R@10 tab. 3.
  • Paper inconsistencies that affect reproduction.
    • Table 16 ("Ours" row) reports MuSiQue EM 0.2960 and 2Wiki F1 0.5506, while Table 2 and Table 4 report 0.3050 and 0.5528 for the same model app. C.7 tab. 16.
    • The relative-gain formula in the scalability section is written as an absolute difference times 100, but the reported values are relative (57.24% = 0.7530 / 0.4789 - 1) app. E.1 tab. 21.
    • Table 21's 10,000-candidate row reuses cells from other tables. ShiftRAG's 0.7530 is its 2Wiki R@10 tab. 1, and RaLa's 0.4789 is Contriever's HotpotQA R@2 app. C.1. Its provenance is unclear.
    • The latency-trade-off text quotes cosine latency of 0.92 s and gap-aware latency of 3.34 s app. D.3, while the table lists 0.93 / 0.96 / 0.87 s and 3.17 / 3.73 / 3.12 s tab. 20.
  • Batch sizes are per device in the code. The paper's global batch sizes of 16 and 8 app. A cannot be matched to the code's per-device settings without the GPU count. The shipped scripts use every visible GPU train_gemma.sh.
  • Cost. Inference uses 24.14 GiB of VRAM at 2.79 s per question tab. 7, against 2.42 s and 20.25 GiB for GritLM. The gain over GritLM is 26.64% EM on HotpotQA §4.7. Latent-CoT adaptations are compared separately app. C.6.
  • Licence of the code. The repository has no LICENSE file, so the code cannot be reused, copied or quoted here. Only the method is described.
  • Limitations stated by the paper. Soft tokens are not readable text, and the method needs white-box access to the backbone §5.

Lineage

  • Builds on: LLM2Vec for bidirectional attention and the masked next-token warm-up §3.1.3 app. B; RaLa for layer contrast in retrieval, whose score the paper replaces with a mean-pooled gap §3.1.1 app. D.2; SPRING for latent-vector initialisation of the anchors §3.2; DSAS and DoLa for layer-wise representations §3.1.1.
  • Compared against: Contriever, GTR, ColBERTv2 and RaLa (encoder retrievers); GTE-Qwen2, LLM2Vec and NV-Embed-v2 (LLM retrievers); OneGen and GritLM (unified models); Coconut, CODI and Sim-CoT (latent chain-of-thought) app. A app. C.6.
  • Superseded by: no later version is recorded.

Source and license

Adapted from “ShiftRAG: Bypassing the Textual Bottleneck via Decoupled Learning and Continuous Soft Tokens” by Youmin Ko, Jihong Jeong, Hyunjoon Kim (arXiv:2611.00871), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/meaningful96/ShiftRAG at commit 75c9277be8df (none (no LICENSE file in the repository)). Any quoted code is shown under that license.