--- kind: concept slug: fill-in-the-middle title: Fill-in-the-middle (infilling) summary: >- Training a left-to-right code model to write the code between a given prefix and suffix. Covered here by one card: StarCoder applies character-level FIM at rate 0.5 in PSM or SPMv2 layout, and reports better single-line infilling than SantaCoder and InCoder. SPMv2 is not defined in the paper, and no other FIM card exists in this wiki yet. tags: [code, infilling, pretraining] papers: ["2305.06161"] related: [code-completion, language-model-pretraining, subword-tokenization, decoding-strategies] status: draft updated_at: 2026-10-24 --- ## Overview Fill-in-the-middle (FIM) lets a model write a span that sits between text it can already see on both sides: code before the cursor and code after it. A plain left-to-right model only sees the text before the cursor. The approach described in the StarCoder card does not change the network. It rewrites part of the training data so that the model learns to produce the missing middle after a prefix and a suffix [arxiv:2305.06161 §5.1] [arxiv:2305.06161 §5.4]. The idea is shared across variants: a training document is cut into three parts, prefix, middle and suffix, and the parts are reordered and tagged with sentinel tokens. The model is trained on the reordered text with the usual next-token loss. Only one paper card in this wiki covers the technique. Other models that the StarCoder paper names as FIM models have no card here, so this page does not compare them. The [code completion page](/concepts/code-completion) gives the wider picture of editor-based code help. ## Variants ### Character-level FIM with PSM or SPMv2 layout (StarCoder) StarCoder applies FIM to source code files only, at character level, with a FIM rate of 0.5. When a file is transformed, the layout is PSM with probability 0.5 and SPMv2 with probability 0.5 [arxiv:2305.06161 §5.1]. The paper's sentinel tokens for this are ``, ``, `` and `` [arxiv:2305.06161 tab. 10]. Code files also carry repository, file and star metadata prefixes before the FIM step [arxiv:2305.06161 §5.1]. The paper does not define the SPMv2 layout. It names the layout and cites the original FIM paper, Bavarian et al. (2022), for the FIM method [arxiv:2305.06161 §5.1]. That paper has no card in this wiki. An implementation that needs SPMv2 must take its definition from the original FIM paper. Card: [StarCoder: may the source be with you!](/papers/2305.06161). ### Sentinel-tagged templates for other sources The same paper uses sentinel tokens for three other data sources: GitHub issues, Jupyter notebooks and Git commits [arxiv:2305.06161 §5.1]. These are templates, not infilling. Commits show the code before a change, the commit message and the code after it, separated by sentinels. They are used to prompt the model to edit code from an instruction [arxiv:2305.06161 §5.1]. This variant is outside the FIM transform itself, so the rest of this page does not cover it. ## When to use which | Situation | Use | Why | |---|---|---| | You need the model to write code between a prefix and a suffix, and you have open weights | FIM training recipe of the StarCoder card, character-level, rate 0.5 [arxiv:2305.06161 §5.1] | The paper reports that StarCoderBase beats SantaCoder and InCoder on single-line infilling for Python, Java and JavaScript [arxiv:2305.06161 §6.2] | | You train your own model and need FIM | PSM only, unless you take the SPMv2 definition from the earlier FIM paper | The SPMv2 layout is not defined in this paper [arxiv:2305.06161 §5.1]. This is an inference, not a result from the paper | | You plan to ship the weights in a product | Check the weights licence first | The StarCoder weights are not under the paper's CC BY-SA licence. They use an OpenRAIL-M-type agreement [arxiv:2305.06161 §1] | | You only need left-to-right continuation | FIM training is not needed for this | The FIM results in this card are about infilling. They say nothing about plain completion [arxiv:2305.06161 §6.2] | The infilling result is single-line, on Python, Java and JavaScript benchmarks built from HumanEval and MultiPL-E style prompts [arxiv:2305.06161 §6.2]. The card does not report multi-line or repository-level infilling, so do not extend the claim to those. ## Implementation sketch The transform applied to a code file, written in our own words. Only the PSM layout is shown, because SPMv2 is not defined in the paper. ```pseudocode # Apply the fill-in-the-middle transform to one code file [§5.1] # Input: code (str); rate (float), 0.5 in StarCoder # Output: training string (str) def fim_transform(code, rate=0.5): # Leave the file unchanged with probability 1 - rate [§5.1] if random() >= rate: return code # Cut the file at two character positions. The paper does not # say how the cut points are sampled [§5.1] i, j = sorted(sample_two_cut_points(len(code))) prefix, middle, suffix = code[:i], code[i:j], code[j:] # PSM layout: prefix, then suffix, then the middle [§5.1] # The order follows from the layout's name return ("" + prefix + "" + suffix + "" + middle) ``` At inference, the prompt is the prefix and suffix in the same layout, ending at the `` token, and the model generates the middle. The paper's text does not show this prompt. It follows from the training format and is an inference. ## Contradictions None known in the cards in this wiki. Only one card covers this topic. ## Papers - [StarCoder: may the source be with you!](/papers/2305.06161): applies character-level FIM at rate 0.5 in PSM or SPMv2 layout during pretraining, with sentinel tokens, and reports single-line infilling results.