MedGemma 1.5 Technical Report

In plain words

MedGemma is an open model that reads medical images and medical text and answers in text. This report describes version 1.5 of the 4B-parameter model. Version 1 handled flat, single images such as one chest X-ray. Version 1.5 adds several harder kinds of input: a 3D scan made of many slices, a pathology slide too large to look at in one piece, a chest X-ray where the model must point to a region, and two X-rays of the same chest taken at different times. It also reads lab reports and electronic health records better.

A 3D scan can have far more slices than the model can take in at once. The authors therefore pick a fixed number of slices spread evenly through the scan. A pathology slide is cut into a grid of square tiles, and a fixed number of tiles is kept as the model's input. Both choices trade detail for a size the model can handle.

The authors compare the new model with its predecessor and with other open and proprietary models, on their own benchmarks. They report large gains on the new imaging tasks and small gains or losses on older ones. The model is released as a research starting point, not as a clinical product.

TL;DR

• MedGemma 1.5 4B keeps the Gemma 3 4B architecture with a frozen MedSigLIP vision encoder, and adds medical pretraining, distillation and RL §1 §2.
• New inputs: 3D CT and MRI as up to 85 axial slices, histopathology whole-slide images as up to 126 patches, chest X-ray bounding boxes, and two-timepoint chest X-ray comparison §2.3 §3.2.
• Author-reported gains over MedGemma 1 4B: 3D CT accuracy 58.2 to 61.1, 3D MRI 51.3 to 64.7, lab-report macro F1 up on three of four datasets, MedQA 64.4 to 69.1 tab. 3 tab. 4.
• Some older benchmarks drop (for example PubMedQA 73.4 to 67.6, SLAKE 72.3 to 59.8), and the general-knowledge score drops (MMLU Pro 39.1 to 33.8) tab. 3 tab. 6.

Problem

Open medical vision-language models were mostly limited to 2D images §1. A 3D study cannot be fed to a 2D image encoder as a whole. Each slice must become a separate image, and the total then exceeds the context budget of a 4B model, so a sampling rule is needed §2.3.1. A whole-slide pathology image is far larger than the encoder input, so tissue must be cut into patches and a subset kept §2.3.2. The report says models and benchmarks for these harder imaging tasks are limited, and aims to cover them in one architecture §1.

Method

Overview. MedGemma 1.5 4B uses the architecture of Gemma 3 4B. The vision encoder is the roughly 400M-parameter MedSigLIP encoder from MedGemma 1 §1 §2.

Pretraining. The vision encoder is frozen. The language decoder receives additional supervised pretraining on text, interleaved image-text data from the Gemma mixture, and new medical image-text data §2.1. The new radiology, pathology, dermatology and document datasets are listed in tab. 1.

Post-training. The language model is refined with distillation and reinforcement learning, using the same recipes as Gemma 3 with extra medical data §2.2. Distillation samples 256 teacher logits per token, weights them by teacher probability, and trains the student with cross-entropy on those samples §2.2. The distillation uses an improved large instruction-tuned teacher plus supplementary teachers trained on CT Dataset 1, MRI Dataset 1 and internal histopathology data §2.2. The paper says these supplementary teachers were trained, but does not say how.

Dataset coverage. tab. 1 lists each training source with its example count and the stages it used (PT = pretraining, Distill, RL). The evaluation sets, with their OOD and public flags, are in tab. 2.

Volumetric CT and MRI input. Each volume is turned into a sequence of 2D axial images, each rescaled to 896 x 896 pixels §2.3.1. Slices can come from several z-stacked volumes per study. The inclusion rules are axial orientation, at most 512 x 512 pixels per slice, equal slice thickness, and at least five slices §2.3.1. A query holds at most 85 slices, which is 21,760 vision tokens. This keeps the full prompt, including the report indication and the findings, under 32K tokens §2.3.1. If the stacked volume has more than 85 slices, the slices are sampled at equal spacing along the z-axis §2.3.1.

# Select the volumes of one study  [§2.3.1]
def select_volumes(study):
    # Keep axial volumes with at least 5 slices and at most 512 x 512 px
    kept = []
    for volume in study.volumes:
        if volume.orientation != "axial":
            continue
        if volume.num_slices < 5:
            continue
        if volume.height > 512 or volume.width > 512:
            continue
        kept.append(volume)
    return kept


# Cap the slices of one query at 85 by equal spacing  [§2.3.1]
def sample_slices(slices, limit=85):
    # Keep everything when the stack already fits
    if len(slices) <= limit:
        return slices
    # Equally spaced indices along z; rounding rule not stated
    indices = equally_spaced_indices(len(slices), limit)
    return [slices[i] for i in indices]

CT intensity mapping. The encoder was trained on 256 intensity levels per channel, so raw Hounsfield units are mapped into the three colour channels with three windows: red (-1024, 1024), green (-135, 215) and blue (0, 80) HU §2.3.1. The paper motivates the windows, but it does not state the exact scaling into the 256 levels.

# Map CT Hounsfield units to an RGB slice  [§2.3.1]
def ct_to_rgb(hounsfield):
    # Three windows, one per channel; the 0-255 scaling is not stated
    red = clip_and_scale(hounsfield, low=-1024, high=1024)
    green = clip_and_scale(hounsfield, low=-135, high=215)
    blue = clip_and_scale(hounsfield, low=0, high=80)
    return stack_channels(red, green, blue)     # (H, W, 3)

MRI intensity mapping. MRI intensities have no physical scale, so no window is applied. Each volume is min-max normalised on its own, and the same value is written to all three channels §2.3.1. The target range of the normalisation is not stated.

Whole-slide image (WSI) input. A tissue mask is computed at 5x magnification in HSV colour space, using the PolyPath segmentation §2.3.2 (arXiv:2502.10536). For each slide, one optical magnification is drawn at random: 5x with probability 0.34, 10x with 0.33, and 20x with 0.33 §2.3.2. The mask is downscaled to the extraction stride. Non-overlapping 896 x 896 patches are taken from a grid over the tissue §2.3.2. The paper caps a slide at 126 patches (32,256 vision tokens), chosen by random subsampling without replacement. The original spatial order of the kept patches is preserved §2.3.2. Each patch is encoded as a PNG image and stored with the slide caption §2.3.2.

# Turn one whole-slide image into a patch sequence  [§2.3.2]
def wsi_to_patches(slide, cap=126):
    # One magnification per slide: 5x 0.34, 10x 0.33, 20x 0.33
    level = sample_magnification({5: 0.34, 10: 0.33, 20: 0.33})
    # Tissue mask at 5x, HSV segmentation (PolyPath)
    mask = tissue_mask(slide, magnification=5)
    # Non-overlapping 896 px grid over tissue, stride = patch size
    patches = grid_patches(slide, level, size=896, mask=mask)
    # Random subset without replacement when over the cap; keep order
    if len(patches) > cap:
        kept = random_subset_sorted(len(patches), cap)
        patches = [patches[i] for i in kept]
    return patches

Evaluation pipelines. For each new task, the model receives a prompt built from the inputs above. The exact templates are in app. B.

• 3D CT and MRI condition classification §3.2.1. Each condition is asked separately as a yes/no question, and the model writes its reasoning before a final answer. Prompts interleave the slices with a SLICE {index} marker and include the report indication. The internal CT and MRI test sets are balanced by sub-sampling. Accuracy is computed per condition and macro-averaged. On the public CT-RATE validation split, F1 is used and macro-averaged.
• WSI report generation §3.2.2. The input is the WSI patches plus the specimen label, for example "colon biopsy". The output is scored with ROUGE against the final-diagnosis section of the original report. Only one WSI and its text is used per query.
• Longitudinal chest X-ray §3.2.3. The model gets a prior and a current image and returns one of three classes: Improved, Stable or Worsened. The score is macro accuracy over five conditions (consolidation, edema, pleural effusion, pneumonia, pneumothorax).
• Anatomical localisation §3.2.4. The model gets one frontal chest X-ray and the question "Where is the {object}?" app. B. It returns a JSON list of objects with a label and a box [y0, x0, y1, x1], normalised to [0, 1] with the origin at the top-left. The metric is mean IoU.
• Lab-report extraction §3.2.5. Each PDF is rendered to an image, and the model returns a JSON list with one object per test. The main text lists the fields name, result, unit, specimen, method and sample collection time. The prompt in app. B also lists range and panel. A multi-phase label matcher pairs predicted and gold labels, and precision, recall and F1 are computed overall and per parameter.
• EHRNoteQA §3.3. Each of 962 multiple-choice questions (five options, MIMIC-IV discharge summaries) is answered with the whole note in the prompt.

Inference settings. Each example is run once. MedGemma 1.5 evaluations use temperature 0.0, chosen by informal tuning on a tune set. Other models use their default temperature and top-k §3. Prompts were revised for MedGemma 1.5 and tuned by hand on train and validation splits §3.1 app. B. The paper states that prompt changes can have large effects §3.

Hyperparameters

Training data tab. 1

Evaluation datasets tab. 2 (OOD = not seen in any development stage; Public = publicly available)

Table 2 gives the CT-RATE count as 1,558; the text in §3.2.1 says 1,564. See Pitfalls.

Results

Author-reported throughout. Setup: a 4B model on held-out data, one run per example, temperature 0.0 for MedGemma 1.5. The comparison models are the MedGemma 1 4B and 27B, Qwen3 VL 4B, Gemma 3 4B and 27B, and Gemini 3.0 Flash and Pro §3. Gemini models are proprietary, so their results cannot be reproduced from this paper.

Original MedGemma 1 tasks tab. 3. Columns: MedGemma 1 4B, MedGemma 1.5 4B, MedGemma 1 27B, Qwen3 VL 4B, Gemini 3 Flash, Gemini 3 Pro.

New tasks tab. 4. Columns: MedGemma 1 4B, MedGemma 1.5 4B, MedGemma 1 27B, Qwen3 VL 4B, Gemma 3 4B, Gemma 3 27B, Gemini 3.0 Flash, Gemini 3.0 Pro, External SOTA (from other papers, protocols differ, see footnotes in tab. 4).

CT-RATE (out-of-distribution, public) tab. 5 app. A. Macro F1: MedGemma 1 4B 23.5, MedGemma 1.5 4B 26.9, Gemini 3.0 Flash 8.5. Only a subset of models was run, because the protocol's runtime is long: 18 queries per condition app. A.

General, non-medical knowledge tab. 6 app. A. MMLU Pro: MedGemma 1 4B 39.1, MedGemma 1.5 4B 33.8, Gemma 3 4B 43.6 (prior report), MedGemma 1 27B 60.2, Gemma 3 27B 67.5 (prior report).

Headline figures in the abstract (author-reported, vs MedGemma 1 4B; see Pitfalls for how they were computed): 3D MRI condition classification +11 %, 3D CT +3 % (absolute), WSI "47 % macro F1 gain", chest X-ray localisation "+35 %" IoU, longitudinal chest X-ray "+4 %", MedQA +5 %, EHRQA +22 %, lab reports +18 % macro F1 on average over four datasets. §1 tab. 3 tab. 4

Figures (no images on this page; described per the wiki rules). fig. 1 shows the model family: MedGemma 1.5 4B covers 3D CT and MRI, WSI, localisation and multi-timepoint analysis, while MedGemma 27B and MedSigLIP stay available for knowledge tasks and classification. fig. 2 summarises averages over text QA, medical image interpretation (seven imaging tasks) and lab-report extraction. fig. 3 carries the caption "overview of MedGemma 1.5 capabilities", noting support for 3D radiology (CT and MRI volumes). Figure numbers follow the order of the LaTeX source.

Paper vs code

No official code. Search: the paper and its arXiv metadata have no code link. The only project link is goo.gle/medgemma, which redirects to the DeepMind MedGemma page; that page links only the google-deepmind GitHub organisation. A GitHub repository search for "medgemma" found Google-Health/medgemma (Apache-2.0, notebooks and a python directory). It is not linked from the paper or the project page, and its contents were not checked for training code, so it was not used. No training, preprocessing or evaluation code is available from the authors, so the pipelines above rest on the paper text alone.

Pitfalls

• Abstract numbers differ from the tables. The abstract gives a "11 %" gain for 3D MRI, but tab. 4 shows 51.3 to 64.7, which is 13.4 points. The "47 % macro F1" WSI gain is actually ROUGE-L in tab. 4 (2.2 to 49.4, 47.2 points). The "35 %" localisation gain is an absolute IoU change (3.1 to 38.0), not a relative one. Quote tab. 4 values, not the abstract percentages.
• Limitations list is incomplete. §4 names only SLAKE and VQA-RAD as regressions. tab. 3 also shows drops on PubMedQA (73.4 to 67.6), CXR14 (50.1 to 48.4), MMLU Med (70.0 to 69.7) and MMLU Pro in tab. 6 (39.1 to 33.8).
• Comparison claim with gaps. §4 says MedGemma 1.5 beats Qwen3 VL 4B on all vision tasks. This holds where both have numbers. Qwen3 has no entries for lab reports ("-") or WSI ("?") in tab. 4, so these tasks cannot be compared.
• Single runs, no variance. Every number is one run at temperature 0.0, with no confidence intervals. Differences of a few points (for example CheXpert 48.1 vs 48.2, or MIMIC MAIRA 40.5 vs 41.5) are within what a rerun could change.
• Prompts matter and are hand-tuned. The paper says prompt changes can move scores a lot, and the prompts were tuned by hand on train and validation splits §3. Reproducing the numbers needs the exact templates in app. B. The MS-CXR-T prompt is not listed in app. B, although the paper says the prompts not listed are the same as MedGemma 1's. Treat it as not stated.
• System prompt contradiction. app. B says MedGemma models have no system instruction, and then adds one ("think silently if needed") for seven benchmarks. Whether that instruction goes into a system turn or the user turn is not stated.
• CT resampling is ambiguous. app. A says CT-RATE was processed "without resampling". §3.2.1 says the volumes were the ones resampled to 480 x 480 x 240 by the dataset authors. These are consistent only if "without resampling" means the authors did not resample themselves.
• CT-RATE count differs. §3.2.1 says the validation split has 1,564 acquisitions from 1,304 patients. tab. 2 lists 1,558 evaluation examples.
• Query count for CT-RATE is unclear. app. A says the model was queried "18 times per condition". §3.2.1 describes one query per condition for 18 conditions. The appendix wording can be read as 18 queries per volume in total, or as 18 per condition. Use the reading that matches your cost budget and check it against the authors' code if released.
• Distillation is under-specified. The paper does not say how the 256 teacher logits are chosen per token beyond "weighted by teacher probabilities", nor the temperature, loss weights or mixture of data sources. The supplementary-teacher training recipe is also not described.
• RL details are missing. For WSI, RL uses token-level ROUGE-L as the reward §2.3.2. The RL algorithm, KL or clipping settings, group size and the reward for the other datasets (for example "RL" in tab. 1) are not stated.
• Lab-report scoring is not reproducible from the text. The multi-phase label matcher and metric code are named but not described. The field list differs between the main text (six fields) and the prompt (eight fields, adding range and panel) §3.2.5 app. B.
• Localisation protocol is partly missing. The paper defers the IoU definition to Wu et al. (chest ImaGenome) and gives no formula. How predicted boxes are matched to ground-truth boxes, and how multiple instances of one label are scored, is not stated.
• Token cost of long inputs. A CT or MRI query can reach 21,760 vision tokens and a WSI query 32,256, with the total kept under 32K. These sizes drive memory use during training, and the paper calls them "manageable" without giving memory figures §2.3.1 §2.3.2.
• Public data is a minority. Many training and several evaluation sets are internal (CT Dataset 1, MRI Dataset 1, CXR-IND1, the WSI set, the EHR sets 2 to 5, and the dermatology sets). Only the public sets in tab. 2 can be reproduced directly.
• Gemini and external baselines. Gemini models are proprietary and cannot be re-run from this paper. The "External SOTA" column uses numbers from other papers with different protocols (footnotes in tab. 4).
• Not for clinical use. fig. 2 says the model is not meant to be deployed without clinical fine-tuning. The paper also says its out-of-the-box functions are not clinical decision-making §4.

Lineage

• Builds on: MedGemma 1 (arXiv:2507.05201) §2 §2.1; Gemma 3 (arXiv:2503.19786) §2 §2.2; MedSigLIP and SigLIP, the vision encoder §2.1.
• Uses the segmentation of PolyPath (arXiv:2502.10536) for WSI tissue masks §2.3.2.
• Compared against: MedGemma 1 4B and 27B, Qwen3 VL 4B, Gemma 3 4B and 27B, Gemini 3.0 Flash and Pro, and external numbers in tab. 4 §3.
• Superseded by: not known from content already in this wiki.