Evolutionary algorithm

Overview

An evolutionary algorithm keeps a set of candidate solutions, scores each one with a fitness function, keeps the better ones as parents, and makes children from them by recombination or mutation. The loop then repeats for a fixed number of rounds or until a stopping rule fires. The three cards below share this loop. They differ in what a candidate is, how a child is made, and what the fitness costs to compute.

The invariant the variants share: each round replaces part of the population with children of selected parents, and the best-scoring member is what the method returns 2208.02400 §3.1 2405.20132 §III-D 2511.07262 §2.

Variants

Evolving the rows of an ensemble

A candidate is one bag: a bootstrap sample of training rows, with one decision tree fitted to it. Crossover swaps the rows each parent gets wrong, a fixed share of fresh random bags enters each round, and a few bags are mutated by swapping rows 2208.02400 §3.1 2208.02400 §3.3 2208.02400 §3.4. Fitness is the bag's performance times a size term (K + bag size) / K 2208.02400 §3.2. The paper leaves unstated the data on which the performance term is measured 2208.02400 §3.2. Card: Evolutionary bagging.

Evolving code written by a language model

A candidate is a Python optimiser. The language model writes the first one from a task prompt, and each later child is a rewrite of the selected parent, guided by the parent's code, its score and its error text [arxiv:2405.20132 §III-D]. The score is the normalised area over the convergence curve (AOCC), averaged over benchmark functions, instances and repeated runs [arxiv:2405.20132 §IV-C]. The paper's main loop is one parent and one child per round, with (1+1) or (1,1) selection 2405.20132 §III-D. Card: LLaMEA.

Evolving solution trees with agents

A candidate is a complete scientific-ML solution, stored in a tree with its score and an analysis report. The best node is always a parent. Three more parents are picked by a majority vote of three selector models [arxiv:2511.07262 §2] (Ensemble-Guided Parent Selection). Each child comes from a debate between a proposer and a critic, then an engineer edits the parent's code, and a debugger repairs it 2511.07262 §2. Card: AgenticSciML.

When to use which

The choice depends on what a fitness evaluation costs, what the mutation operator costs, and how much of the search space you can express in code.

Notes on reading the numbers:

• All results are author-reported. None of the three papers reports seeds or repeated runs for the headline comparison, except that LLaMEA averages five runs per configuration 2405.20132 §V.
• The benchmarks differ, so the scores cannot be compared across rows of the table. The bagging result uses tabular datasets, LLaMEA uses noiseless BBOB functions, and AgenticSciML uses six PDE and operator-learning tasks 2208.02400 §4.2 2405.20132 §IV-B 2511.07262 §3.
• If fitness is computed on the same rows a model was trained on, the score is optimistic. The evolutionary bagging fitness can work this way, and its paper does not say which data is used 2208.02400 §3.2.
• In the agent card, selection and reporting use one validation set, so the champion number is optimistic 2511.07262 §S1.1.1. Re-evaluate on held-out data before comparing methods.

Implementation sketch

The loop shared by the three variants. Each variant changes the vary and select steps, and the cost is in fitness and vary.

# Common loop of the three variants (generic sketch, not a released function)

def evolve(population, fitness, select, vary, rounds):
    # Score every member. The score is a training accuracy, a benchmark
    # AOCC or a validation score, depending on the variant [arxiv:2208.02400 §3.2]
    scores = [fitness(m) for m in population]
    for _ in range(rounds):
        # Choose parents. Top-ranked [arxiv:2405.20132 §III-D], or the best
        # node plus votes [arxiv:2511.07262 §2]
        parents = select(population, scores)
        # Make children by recombination, mutation or an LLM rewrite
        # [arxiv:2208.02400 §3.4] [arxiv:2405.20132 §III-D] [arxiv:2511.07262 §2]
        children = [vary(p) for p in parents]
        # Score the children. This is the costly step in every variant
        child_scores = [fitness(c) for c in children]
        # Keep the best of the old and new members. Ties are handled
        # differently in the code than in the printed algorithm, see below
        population, scores = keep_best(population + children,
                                       scores + child_scores)
    return best_of(population, scores)

Where the variants differ:

• Crossover. The row-level variant gives each parent's correct rows to itself and passes its wrong rows to the other parent 2208.02400 §3.4.
• Mutation. The row-level variant swaps a fixed number of rows in a few bags 2208.02400 §3.4. The LLM variants change the candidate by asking a model for an edit 2405.20132 §III-D.
• Ties. The printed update rule in the LLM paper accepts an offspring on a tie 2405.20132 alg. 1. The released code keeps the parent on a tie see the card's Paper vs code section.

Contradictions

• Parent selection in the bagging paper. The text names rank selection in one place 2208.02400 §3.4 and probabilistic selection by fitness in another 2208.02400 §3.3. The released code takes the top-ranked bags.
• Two-spiral gain in the bagging paper. The text gives 11.6% over bagging and 18.3% over random forest 2208.02400 §4.3. The main table gives 8.4 and 12.0 percentage points 2208.02400 tab. 4. Both are reported; this page uses the table.
• Ensemble size from the test set. The bagging paper chooses the number of bags on the test set 2208.02400 §4.2. The released code uses cross-validation on the training data see the card's Paper vs code section.
• Generalisation of the best LLM method. The abstract of the LLM paper says several generated algorithms beat CMA-ES 2405.20132 §I. The text says two algorithms beat tuned CMA-ES after the full budget, but Table I lists the second one below it 2405.20132 §VI-A 2405.20132 tab. I.
• Model line-up in the agent paper. The cost tables label the critic and debugger as GPT-4o-Mini, and the configuration tables say GPT-5 Mini 2511.07262 tab. S2 2511.07262 tab. S1. Both are reported here; the card's Pitfalls section lists the rest of the discrepancies.
• Benefit of the knowledge base. The agent paper reports one ablation with one run per setting and no stated seed 2511.07262 §S2.2. No other result in the cited set tests the same effect.

Papers

• Evolutionary bagging: evolves the training rows of each bagged tree with crossover, mutation and fresh bags.
• LLaMEA: a language model writes and rewrites Python optimisers, scored on 5-D BBOB.
• AgenticSciML: an LLM-agent tree search for scientific-ML models, with ensemble-chosen parents.