Carbon-aware scheduling

Overview

A cloud GPU job emits CO2 in proportion to the energy it uses and the carbon intensity of the electricity at the place and time it runs. The same job can therefore emit different amounts depending on its placement [arxiv:2206.05229 §3.1]. Carbon-aware scheduling means choosing that placement on purpose. The key quantity is the marginal grid intensity, not the average one, because it reflects what an extra unit of demand changes 2206.05229 §2 [arxiv:2206.05229 §3.1].

The only invariant across the variants is the operational estimate: energy for a step times the intensity for that step, summed over the run 2206.05229 §3.1. The variants differ only in which steps the job is allowed to use.

Variants

All three variants below are simulations on historical intensity data with perfect foresight. The paper treats the results as upper bounds, not as deployed gains 2206.05229 §6.

Region choice

Run the job in the cloud region with the cleanest grid. This is a placement decision made once, not a schedule. Across 16 Azure regions, the paper reports an average of 34 to 128 gCO2eq per GPU-hour for the most and least efficient region, and 200 to 755 gCO2eq per kWh 2206.05229 §5.1. The paper reports region choice as the largest operational lever of the three 2206.05229 §6.1. The intensity data covers 2020 only, and regions outside North America, Europe and Australia are absent 2206.05229 §6. Card: Measuring the Carbon Intensity of AI in Cloud Instances.

Flexible Start

Pick a start time within the next N hours that minimises the job's total emissions, then run to completion without interruption 2206.05229 §6. In the paper's 24 h window, the average gain over the year and 16 regions is 14.5% for BERT fine-tuning and 26.8% for DenseNet 121, but 0.5% for a 6.1 B parameter Transformer 2206.05229 tab. 4. The gain is largest for short jobs and smallest for long ones, which matches the paper's note that the effect depends on job length 2206.05229 §6.1. Card: Measuring the Carbon Intensity of AI in Cloud Instances.

Pause and Resume

Run the job only in the cleanest intervals inside a window of N hours plus the job's own duration, and pause in between 2206.05229 §6. The paper assumes that pausing and resuming is instant and uses no extra energy, which is an assumption and not a measurement 2206.05229 §6. With a 24 h window the gain is 19.0% for BERT fine-tuning and 27.7% for DenseNet 121, but 2.5% for the 6.1 B model 2206.05229 tab. 4. With a window equal to 100% of the job's duration, most workloads gain about 9.5% to 11.4%, while DenseNet 121 gains only 2.0% 2206.05229 tab. 5. The largest gains in the paper come from Pause and Resume, but they rely on a long window and a perfect-foresight oracle. Card: Measuring the Carbon Intensity of AI in Cloud Instances.

When to use which

Implementation sketch

The paper's Flexible Start and Pause and Resume both reduce to a choice over a window of per-interval intensities. The sketch below is our own, using 5-minute steps as the paper does 2206.05229 §6.

# Operational emissions of one job  [arxiv:2206.05229 §3.1]
# Input:  energy (T,) kWh per 5-min step; intensity (T,) gCO2eq/kWh
def job_emissions(energy, intensity):
    # Sum of energy times intensity over all steps
    return sum(energy * intensity)

# Flexible Start: one contiguous run, best start in the window  [arxiv:2206.05229 §6]
def best_start(intensity, job_energy):
    best, best_total = None, float("inf")
    for start in range(len(intensity) - len(job_energy) + 1):
        # Replay the job's energy profile from this start step
        total = sum(job_energy[k] * intensity[start + k]
                    for k in range(len(job_energy)))
        if total < best_total:
            best, best_total = start, total
    return best

# Pause and Resume: cleanest steps until the run time is covered  [arxiv:2206.05229 §6]
def cleanest_steps(intensity, duration_steps):
    # Rank steps by marginal intensity and keep the lowest ones
    ranked = sorted(range(len(intensity)), key=lambda i: intensity[i])
    return sorted(ranked[:duration_steps])

Where the variants differ: Flexible Start keeps the job contiguous; Pause and Resume may split it. The paper does not say how a measured energy profile maps onto non-contiguous steps. The sketch assumes the profile is replayed in order, which is our inference 2206.05229 §6.

Contradictions

Within the single source so far. No second paper is in the wiki yet.

• Relative-extension gain. The caption of a DenseNet/6 B figure says doubling the duration can save "up to about 25%" 2206.05229 fig. 4. The averaged 100% result in the table is 11.4% for the 6.1 B model 2206.05229 tab. 5. The caption does not say whether the 25% refers to one region or one month.
• BERT fine-tune energy. The text gives 3.2 kWh 2206.05229 §4; the table gives 3.1 kWh 2206.05229 tab. 2.
• BERT size. The energy measurement uses BERT-base 2206.05229 §3.2; the pretraining and finetuning runs are described as BERT-small 2206.05229 §4. The paper does not say which size gives the 37.3 kWh pretraining energy.

Papers

• Measuring the Carbon Intensity of AI in Cloud Instances: defines the operational emissions estimate on Azure GPU jobs and simulates region choice, Flexible Start and Pause and Resume over 16 regions.