Carbon accounting for AI compute

Overview

The question is how much CO2 one job emits when it runs on rented GPUs. The shared idea is that emissions equal the energy the job consumes multiplied by the carbon intensity of the electricity at the time and place it is consumed. The paper calls the part it measures the operational term, O = E × I, where E is energy in kWh and I is a location-based marginal intensity in gCO2eq per kWh 2206.05229 §3.1 2206.05229 eq. 2. Embodied carbon of the hardware is left out and named as future work 2206.05229 §3.1 2206.05229 §8.

Only one paper in this wiki covers the topic so far, so the variants below describe that paper's method, its scope choices and the alternative it argues against. Card: Measuring the Carbon Intensity of AI in Cloud Instances.

Variants

Measured GPU energy times marginal intensity

Energy is read from the GPU with nvidia-smi during the run. CPU and memory power are measured too, but the emission estimate counts GPU energy only 2206.05229 §3.2. Each interval's energy is multiplied by the marginal intensity for that interval, and the products are summed to give the job's total 2206.05229 §5. The intensity series is WattTime marginal data at 5-minute resolution 2206.05229 §6. Reported results range from 34 to 128 gCO2eq per GPU-hour and from 200 to 755 gCO2eq per kWh across 16 Azure regions, averaged over a year 2206.05229 §5.1.

Shifting the job in time or region

The same intensity signal can drive a scheduler. Flexible Start picks the start time inside a window that gives the lowest total emissions. Pause and Resume runs the job only in the cleanest intervals of a window 2206.05229 §6. Both are simulated on historical data with perfect foresight, and the paper treats the gains as upper bounds on what a real scheduler could reach 2206.05229 §6. The largest reported gain is 27.7% for DenseNet 121 under Pause and Resume with a 24-hour extension 2206.05229 tab. 4.

Averaged or post-hoc estimates (argued against)

The paper says earlier estimates often used coarse regional averages or data that was not public 2206.05229 §1. It argues for the marginal rather than the average signal, because the marginal signal shows what a change in demand does to the grid 2206.05229 §2 2206.05229 §3.1. In its own comparison, it expects prior figures to be higher because they include data-centre overhead such as PUE, and it finds most prior estimates inside its own ranges 2206.05229 §5.1. The paper does not run an average-intensity estimator of its own on these workloads.

When to use which

Gains from pausing depend on job length and on the allowed extension. For the 6B model the Pause and Resume gain is 2.5% at a 24-hour extension and 11.4% at a 100% extension 2206.05229 tab. 4 2206.05229 tab. 5.

Implementation sketch

Operational emissions of one job, from energy and intensity on the same 5-minute grid 2206.05229 §3.1 2206.05229 §5:

# Operational emissions of one GPU job  [§3.1] [eq. 2] [§5]
# Input:  energy (T,) kWh per 5-min step, GPU only  [§3.2]
#         intensity (T,) gCO2eq per kWh, marginal, same steps  [§6]
# Output: total gCO2eq

def operational_emissions(energy, intensity):
    # Each step: energy used times the marginal intensity of that step  [eq. 2]
    total = 0.0
    for e, i in zip(energy, intensity):
        total += e * i
    # Cumulative emission is the sum over all steps  [§5]
    return total

Implementation notes:

• Align the two series before multiplying. The paper's results depend on both being on the same 5-minute grid 2206.05229 §6.
• Keep units explicit. The paper reports both per GPU-hour and per kWh, and its time-of-day table has no units 2206.05229 §5.1 2206.05229 tab. 3.
• The paper's scheduling simulations replay the job's measured energy profile from its start. Replaying it over non-contiguous paused steps is not described in the paper; treat that mapping as your own choice 2206.05229 §6.

Contradictions

Within the single paper, several numbers disagree:

• Finetuning energy. The text gives 3.2 kWh 2206.05229 §4, while 2206.05229 tab. 2 and the appendix caption 2206.05229 fig. 6 give 3.1 kWh.
• Full 6B run. The text extrapolates (60/8) × 13.8 MWh ≈ 103.5 MWh 2206.05229 §4. The caption of 2206.05229 tab. 2 gives 103,593 kWh. The extrapolation assumes linear scaling from a run stopped at about 13% of completion, which the paper does not validate.
• Duration saving. The caption of 2206.05229 fig. 4 says doubling the 6B run's duration can save "up to about 25%". The averaged 100% result in 2206.05229 tab. 5 is 11.4%. The caption does not say whether the 25% refers to one region or month.
• Model size. The energy measurement run uses BERT-base 2206.05229 §3.2 2206.05229 tab. 1. The pretraining and finetuning runs are described as BERT-small 2206.05229 §4. The paper does not say which model the 37.3 kWh pretraining energy comes from.

No cross-paper disagreements are known yet.

Papers

• Measuring the Carbon Intensity of AI in Cloud Instances: defines the operational carbon of a cloud GPU job as measured GPU energy times marginal grid intensity, and simulates region and time shifting over 11 workloads on Azure.