AI auditing

Overview

An audit, in this sense, is a systematic and independent process that gathers and evaluates evidence about an entity and reports the result to stakeholders. The entity can be a system, an organisation, a process, or a mix of these 2302.08500 §3.1. The paper argues that large language models need audits at more than one level. Checking one application against the sector's rules is not enough, because one model serves many downstream tasks 2302.08500 §1 2302.08500 §3.2.

The proposal has three layers that feed each other: governance audits of the provider, model audits before any product adapts the model, and application audits of each product 2302.08500 §4.1. The "Variants" section below describes each layer and the access it needs.

The sources are one position paper. It has no experiments, and it reports no pass or fail thresholds for any characteristic 2302.08500 app. A 2302.08500 §4.3. Its evidence is a literature review and more than 20 expert interviews, and it does not claim that an audit makes a system ethical or legal in any universal sense 2302.08500 §2.4. Treat the layers as a checklist for what to examine and who should see the results.

Variants

The paper's three layers are the variants. They differ in what is examined, when, and how much access the auditor needs.

Governance audit of the provider

This audit looks at the organisation that builds the model. It covers three tasks: whether governance and quality management can capture LLM-specific risks and separate accountability from blame; an audit trail of design choices, training data, testing and documents such as model and system cards; and a map of roles so that each foreseeable risk has an owner 2302.08500 §4.2. The output is a full report to management and selected evidence for law enforcers and downstream developers 2302.08500 §4.2. It needs white-box access. The paper calls the non-disclosure and data-sharing agreements this requires a logistical burden 2302.08500 §4.2.

Model audit before adaptation

This audit runs after pre-training and before any application adapts the model 2302.08500 §4.1 2302.08500 §4.3. Its two tasks are to find limits that should shape redesign and licence terms, and to tell downstream developers what the model can and cannot do, for example through interactive model cards and information about the training data 2302.08500 §4.3.

The paper gives two modes. The intrinsic mode inspects the model's characteristics, such as its training data, without a use case in mind. The indirect mode tests the model across possible downstream uses, links the results to characteristics, and combines them with weights 2302.08500 §4.3.

The paper names four characteristics as a minimum. It chose them for being socially relevant, predictably transferable to applications, and measurable with available tools 2302.08500 §4.3:

• Performance, measured on standard suites such as GLUE, SuperGLUE and BIG-bench. The paper warns that benchmarks saturate 2302.08500 §4.3.
• Robustness, meaning behaviour on unexpected inputs, out-of-domain inputs and adversarial attacks. Named tools include AdvGLUE, ANLI, Robustness Gym and Dynabench, and the paper asks for several of them, including group robustness 2302.08500 §4.3.
• Information security, meaning how easily training data can be extracted, measured by a memorisation score called exposure 2302.08500 §4.3.
• Truthfulness, meaning the model's overall truthfulness rather than single statements. TruthfulQA is the example benchmark 2302.08500 §4.3.

The paper also asks for a review of the training datasets for gaps and bias, and names documentation, representativeness criteria and bias metrics for word embeddings as tools for it. It notes that large datasets are costly to inspect 2302.08500 §4.3. This audit needs medium access: the auditor may change model parameters and review the learning procedure and task objectives 2302.08500 §4.3.

Application audit of a product

This audit looks at one product built on the model. It has two parts 2302.08500 §4.4:

• Functionality audit: whether the intended purpose is lawful and ethical, and whether it fits the model's intended use and documented limits.
• Impact audit: the effects of outputs on users, groups and the environment. Before launch, the paper suggests sandbox testing and an ethical impact assessment or foresight analysis. After launch, it suggests continuous monitoring, either by periodic review or by automated alerts when predefined tolerance spans are exceeded 2302.08500 §4.4.

Black-box access is enough for input-output analysis, and public information may be enough for compliance checks 2302.08500 §4.4. The paper wants results published at least in summary and favours registries for them. It says these audits should be used more selectively than the other two layers 2302.08500 §4.4. See the Contradictions section on this point.

Links between the layers

The paper asks that findings flow between layers 2302.08500 §4.5:

• Model audit reports inform application audits, which check whether the known model limits were taken into account in the product.
• Output logs from products inform model redesign and the model card.
• Governance audits check that the provider has a process for acting on application feedback.

The paper names these paths but does not specify their format, owners or timing 2302.08500 §4.5.

When to use which

The paper gives no measured rules, so these rows follow the constraints the paper states. They are not tested recommendations.

Implementation sketch

The paper gives no code. The sketch restates the three layers as one procedure, in the card's words, and marks which step each part comes from. Names are ours.

# Three-layer audit of an LLM and the products built on it  [arxiv:2302.08500 §4.1]
# Input:  provider, model after pre-training, downstream applications
# Output: governance report, model report, application reports and logs

def audit_llm_stack(provider, model, applications):
    # Governance audit of the provider, white-box access  [§4.2]
    governance_report = governance_audit(provider)

    # Model audit before adaptation, medium access  [§4.3]
    model_report = model_audit(model)

    # Model limits go to the provider and to downstream developers  [§4.5]
    publish_limits(model_report)

    # Which applications to audit is left to the auditor  [§4.4]
    application_reports = []
    for app in applications:
        # Functionality and impact audit, before and after launch  [§4.4]
        report = application_audit(app, model_report)
        # Product logs go back to the model card and to redesign  [§4.5]
        feed_back(report.logs, model)
        application_reports.append(report)

    # Governance audit checks that the feedback path exists  [§4.5]
    check_feedback(governance_report, application_reports)
    return governance_report, model_report, application_reports

Contradictions

Only one source is on this page, so no two papers disagree. The paper has three internal inconsistencies, and an implementer has to settle each one.

• Number of design claims. Section 3.2 derives seven claims 2302.08500 §3.2. The methodology appendix refers to six key claims in two places 2302.08500 app. A. The paper does not reconcile the two counts.
• Obligatory or selective application audits. Section 4.4 says governance and model audits should be obligatory, and application audits should be used more selectively 2302.08500 §4.4. The conclusion asks providers to require application audits from products built on their models 2302.08500 §6.
• Thresholds and frequency. The paper asks for predefined tolerance spans and periodic review, but it gives no pass or fail values, tolerance values or frequency 2302.08500 §4.3 2302.08500 §4.4. This is a gap rather than a disagreement; each deployment must choose and record its own values.

Papers

• Auditing large language models: a three-layered approach (2302.08500): proposes governance, model and application audits for LLMs and argues that audits must be external, collaborative and continuous. Position paper; no experiments, no thresholds.