Bayesian network

Overview

A Bayesian belief network (BBN) is a probabilistic model with a structure S and parameters θ. Two papers on this wiki use it, for different jobs.

In the first job the parameters are random: they carry a prior, and learning from data updates it. A prediction averages over all parameter settings rather than using one fitted set 2605.04664 p. 2. Learning is simplified by two assumptions the paper states, parameter independence and conjugate priors, so the posterior keeps the prior's form 2605.04664 p. 2. This averaging is the step the anomaly-test variants below share.

The paper uses the model for a conditional anomaly test. For a case, the target attribute is one decision A(x) and C(x) holds the other attributes. The statistic is the predictive probability p(A(x) | C(x), M), and the decision is flagged when it falls below a fixed threshold 2605.04664 p. 2. The model is fitted on similar past cases only. Scoring against all cases is biased when the target sits in a small subpopulation, because the routine decisions of that group look unusual against the majority 2605.04664 p. 2.

In the second job the network is learned over every attribute of a tabular dataset, and its edges are read as dependencies. No predictive probability is computed from it 2110.00530 §2.2.

Design choices sit on top of these models: which past cases count as similar, which model to fit, and, for the dependency graph, how to discretise and how strongly to penalise edges. The anomaly paper crosses three selections (all cases, Mahalanobis distance, weighted Mahalanobis distance) with two models (a BBN and Naive Bayes) 2605.04664 tab. 2. A Bayesian neural network is a different object, with weights as random variables: see Bayesian neural network.

Variants

BBN on all past cases

The structure is learned greedily with a maximal marginal-likelihood score, on 2187 cases, and then held fixed. Parameters are re-learned for each set of cases used 2605.04664 p. 4. With all cases, the table reports AUC-ROC 0.81 and AUC-PR 0.56, the best AUC-ROC of any row 2605.04664 tab. 2. Card: Evidence-based anomaly detection in clinical domains.

BBN on similar cases

Take the k = 40 past cases nearest the target, under a Mahalanobis or weighted Mahalanobis distance over C(x), and learn the parameters from them alone 2605.04664 p. 3 2605.04664 p. 4. Reported AUC-ROC is 0.79 with Mahalanobis selection and 0.80 with weighted selection; AUC-PR is 0.52 and 0.56 2605.04664 tab. 2. The paper's explanation for the gap to the all-cases BBN is model complexity together with small samples: a smaller sample fits the target better but estimates the parameters less accurately [arxiv:2605.04664 p. 5].

Naive Bayes

A simpler model with Hospitalization as the class variable 2605.04664 p. 4. It serves as the baseline 2605.04664 p. 5. Its reported AUC-ROC and AUC-PR rise with each selection step: all cases 0.74 and 0.42, Mahalanobis 0.77 and 0.49, weighted 0.80 and 0.60 2605.04664 tab. 2. Weighted selection gives the highest AUC-PR in the table.

Dependency graph over dataset attributes

Learn a directed graph over all attributes of a dataset, with the class attribute constrained to be a leaf. The graph then shows which features bear on the class 2110.00530 §2.2 2110.00530 eq. 2. Numerical attributes are discretised before learning, with bins set per dataset [arxiv:2110.00530 §2.2]. The paper reads a direct or indirect edge from a protected attribute to the class as a sign that the data may be biased 2110.00530 §2.2. The authors apply this to each of 15 tabular fairness datasets as part of a dataset survey; the same card reports only logistic-regression scores, not network outputs 2110.00530 §4.1. Card: A survey on datasets for fairness-aware machine learning.

When to use which

The anomaly evidence is one retrospective study: binary attributes, one target decision, 2287 PORT patients, and an expert panel as gold standard 2605.04664 p. 3. Transfer to other settings is untested. The results are one leave-one-out run over 100 cases with 23 positives, with no variance reported 2605.04664 p. 4. Differences such as 0.56 against 0.52 in AUC-PR should not be read as significant.

The dependency-graph evidence is one survey. Its objective includes a penalty, but the paper gives no value for it and names no search algorithm 2110.00530 §2.2. Learned edges therefore depend on choices the paper does not fix, so an edge is a prompt for inspection rather than a finding.

Implementation sketch

Pseudocode for the predictive statistic only, written from the text. The paper gives no pseudocode.

# Predictive probability of a decision under a BBN  [arxiv:2605.04664 p. 2]
# Input:  past_cases: rows over (decision, C(x) attributes), e.g. the k
#                     nearest cases, or all cases  [p. 3] [p. 4]
#         structure: fixed BBN structure S, learned beforehand  [p. 4]
#         prior: prior over parameters; family not stated  [p. 2]
# Output: p(A(x) | C(x)), averaged over parameter settings

def predictive_probability(past_cases, target_other, structure, prior):
    # Parameter independence and conjugate priors reduce learning to
    # updating sufficient statistics  [p. 2]
    posterior = update_sufficient_stats(prior, structure, past_cases)
    # Average over θ rather than plugging in one estimate  [p. 2]
    return posterior_predictive(posterior, structure, target_other)

Points the paper leaves open, each one an implementer's choice:

• Prior family and hyperparameters. Not stated p. 2. The predictive probability depends on this choice.
• Attribute weights. The paper says the weights come from a Wilcoxon rank-sum statistic per attribute, but not how that becomes a weight, nor whether the statistic used the target case 2605.04664 p. 3 2605.04664 p. 4. Compute it on training cases only, or the weights leak the outcome.
• Covariance. The sample behind Σ is not stated. With 40 neighbours and 18 binary attributes the estimate may be ill-conditioned, and the paper discusses no regularisation 2605.04664 p. 3.
• Threshold p_ε. Not reported for any setting; the paper sweeps it over [0, 1] to draw its curves 2605.04664 p. 4.
• Rejection rule. The 2-standard-deviation rule for dropping a subpopulation is described, but the paper does not say it was used 2605.04664 p. 3.
• Dependency graph: penalty and discretisation. The penalty value and the search algorithm are not stated, and the bins are set per dataset 2110.00530 §2.2. Fix and report both before reading the edges.

Contradictions

No disagreement between the two papers, because they answer different questions. They do learn structure differently: the anomaly paper uses a greedy search with a maximal marginal-likelihood score 2605.04664 p. 4, while the survey uses a penalised objective with a leaf constraint on the class and names no algorithm 2110.00530 §2.2. Neither reports a comparison of structure learners.

Within the anomaly paper, the ranking depends on the metric: BBN on all cases has the best AUC-ROC (0.81) and Naive Bayes with weighted selection has the best AUC-PR (0.60) 2605.04664 tab. 2. The headline precision of 0.5 at 0.53 recall 2605.04664 p. 1 and the corrected estimate of 0.32 precision at 0.3 recall on the full PORT data 2605.04664 p. 5 are not on the same set. The authors say precision may be overstated on the enriched set, and do not reconcile the two figures.

Papers

• A survey on datasets for fairness-aware machine learning: learns a Bayesian network over each of 15 tabular datasets, with the class as a leaf, to show links between protected attributes and the outcome.
• Evidence-based anomaly detection in clinical domains: flags a clinical decision as anomalous when a BBN or Naive Bayes model, fitted on the most similar past patients, gives it low predictive probability.