Inference

Inference answers any probabilistic queries: given some observed evidence, what is the distribution over one or more query variables? infer computes this posterior exactly, by variable elimination, on a discrete network, i.e. a Bayesian Network (BN) or a Credal Network (CN). An Enhanced Bayesian Network (eBN) is not queried directly: it is first reduced to one of these (see the Reduction & Structural Reliability Problem chapter).

Evidence is given as an Evidence, a Dict{Symbol,Symbol} mapping node names to observed states, and the query as a single node name or a vector of them.

W = DiscreteNode(:W)
W[:W => :sunny] = 0.5
W[:W => :cloudy] = 0.5
S = DiscreteNode(:S, [:W])
S[:W => :sunny,  :S => :on] = 0.9
S[:W => :sunny,  :S => :off] = 0.1
S[:W => :cloudy, :S => :on] = 0.2
S[:W => :cloudy, :S => :off] = 0.8
bn = BayesianNetwork([W, S])
add_child!(bn, :W, :S)
order!(bn)

infer(bn, :S, Evidence(:W => :sunny))       # posterior P(S | W = sunny)

Posterior P(S | W=sunny)

S	Probability
------------------------
on	0.9
off	0.1

The result is a Posterior: the labelled probability table over the query, together with the schema, query, and evidence used to produce it. Passing an empty Evidence() returns the prior marginal:

infer(bn, :S, Evidence())                   # prior marginal P(S)

Posterior P(S)

S	Probability
------------------------
on	0.55
off	0.45

The query must not overlap the evidence, and both must name states that actually exist in the network.

Variable elimination

infer uses variable elimination [10]: the network's conditional probability tables become factors, the evidence restricts every factor that mentions an observed variable to its observed state, and each non-query, non-evidence variable is then summed out in turn, the factors mentioning it are multiplied together, and the variable is marginalised away. Multiplying the surviving factors and normalising yields the posterior over the query. The cost of the computation is dominated by the largest intermediate factor built along the way, which depends heavily on the order in which variables are eliminated.

Formally, write the joint distribution as a product of factors (potentials), initially one per node, its CPT ${\varphi }_i=p(y_i\mid \mathrm{Pa}(Y_i))$:

$$p(y_1,\dots ,y_n)=\prod _{i=1}^n{\varphi }_i.$$

To answer a query over the variables $Q$ given evidence $E=e$, every factor is first restricted to the observed states, ${\varphi }_i{|}_{E=e}$. Each remaining variable that is neither in $Q$ nor in $E$ is then eliminated with the two elementary factor operations, the product of all factors that mention it, followed by marginalization, i.e. summing the variable out of that product. Eliminating a variable $Y_k$ yields a single new factor

$$\psi =\sum _{y_k}\prod _{i:Y_k\in \mathrm{scope}({\varphi }_i)}{\varphi }_i,$$

which no longer mentions $Y_k$ and replaces the factors it was built from. Once every non-query, non-evidence variable has been eliminated, the product of the surviving factors is the unnormalized joint $p(Q,E=e)$; dividing by its total gives the posterior:

$$p(Q\mid E=e)=\frac{{\displaystyle \sum _{\mathbf{h}}\prod _i{\varphi }_i{|}_{E=e}}}{{\displaystyle \sum _Q\sum _{\mathbf{h}}\prod _i{\varphi }_i{|}_{E=e}}},$$

where $\mathbf{h}$ ranges over the variables other than $Q$ and $E$, and the denominator is the normalizing constant $p(E=e)$.

Progress bar

On a large network variable elimination is itself expensive — the cost is dominated by the largest intermediate factor. infer takes a progress keyword that shows a progress bar over the variables as they are eliminated. It defaults to isinteractive() (shown in the REPL, silent in scripts, tests, and this documentation); pass progress = true or progress = false to force it.

Elimination order

The elimination order is chosen greedily on the network's interaction graph (the moral graph): repeatedly eliminate the remaining node with the lowest score, adding the fill-in edges its removal induces, until none are left. Ren et al. [11] survey heuristics for constructing a good order and find the minimum-increase-in-complexity search, greedily removing the node whose elimination least increases the problem's complexity, to outperform the alternatives.

EnhancedBayesianNetworks.jl adapts this idea into a mixed scored function. Rather than committing to a single complexity measure, it exposes two and combines them:

• fill_score, min-fill heuristic: the ratio of fill-in edges an elimination would add to the edges it would remove, favouring nodes that introduce few new dependencies.

• factor_score, min-factor heuristic: the size of the factor an elimination would create, the product of the state-space sizes of the node and its current neighbours.

Writing $N(Y)$ for the current neighbours of a node $Y$ in the interaction graph $(V,E)$, and $|\mathrm{dom}(Y^{\prime })|$ for the number of states of a neighbour $Y^{\prime }$, the two scores are

$$s_{\mathrm{fill}}(Y)=\frac{|\{\{a,b\}\subseteq N(Y):\{a,b\}\notin E\}|}{|N(Y)|},$$$$s_{\mathrm{factor}}(Y)=|\mathrm{dom}(Y)|\prod _{Y^{\prime }\in N(Y)}|\mathrm{dom}(Y^{\prime })|,$$

with $s_{\mathrm{fill}}(Y)=0$ when $Y$ has no neighbours. The numerator of $s_{\mathrm{fill}}$ counts the neighbour pairs not yet adjacent (the fill-in edges elimination would add) and its denominator is the degree $|N(Y)|$ (the edges it would remove); $s_{\mathrm{factor}}$ is the number of entries of the factor that elimination would build.

The default, fill_factor_score, is the mix of the two: a tuple (fill_score, factor_score, node) compared lexicographically, so the min-fill score decides, ties are broken by the smaller resulting factor, and the node id breaks any remaining tie for a deterministic order. Either single heuristic can be selected instead by passing it as the last argument to perform inference:

infer(bn, :S, Evidence(:W => :sunny), fill_score)   # min-fill ordering only

Posterior P(S | W=sunny)

S	Probability
------------------------
on	0.9
off	0.1

The chosen heuristic changes only the order of elimination, never the computed posterior, exact inference is invariant to it; the order affects only how much work is done to reach the answer.

Complete-scenario probability

When every node is observed there is nothing to marginalise, and the joint probability of the full scenario is just the product of each node's conditional entry given its parents. joint_probability computes this directly, without running variable elimination:

joint_probability(bn, Evidence(:W => :sunny, :S => :on))   # 0.5 * 0.9 = 0.45

0.45

It requires a complete scenario, i.e. one state per node; for partial evidence or marginals, use infer.

Credal inference

On a Credal Network the local tables are interval-valued, so the network stands for a whole credal set of Bayesian networks [6]. Inference then returns a CredalPosterior carrying lower and upper posterior probabilities. These are obtained by running variable elimination over the extreme Bayesian networks of the credal sets, and taking the element-wise minimum and maximum over the resulting posteriors.

Extreme networks under which the evidence has probability zero are left out of that minimum and maximum. Under such a network the posterior is a $0/0$ ratio: the evidence could not have been observed, so the network says nothing about the conditional, and folding it in would destroy both bounds rather than widen them. Discarding it loses no information, because $A\cap E\subseteq E$ forces $P(A\cap E)=0$ wherever $P(E)=0$, so such a network contributes to neither side of the ratio. Conditioning this way is known as regular extension, introduced in Appendix J of [12] and stated for lower previsions in section 2.8 of [13]. A CredalPosterior keeps the surviving posteriors and reports, when displayed, how many extreme networks were discarded.

This matters as soon as a node is vacuous — a continuous node discretized from a bare Interval carries $[0,1]$ masses (see Imprecise node), its credal set is the whole probability simplex, and its extreme points are the degenerate distributions putting all the mass on a single state. Conditioning on one of its states is then only meaningful under regular extension.

If no extreme network admits the evidence, its upper probability is zero: the credal set holds that the evidence could not have occurred under any of its measures. The definition of regular extension covers this case by returning the vacuous $[0,1]$; EnhancedBayesianNetworks deliberately departs from it there and has infer raise an error instead, because evidence that is impossible under every measure of the credal set is in practice a modelling mistake, and a vacuous interval would bury it in a plausible-looking result. The tol keyword sets the threshold at or below which an extreme network counts as inadmissible; it defaults to 0.0, which discards exactly the impossible ones. Raising it prunes extreme networks whose evidence probability is merely very small, and so changes which measures the bounds range over — a modelling choice rather than a numerical one.

Regular versus natural extension

Regular extension is the standard pragmatic rule, but it is not the only defensible one. Natural extension is more conservative: it returns the vacuous $[0,1]$ whenever the evidence has lower probability zero, on the grounds that the model does admit measures under which the evidence was impossible. The two rules agree except when the evidence has lower probability zero and upper probability positive — exactly the case a vacuous node creates — where natural extension is vacuous and regular extension is far more informative. EnhancedBayesianNetworks implements regular extension, with the one deviation noted above when the upper probability is zero as well.

Wc = DiscreteNode(:Wc)
Wc[:Wc => :sunny] = 0.5
Wc[:Wc => :cloudy] = 0.5
Sc = DiscreteNode(:Sc, [:Wc])
Sc[:Wc => :sunny,  :Sc => :on] = Interval(0.8, 0.95)
Sc[:Wc => :sunny,  :Sc => :off] = Interval(0.05, 0.2)
Sc[:Wc => :cloudy, :Sc => :on] = 0.2
Sc[:Wc => :cloudy, :Sc => :off] = 0.8
cn = CredalNetwork([Wc, Sc])
add_child!(cn, :Wc, :Sc)
order!(cn)

infer(cn, :Sc, Evidence(:Wc => :sunny))     # CredalPosterior: [lower, upper] bounds

CredalPosterior P(Sc | Wc=sunny)

Sc	Interval
------------------------
on	[0.8, 0.95]
off	[0.05, 0.2]

Extreme posteriors: 2

The cost grows with the number of extreme networks, one per combination of interval endpoints, so credal inference is exact but exponential in the number of imprecise entries.

Progress bar

Because each extreme network is a full variable-elimination pass, credal inference over many of them can be slow. The same progress keyword shows a progress bar over the extreme networks as they are solved (the inner per-network elimination bars are suppressed), likewise defaulting to isinteractive().

Sampling

Besides computing posteriors, a discrete network can be sampled with sample, which draws realizations rather than marginals:

• sample(bn::BayesianNetwork, n): draws n joint samples by visiting the nodes in topological order and drawing each from its CPT given its already-sampled parents. The result is a DataFrame with one column per node.

• sample(node::DiscreteNode, evidence::Evidence) draws a single state of one node, given an Evidence that fixes its parents.

sample(bn, 4)                               # 4 joint draws, one row per sample

4×2 DataFrame

Row	W	S
	Symbol	Symbol
1	sunny	off
2	sunny	on
3	cloudy	off
4	sunny	off

Sampling a node whose entries are imprecise raises an error — there is no single distribution to draw from, so sample applies to precise (Bayesian) networks and nodes.