Inference

Inference answers probabilistic queries: given some observed evidence, what is the distribution over one or more query variables? infer computes this posterior exactly, by variable elimination, on a discrete network — a BayesianNetwork or a CredalNetwork [2]. An EnhancedBayesianNetwork is not queried directly: it is first reduced to one of these (see the Reduction & Reliability Analysis chapter).

Evidence is given as an Evidence — a Dict{Symbol,Symbol} mapping node names to observed states — and the query as a single node name or a vector of them.

W = DiscreteNode(:W); W[:W => :sunny] = 0.5; W[:W => :cloudy] = 0.5
S = DiscreteNode(:S, [:W])
S[:W => :sunny,  :S => :on] = 0.9; S[:W => :sunny,  :S => :off] = 0.1
S[:W => :cloudy, :S => :on] = 0.2; S[:W => :cloudy, :S => :off] = 0.8
bn = BayesianNetwork([W, S]); add_child!(bn, :W, :S); order!(bn)

infer(bn, :S, Evidence(:W => :sunny))       # posterior P(S | W = sunny)

Posterior P(S | W=sunny)

S	Probability
------------------------
on	0.9
off	0.1

The result is a Posterior: the labelled probability table over the query, together with the schema, query, and evidence used to produce it. Passing an empty Evidence() returns the prior marginal:

infer(bn, :S, Evidence())                   # prior marginal P(S)

Posterior P(S)

S	Probability
------------------------
on	0.55
off	0.45

The query must not overlap the evidence, and both must name states that actually exist in the network.

Variable elimination

infer uses variable elimination [10]: the network's conditional probability tables become factors, the evidence restricts every factor that mentions an observed variable to its observed state, and each non-query, non-evidence variable is then summed out in turn — the factors mentioning it are multiplied together and the variable is marginalised away. Multiplying the surviving factors and normalising yields the posterior over the query. The cost of the computation is dominated by the largest intermediate factor built along the way, which depends heavily on the order in which variables are eliminated.

Formally, write the joint distribution as a product of factors (potentials) — initially one per node, its CPT ${\varphi }_i=p(y_i\mid \mathrm{Pa}(Y_i))$:

$$p(y_1,\dots ,y_n)=\prod _{i=1}^n{\varphi }_i.$$

To answer a query over the variables $Q$ given evidence $E=e$, every factor is first restricted to the observed states, ${\varphi }_i{|}_{E=e}$. Each remaining variable that is neither in $Q$ nor in $E$ is then eliminated with the two elementary factor operations — the product of all factors that mention it, followed by marginalization, i.e. summing the variable out of that product. Eliminating a variable $Y_k$ yields a single new factor

$$\psi =\sum _{y_k}\prod _{i:Y_k\in \mathrm{scope}({\varphi }_i)}{\varphi }_i,$$

which no longer mentions $Y_k$ and replaces the factors it was built from. Once every non-query, non-evidence variable has been eliminated, the product of the surviving factors is the unnormalized joint $p(Q,E=e)$; dividing by its total gives the posterior:

$$p(Q\mid E=e)=\frac{{\displaystyle \sum _{\mathbf{h}}\prod _i{\varphi }_i{|}_{E=e}}}{{\displaystyle \sum _Q\sum _{\mathbf{h}}\prod _i{\varphi }_i{|}_{E=e}}},$$

where $\mathbf{h}$ ranges over the variables other than $Q$ and $E$, and the denominator is the normalizing constant $p(E=e)$.

Elimination order

The elimination order is chosen greedily on the network's interaction graph (the moral graph): repeatedly eliminate the remaining node with the lowest score, adding the fill-in edges its removal induces, until none are left. Ren et al. [11] survey heuristics for constructing a good order and find the minimum-increase-in-complexity search — greedily removing the node whose elimination least increases the problem's complexity — to outperform the alternatives.

EnhancedBayesianNetworks.jl adapts this idea into a mixed scored function. Rather than committing to a single complexity measure, it exposes two and combines them:

• fill_score — a min-fill heuristic: the ratio of fill-in edges an elimination would add to the edges it would remove, favouring nodes that introduce few new dependencies.

• factor_score — a min-factor heuristic: the size of the factor an elimination would create, the product of the state-space sizes of the node and its current neighbours.

Writing $N(Y)$ for the current neighbours of a node $Y$ in the interaction graph $(V,E)$, and $|\mathrm{dom}(Y^{\prime })|$ for the number of states of a neighbour $Y^{\prime }$, the two scores are

$$s_{\mathrm{fill}}(Y)=\frac{|\{\{a,b\}\subseteq N(Y):\{a,b\}\notin E\}|}{|N(Y)|},$$$$s_{\mathrm{factor}}(Y)=|\mathrm{dom}(Y)|\prod _{Y^{\prime }\in N(Y)}|\mathrm{dom}(Y^{\prime })|,$$

with $s_{\mathrm{fill}}(Y)=0$ when $Y$ has no neighbours. The numerator of $s_{\mathrm{fill}}$ counts the neighbour pairs not yet adjacent (the fill-in edges elimination would add) and its denominator is the degree $|N(Y)|$ (the edges it would remove); $s_{\mathrm{factor}}$ is the number of entries of the factor that elimination would build.

The default, fill_factor_score, is the mix of the two: a tuple (fill_score, factor_score, node) compared lexicographically, so the min-fill score decides, ties are broken by the smaller resulting factor, and the node id breaks any remaining tie for a deterministic order. Either single heuristic can be selected instead by passing it as the last argument to infer:

infer(bn, :S, Evidence(:W => :sunny), fill_score)   # min-fill ordering only

Posterior P(S | W=sunny)

S	Probability
------------------------
on	0.9
off	0.1

The chosen heuristic changes only the order of elimination, never the computed posterior — exact inference is invariant to it; the order affects only how much work is done to reach the answer.

Complete-scenario probability

When every node is observed there is nothing to marginalise, and the joint probability of the full scenario is just the product of each node's conditional entry given its parents. joint_probability computes this directly, without running variable elimination:

joint_probability(bn, Evidence(:W => :sunny, :S => :on))   # 0.5 * 0.9 = 0.45

0.45

It requires a complete scenario — one state per node; for partial evidence or marginals, use infer.

Credal inference

On a CredalNetwork the local tables are interval-valued, so the network stands for a whole credal set of Bayesian networks [6]. infer then returns a CredalPosterior carrying lower and upper posterior probabilities. These are obtained by enumerating the extreme Bayesian networks of the credal set — every combination of the interval endpoints of its imprecise nodes — running variable elimination on each, and taking the element-wise minimum and maximum over the resulting posteriors.

Wc = DiscreteNode(:Wc); Wc[:Wc => :sunny] = 0.5; Wc[:Wc => :cloudy] = 0.5
Sc = DiscreteNode(:Sc, [:Wc])
Sc[:Wc => :sunny,  :Sc => :on] = Interval(0.8, 0.95); Sc[:Wc => :sunny,  :Sc => :off] = Interval(0.05, 0.2)
Sc[:Wc => :cloudy, :Sc => :on] = 0.2;                 Sc[:Wc => :cloudy, :Sc => :off] = 0.8
cn = CredalNetwork([Wc, Sc]); add_child!(cn, :Wc, :Sc); order!(cn)

infer(cn, :Sc, Evidence(:Wc => :sunny))     # CredalPosterior: [lower, upper] bounds

CredalPosterior P(Sc | Wc=sunny)

Sc	Interval
------------------------
on	[0.8, 0.95]
off	[0.05, 0.2]

Extreme posteriors: 2

The cost grows with the number of extreme networks — one per combination of interval endpoints — so credal inference is exact but exponential in the number of imprecise entries.

Sampling

Besides computing posteriors, a discrete network can be sampled with sample, which draws realizations rather than marginals:

• sample(bn::BayesianNetwork, n) performs ancestral sampling — it draws n joint samples by visiting the nodes in topological order and drawing each from its CPT given its already-sampled parents. The result is a DataFrame with one column per node.

• sample(node::DiscreteNode, evidence::Evidence) draws a single state of one node, given an Evidence that fixes its parents.

sample(bn, 4)                               # 4 joint draws, one row per sample

4×2 DataFrame

Row	W	S
	Symbol	Symbol
1	sunny	on
2	cloudy	off
3	cloudy	on
4	cloudy	off

Sampling a node whose entries are imprecise raises an error — there is no single distribution to draw from — so sample applies to precise (Bayesian) networks and nodes.