Parameter Learning

Parameter learning fills in the conditional probability tables of a Bayesian Network (BN) from data, given its structure. You supply the graph, meaning which nodes exist and how they depend on one another, and a dataset of observations. The learned network is a fully-specified BN, ready for inference once ordered.

The structure is described by a DirectAcyclicGraph: nodes and edges, plus optional declared states, but no probabilities. It is a lightweight input type, not a network, it supports add_node!, parents, children, and gplot, but none of the operations that need CPTs (order!, infer, sample).

Declaring the structure

Build the DAG top-down with add_node!, listing each node's parents (which must already be present). Edges are wired for you as you declare the parents.

using EnhancedBayesianNetworks

dag = DirectAcyclicGraph()
add_node!(dag, :W)                          # root
add_node!(dag, :R; parents = [:W])          # edge :W -> :R wired here
add_node!(dag, :P; parents = [:W])
add_node!(dag, :G; parents = [:R, :P])
dag

DirectAcyclicGraph(DiscreteNode[DiscreteNode(W, parents=Symbol[], states=Symbol[]), DiscreteNode(R, parents=[:W], states=Symbol[]), DiscreteNode(P, parents=[:W], states=Symbol[]), DiscreteNode(G, parents=[:R, :P], states=Symbol[])], Dict(:P => 3, :R => 2, :G => 4, :W => 1), sparse([1, 1, 2, 3], [2, 3, 4, 4], Bool[1, 1, 1, 1], 4, 4), Dict{Symbol, Vector{Symbol}}())

Each node's domain is taken from the data at learn time. Any states passed to add_node! are added to that domain. Those are the states you want to keep even if they never occur in the dataset. They end up in the learned CPT with probability 0 (or an alpha-smoothed mass):

dag = DirectAcyclicGraph()
add_node!(dag, :W, [:foggy])                # :sunny/:cloudy come from data; :foggy guaranteed
dag

DirectAcyclicGraph(DiscreteNode[DiscreteNode(W, parents=Symbol[], states=Symbol[])], Dict(:W => 1), sparse(Int64[], Int64[], Bool[], 1, 1), Dict(:W => [:foggy]))

Learning the parameters

learn estimates the CPTs and chooses the algorithm from the data: complete data uses maximum likelihood, data with any missing entries uses Expectation-Maximization. It returns a BN; call order! on it before inference or sampling.

using DataFrames
df = DataFrame(
    W = [:sunny, :sunny, :cloudy, :cloudy],
    S = [:on, :off, :on, :off]
    )

dag = DirectAcyclicGraph()
add_node!(dag, :W)
add_node!(dag, :S; parents = [:W])

learned = learn(dag, df)                    # complete data -> MLE

BayesianNetwork

Nodes: 2
Edges: 1

Topology:

#   Node            States              Parents
--------------------------------------------------------------------------------
1   W               cloudy, sunny       -
2   S               off, on             W

order!(learned)

alpha is a Laplace/Dirichlet pseudo-count applied by both algorithms; max_iter and tol control EM's convergence:

learn(dag, df; alpha = 1)                            # add-one smoothing

BayesianNetwork

Nodes: 2
Edges: 1

Topology:

#   Node            States              Parents
--------------------------------------------------------------------------------
1   W               cloudy, sunny       -
2   S               off, on             W

df_with_missing = DataFrame(
    W = [:sunny, :sunny, :cloudy, :cloudy, :cloudy, :sunny],
    S = [:on, :off, :on, :off, :missing, :on]
    )

learn(dag, df_with_missing; tol = 1e-6, max_iter = 500)

BayesianNetwork

Nodes: 2
Edges: 1

Topology:

#   Node            States              Parents
--------------------------------------------------------------------------------
1   W               cloudy, sunny       -
2   S               missing, off, on    W

To force a specific algorithm, call it directly.

Maximum likelihood

learn_parameters_mle is the closed-form estimator for complete data. For every node $Y_i$, each of its states $s$, and each configuration $c$ of its parents $\mathrm{Pa}(Y_i)$, it sets

$$P(Y_i=s\mid \mathrm{Pa}(Y_i)=c)=\frac{N(s,c)+\alpha }{N(c)+\alpha k_i},$$

where $N(s,c)$ counts the training rows with $Y_i=s$ and $\mathrm{Pa}(Y_i)=c$, $N(c)$ those with $\mathrm{Pa}(Y_i)=c$, $k_i$ is the number of states of $Y_i$, and alpha the pseudo-count (alpha = 0 is pure maximum likelihood; alpha > 0 smooths, keeping never-observed states off exactly 0). A parent configuration that never appears in the data falls back to a uniform distribution.

learned = learn_parameters_mle(dag, df; alpha = 1)

BayesianNetwork

Nodes: 2
Edges: 1

Topology:

#   Node            States              Parents
--------------------------------------------------------------------------------
1   W               cloudy, sunny       -
2   S               off, on             W

order!(learned)

Expectation-Maximization

When some entries are missing, learn_parameters_em estimates the CPTs iteratively by Expectation-Maximization [14]. Starting from uniform tables, each iteration performs:

• an E-step: every row with missing values is expanded into all completions of its missing variables, each weighted by $P(\text{missing}\mid \text{observed})$ under the current network; fully-observed rows keep weight 1;

• an M-step: the CPTs are re-estimated by the same counting as maximum likelihood, summing these fractional weights instead of counting whole rows.

Iteration stops when no CPT entry moves by more than tol, or after max_iter steps. Because EM converges only to a local optimum, the (uniform) initialization matters. With no missing values it reduces exactly to learn_parameters_mle.

learned = learn_parameters_em(dag, df_with_missing; alpha = 1, tol = 1e-6)

BayesianNetwork

Nodes: 2
Edges: 1

Topology:

#   Node            States              Parents
--------------------------------------------------------------------------------
1   W               cloudy, sunny       -
2   S               missing, off, on    W

order!(learned)

Both estimators leave the input dag untouched, so the same structure can be relearned on different datasets or with different smoothing.

Progress bar

On a large network or dataset — and especially for EM, which reruns the E/M steps until convergence — learn can be slow. It takes a progress keyword that shows a progress bar: over the nodes as their CPTs are fitted for maximum likelihood, and over the iterations (annotated with the current maximum CPT change) for EM. It defaults to isinteractive() (shown in the REPL, silent in scripts, tests, and this documentation); pass progress = true or progress = false to force it.