Parameter Learning
Index
EnhancedBayesianNetworks.ImpreciseDirichletModelEnhancedBayesianNetworks.ParameterEstimatorEnhancedBayesianNetworks.PreciseDirichletModelEnhancedBayesianNetworks.QuesenberryHurstEnhancedBayesianNetworks.learn
Types
EnhancedBayesianNetworks.ParameterEstimator Type
ParameterEstimatorAbstract supertype of the estimators accepted by learn and by the estimator keyword of add_node!. An estimator fixes how the probabilities of one parent configuration are derived from the data, and with it the kind of network returned: a precise estimator gives one probability per node state and a BayesianNetwork, an imprecise one an Interval per node state and a CredalNetwork.
Every estimator works on one parent configuration: one node together with one configuration of its parents' states. Three quantities describe it:
k, how many states the node has.cnt, how many data rows match the parent configuration and have the node in the state being estimated.total, how many such rows there are over allkstates, sototalis alwayssum(cnt). On incomplete data it counts only the rows where the node and every one of its parents were observed.
Available estimators are PreciseDirichletModel, ImpreciseDirichletModel and QuesenberryHurst.
EnhancedBayesianNetworks.PreciseDirichletModel Type
PreciseDirichletModel(α = 0)CPT entries as (cnt + α) / (total + α * k), the mean of a symmetric Dirichlet posterior with pseudo-count α ≥ 0 in each of the k states. Here k is the number of states of the node, cnt the number of rows matching the parent configuration with the node in the state being estimated, and total the number of rows matching it in any state.
α = 0 is maximum likelihood, the relative frequency, and gives exactly 0 to a state never observed in a configuration; any α > 0 is additive smoothing, Laplace smoothing at α = 1, pulling towards the uniform. Use ImpreciseDirichletModel to report that uncertainty as an interval rather than smooth it away. A parent configuration absent from the data gives the uniform distribution.
On incomplete data learn reaches this estimator through expectation maximisation, which assumes the data are missing at random; max_iter and tol control the iteration.
Examples
bn = learn(dag, df, PreciseDirichletModel()) # maximum likelihood
bn = learn(dag, df, PreciseDirichletModel(1)) # Laplace smoothing
order!(bn)EnhancedBayesianNetworks.ImpreciseDirichletModel Type
ImpreciseDirichletModel(s = 1)CPT entries as the interval [cnt / (total + s), (cnt + s) / (total + s)], with s > 0 the prior strength: s extra observations whose states are left unspecified, placed outside the estimated state for the lower bound and inside it for the upper. Here k is the number of states of the node, cnt the number of rows matching the parent configuration with the node in the state being estimated, and total the number of rows matching it in any state. s = 1 and s = 2 are the usual choices; larger s gives wider intervals.
It is PreciseDirichletModel with the prior mean left free instead of pinned to the uniform, so the interval always contains PreciseDirichletModel(s / k). Two properties make it the default for credal learning: the credal set is never empty, the lower bounds of a configuration summing to at most 1 and the upper bounds to at least 1; and a configuration absent from the data gives [0, 1] with no special case, which infer handles through the regular extension.
On incomplete data the interval widens to the envelope of the estimates over every completion of the data consistent with what was observed, so nothing is assumed about why a value is absent. In particular not that it is missing at random. Each bound gains a virtual frequency counting the incomplete rows it may ascribe to the configuration; a row contradicted by any observed parent is excluded however much else is unknown.
References: Walley, P. (1996), Inferences from Multinomial Data: Learning About a Bag of Marbles, Journal of the Royal Statistical Society Series B 58(1), 3–34. For incomplete data, Ramoni, M. & Sebastiani, P. (2001), Robust Learning with Missing Data, Machine Learning 45(2), 147–170, whose robust Bayesian estimator gives the bounds in closed form.
Examples
cn = learn(dag, df, ImpreciseDirichletModel()) # s = 1
cn = learn(dag, df, ImpreciseDirichletModel(2))
order!(cn)EnhancedBayesianNetworks.QuesenberryHurst Type
QuesenberryHurst(confidence = 0.95)CPT entries as the simultaneous confidence region for multinomial proportions projected onto one coordinate, obtained by inverting Pearson's score test:
θ = [(A + 2 * cnt) ± sqrt(A * (A + 4 * cnt * (total - cnt) / total))] / (2 * (total + A))with A = quantile(Chisq(k - 1), confidence). Here k is the number of states of the node, cnt the number of rows matching the parent configuration with the node in the state being estimated, and total the number of rows matching it in any state. The k - 1 degrees of freedom are what make the coverage simultaneous over the node's states rather than one state at a time; at k = 2 this is exactly the Wilson score interval. A configuration absent from the data gives [0, 1], and coherence holds whenever total > 0.
Unlike ImpreciseDirichletModel this is a frequentist confidence region read as a credal set: it carries repeated-sampling coverage, not a behavioural interpretation, so results should say which of the two produced them.
learn refuses this estimator on incomplete data, where the envelope over the completions has no closed form and the simultaneity above would no longer hold; use ImpreciseDirichletModel.
Reference: Quesenberry, C. P. and Hurst, D. C. (1964), Large Sample Simultaneous Confidence Intervals for Multinomial Proportions, Technometrics 6(2), 191–195.
Examples
cn = learn(dag, df, QuesenberryHurst())
cn = learn(dag, df, QuesenberryHurst(0.99))Methods
EnhancedBayesianNetworks.learn Function
learn(dag::DirectAcyclicGraph, df::DataFrame, estimator::ParameterEstimator; max_iter::Int = 100, tol::Real = 1.0e-4, progress::Bool = isinteractive())
learn(dag::DirectAcyclicGraph, df::DataFrame; max_iter::Int = 100, tol::Real = 1.0e-4, progress::Bool = isinteractive())Learn the CPTs of dag from df. Either pass one estimator for every node, or declare one per node with add_node! and call the two-argument form. The two exclude each other, and a DAG declaring an estimator for only some of its nodes is rejected here rather than at add_node!, since a DAG under construction is legitimately partial.
Every node of dag needs a Symbol column in df; other columns are ignored with a warning. A node's domain is the states observed in df together with any declared through add_node!. A precise estimator (PreciseDirichletModel) writes numbers and an imprecise one (ImpreciseDirichletModel, QuesenberryHurst) writes Intervals; one interval anywhere gives a CredalNetwork, otherwise a BayesianNetwork.
missing entries are allowed, and the estimators fix how they are treated:
every node precise: expectation maximisation, which assumes the data are missing at random.
max_iterandtolbound the iteration and are read on no other path.every node imprecise: every entry is bounded over all completions of the data, assuming nothing about the missingness.
QuesenberryHurston incomplete data, or a DAG mixing precise and imprecise nodes on incomplete data: refused with an error.
The returned network is fully specified but unordered, so call order! before inference or sampling. progress defaults to isinteractive(), silent in scripts, tests and docs.
Examples
One estimator for the whole DAG:
bn = learn(dag, df, PreciseDirichletModel()) # maximum likelihood
bn = learn(dag, df, PreciseDirichletModel(1)) # Laplace smoothing
cn = learn(dag, df, ImpreciseDirichletModel()) # IDM with s = 1
cn = learn(dag, df, QuesenberryHurst(0.99)) # 99% simultaneous intervals
order!(cn)Or one per node, where a single imprecise node makes the result credal:
dag = DirectAcyclicGraph()
add_node!(dag, :W; estimator = PreciseDirichletModel())
add_node!(dag, :R; parents = [:W], estimator = ImpreciseDirichletModel())
add_node!(dag, :G; parents = [:R], estimator = PreciseDirichletModel())
cn = learn(dag, df) # -> CredalNetwork, only :R is imprecise
order!(cn)