Probabilistic graphical models

Summary

A probabilistic graphical model pairs a graph with factors defining a joint probability distribution. Nodes represent variables, not necessarily causal mechanisms. The graph constrains independence; probability tables or densities supply the numerical parameters.

Ask whether observing a middle node blocks or opens information between two endpoints. Our teaching fixture uses independent fair binary variables $A,B$ and their common effect $C=A\mathrm{OR}B$.

Two main families

Bayesian networks (directed acyclic graphs)

Each node has a conditional distribution given its parents. The joint factorizes as

The local Markov property makes each node independent of its non-descendants given its parents. For our collider, $p(a,b,c)=p(a)p(b)p(c\mid a,b)$. The last factor is one when $c=a\mathrm{OR}b$, zero otherwise.

Markov random fields (undirected graphs)

The joint factorizes over cliques $C\subseteq \text{nodes}$:

Potentials ${\psi }_C$ are nonnegative; require finite, positive $Z$. For continuous variables, replace the sum defining $Z$ with an integral. An undirected separator blocks every path between the sets whose independence it establishes. Strict positivity is required for the general equivalence between undirected Markov properties and clique factorization.

d-separation (Bayesian networks)

A path is blocked by an observation set $O$ if either:

• A non-collider on that path is observed.
• A collider and all its descendants are unobserved.

At a collider, both path arrows point toward the middle node. All paths must be blocked for d-separation to imply conditional independence. Chains $A\to M\to B$ and forks $A\leftarrow M\to B$ both close when $M$ is observed.

An observed collider or its descendant instead opens that path. An open path permits association; it does not guarantee association for every choice of parameters.

Normalize the collider

The four input pairs each have probability $1/4$. Observing $C=1$ excludes only $(0,0)$, so $p(C=1)=3/4$. The three remaining pairs each receive conditional mass $1/3$:

Further observing $B=1$ leaves two equally likely pairs, giving $p(A=1\mid C=1,B=1)=1/2$. Observing $B=0$ instead forces $A=1$. This is explaining away within the specified model, not an observational proof of causation.

Observe a non-collider

New task: use $A\to M\to B$, with fair $A$ and binary copying probabilities $0.8$, then $0.9$.

Picture and change: replace the common effect with a middle transition. Before observing $M$,

$p(B=1\mid A=1)=0.8(0.9)+0.2(0.1)=\mathrm{0.74.}$

Check: given $M=1$, the probability of $B=1$ is $0.9$ regardless of $A$. Conditioning blocks this path rather than opening it.

Use when: the observed middle node is a non-collider on the relevant path.

Observe a collider’s descendant

New task: add $C\to D$ with deterministic $D=C$ and observe $D=1$.

Picture and change: evidence arrives downstream instead of at $C$ directly.

Check: the same three OR cases survive, so $p(A=1\mid D=1)=2/3$.

Use when: selection or measurement depends on a common effect’s descendant.

An active path need not produce association

Keep the collider graph, but set $p(C=1\mid A,B)=1/2$ for every input pair. Observing $C$ then leaves $A,B$ independent. This parameterization is Markov to the graph but not faithful: it has extra independence beyond graphical separation.

Related

• Markov chains. Simplest sequential PGM.
• Bayes’ rule and the posterior. Foundation for PGM inference.