CRF Inference Problem CRF over variables CRF distribution MAP inference MPM maximum posterior marginals inference Other notation Unnormalized distribution Variational distribution ID: 339894
Download Presentation The PPT/PDF document "Mean field approximation for CRF inferen..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Mean field approximation for CRF inference
Slide2
CRF Inference Problem
CRF over variables:
CRF distribution:
MAP inference:MPM (maximum posterior marginals) inference:Slide3
Other notation
Unnormalized
distribution
Variational distributionExpectationEntropySlide4
Variational Inference
Inference => minimize KL-divergence
General Objective FunctionSlide5
Mean field approximation
Variational
distribution => product of independent
marginals:Expectations:Entropy:Slide6
Mean field objective
ObjectiveSlide7
Local optimality conditions
Lagrangian
Setting derivatives to 0 gives conditions for local optimalitySlide8
Coordinate ascent
Sequential coordinate ascent
Initialize
Q_i’s to uniform distributionFor i = 1...N, update vector Q_i by summing expectations over all cliques involving X_i (while fixing all Q_j
, j!=i)Parallel updates algorithmAs above, but perform updates in step 2 for all
Q_i’s in parrallel (i.e. Generating Q^1, Q^2...)Slide9
Comparison with belief propagation
Objective
Factored energy functional
Local polytope Slide10
Comparison with belief propagation
Message updates:
Extracting beliefs (after convergence):Slide11
Comparison with belief propagation
- = => Bethe free energy for
pairwise
graphsBethe cluster graphs:
General:
Pairwise
:Slide12
Mean field updates
Updates in dense CRF (
Krahenbuhl
NIPS ’11)Evaluate using filtering =Slide13
Higher-order potentials
Pattern-based potentials
P^n
-Potts potentialsSlide14
Higher-order potentials
Co-occurrence potentials
L
(X) = set of labels present in X{Y_1,...Y_L} = set of binary latent variables