With Bayesian conditional density estimation Problem Analytic expressions for likelihood of parameters is not available with simulation based models Approximate Bayesian Computation ABC Provides likelihood free inference ID: 738065
Download Presentation The PPT/PDF document "Fast -free Inference of Simulation Mode..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Fast -free Inference of Simulation Models
With Bayesian conditional density estimationSlide2
ProblemAnalytic expressions for likelihood of parameters is not available with simulation based modelsSlide3
Approximate Bayesian Computation (ABC)Provides likelihood free inferenceSimulate model repeatedlyAccept parameter settings which generate synthetic data that is close to realSlide4
ABC approachesRejection ABC:Only accept samples that are
-close to realAccepted samples form an approximate posteriorMCMC-ABC:
Explores the sample space by perturbing most recent parameters
Sequential Monte Carlo ABC (SMC-ABC)
Importance sampling to estimate a sequence of slowly changing distributions
Last is an approximation to the posterior
Slide5
Issues with ABC approachesRepresents the posterior as a set of samplesDifficult to combine posteriors from separate analysis
Samples aren’t from the correct posteriorCome from pseudo-observations in an
-ball neighborhood to actuals
Reduction in
can make simulation difficult to impossible
Slide6
Conditional Density EstimationParametric alternativeLearns an approximation to the exact posteriorImplemented using Bayesian neural networksParametric density estimation
Stochastic Variational Inference (SVI)Recognition networksSlide7
Model Definition
vector of parametersx = observed dataImplicitly defined likelihood:
Prior:
Posterior:
Slide8
Basic ProcessUse simulation to directly estimate posteriorChoose a loose prior (called a proposal prior by authors)
Form a consistent estimate of the exact posterior
Select a flexible family of conditional densities:
Slide9
Detailed ProcessEach set of N pairs
independently generated
Limit as
probability of the parameter vectors is max w.r.t.
iif:
Slide10
Choice of Conditional Densityq should be:Flexible enough to represent the posteriorEast to train with maximum likelihood
should be:
Easy to evaluate and sample from
Authors take:
q to be a mixture of K Gaussians
Proposal prior to be Gaussian
Slide11
SVISubsample one or more data pointsAnalyze the subsample using the current variational parametersImplement a closed form update of the parametersRepeat Generalization of the EM algorithm:
Probabilistic models amenable to closed form coordinate descent