Approximations in Probabilistic Programming The computing stack approximation Algorithms Compiler and runtime Architecture The APPROX view with probabilities and approximations The computing stack ID: 420733
Download Presentation The PPT/PDF document "Principled" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Principled
Approximations in Probabilistic ProgrammingSlide2
The computing
stack (approximation)
AlgorithmsCompiler and runtimeArchitecture
The APPROX view: with
probabilities
and approximations!Slide3
The computing stack
Algorithms
Compiler and runtimeArchitecture
The APPROX view: with
probabilities
and approximations!
Probabilistic program
Program synthesis
Hardware for samplingSlide4
DPMM Clustering
In each iteration, update the cluster assignments of data points one at a time
For each data point, compute the probability of it belonging to all existing clusters and an unseen cluster
Sample from this probability distribution
Gaussian Dirichlet Process Mixture Model (DPMM)
Gibbs Sampling Algorithm
Cluster 3
Cluster 2
Cluster 1
Cluster 2
Cluster 1
Cluster 2
Cluster 1
New
Cluster
ModelSlide5
Sampling
Probabilities
A Distribution
1
2
3
4
5
6
7
0
5
Call to Sampler
A Sample
3
2
7
2
4
Samples
Specified DistributionSlide6
Sampling in Hardware
Prefix Sum
Comparators
PRNG
Encoder
CLK
Probabilities
Sample
32b
35b
1
b
3
b
32b
Idea: exploit errors!Slide7
Robustness to hardware faults
Stuck-at Faults
Transient Faults
Sampler
Clustering using DPMM
Deka, Biplab. “
On Fault Tolerance of Hardware Samplers”
. Masters Thesis, University of Illinois at Urbana Champaign, 2014 Slide8
Voltage-Error Rate Tradeoff
Sloan, J.;
Kesler
, D.; Kumar, R.;
Rahimi
, A., "A numerical optimization-based methodology for application
robustification: Transforming applications for error tolerance," Dependable Systems and Networks (DSN), 2010Slide9
Approximating compilers
Traditional compiler:
Input: Deterministic program
Goal:
Executable semantically equivalent to source
Method:
Syntax-guided translationApproximating compiler:Input: Probabilistic program
Goal:
Satisfy basic boolean invariantsMinimize quantitative errorMethod:
Program synthesisSlide10
Probabilistic programs
heightMan
= Gaussian(177,64);
heightWoman
=
Gaussian
(164,64);assume(heightWoman
> heightMan
);return heightMan
Source: Tutorial on
Infer.NET
by John Winn and Tom Minka
Addition: assertions,
angelic
nondeterminism
More complex example: clusteringSlide11
Probabilistic programming ++
Random variables X: range over distributions
Deterministic variables yEither holes or temporaries
Functions f(X
1
,...,
Xk, y1,..., yk)Can map random variables to deterministic onesExpectation, probability Assertions
Pareto-optimality goalsSlide12
Example
X = Gaussian(
??, 10
);
assume (X < 10
);
c = Pr(X > 0
);
assert (c > 0.7);
minimize (c);
One synthesis
algorithm in [CCS14]
Based on probabilistic abstract interpretation
Hole
[CCS14]
Chaudhuri
, Clochard, Solar-
Lezama. Bridging
boolean
and
quantitative synthesis
using smoothed proof search. POPL 2014.Slide13
Use in approximation
Holes = degree of approximation
Assertions = invariants, hard boundsDeclarative error minimization
Deterministic temporaries track
resource consumption
S
ynthesis = CompilationSlide14
Thank you
!
Questions?