Programming using Halfspace Queries and Multiscale Monge Decomposition Charalampos Babis E Tsourakakis ctsourakmathcmuedu SODA 2011 25 ID: 619227
Download Presentation The PPT/PDF document "Approximate Dynamic" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Approximate Dynamic Programming using Halfspace Queries and Multiscale Monge Decomposition
Charalampos (Babis) E. Tsourakakis ctsourak@math.cmu.edu
SODA 2011
25
th January ‘11
SODA '11
1Slide2
Joint work
Richard Peng
SCS, CMUGary L. Miller SCS, CMU
Russell SchwartzSCS & BioSciences
CMUSODA '112Slide3
OutlineMotivationRelated Work“Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
3Slide4
Motivation
SODA '114
Array based comparative genomic hybridization (aCGH)
log T/Rfor humans
R=2
GenomeSlide5
MotivationNear-by probes (genomic positions) tend to have the same DNA copy
number.Treat the data as 1d time series.Fit piecewise constant segments.
SODA '115
log T/R
for humansR=2
GenomeSlide6
MotivationOther applicationsHistogram constructionSpeech recognitionData mining
Biology and many more…SODA '11
6Slide7
Problem FormulationInput: Noisy sequence (P1
,..,Pn)Output: (F1,..,Fn) which minimizes
Digression: Constant C is determined by training on data with ground truth.
SODA '11
7
Goodness of fit
Regularization/avoid
overfittingSlide8
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
8Slide9
Related Work
SODA '119
Don KnuthOptimal BSTs in O(n2) time
Frances Yao
Recurrence
Quadrangle
inequality
(
Monge
)
Then we can turn the naïve O(n
3
) algorithm to O(n
2
) Slide10
Related WorkGaspard Monge
SODA '1110
Quadrangle
inequality
Inverse
Quadrangle
inequality
Transportation
problems (1781)Slide11
Related WorkSODA '11
11
Eppstein
Galil
Giancarlo
Larmore
Schieber
time
time
Slide12
Related WorkSMAWK algorithm : finds all row minima of a totally monotone matrix NxN in O(N) time!
Bein, Golin, Larmore, Zhang showed that the Knuth-Yao technique is implied by the SMAWK algorithm.
Weimann [PhD thesis] improved state of the art results in several problems on planar graphs.SODA '1112Slide13
Related WorkSODA '11
13Guha
Koudas
Shim
Synopsis of data distributions: fit K segments to 1d
time series.
Monotonicity properties of the key quantities involved.
(1+
ε)
approximation O(n
+Κ
3
logn+K
2
/
ε
) timeSlide14
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
14Slide15
Vanilla DPRecurrence for our optimization problem:
SODA '1115Slide16
Vanilla DPCompute nxn matrix M where Mj,i = mean squared error for fitting a segment from point j to point i.
This can be done in O(n2) time by keeping first and second moments in “online” waySODA '11
16Slide17
Vanilla DPSODA '11
17
First Moments
Squared Errors
RecurseSlide18
QuestionsIs the O(n2) running time tight? Probably not
What do halfspace queries have to do with this problem?What is Multiscale Monge analysis?
SODA '1118Slide19
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
19Slide20
Our contributionsTechnique 1: Using halfspace queries we get an approximation algorithm with
ε additive error which runs in O(n4/3+δ log(U/ε)
) time.Technique 2: break carefully the original problem into a “small” number of Monge optimization problems. Approximates the optimal answer for the shifted objective within a factor of
(1+ε), O(nlogn/ε) time.
SODA '1120
~Slide21
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
21Slide22
Analysis of our RecurrenceRecurrence for our optimization problem:
Equivalent formulation where
SODA '11
22Slide23
Analysis of our RecurrenceLet Claim:
SODA '11
23
This term kills the
Monge
property! Slide24
Why it’s not Monge?Basically, because we cannot be searching for the optimum breakpoint in a restricted range. E.g., for C=1 and the sequence (0,2,0,2,….,0,2) : fit a segment per point
(0,2,0,2,….,0,2,1): fit one segment for all points
SODA '1124Slide25
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
25Slide26
Notation
the quantity we wish to approximate
the approximate value of
the optimum value of the recurrence by examining the values
SODA '11
26
~
-
~Slide27
Halfspaces and DPSODA '11
27Do binary searches to approximate
DPi for every i=1,..,n. LetWe do enough iterations in order to get . O(logn log(U/ε))
iterations suffice where
C}
By induction we can show that
,i.e., additive
ε
error
approximation
Slide28
Halfspaces and DPSODA '11
28
i fixed, binary search query
constant
~
~Slide29
Dynamic Halfspace Reporting
29
Eppstein
Agarwal
Matousek
Halfspace emptiness query
Given a set of points S in R
4
the halfspace
range reporting problem can be solved
query time
space and preprocessing time
u
pdate time
Slide30
Halfspaces and DPHence the algorithm iterates through the indices i=1..n, and maintains the Eppstein et al. data structure containing one point for every j<i.
It performs binary search on the value, which reduces to emptiness queriesIt provides an answer within ε additive error from the optimal one.Running time: O(n4/3+δ
log(U/ε) )SODA '1130
~Slide31
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
31Slide32
Multiscale Monge Decomposition By simple algebra we can write our weight function w(
j,i) as w’(j,i)/(i-j)+C where The weight function w’ is Monge!
Key Idea: approximate i-j by a constant! But how? SODA '1132Slide33
Multiscale Monge Decomposition For each i, we break the choices of j into intervals [l
k,rk] s.t i-lk and i-rk differ by at most 1+
ε.Ο(logn/ε) such intervals suffice to get a 1+ε approximation. However, we need to make sure that when we solve a specific subproblem, the optimum lies in the desired interval. How?
SODA '11
33Slide34
Multiscale Monge Decomposition
M is a sufficiently large positive constantRunning time O(nlogn/ε)
using an O(n) time persubproblemSODA '1134
Larmore
SchieberSlide35
OutlineMotivationRelated Work “Vanilla” DP algorithm
Our contributionsAnalysis of the recurrenceHalfspaces and DP
Multiscale Monge optimization ConclusionsSODA '11
35Slide36
SummaryTwo new techniques for approximate DP for a recurrence not treated by existing methods:
Halfspace emptiness queries Multiscale Monge Decomposition
SODA '1136Slide37
ProblemsOther problems where our techniques are directly or almost directly applicable?O(n
2) is unlikely to be tight. E.g., if two points Pi and Pj satisfy
then they belong in different segments.
Can we find a faster exact algorithm?
SODA '11
37Slide38
Thanks a lot!
SODA '1138