Yining Wang YuXiang Wang Aarti Singh Machine Learning Department Carnegie mellon university 1 Subspace Clustering 2 Subspace Clustering Applications Motion Trajectories tracking 1 ID: 617853
Download Presentation The PPT/PDF document "Noisy Sparse Subspace Clustering with di..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Noisy Sparse Subspace Clustering with dimension reduction
Yining Wang, Yu-Xiang Wang, Aarti SinghMachine Learning DepartmentCarnegie mellon university
1Slide2
Subspace Clustering
2Slide3
Subspace Clustering Applications
Motion Trajectories tracking1
1 (
Elhamifar and Vidal, 2013), (Tomasi and
Kanade, 1992)
3Slide4
Subspace Clustering Applications
Face Clustering11 (
Elhamifar and Vidal, 2013), (
Basri and Jacobs, 2003)
Network hop counts, movie ratings, social graphs, …
4Slide5
Sparse Subspace Clustering
(Elhamifar and Vidal, 2013), (Wang and Xu, 2013).Data:
Key idea:
similarity graph
based on
l
1
self-regression
No False Connections
5Slide6
Sparse Subspace Clustering
(Elhamifar and Vidal, 2013), (Wang and Xu, 2013).Data:
Key idea:
similarity graph
based on
l
1
self-regression
s.t.
Noiseless
data
Noisy
data
6Slide7
SSC with dimension reduction
Real-world data are usually high-dimensionalHopkins-155:
Extended Yale Face-B:
Computational concerns
Data availability: values of some features might be missingPrivacy concerns: releasing the raw data might cause privacy breaches.
7Slide8
SSC with dimension reduction
Dimensionality reduction:
How small can
p
be?
A
trivial result:
is OK.
L
: the number of subspaces (clusters)
r
: the intrinsic dimension of each subspace
Can we do better?
8Slide9
Main Result
,if
is a
subspace embedding
Random Gaussian projection
Fast Johnson-
Lindenstrauss
Transform (FJLT)
Uniform row subsampling under incoherence conditions
Sketching
……
Lasso SSC should be used
even if data are noiseless.
9Slide10
Proof sketch
Review of deterministic success conditions for SSC (Soltanolkotabi and Candes, 12)(Elhamifar
and Vidal, 13)Subspace incoherence
InradiusAnalyze perturbation under dimension reduction
Main results for noiseless and noisy cases.10Slide11
Review of SSC success condition
Subspace incoherenceCharacterizing inter-subspace separation
where
solves
Dual problem of Lasso SSC
11
Lasso SSC formulationSlide12
Review of SSC success condition
InradiusCharacterzing inner-subspace data point distribution
Large
inradius
Small
inradius
12Slide13
Review of SSC success condition
(Soltanolkotabi &
Candes, 2012)Noiseless SSC succeeds (similarity graph has no false connection) if
With dimensionality reduction:
Bound
13Slide14
Perturbation of subspace incoherence
We know that
…
So
because of
strong convexity
14Slide15
Perturbation of
inradius
Main idea: linear operator transforms a ball to an ellipsoid
15Slide16
Main result
SSC with dimensionality reduction succeeds (similarity graph has no false connection) if
Error of approximate isometry
O(1) if
16
Regularization parameter of Lasso
Lasso SSC required even for noiseless problem
Takeaways: the
geometric gap
is a resource that can be traded-off for data dimension reduction
Noisy case: (
is the adversarial noise level)
Slide17
Simulation results (Hopkins 155)
17Slide18
Conclusion
SSC provably succeeds with dimensionality reductionDimension after reduction can be as small as
Lasso SSC is required for provable results.
Questions?
18Slide19
References
M. Soltanolkotabi and E. Candes. A Geometric Analysis of Subspace Clustering with Outliers. Annals of Statistics, 2012.E.
Elhamifar and R. Vidal. Sparse Subspace Clustering: Algorithm, Theory and Applications. IEEE TPAMI, 2013
C. Tomasi and T. Kanade. Shape and Motion from Image Streams under Orthography.
IJCV, 1992.R. Basri and D. Jacobs. Lambertian
Reflection and Linear Subspaces.
IEEE TPAMI
, 2003.
Y.-X., Wang and H., Xu. Noisy Sparse Subspace Clustering.
ICML
, 2013.
19