CS5670 Computer Vision Announcements Project 1 code due Thursday 225 at 1159pm Turnin via Github Classroom Project 1 artifact due Monday 31 at 1159pm Quiz this Wednesday 224 via Canvas ID: 1019255
Download Presentation The PPT/PDF document "Local features & Harris corner detec..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Local features & Harris corner detectionCS5670: Computer Vision
2. AnnouncementsProject 1 code due Thursday, 2/25 at 11:59pmTurnin via Github ClassroomProject 1 artifact due Monday, 3/1 at 11:59pmQuiz this Wednesday, 2/24, via CanvasStarts Weds 9am EST. Open until 3:10pm EST.10-minute time limitCovers lecture material through today (Monday)Closed book / closed note
3. ReadingSzeliski: 4.1
4. Last timeSampling & interpolationKey points:Subsampling an image can cause aliasing. Better is to blur (“pre-filter”) to remote high frequencies then downsampleIf you repeatedly blur and downsample by 2x, you get a Gaussian pyramidUpsampling an image requires interpolation. This can be posed as convolution with a “reconstruction kernel”
5. Today: Feature extraction—Corners and blobs
6. Motivation: Automatic panoramasCredit: Matt Brown
7. GigaPan:http://gigapan.com/Also see Google Zoom Views: https://www.google.com/culturalinstitute/beta/project/gigapixels Motivation: Automatic panoramas
8. Why extract features?Motivation: panorama stitchingWe have two images – how do we combine them?
9. Why extract features?Motivation: panorama stitchingWe have two images – how do we combine them?Step 1: extract featuresStep 2: match features
10. Why extract features?Motivation: panorama stitchingWe have two images – how do we combine them?Step 1: extract featuresStep 2: match featuresStep 3: align images
11. Application: Visual SLAM(aka Simultaneous Localization and Mapping)
12. Image matchingby Diva Sianby swashfordTexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA
13. Harder caseby Diva Sianby scgbt
14. Harder still?
15. NASA Mars Rover imageswith SIFT feature matchesAnswer below (look for tiny colored squares…)
16. Feature matching for object search
17. Feature Matching
18. Invariant local featuresFind features that are invariant to transformationsgeometric invariance: translation, rotation, scalephotometric invariance: brightness, exposure, …Feature Descriptors
19. Advantages of local featuresLocality features are local, so robust to occlusion and clutterQuantityhundreds or thousands in a single imageDistinctiveness: can differentiate a large database of objectsEfficiencyreal-time performance achievable
20. More motivation… Feature points are used for:Image alignment (e.g., mosaics)3D reconstructionMotion tracking (e.g. for AR)Object recognitionImage retrievalRobot/car navigation… other
21. Approach1. Feature detection: find it2. Feature descriptor: represent it3. Feature matching: match itFeature tracking: track it, when motion
22. Local features: main componentsDetection: Identify the interest pointsDescription: Extract vector feature descriptor surrounding each interest pointMatching: Determine correspondence between descriptors in two viewsCredit: Kristen Grauman
23. Snoop demoWhat makes a good feature?
24. Want uniquenessLook for image regions that are unusualLead to unambiguous matches in other imagesHow to define “unusual”?
25. Local measures of uniquenessSuppose we only consider a small window of pixelsWhat defines whether a feature is a good or bad candidate?Credit: S. Seitz, D. Frolova, D. Simakov
26. Local measures of uniqueness“flat” region:no change in all directions“edge”: no change along the edge direction“corner”:significant change in all directionsHow does the window change when you shift it?Shifting the window in any direction causes a big changeCredit: S. Seitz, D. Frolova, D. Simakov
27. Consider shifting the window W by (u,v)how do the pixels in W change?compare each pixel before and after bysumming up the squared differences (SSD)this defines an SSD “error” E(u,v):We are happy if this error is highSlow to compute exactly for each pixel and each offset (u,v)Harris corner detection: the mathW(u,v)Chris Harris and Mike Stephens (1988). "A Combined Corner and Edge Detector". Alvey Vision Conference.
28. Taylor Series expansion of I:If the motion (u,v) is small, then first order approximation is goodPlugging this into the formula on the previous slide…Small motion assumption
29. Corner detection: the mathConsider shifting the window W by (u,v)define an SSD “error” E(u,v):W(u,v)
30. Corner detection: the mathConsider shifting the window W by (u,v)define an SSD “error” E(u,v):Thus, E(u,v) is locally approximated as a quadratic error functionW(u,v)
31. The surface E(u,v) is locally approximated by a quadratic form. The second moment matrixLet’s try to understand its shape.
32. Horizontal edge: uvE(u,v)
33. Vertical edge: uvE(u,v)
34. General caseWe can visualize H as an ellipse with axis lengths determined by the eigenvalues of H and orientation determined by the eigenvectors of Hdirection of the slowest changedirection of the fastest change(max)-1/2(min)-1/2Ellipse equation:max, min : eigenvalues of H
35. Quick eigenvalue/eigenvector reviewThe eigenvectors of a matrix A are the vectors x that satisfy:The scalar is the eigenvalue corresponding to xThe eigenvalues are found by solving:In our case, A = H is a 2x2 matrix, so we haveThe solution:Once you know , you find x by solving
36. Corner detection: the mathEigenvalues and eigenvectors of HDefine shift directions with the smallest and largest change in errorxmax = direction of largest increase in Emax = amount of increase in direction xmaxxmin = direction of smallest increase in E min = amount of increase in direction xmin xmin xmax
37. Corner detection: the mathHow are max, xmax, min, and xmin relevant for feature detection?What’s our feature scoring function?
38. Corner detection: the mathHow are max, xmax, min, and xmin relevant for feature detection?What’s our feature scoring function?Want E(u,v) to be large for small shifts in all directionsthe minimum of E(u,v) should be large, over all unit vectors [u v]this minimum is given by the smaller eigenvalue (min) of H
39. Interpreting the eigenvalues12“Corner”1 and 2 are large, 1 ~ 2;E increases in all directions1 and 2 are small;E is almost constant in all directions“Edge” 1 >> 2“Edge” 2 >> 1“Flat” regionClassification of image points using eigenvalues of M:
40. Corner detection summaryHere’s what you do:Compute the gradient at each point in the imageFor each pixel:Create the H matrix from the entries in the gradientCompute the eigenvalues. Find points with large response (min > threshold)Choose those points where min is a local maximum as features
41. Corner detection summaryHere’s what you do:Compute the gradient at each point in the imageFor each pixel:Create the H matrix from the nearby gradient valuesCompute the eigenvalues. Find points with large response (min > threshold)Choose those points where min is a local maximum as features
42. The Harris operatormin is a variant of the “Harris operator” for feature detectionThe trace is the sum of the diagonals, i.e., trace(H) = h11 + h22Very similar to min but less expensive (no square root)Called the “Harris Corner Detector” or “Harris Operator”Lots of other detectors, this is one of the most popular
43. The Harris operatorHarris operator
44. Harris detector example
45. f value (red high, blue low)
46. Threshold (f > value)
47. Find local maxima of f (non-max suppression)
48. Harris features (in red)
49. Weighting the derivativesIn practice, using a simple window W doesn’t work too wellInstead, we’ll weight each derivative value based on its distance from the center pixel
50. Harris Detector [Harris88]Second moment matrix501. Image derivatives2. Square of derivatives3. Gaussian filter g(sI)IxIyIx2Iy2IxIyg(Ix2)g(Iy2)g(IxIy)4. Cornerness function – both eigenvalues are stronghar5. Non-maxima suppression
51. Harris Corners – Why so complicated?Can’t we just check for regions with lots of gradients in the x and y directions?No! A diagonal line would satisfy that criteriaCurrent Window
52. Questions?