Devi Parikh Disclaimer Many slides have been borrowed from Kristen Grauman who may have borrowed some of them from others Any time a slide did not already have a credit on it I have credited it to Kristen So there is a chance some of these credits are inaccurate ID: 1002839
Download Presentation The PPT/PDF document "Local features and image matching" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Local features and image matchingDevi ParikhDisclaimer: Many slides have been borrowed from Kristen Grauman, who may have borrowed some of them from others. Any time a slide did not already have a credit on it, I have credited it to Kristen. So there is a chance some of these credits are inaccurate.
2. AnnouncementsPS2 due last nightProject proposalsDue in a ~week (October 10th)PS3 outDue in ~3 weeks (October 24th)Slide credit: Kristen Grauman2
3. Video from last time
4. University of Southern Californiahttp://iris.usc.edu/Projects/detect/detection-result.htmlExample using binary image analysis:Bg subtraction + blob detectionSlide credit: Kristen Grauman4
5. Topics overviewIntroMultiple views and motionFeatures & filtersFiltersGradientsEdgesBlobs/regionsLocal invariant featuresGrouping & fittingRecognitionVideo processing5Slide credit: Kristen Grauman
6. Topics overviewIntroMultiple views and motionFeatures & filtersFiltersGradientsEdgesBlobs/regionsLocal invariant featuresGrouping & fittingRecognitionVideo processing6Slide credit: Kristen Grauman
7. Topics overviewIntroMultiple views and motionLocal invariant featuresFeatures & filtersFiltersGradientsEdgesBlobs/regionsLocal invariant featuresGrouping & fittingRecognitionVideo processing7Slide credit: Kristen Grauman
8. Numerical IssuesWhen computing HSay true match is [50 100 1] [50 100][50.5 100 1][50.5 100][50 100 1.5][33 67]Scale co-ordinates to lie between 0 and 28
9. Topics overviewFeatures & filtersGrouping & fittingMultiple views and motionHomography and image warpingLocal invariant featuresImage formationEpipolar geometryStereo and structure from motionRecognitionVideo processing9Slide credit: Kristen Grauman
10. EarlierImage mosaicsFitting a 2D transformationAffine, Homography2D image warpingComputing an image mosaic
11. Robust feature-based alignmentSource: L. Lazebnik
12. Robust feature-based alignmentExtract featuresSource: L. Lazebnik
13. Robust feature-based alignmentExtract featuresCompute putative matchesSource: L. Lazebnik
14. Robust feature-based alignmentExtract featuresCompute putative matchesLoop:Hypothesize transformation T (small group of putative matches that are related by T)Source: L. Lazebnik
15. Robust feature-based alignmentExtract featuresCompute putative matchesLoop:Hypothesize transformation T (small group of putative matches that are related by T)Verify transformation (search for other matches consistent with T)Source: L. Lazebnik
16. Robust feature-based alignmentExtract featuresCompute putative matchesLoop:Hypothesize transformation T (small group of putative matches that are related by T)Verify transformation (search for other matches consistent with T)Source: L. Lazebnik
17. TodayHow to detect which features to match?
18. Boundary extensionWide-Angle Memories of Close-Up Scenes, Helene Intraub and Michael Richardson, Journal of Experimental Psychology: Learning, Memory, and Cognition, 1989, Vol. 15, No. 2, 179-187
19. Detecting local invariant featuresDetection of interest pointsHarris corner detectionScale invariant blob detection: LoG (next time)Description of local patches (next time)
20. Local features: main componentsDetection: Identify the interest pointsDescription:Extract vector feature descriptor surrounding each interest point.Matching: Determine correspondence between descriptors in two viewsKristen Grauman
21. Local features: desired propertiesRepeatabilityThe same feature can be found in several images despite geometric and photometric transformations SaliencyEach feature has a distinctive descriptionCompactness and efficiencyMany fewer features than image pixelsLocalityA feature occupies a relatively small area of the image; robust to clutter and occlusion
22. Goal: interest operator repeatabilityWe want to detect (at least some of) the same points in both images.Yet we have to be able to run the detection procedure independently per image.No chance to find true matches!
23. Goal: descriptor distinctivenessWe want to be able to reliably determine which point goes with which.Must provide some invariance to geometric and photometric differences between the two views.?
24. Local features: main componentsDetection: Identify the interest pointsDescription:Extract vector feature descriptor surrounding each interest point.Matching: Determine correspondence between descriptors in two views
25. What points would you choose?
26. Corners as distinctive interest pointsWe should easily recognize the point by looking through a small windowShifting a window in any direction should give a large change in intensity“edge”:no change along the edge direction“corner”:significant change in all directions“flat” region:no change in all directionsSlide credit: Alyosha Efros, Darya Frolova, Denis Simakov
27. Corners as distinctive interest points2 x 2 matrix of image derivatives (averaged in neighborhood of a point).Notation:
28. First, consider an axis-aligned corner:What does this matrix reveal?
29. First, consider an axis-aligned corner:This means dominant gradient directions align with x or y axisLook for locations where both λ’s are large.If either λ is close to 0, then this is not corner-like.What does this matrix reveal?What if we have a corner that is not aligned with the image axes?
30. What does this matrix reveal?Since M is symmetric, we haveThe eigenvalues of M reveal the amount of intensity change in the two principal orthogonal gradient directions in the window.
31. What does this matrix reveal?The eigenvalues of M reveal the amount of intensity change in the two principal orthogonal gradient directions in the window.
32. Corner response function“flat” region1 and 2 are small;“edge”:1 >> 22 >> 1“corner”:1 and 2 are large, 1 ~ 2;12
33. Harris corner detectorCompute M matrix for each image window to get their cornerness scores.Find points whose surrounding window gave large corner response (f> threshold)Take the points of local maxima, i.e., perform non-maximum suppression
34. Example of Harris applicationKristen Grauman
35. Compute corner response at every pixel.Example of Harris applicationKristen Grauman
36. Example of Harris applicationKristen Grauman
37. Properties of the Harris corner detectorRotation invariant? Scale invariant?Yes
38. Properties of the Harris corner detectorRotation invariant? Scale invariant?All points will be classified as edgesCorner !YesNo
39. Harris Detector: StepsSlide credit: Kristen Grauman39
40. Harris Detector: StepsCompute corner response fSlide credit: Kristen Grauman40
41. Harris Detector: StepsFind points with large corner response: f > thresholdSlide credit: Kristen Grauman41
42. Harris Detector: StepsTake only the points of local maxima of fSlide credit: Kristen Grauman42
43. Harris Detector: StepsSlide credit: Kristen Grauman43
44. SummaryExtracting interest points (and features) from imagesImage warping to create mosaic, given homographyInterest point detectionHarris corner detectorNext time: Laplacian of Gaussian, automatic scale selection
45. Questions?