EECS 442 Prof David Fouhey Winter 2019 University of Michigan httpwebeecsumichedufouheyteachingEECS442W19 Administrivia Homework 4 due Sunday Homework 4 due Thursday 1 week ID: 1025234
Download Presentation The PPT/PDF document "Intro to 3D + Camera Calibration" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Intro to 3D +Camera CalibrationEECS 442 – Prof. David FouheyWinter 2019, University of Michiganhttp://web.eecs.umich.edu/~fouhey/teaching/EECS442_W19/
2. Administrivia Homework 4 due SundayHomework 4 due Thursday (1 week)See debugging / speed hintsBe carefulOffice hours in BBB Learning center on Friday (I’m out of town)
3. Our goal: Recovery of 3D structureJ. Vermeer, Music Lesson, 1662A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of paintings, Proc. Computers and the History of Art, 2002
4. Next few classesFirst: some intuitions and examples from biological vision about 3D perceptionBut first, a brief review
5. Let’s Take a Picture!Slide inspired by S. Seitz; image from Michigan EngineeringPhotosensitive Material
6. Projection MatrixProjection (fx/z, fy/z) is matrix multiplicationSlide inspired from L. Lazebnik Of
7. Single-view AmbiguityxX?X?X?Diagram credit: S. LazebnikGiven a calibrated camera and an image, we only know the ray corresponding to each pixel.Nowhere near enough constraints for X
8. Single-view Ambiguityhttp://en.wikipedia.org/wiki/Ames_roomSlide Credit: J. Hays
9. Single-view AmbiguityDiagram credit: J. Hays
10. Single-view AmbiguityRashad Alakbarov shadow sculptures
11. Resolving Single-view AmbiguityShoot light (lasers etc.) out of your eyes!Con: not so biologically plausible, dangerous?
12. Resolving Single-view AmbiguityShoot light (lasers etc.) out of your eyes!Con: not so biologically plausible, dangerous?
13. Resolving Single-view AmbiguityXStereo: given 2 calibrated cameras in different views and correspondences, can solve for XxxOriginal diagram credit: S. Lazebnik
14. Human stereopsis: disparityHuman eyes fixate on point in space – rotate so that corresponding images form in centers of fovea.
15. Disparity occurs when eyes fixate on one object; others appear at different visual anglesHuman stereopsis: disparity
16. Stereo photography and stereo viewersImage from fisher-price.comTake two pictures of the same subject from two slightly different viewpoints and display so that each eye sees only one of the images.Slide credit: J. HaysInvented by Sir Charles Wheatstone, 1838
17. http://www.johnsonshawmuseum.orgSlide credit: J. Hays
18. http://www.well.com/~jimg/stereo/stereo_list.htmlSlide credit: J. Hays
19. http://www.well.com/~jimg/stereo/stereo_list.htmlSlide credit: J. Hays
20. AutostereogramsExploit disparity as depth cue using single image.(Single image random dot stereogram, Single image stereogram)Slide credit: J. Hays, Images from magiceye.com
21. AutostereogramsSlide credit: J. Hays, Images from magiceye.com
22. Yeah, yeah, but…Not all animals see stereo:Prey animals (large field of view to spot predators)Stereoblind people
23. Resolving Single-view AmbiguityxXOne option: move, find correspondence.If you know how you moved and have a calibrated camera, can solve for XR,tOriginal diagram credit: S. Lazebnik
24. Knowing R,tHow do you know how far you moved?Can solve via visionCan solve via earsWhy does your inner ear have 3 ducts?Can solve via signals sent to muscles
25. Yeah, yeah, but…You haven’t been here before, yet you probably have a fairly good understanding of this scene.
26. Pictorial Cues – Shading[Figure from Prados & Faugeras 2006]
27. Pictorial Cues – Texture[From A.M. Loh. The recovery of 3-D structure using visual texture patterns. PhD thesis]
28. Pictorial Cues – Perspective effectsImage credit: S. Seitz
29. Pictorial Cues – Familiar ObjectsMonitor: probably not 12 feet wide. Desk surface: probably flat
30. Reality of 3D Perception3D perception is absurdly complex and involves integration of many cues:Learned cues for 3DStereo between eyesStereo via motionIntegration of known motion signals to muscles (efferent copy), acceleration sensed via earsPast experience of touching objectsAll connect: learned cues from 3D probably come from stereo/motion cues in large partReally fantastic article on cues for 3D from Cutting and Vishton, 1995: https://pmvish.people.wm.edu/cutting%26vishton1995.pdf
31. How are Cues Combined?Gehringer and Engel, Journal of Experimental Psychology: Human Perception and Performance, 1986Ames illusion persists (in a weaker form) even if you have stereo vision –gussing the texture is rectilinear is usually incredibly reliable
32. More Formally
33. Multi-view geometry problemsCamera 1K?Slide credit: Noah SnavelyCalibration:We need camera intrinsics / K in order to figure out where the rays are
34. Multi-view geometry problemsCamera 3R3,t3Slide credit: Noah Snavely?Camera 1Camera 2R1,t1R2,t2Recovering structure:Given cameras and correspondences, find 3D.
35. Multi-view geometry problemsCamera 3R3,t3Camera 1Camera 2R1,t1R2,t2Slide credit: Noah SnavelyStereo/Epipolar Geomery:Given 2 cameras and find where a point could be
36. Multi-view geometry problemsCamera 1Camera 2Camera 3R1,t1R2,t2R3,t3???Slide credit: Noah SnavelyMotion:Figure out R, t for a set of cameras given correspondences
37. Outline(Today) Calibration: Getting intrinsic matrix/KSingle view geometry: measurements with 1 imageStereo/Epipolar geometry: 2 pictures → depthmapStructure from motion (SfM): 2+ pictures → cameras, pointcloud
38. Typical Perspective Model focal lengthprincipal point (image coords of camera origin on retina)Just moves camera origin2D Projection of X rotationtranslation3D point
39. Camera Calibration If I can get pairs of [X,Y,Z] and [u,v] → equations to constrain MHow do I get [X,Y,Z], [u,v]
40. Camera CalibrationA funny object with multiple planes.
41. Camera Calibration TargetsUsing a tape measure312.747 309.140 30.086305.796 311.649 30.356307.694 312.358 30.418310.149 307.186 29.298311.937 310.105 29.216311.202 307.572 30.682307.106 306.876 28.660309.317 312.490 30.230307.435 310.151 29.318308.253 306.300 28.881306.650 309.301 28.905308.069 306.831 29.189309.671 308.834 29.029308.255 309.955 29.267307.546 308.613 28.963311.036 309.206 28.913307.518 308.175 29.069309.950 311.262 29.990312.160 310.772 29.080311.988 312.709 30.514880 214 43 203270 197886 347745 302943 128476 590419 214317 335783 521235 427665 429655 362427 333412 415746 351434 415525 234716 308602 187Known 3d locationsKnown 2d image coordsImage credit: J. Hays
42. Camera Calibration Targets…A set of views of a plane (not covered today)
43. Camera Calibration TargetsA single, huge plane. What’s this for?
44. Camera calibrationGiven n points with known 3D coordinates Xi and known image projections pi, estimate the camera parametersXipiSlide credit: S. Lazebnik
45. Camera Calibration: Linear Method Remember (from geometry): this implies MXi pi are scaled copies of each other Remember (from homography fitting): this implies their cross product is 0
46. Camera Calibration: Linear Method …Some tedious math occurs…(see Homography deriviation)
47. Camera Calibration: Linear Method How many linearly independent equations?2How many equations per [u,v] + [X,Y,Z] pair?2If M is 3x4, how many degrees of freedom?11
48. Camera Calibration: Linear Method Derivation from L. Lazebnik; note we negate one of the equations from the cross productHow do we solve problems of the form ?Eigenvector of ATA with smallest eigenvalue
49. In PracticeDegenerate configurations (e.g., all points on one plane) an issue. Usually need multiplane targets.
50. In PracticeI pulled a fast one. We want:We get:What’s the difference between K[R,t] and M?Solution: QR-decomposition on left-most 3x3 matrix → finite options of a upper triangular matrix * rotation
51. In PracticeIf pi = Mxi is overconstrained, the objective function isn’t actually the one you care about. Instead: initialize parameters with linear model2) Apply off-the-shelf non-linear optimizer to:Advantage: can also add radial distortion, not optimize over known variables, add constraints
52. What Does This Get You?Given projection pi of unknown 3D point X in two or more images (with known cameras Mi), find X
53. Triangulationp1p2X?Given projection pi of unknown 3D point X in two or more images (with known cameras Mi), find XWhy is the calibration here important?
54. TriangulationRays in principle should intersect, but in practice usually don’t exactly due to noise, numerical errors.p1p2X?
55. Triangulation – Geometryp1p2XFind shortest segment between viewing rays, set X to be the midpoint of the segment.
56. Triangulation – Non-linear Optim.p1p2XFind X minimizing jM1XM2X
57. Triangulation – Linear Optimization Two eqns per camera for 3 unkn. in XCross Prod.as matrix