The corr espondence pr oblem Wh yi st he corr espondence pr oblem difcult S ome points - PDF document

The correspondence problemƒWhyisthe correspondence problem difŒcult?-Some points in each image will have nocorresponding points in the other image.(1) the cameras might have different Œelds of view.(2) due to occlusion.-Astereo system must be able to determine the image parts that should not bematched. -2-ƒMethods for establishing correspondence-There are twoissues to be considered:*how toselect candidate matches?*how todetermine the goodness of a match?-Two main classes of algorithms:Correlation-based: attempt to establish a correspondence by matching imageintensities.Feature-based: attempt to establish a correspondence by matching a sparse sets ofimage features.Correlation-based Methods-Match image subwindows between the twoimages usingimage correlation(the old-est technique for Œnding the correspondence between pixels of twoimages).-Scene points must have the same intensity in each image (strictly accurate for per-fectly matte surfaces only). -3-AlgorithmInputs:(1)IlandIr(2) the width of the subwindow2W+1(3) the search region in the right imageR(pl)associatedwith a pixelplin the left imageForeach pixelpl=(i,j)inthe left image:1. for each displacementd=(d1,d2)ÎR(pl)computec(d)=Wk=-WSWl=-WSIl(i+k,j+l)Ir(i+k-d1,j+l-d2)(cross-correlation)2. the disparity ofplis the vectord=(d1,d2)that maximizesc(d)overR(pr)d=argmaxdÎR[c(d)]-Usually,wenormalizec(d)bydividing it by the standard deviation of bothIlandIr(normalized cross-correlation,e.g.,Î[0,1])c(d)=Wk=-WSWl=-WS(Il(i+k,j+l)-Il)(Ir(i+k-d1,j+l-d2)-Ir)Ö` ```````````````````Wk=-WSWl=-WS(Il(i+k,j+l)-Il)2Wk=-WSWl=-WS(Ir(i+k-d1,j+l-d2)-Ir)2whereIlandIrare the average pixel values in the left and right windows.-Analternative similarity measure is thesum of squared differences(SSD):c(d)=-Wk=-WSWl=-WS(Il(i+k,j+l)-(Il(i+k-d1,j+l-d2))2 -4-ƒImprovements-Instead of using the image intensity values, the accuracyofcorrelation is improvedby usingthresholded signed gradient magnitudesat each pixel.-Compute the gradient magnitude at each pixel in the twoimages withoutsmoothing.-Map the gradient magnitude values into three values: -1, 0, 1 (i.e., by threshold-ing the gradient magnitiude)-More sensitive correlations are produced this way.ƒSome comments-The success of correlation-based methods depends on whether the image windowinone image exhibits a distinctive structure that occurs infrequently in the search regionof the other image.-How tochoose the size of the window(i.e.,W)?*too small a windowmay not capture enough image structure, and may be toonoise sensitive (i.e., manyfalse matches).*too large a windowmakes matching less sensitive tonoise (desired) but also toactual variations of image intensity (undesired -- it causes discontinuities in thedisparity map).*anadaptive searching windowhas been proposed in the literature.((a) original, (b) 3x3, (c) 7x7, (d) adaptive) -5-(3D info recovered using adaptive window)-How tochoose the size and location ofR(pl)?*ifthe distance of the Œxating point from the cameras is much larger than thebaseline, the location ofR(pl)can be chosen to be the same as the location ofpl.*the size ofR(pl)can be estimated from the maximum range of distances weexpect to Œnd in the scene.*wewill see that the search regioncan always be reduced to a line !! -6-Feature-based Methods-Look for a feature in an image that matches a feature in the other.-Typical features used are:*edge points*line segments*corners-Aset offeaturesis used for matching; a line feature descriptor,for example, couldcontain:*the length,l*the orientation,*the coordinates of the midpoint,m*the average intensity along the line,i-Similarity measures are based on matching feature descriptors:S=1w0(ll-lr)2+w1(l-r)2+w2(ml-mr)2+w3(il-ir)2wherew0,...,w3are weights (determining the weights that yield the best matches is anontrivial task). -7-AlgorithmInputs:(1)IlandIr(2) features and their descriptors in both images(3) the search region in the right imageR(fl)associatedwith a featureflin the left imageForeach featureflin the left image:1. Compute the similarity betweenfland each image feature inR(fl)2. Select the right-image featurefr,that maximizes the similarity measure.3. Save the correspondence and disparityd(fl,fr)ƒCorrelation-based vs feature-based approachesCorrelation-based methods-Easier to implement than feature-based methods.-Provide a dense disparity map (useful for reconstructing surfaces).-Need textured images to work well (manyfalse matches otherwise).-Don'twork well when viewpoints are very different (due to forshorteningand change in illumination direction).Feature-based methods:-Suitable when good features can be extracted from the scene.-Faster than correlation-based methods.-Provide sparse disparity maps (OK for applications likevisual navigation)-Relatively insensitive toillumination changes. -8-ƒStructured lighting-Feature-based methods are not applicable when the objects have smooth surfaces(i.e., sparse disparity maps makesurface reconstruction difŒcult).-Patterns of light are projected onto the surface of objects, creating interesting pointsev eninregions which would be otherwise smooth.-Finding and matching such points is simpliŒed by knowing the geometry of the pro-jected patterns.