277K - views

The amount of light coming to the eye from an object depends on the amount of light striking the surface and on the proportion of light that is reflected

If a visual system only made a single measurement of luminance acting as a pho tometer then there would be no way to distinguish a white surface in dim light from a black surface in bright light et humans can usually do so and this skill is known as

Tags : visual system
Embed :
Pdf Download Link

Download Pdf - The PPT/PDF document "The amount of light coming to the eye fr..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

The amount of light coming to the eye from an object depends on the amount of light striking the surface and on the proportion of light that is reflected

Presentation on theme: "The amount of light coming to the eye from an object depends on the amount of light striking the surface and on the proportion of light that is reflected"— Presentation transcript:

339 The amount of light coming to the eye from an objectdepends on the amount of light striking the surface, and onthe proportion of light that is reflected. If a visual system onlymade a single measurement of luminance, acting as a pho-tometer, then there would be no way to distinguish a whitesurface in dim light from a black surface in bright light. Yethumans can usually do so, and this skill is known as lightnessconstancy.The constancies are central to perception. An organismneeds to know about meaningful world-properties, such ascolor, size, shape, etc. These properties are not explicitlyavailable in the retinal image, and must be extracted by visu-al processing. The gray shade of a surface is one such prop-erty.To extract it, luminance information must be combinedacross space. Figure 24.1 shows the well-known simultane-ous contrasteffect, which demonstrates a spatial interactionin lightness perception. The two smaller squares are the sameshade of gray. However, the square in the dark surroundappears the square in the light Illusions like these are sometimes viewed as quirky failuresof perception, but they help reveal the inner workings of asystem that functions remarkably well. Here we will consid-er how lightness illusions can inform us about lightness per-ception. Levels of processingThe visual system processes information at many levels ofsophistication. At the retina, there is low-level vision, includ-ing light adaptation and the center-surround receptive fieldsof ganglion cells. At the other extreme is high-level vision,which includes cognitive processes that incorporate knowl-edge about objects, materials, and scenes. In between there ismid-level vision. Mid-level vision is simply an ill-definedregion between low and high. The representations and theprocessing in the middle stages are commonly thought toinvolve surfaces, contours, grouping, and so on. Lightnessperception seems to involve all three levels of processing. The low-level approach to lightness is associated withEwald Hering. He considered adaptation and local interac-tions, at a physiological level, as the crucial mechanisms.This approach has long enjoyed popularity because it offersan psy-chophysics. Figure 24.2(a) shows the receptive field of anidealized center-surround cell. The cell exhibits lateral inhi-bition: light in the center is excitatory while light in the sur-round is inhibitory.Across-section of the receptive field isshown in figure 24.2(b). This cell performs a local compari-son between a given luminance and the neighboring lumi-nances, and thus offers machinery that can help explain thesimultaneous contrast (SC) illusion. This idea was formalizedby Mach, who proposed a Laplacian derivative operator asthe mechanism. One of Mach's inspirations was an illusion now known asthe Mach band. When a spatial ramp in luminance abruptlychanges slope, an illusory light or dark band appears. Avari-ant of the Mach band has been used by the op artist Vasarely,as shown in figure 24.3(a). The image consists of a set of EDWARD H. ADELSON Department of Brain and CognitiveSciences, Massachusetts Institute of Technology, Cambridge, Mass. 24 Lightness Perception andLightness Illusions EDWARD H. ADELSON ABSTRACT:Agray surface in sunlight may have much higherluminance than it has in the shade, but it still looks gray.To achievethe task of Òlightness constancy,Ó the visual system must discountthe illumination and other viewing conditions and estimate thereflectance. Many different physical situations, such as shadows, fil-ters, or haze, can be combined to form a single, simple mappingfrom luminance to reflectance. The net effect of the viewing condi-tions, including additive and multiplicative effects, may be termedan reflectance into luminance. To correctly estimate lightness, a visualsystem must determine a Òlightness transfer functionÓ that performsthe inverse. Human lightness computation is imperfect, but per-forms well in most natural situations. Lightness illusions can revealthe inner workings of the estimation process, which may involvelow-level, mid-level, and high-level mechanisms. Mid-level mech-anisms, involving contours, junctions, and grouping, appear to becritical in explaining many lightness phenomena. Every light is a shade, compared to the higher lights, till you cometo the sun; and every shade is a light, compared to the deepershades, till you come to the night.ÑJohn Ruskin (1879). Adelson, E.H. Lightness Perception and Lightness Illusions. In The New CognitiveNeurosciences, 2nd ed., M. Gazzaniga, ed. Cambridge, MA: MIT Press, pp. 339-351, (2000). nested squares. Each square is a constant luminance. Thepattern gives the illusion of a glowing X along the diagonals,even though the corners of the squares are no brighter thanthe straight parts. When a center-surround filter is run overthis pattern (i.e., is convolved with it) it produces the imageshown in figure 24.3(b). The filter output makes the brightdiagonals explicit.Acenter-surround filter cannot explain a percept byitself: perception involves the whole brain. However, it isinteresting that center-surround responses can go a long wayto explaining certain illusions. Derivative operators respond especially well to sharpintensity transitions such as edges. The importance of edges,and the lesser importance of slow gradients, is indicated bythe Craik-OÕBrien-Cornsweet effect (COCE) named after itsseveral discoverers. Figure 24.4 shows one of several COCEvariants. The figure appears to contain a dark square next toa light square. Actually, the two squares are ramps, and theyare identical, as shown by the luminance profile underneath(the dashed lines show constant luminances). The responseof a center-surround cell to this pattern will be almost thesame as its response to a true step edge: there will be a bigresponse at the edge, and a small response elsewhere. Whilethis doesnÕt explain why the image looks as it does, it mayhelp explain why one image looks similar to the other(Cornsweet, 1970). Center-surround processing is presumably in place for agood reason. Land and McCann (1971) developed a modelthey called Retinex, which placed the processing in a mean-ingful computational context. Land and McCann began by considering the nature ofscenes and images. They argued that reflectance tends to beconstant across space except for abrupt changes at the tran-sitions between objects or pigments. Thus a reflectancechange shows itself as step edge in an image, while illumi-nance will change only gradually over space. By this argu-ment one can separate reflectance change from illuminancechange by taking spatial derivatives: high derivatives are dueto reflectance and low ones are due to illuminance.The Retinex model applies a derivative operator to theimage, and thresholds the output to remove illuminance vari-ation. The algorithm then reintegrates edge information overspace to reconstruct the reflectance image.The Retinex model works well for stimuli that satisfy itsassumptions, including the Craik-OÕBrien-Cornsweet dis-play, and the ÒMondriansÓ that Land and McCann used. AMondrian (so-called because of its loose resemblance topaintings by the artist Mondrian) is an array of randomly col-ored, randomly placed rectangles covering a plane surface,and illuminated non-uniformly. 340 SENSORYSYSTEMS F IGURE 24.1 The simultaneous contrast effect. F IGURE 24.2 Center-surround inhibition. F IGURE 24.3 An illusion by Vasarely (a) and a bandpass filtered ver-sion (b) F IGURE 24.4 One version of the Craik-OÕBrien-Cornsweet Effect The real world is more complex than the Mondrianworld, of course, and the Retinex model has its limits. In itsoriginal form it cannot handle the configural effects to bedescribed later in this chapter. However, the Land-McCannresearch program articulated some important principles.Vision is only possible because there are constraints in theworld, i.e., images are not formed by arbitrary randomprocesses. To function in this world, the visual system mustexploit the ecology of imagesÑit must ÒknowÓ the likeli-hood of various things in the world, and the likelihood that agiven image-property could be caused by one or anotherworld-property.This world-knowledge may be hard-wiredor learned, and may manifest itself at various levels of pro-cessing.Limits on low-level processesThe high-level approach is historically associated withHelmholtz, who argued that perception is the product ofunconscious inference. His dictum was this: what we per-ceive is our visual systemÕs best guess as to what is in theworld. The guess is based on the raw image data plus ourprior experience. In the Helmholtz view, lightness constancyis achieved by inferring, and discounting, the illuminant.From this standpoint the details of low-level processing arenot the issue. Alightness judgment involves the workings ofthe whole visual system, and that system is designed to inter-pret natural scenes. Simultaneous contrast and other illusionsare the byproduct of such processing. Hochberg and Beck (1954), and Gilchrist (1977), showedthat 3D cues could greatly change the lightness perception ina scene, even when the retinal image remained essentiallyunchanged, in accord with HelmholtzÕs approach.The importance of scene interpretation is also shown bya recent variant on the Craik-OÕBrien-Cornsweet effect,devised by Knill and Kersten (1991). In figure 24.5(a), onesees two identical cylinders. In figure 24.5(b) one sees abrick painted with two shades of paint. Embedded withineach image is a COCE pattern. The two ramps are interpret-ed as shading in figure 24.5(a), but as paint in figure 24.5(b).The Gestalt approachThe Gestalt psychologists approached lightness percep-tion, and perception generally, in a different manner than theHering or the Helmholtz schools. They emphasized theimportance of perceptual organization, much of it based onmechanisms that might be characterized as mid-level. Thekey concepts include grouping, belongingness, good contin-uation, proximity, and so on. Koffka offered an example of how simultaneous contrastcan be manipulated by changing spatial configuration. Thering in figure 24.6(a) appears almost uniform in color.Whenthe stimulus is split in two, as shown in figure 24.6(b), thetwo half-rings appear to be different shades of gray.The twohalves now have separate identities, and each is perceivedwithin its own context. An interesting variant that involves transparency isshown in figure 24.6(c). The left and right half-blocks areslid vertically, and the new configuration leads to a very dif-ferent perceptual organization and a strong lightness illusion.We will return to this stimulus in our later discussion.Some terminologyHaving outlined some basic phenomena, we now return tothe basic problems. First, we will clarify some terminology.More complete definitions can be found in books on pho-tometry and colorimetry.Luminanceis the amount of visible light that comes tothe eye from a surface. Illuminanceis the amount of light incident on a surface. Reflectanceis the proportion of incident light that isreflected from a surface. Reflectance, also calledalbedo, varies from 0 to 1, or equiv-alently from 0% to 100%. 0% is ideal black; 100% is idealwhite. In practice, typical black paint is about 5% and typi-cal white paint about 85%. (To keep things simple, we onlyconsider matte surfaces, for which a single reflectance value ADELSON : LIGHTNESSPERCEPTIONANDLIGHTNESSILLUSIONS 341 F IGURE 24.5 Knill and KerstenÕs illusion. Both figures contain thesame COCE ramps, but the interpretations are quite different. offers a complete description.)Luminance, illuminance, and reflectance, are physicalquantities that can be measured by physical devices. Thereare also two subjective variables that must be discussed.Lightnessis defined as the perceived reflectance of a sur-face. It represents the visual systemÕs attempt to extractreflectance based on the luminances in the scene. Brightnessis defined as the perceived intensity of lightcoming from the image itself, rather than any property of theportrayed scene. Brightness is sometimes defined as per-ceived luminance. These terms may be understood by reference to figure 24.7.The block is made of a 2x2 set of cubes, each colored eitherlight or dark gray. We call this the Òchecker- b l o c k . ÓIllumination comes from an oblique angle, lighting differentfaces differently.The luminance image can be considered tobe the product of two other images: the reflectance imageand the illuminance image, shown below.These underlyingimages are termed intrinsic imagesin machine vision(Barrow and Tenenbaum, 1978). Intrinsic image decomposi-tions have been proposed for understanding lightness per-ception (Arend, 1994; Adelson and Pentland, 1996)Patches pand qhave the same reflectance, but differentluminances. Patches qand rhave different reflectances andd i fferent luminances; they share the same illuminance.Patches pand rhappen to have the same luminance, becausethe lower reflectance of pis counterbalanced by its higherilluminance. Faces pand qappear to be painted with the same gray,and thus they have the same lightness. However, it is clearthat phas more luminance than qin the image, and so thepatches differ in brightness. Patches pand rdiffer in bothlightness and brightness.The problem of lightness constancyFrom a physical point of view, the problem of lightness con-stancy is as follows. An illuminance image, E(x,y), and areflectance image, R(x,y), are multiplied to produce a lumi-nance image, L(x,y):An observer is given Lat each pixel, and attempts todetermine the two numbers Eand Rthat were multiplied tomake it. Unfortunately, numbers isimpossible. If E(x,y)and R(x,y)are arbitrary functions, thenfor any E(x,y)there exists an R(x,y)that produces theobserved image. The problem appears impossible, buthumans do it pretty well. This must mean that illuminanceand reflectance images are not arbitrary functions. They areconstrained by statistical properties of the world, as pro-posed by Land and McCann.Note that Land and McCannÕs constraints fail whenapplied to the checker-block image. Figure 24.8(a) showstwo light-dark edges. They are exactly the same in the 342 SENSORYSYSTEMS F IGURE 24.6 Variants on the Koffka ring. (a) The ring appears aboutuniform. (b) When split, the two half-rings appear distinctly differ-ent. (c) When shifted, the two half-rings appear quite different. F IGURE 24.7 The Òchecker-blockÓ and its analysis into two intrinsicimages. L(x,y) = E(x,y)R(x,y). image, and any local edge detector or filter will respond tothem in the same way. Retinex will classify both asreflectance steps. Yet they have very different meanings.One is caused by illuminance (due to a change in surfacenormal); the other is caused by reflectance. To interpret the edges, the visual system must considerthem in a larger context. One good source of information isthe junctions, such as those labeled in figure 24.8(b). Ajunc-tion is a place where two or more contours come together. X,Y, L, T, and y, as shown, are some of the simple junctiontypes. The configuration of a junction, as well as the graylevels forming the junction, can offer cues about the shadingand reflectance of a surface. Particularly strong constraints are imposed by a y-junc-tion, like that in figure 24.8(b). The vertical spine appears tobe a dihedral with different illuminance on the two sides.The angled arms appear to represent a reflectance edge thatcrosses the dihedral. The ratios of the gray levels, and theangles of the arms, are consistent with this interpretation. The influence of a y-junction can propagate along thecontours that meet at the y.Asingle light-dark edge,ambiguous by itself, can be pushed toward a particular inter-pretation by adjoining yÕs (Sinha and Adelson, 1993). In figure 24.9, the dashed rectangle encloses a set of hor-izontal light and dark stripes. If one only considers theregion within the dashed rectangle, it is impossible to deter-mine the physical sources of the stripes. However, if onecovers the right side of the figure and views the left side, itappears that the stripes are due to paint. If one covers the leftside and views the right, it appears that the stripes are due thedifferent lighting on the stairsteps. If one views both sides,the percept flip-flops according to where one looks. The y-junctions seem to be in control here. If one fol-lows a stripe to the left, it connects to a y with a verticalspine, and becomes an arm of that y.The junction configu-ration, along with the junction gray levels, suggest that thestripe is due to reflectance. When the same strip is followedto the right, it joins a ywith a horizontal stem. Again, theconfiguration and gray levels suggest that illuminance is thecause. Configurations involving yÕs can modulate brightnessillusions. Figure 24.10 shows a stimulus we call the corru-gated plaid(Adelson, 1993). In figure 24.10(a) the twomarked patches are the same shade of gray.The upper patchappears slightly darker. Figure 24.10(b) shows another arrayof gray patches that have the same gray levels at the samepositions as in figure 24.10(a), i.e., the same raster sequenceof grays. Only the geometry has been changed, parallelo-grams having been substituted for squares and vice-versa.The illusion is much enhanced, the upper patch appearingmuch darker than the lower one. In the laboratory the appar-ent luminance difference is increased threefold. Alow-level filtering mechanism, or a mechanism basedon local edge interactions, cannot explain the change in theillusion. We proposed (Adelson, 1993) a Helmholtzianexplanation based on intrinsic images: the change in y-junc-tions causes a change in the perception of 3D surface orien-tation and shading. In figure 24.10(a) the two test patchesappear to be in the same illumination, but in figure 24.10(b) ADELSON : LIGHTNESSPERCEPTIONANDLIGHTNESSILLUSIONS 343 F IGURE 24.8 (a) The local ambiguity of edges. (b) Avariety of junc-tions. F IGURE 24.9 The impossible steps. On the left, the horizontal stripesappear to be due to paint; on the right, they appear to be due to shad-ing. they are differently illuminated. Abrightly lit patch of darkgray looks quite different from a dimly lit patch of light gray.This lightness computation could have a strong influence onbrightness judgments. Thus a 3D shaded model can help explain the phenome-non, but is it necessary? Todorovic has devised a variant,shown in figure 24.10(c), that suggests not. The figure wasmade by mirror reversing the bottom two rows. The illusionremains strongÑnearly as strong as figure 24.10(b)Ñformany subjects. However, there is no reasonable interpreta-tion in terms of a 3D shaded model. The two strips contain-ing the test patches appear to lie in parallel planes, and sothey should be receiving similar illumination. Perhaps the intrinsic image story can be saved by appeal-ing to the notion of local consistency without global consis-tency, such as occurs in figure 24.9. However, it may be thatthe main effects are the result of simpler 2D computations.The y-junctions, taken as 2D configurations, could be usedas grouping cues that define the context in which lightness isassessed, as indicated in figure 24.10(d). If this is correct,then the Helmholtzian approach is overkill. Anumber of investigators have lately argued for modelsbased on Gestalt-like grouping mechanisms (e.g., Ross andPessoa, in press). Gilchrist, who in earlier years took aHelmholtzian stance (Gilchrist et al, 1983), has recently pro-posed a model of lightness that emphasizes 2D configurationand grouping mechanisms (Gilchrist et al., in press).Anchoring and frameworksGilchristÕs new model took shape in the course of his inves-tigations into anchoring.The anchoring problem is this.Suppose an observer determines that patch xhas four timesthe reflectance of patch y.This solves part of the lightnessproblem, but not all of it: the absolute reflectances remainunknown. An 80% near-white is four times a 20% gray, buta 20% gray is also four times a 5% black. For absolute judg-ments one must tie down the gray scale with an anchor, i.e.,a luminance that is mapped to a standard reflectance such asmidgray or white.Land and McCann had encountered this problem withRetinex, and they proposed that the highest luminanceshould be anchored to white. All other grays could then bescaled relative to that white. This is known as the highestluminance rule.Li and Gilchrist (in press) tested the highest-luminancerule using bipartite ganzfelds. They painted the inside of alarge hemispherical dome with two shades of gray paint.When subjects put their heads inside, their entire visualfields were filled with only two luminances. Abipartite fieldpainted black and gray appeared to be a bipartite field paint-ed gray and white, as predicted by the highest luminancerule. By manipulating the relative areas of the light and darkfields, Gilchrist and Cataliotti (1994) found evidence for asecond, competing anchoring rule: the largest area tends toappear white. They argue that the actual anchor is a compro-mise between these rules. 344 SENSORYSYSTEMS F IGURE 24.10 Variations on the corrugated plaid. (a) The two patch-es appear nearly the same. (b) The patches appear quite different. (c)The patches appear quite different, but there is no plausible shadedmodel. (d) Possible grouping induced by junctions. Gilchrist also emphasizes the importance of articulationand insulationin anchoring. Articulation is a term used ear-lier by Katz (1935); it refers to the number of distinct sur-faces or patches within a region. Katz observed that greaterarticulation leads to better lightness constancy, and Gilchristproposes that it leads to better local anchoring. We candemonstrate the effect of articulation with a simultaneouscontrast display, as in figure 24.11. Above is a standard dis-play. Below is an articulated version. The surround meanluminances have not been changed, but the surrounds havebeen broken into many squares. The articulated versiongives a stronger illusion. In our laboratory we find that thestrength of the illusory contrast can be doubled. (As with allthe demonstrations in this chapter, the effect may be weakeron the printed page due to the small image size and limita-tions in the printing process).In GilchristÕs model, anchoring occurs within a frame-workwhich is a region containing patches that are grouped.Frameworks can be local or global. In figure 24.11, a localframework would be the patches surrounding the test square,and the global framework would be the entire page, and eventhe room in which the page is viewed.If a local framework is well insulated, it has strong con-trol over the anchoring. Insulation occurs when the localframework is strongly grouped as a separate entity from theglobal framework. Statistical EstimationThe various lightness principles might be thought of asheuristics that the visual system has arbitrarily adopted.These principles begin to make sense, however, if we con-sider the lightness problem from the standpoint of statisticalestimation. Suppose that the world consisted of a set of gray patchesrandomly drawn from some distribution. Then, under a givenilluminance, one would observe a distribution of luminancesamples such as that shown in figure 24.12(a). If the illumi-nation were dimmed by half, then the luminances would fol-low suit, as shown in figure 24.12(b). The arrows bracketingthe distributions represent the extremes of 0% and 100%reflectance, i.e., the luminances mapping to ideal black andideal white.The observed luminances can also be changed by anadditive haze or glare, which slides the distribution upward,illustrated in figure 24.12(c). Again one can estimate whichluminance corresponds to which reflectance, i.e., one canestimate the mapping between the observed luminance andthe underlying reflectance. We use the term atmosphere torefer to the combined effects of a multiplicative process(e.g., illuminance) and an additive process (e.g., haze). If one has prior reflectances and atmospheres, then one can construct opti-mal estimates of the locations of various reflectances alongthe luminance axis. That is, one can estimate the mappingbetween luminance and reflectance, as is required for light-ness constancy.Estimating this mapping is a central task of lightness per- ADELSON : LIGHTNESSPERCEPTIONANDLIGHTNESSILLUSIONS 345 F IGURE 24.11Simultaneous contrast is enhanced with articulatedsurrounds, as shown below. F IGURE 24.12 Acollection of random gray surfaces will lead to adifferent luminance distribution in different viewing conditions. ception. The image luminance is given and the perceivedreflectance (lightness) must be derived. Anchoring is a wayof describing part of this process. We will return to this prob-lem when we discuss lightness transfer functions. Adaptive windowsAlarger number of samples will lead to better estimates ofthe lightness mapping. To increase N, the visual system cangather samples from a larger window. However, the atmos-phere can vary from place to place, so there is a counter-argument favoring small windows.Suppose that the visual system uses an adaptive windowto deal with this tradeoff. The window grows when there aretoo few samples, and shrinks when there are more thanenough. Consider the examples shown in figure 24.13. In theclassical SC display, figure 24.13(a), there are only a fewlarge patches, so the window will tend to grow. In the artic-ulated SC display, figure 24.13(b), the window can remainfairly small. Lightness estimates are computed based on the statisticswithin the adaptive window. In the classic SC display, thewindow becomes so large that the statistics surroundingeither of the test patches are rather similar. (In GilchristÕs ter-minology, the global framework dominates). In the articulat-ed display, the windows can be small, so that they will notmix statistics from different atmospheres. This predicts theenhancement in the illusion.It is reasonable to assume that the statistical window hassoft edges. For example, it could be a 2D Gaussian humpcentered at the location of interest. Since nearby patches arelikely to share the same atmosphere, proximity should leadto high weights, with more distant patches getting lowerweights (cf. Reid and Shapley 1988, Spehar et al., 1996).The dashed lines in figure 24.13 would indicate a level lineof the Gaussian hump.Afurther advantage occurs if the adaptive window canchange shape. For example, in figure 24.13(c), it would beprudent to keep the statistical pooling within the horizontalregion shown by the ellipse. This will avoid mixing lumi-nances from the adjacent regions, which are in differentlighting.This reasoning might explain why y-junctions are effec-tive at insulating one region from anotheryÕs alonga contour (and with the appropriate gray levels) gives astrong cue that the contour is an atmospheric boundary.Thestatistical window should avoid crossing such a boundary inorder to avoid mixing distributions. Thus the window shouldconfigure itself into a shape like that in figure 24.13(c).AtmospheresAs noted above, illuminance is only one of the factors deter-mining the luminance corresponding to a given reflectance.Other factors could include interposed filters (e.g., sunglass-es), scattering, glare from a specular surface such as a wind-shield, and so on. It turns out that most physical effects willlead to linear transforms. Therefore the combined effects canbe captured by a single linear transform (characterized bytwo parameters). This is what we call an atmosphere.The equation we use is,where Land Rare luminance and reflectance, mis a multi-plier on the reflectance, and eis an additive source of light.The value of mis determined by the amount of light fallingon the surface, as well as the proportion of light absorbed bythe intervening media between the surface and the eye. The equation here is closely related to the linear equationunderlying Metelli's episcotister model (Metelli, 1974) fortransparency, except that there is no necessary couplingbetween the additive and multiplicative terms. The parame-ters mand eare free to take on any positive values.An atmosphere may be thought of as a single transparentlayer, except that it allows a larger range of parameters. Itcan be amplifying rather than attenuating, and it can have anarbitrarily large additive component.In our usage, ÒatmosphereÓ simply refers to the mapping,i.e., the mathematical properties established by the viewingconditions the processes. Putting on sunglasses or dimming the lights hasthe same effect on the luminances, and so leads to the sameeffect on atmosphere. To be more explicit about this mean-ing, we define the Atmospheric Transfer Function, or ATF, asthe mapping between reflectance and luminance.Figure 24.14 shows a set of random vertical lines viewed 346 SENSORYSYSTEMS F IGURE 24.13 Lightness computations may employ adaptive win-dows. L= m R + e, in three different atmospheres. The large outer region is insome default atmosphere. The left disk is in an attenuatingatmosphere (compared to the default). The right disk is in ahazy atmosphere.The ATF for the main atmosphere is shown in figure24.14(a). It passes through the origin, meaning that eis zero.The slope is specified by m. (Note: Since reflectance andluminance are in different units, there is also a scale constantthat depends on the chosen units.) The small arrows in thepanels show how the various reflectances are mapped totheir corresponding luminances. The shaded area within thearrows shows how a typical range of reflectances will bemapped into the corresponding range of luminances.Figure 24.14(b) shows the ATF for the dimmer atmos-phere. The slope is reduced, and the intercept remains zero.On the right, in figure 24.14(c), is the ATF of the hazy atmos-phere. The output luminance range is compressed by mandshifted up by e.Note that there is no such thing as a Ònon-atmosphere.ÓAn observer cannot see the reflectances Òdirectly,Ó but ratherrequires function reflectances to luminances. The parameters of the ATFalways have values.Finally, note that the (m,e) parameterization has no priv-ileged status. Any two numbers will do. For example, a use-ful alternative would be the white-point and the black-point.Since the atmosphere maps a reflectance to a luminance, theobserver must implicitly reverse the mapping, turning aluminance into a perceived reflectance, as illustrated in fig-ure 24.15. The inverting function, for a given observer in agiven condition, may be called the lightness transferfunc-tionor LTF.The LTF is subjective; it need not be linear andneed not be the correct inverse of the ATF. For a givenobserver it must be determined empirically.Atmospheres and X-junctionsThe connection between X-junctions and atmospheres isshown in figure 24.16. Different types of atmospheres leadto different ordinal categories of X-junctions (cf. Beck et al,1984; Adelson and Anandan, 1990; Anderson, 1997).Figure 24.16(a) shows a region with two shades of graypaint. The large light square has 75% reflectance and thesmall dark square in the corner has 25% reflectance. The tworeflectances are marked with arrows on the abscissa of thecorresponding ATF diagram, below. Figure 24.16(b) showswhat happens when a new atmosphere is introduced in thecentral patch. The new ATF is shown in a dashed line in theATF diagram; it might be produced by a dark filter or a shad-ow.The resulting luminances form an X-junction of theÒsign-preservingÓ or Ònon-reversingÓ type (Adelson andAnandan, 1990), which is consistent with transparency.Figure 24.16(c) shows a different category of X-junction:the single-reversing X. It gives the impression of a murky orhazy medium. For a single-reversing X, the new ATF mustcross the at a point between the tworeflectances. Acrossover ATF can only arise from an addi-tive process combined with an attenuative process, such aswould occur with smoke or a dirty window.Another differ-ence between single-reversing and sign-preserving XÕs isthat either edge of a sign-preserving Xis potentially anatmospheric boundary, while only one edge of a single-reversing edge can be an atmospheric boundary. For this rea-son, the depth ordering of a single-reversing Xis unambigu-ous. Finally, figure 24.16(d) shows a double-reversing X,which does not look transparent. The ATF needed to producethis Xwould need a negative slope. This cannot occur in ADELSON : LIGHTNESSPERCEPTIONANDLIGHTNESSILLUSIONS 347 F IGURE 24.14 Lines of random gray, viewed under three differentatmospheres. The ATFÕs, shown below, determine the mapping fromreflectance to luminance. F IGURE 24.15 The inverse relation betweent he atmospheric trans-fer function and the ideal lightness transfer fuction. normal physical circumstances. Double-reversing X-junc-tions do not signal atmospheric boundaries to the visual sys-tem, and they typically look like paint rather than trans-parency.The junctions in a checkerboard are double-revers-ing XÕs.The ATF diagrams offer a simple graphical analysis ofdifferent X-junction types, and show how the X-junctionscan be diagnostic of atmospheric boundaries.Figure 24.17 shows an illusion using X-junctions to makeatmospheres perceptible as such. The centers of the two dia-mond shaped regions are physically the same shade of lightgray. However, the upper one seems to lie in haze, while thelower one seems to lie in clear air.The single-reversing XÕs surrounding the lower diamondindicate that it is a clearer region within a hazier region. Thestatistics of the upper region are elevated and compressed,indicating the presence of both attenuative and additiveprocesses. Thus the statistical cues and the configural cuespoint in the same direction: the lower atmosphere is clearwhile the upper one is hazy.The shifted Koffka ringsIt is useful at this point to recall the modified Koffka displayof figure 24.6(c). When the two halves are slid vertically, aset of sign-preserving X-junctions is created along the verti-cal contour.The junctions are consistent with transparency,and the contour becomes a strong atmospheric boundarybetween the left and right regions. The two semicircles areseen within different frameworks. The statistics on the twosides are different. In addition, grouping cues such as goodcontinuation indicate that the left semicircle is connected tothe light region on the right, and the right semicircle is con-nected to the dark region on the left. Thus there are severalcues that conspire to make the two semicircles look quite dif-ferent. T-junctions and WhiteÕs illusionWhiteÕs illusion is shown in figure 24.18. The gray strips arethe same. This is surprising: by local contrast, the left onesshould look darker than the right ones. The left strips have along border with white and a short border with black. Theillusion is reversed from the usual direction. This effect hasbeen interpreted in terms of the T-junctions (Todorovic,1997, Gilchrist et al., in press). Patches straddling the stemof a Tare grouped together for the lightness computation,and the cross-bar of the Tserves as an atmospheric bound-ary. (cf. Anderson, 1997, for an alternative approach).Zaidi et al. (1997) have shown that the action of T-junc-tions can be so strong that it overpowers traditional groupingcues such as coplanarity.Therefore the grouping rules for thelightness computation evidently differ from those underlyingsubjective belongingness. Constructing a new illusionOne can intentionally combine statistical and configural cues 348 SENSORYSYSTEMS F IGURE 24.16 Transparency involves the imposition of a newatmosphere. The resulting X-junctions category depends on theatmospheric transfer function. F IGURE 24.17 The haze illusion. The two marked regions are iden-tical shades of gray. One appears clear and the other appears hazy. to produce large contrast illusions. In the Òcriss-crossÓ illu-sion of figure 24.19, the small tilted rectangles in the middleare all the same shade of gray. Many people find this hard tobelieve. The figure was constructed by the following princi-ples: The multiple y-junctions along the vertical edgesestablish strong atmospheric boundaries. Within each verti-cal strip there are three luminances and multiple edges toestablish articulation. The test patch is the maximum of thedistribution within the dark vertical strips, and the minimumof the distribution within the light vertical strips. The com-bination of tricks leads to a strong illusion.Each y-junction, by itself, would offer evidence of a 3Dfold with shading. However, along a given vertical contourthe yÕs point in opposite directions, which discourages thefolded interpretation. Some subjects see the image in termsof transparent strips; others see it merely as a flat painting.However, all subjects see a strong illusion instantly.Thus, a3D folded percept is not necessary: the illusion works evenwhen Òa yis just a yÓ (Hupfeld, 1931).The snake illusionSimilar principles can be used to construct a figure with X-junctions. Figure 24.20(a) shows an illusion we call thesnake illusion (Somers and Adelson, 1997). The figure is amodification of the simultaneous contrast display shown atthe right. The diamonds are the same shade of gray and theyare seen against light or dark backgrounds. Aset of half-ellipses have been added along the horizontal contours. TheX-junctions aligned with the contour are consistent withtransparency, and they establish atmospheric boundariesbetween strips. The statistics within a strip are chosen so thatthe diamonds are at extrema within the strip distributions.Note that the ellipses do not touch the diamonds, so the edgeits unchanged. Figure 24.20(b) shows a different modification in whichthe half-ellipses create a sinuous pattern with no junctionsand no sense of transparency.The contrast illusion is weak;for most subjects it is almost gone. Thus, while figures 20(a)and (b) have the same diamonds against the same surrounds,the manipulations of the contour greatly change the lightnesspercept. In effect, we can turn the contrast effect up or downby remote control. ADELSON : LIGHTNESSPERCEPTIONANDLIGHTNESSILLUSIONS 349 F IGURE 24.18 WhiteÕs illusion. The gray rectangles are all the sameshade of gray. F IGURE 24.19 The crisscross illusion. The small tilted rectangles areall the same shade of gray. F IGURE 14.20 The snake illusion. All diamonds are the same shadeof gray. (a) The regular snake: the diamonds appear quite different.(b) The Òanti-snakeÓ: the diamonds appear nearly the same. Thelocal contrast relations between diamonds and surrounds are thesame in both (a) and (b). Why should the illusion of figure 24.20(b) be weakerthan in the standard SC? We have various observations sug-gesting that the best atmospheric boundaries are straight, andthat curved contours tend to be interpreted as reflectance.The sinuous contours of figure 24.20(b) are not seen asatmospheric boundaries, and therefore the adaptive windowis free to grow and to mix statistics from both light and darkstrips. SummaryIllusions of lightness and brightness can help reveal thenature of lightness computation in the human visual system.It appears that low-level, mid-level, and high-level factorscan all be involved. In this chapter we have emphasized thephenomena related to mid-level processing.Our with the otherresearchers, supports the notion that statistical and configur-al information are combined to estimate the lightness map-ping at a given image location. In outline, picture looks likethis:¥At every point in an image, there exists an apparentatmospheric transfer function (ATF) mapping reflectanceinto luminance. To estimate reflectance given luminance, thevisual system must invert the mapping, implicitly or explic-itly.The inverting function at each point may be called thelightness transfer function (LTF).¥The lightness of a given patch is computed by compar-ing its luminance to a weighted distribution of neighboringluminances. The exact computation remains unknown. ¥Classical mechanisms of perceptual grouping can influ-ence the weights assigned to patches during the lightnesscomputation. The mechanisms may include proximity, goodcontinuation, similarity, and so on. However, the groupingused by the lightness system apparently differs from ordi-nary perceptual grouping. ¥The luminance statistics are gathered within an adaptivewindow.When the samples are plentiful the window remainssmall, but when the samples are sparse the window expands.The window is soft-edged. ¥The adaptive window can change shape and size inorder to avoid mixing information from different atmos-pheres. ¥Certain junction types offer evidence that a given con-tour is the result of a change in atmosphere. The contour thenacts as an atmospheric boundary, preventing the informationon one side from mixing with that on the other.Aseries ofjunctions aligned consistently along a contour produce astrong atmospheric boundary. Some evidence suggests thatstraight contours make better atmospheric boundaries thancurved ones. REFERENCESA DELSON ,E. H., 1993. Perceptual organization and the judgment ofbrightness. Science262:2042-2044.A DELSON ,E. H., and P.A NANDAN , 1990. Ordinal characteristics oft r a n s p a r e n c y. P roceedings the AAAI-90 Workshop onQualitative Vision, pp. 77-81.A DELSON ,E. H., and A.P. P ENTLAND , 1996. The Perception ofShading and Reflectance. In Perception as Bayesian Inference,D. Knill and W. New York: A NDERSON ,B. L., 1997. Atheory of illusory lightness and trans-parency in monocular and binocular images: the role of contourjunctions. Perception26:419-453.A R E N D , L., 1994. Surface Colors, Illumination, and SurfaceHuman ColorPerception. In Lightness, Brightness, and Transparency,A.Gilchrist, ed. Hillsdale, N.J.:Erlbaum, pp. 159-213. B ARROW ,H. G., and J. T ENENBAUM , 1978. Recovering intrinsicscene characteristics from images. In Computer Vision Systems,A. R. Hanson and E. M. Riseman, eds. New York: AcademicPress, pp. 3-26.B ECK , J., K. P RAZDNY ,and R. I VRY , 1984. The perception of trans-parency with P e rcept. G ILCHRIST ,A. L., 1977. Perceived lightness depends on perceivedspatial arrangement. Science195(4274): 185-187.G ILCHRIST ,A. L., and A. J ACOBSEN , 1983. Lightness constancythrough a veiling luminance. J. Exp. Psych.9(6):936-944.G ILCHRIST ,A. L., C. K OSSYFIDIS , F. B ONATO ,T.A GOSTINI , J. X. L.C ATALIOTTI , B. S PEHAR ,andJ. S ZURA , in press. Anew theory oflightness perception. Psych. Rev.H OCHBERG ,J. E., and J. B ECK ,1954. Apparent spatial arrangementand perceived brightness. J. Exp. Psych.47:263-266.H UPFELD ,H, 1931. Lyrics to ÒAs Time Goes By,Ó Harms, Inc.,ASCAP. See also Casablanca,Warner Brothers, 1942.K ATZ , D., 1935. The World of Colour.London: Kegan Paul.K NILL , D., and D. K ERSTEN , 1991. Apparent surface curvatureaffects lightness perception. Nature351: 228-230.L AND , E. H., and J. J. M C C ANN , 1971. Lightness and retinex theo-ry.J. Opt. Soc. Amer. 61:1-11.L I ,X., and A. G ILCHRIST , in press. Relative area and relative lumi-nance combine to anchor surfacelightness values. Percept.Psychophysics.M ETELLI , F., 1974. Achromatic color conditions in the perception oftransparency. In Perception, Essays in Honor of J.J. Gibson, R.B. McLeod and H. L. Pick, eds. Ithaca, NY: Cornell UniversityPress, pp. 93-116.R EID , R. C., and R. S HAPLEY , 1988. Brightness induction by localcontrast and the spatial dependence of assimilation. Vision Res.28:115-132.R OSS ESSOA , in press. Contrast/filling-in model of 3-Dlightness perception. Percept. Psychophysics.S INHA , P., and E. H. A DELSON , 1993. Recovering reflectance in aworld of P roceedings Fourt h 350 SENSORYSYSTEMS International Conference on Computer Vision, Berlin; May 11-14, 1993; Los Alamitos, Calif: IEEE Computer Society Press,pp.156-163.S OMERS , D. C., and A DELSON , E. H., 1997. Junctions, transparency,and brightness. Invest. Ophthalmol. Vis. Sci. (Suppl.)38:S453.S PEHAR , B., I. D E B ONET , and Q. Z AIDI , 1996. Brightness inductionfrom uniform and complex surrounds: Ageneral model. VisionRes., 36, 1893-1906.T O D O R O V I C , D., 1997. P e rc e p t i o n26(4):379-394.Z AIDI , Q., B. S PEHAR , and M. S HY , 1997. Induced effects of back-grounds and foregrounds in three-dimensional configurations:The role of T-junctions. Perception26(4):395-408. ADELSON : LIGHTNESSPERCEPTIONANDLIGHTNESSILLUSIONS 351