Binocular Vision

Abstract

Binocular vision is a form of visual perception: the coordinated use of both eyes, whose slightly different viewpoints let the brain recover the depth of a scene. Because the two eyes are separated horizontally, any object projects to slightly different positions on the two retinas, and this binocular disparity is the signal from which stereoscopic depth, or stereopsis, is computed. Matching the features seen by one eye to those seen by the other poses the correspondence problem, solved by cortical circuits containing disparity-selective neurons first identified in the primary visual cortex. This article covers the geometry of disparity, the correspondence problem and its computational solutions, the neurons that encode depth, binocular rivalry between incompatible images, and the development and disorders of binocular vision, with interactive demonstrations.

Keywords: binocular vision, stereopsis, binocular disparity

Binocular vision is the use of two forward-facing eyes as a single perceptual system. Its central achievement is stereopsis, the vivid impression of depth and solidity that arises because the two eyes, separated by roughly six and a half centimetres, view the world from slightly different vantage points. Charles Wheatstone made this dependence concrete in 1838 when he built the stereoscope and showed that presenting each eye with a flat drawing of what it alone would see, computed for two separated viewpoints, produces a compelling perception of three-dimensional depth from two two-dimensional images (Wheatstone, 1838). Binocular vision matters to cognitive psychology because it is a case in which the brain manufactures a percept, the depth of a scene, that is present in neither retinal image on its own but only in the relation between them. It also serves the wider ends of combining two noisy images into one cleaner estimate and of seeing around partial occluders, and its mechanisms recur, in varied form, across much of the animal kingdom (Read, 2021).

Binocular Disparity and the Geometry of Stereopsis

The geometry of two eyes fixing a common point defines the whole problem. When the eyes converge on a target, its images fall on corresponding points, the paired retinal locations that map to the same visual direction, and it is seen as single. The set of points in space that project to corresponding points, and so are also seen single at that fixation, forms a curved surface called the horopter. Objects nearer or farther than the horopter project to non-corresponding points, and the angular mismatch between the two retinal images is the binocular disparity of that object. Around the horopter lies a narrow zone, Panum's fusional area, within which the two images are still fused into one; disparities larger than this zone allows are seen as double, a state called diplopia.

Disparity carries a sign as well as a magnitude, and the sign specifies depth (Table 1). An object closer than the fixation point produces crossed disparity, so named because its images are displaced toward each eye's nose and would require the eyes to cross to fixate it; an object beyond the fixation point produces uncrossed disparity in the opposite direction. The visual system reads the sign to place objects in front of or behind the horopter and reads the magnitude to judge how far. The finest disparity that yields a reliable depth judgement, the stereoacuity of the observer, can reach a few arc seconds under good conditions, a spatial precision finer than the spacing of the photoreceptors and one of the sharpest thresholds in human perception (Read, 2021). That two flat images can be combined into depth at all was Wheatstone's founding demonstration, and every later account of stereopsis is a theory of how the disparity between them is measured and used (Wheatstone, 1838).

The Correspondence Problem

Before disparity can be measured, the visual system must decide which feature in the left image corresponds to which feature in the right, and this matching is far harder than it sounds. Béla Julesz brought the difficulty into sharp relief with the random-dot stereogram, a pair of images each consisting of the same random field of black and white dots except that a central region is shifted horizontally in one image (Julesz, 1960). Neither image alone contains any figure, edge, or object; only when the two are fused does a shape spring out in depth. The stereogram proves that stereopsis can operate on disparity alone, before any monocular form is recognized, and it poses the correspondence problem in its purest form: with thousands of identical dots, each dot in one eye could in principle match many dots in the other, and the system must select the one globally consistent set of matches.

David Marr and Tomaso Poggio treated this as a computation to be specified independently of the hardware that runs it. Their cooperative algorithm exploited two constraints drawn from the physics of surfaces: each point on a surface has a single depth, so a given feature should take only one match (uniqueness), and depth varies smoothly across most of a surface, so neighbouring points should take similar disparities (continuity). A network in which candidate matches excite neighbours of like disparity and inhibit alternative matches of the same feature settles onto the correct global solution (Marr & Poggio, 1976). They later proposed a second, more biological scheme in which disparity is measured coarsely by large, low-resolution channels and then refined by finer ones, with eye movements bringing features into the range of the fine channels, an approach that resolves ambiguity by working from coarse to fine scales (Marr & Poggio, 1979). Both algorithms show that the correspondence problem is soluble given the right assumptions about surfaces, and both anticipated the disparity-tuned machinery later found in cortex.

Unpaired Points and da Vinci Stereopsis

Not every point in a scene has a match, and the unmatched points turn out to carry information rather than noise. At the edge of any opaque object, a strip of the background is visible to one eye but hidden from the other, because the near object occludes slightly different parts of the far surface for each viewpoint. Ken Nakayama and Shinsuke Shimojo showed that the visual system treats these half-occluded, unpaired points as a valid depth cue in their own right, assigning them a depth consistent with the occlusion geometry and even generating subjective occluding contours and a definite depth order where no disparity as such exists (Nakayama & Shimojo, 1990). They named the effect da Vinci stereopsis, after Leonardo's observation that each eye sees a little way around a near object. Its lesson is that binocular vision is not merely a disparity-matching machine but an interpretation of the scene as a layout of surfaces at different depths: the very regions one eye cannot see are used to recover which surface lies in front and how the hidden surface continues behind it. This reframes the correspondence problem, since a complete account must decide not only how to match features but which features are unmatched by design and what their being unmatched implies about depth.

Disparity-Selective Neurons

The neural basis of stereopsis is a population of neurons each tuned to a preferred binocular disparity. Recording in the cat visual cortex, Horace Barlow, Colin Blakemore, and Jack Pettigrew found single cells that responded best when a bar fell on the two retinas with a particular disparity, firing for a specific depth relative to fixation and little for others; different cells preferred different disparities, so that depth was encoded across the population (Figure 1). This was the first direct evidence that the brain contains detectors for binocular depth (Barlow et al., 1967). Gian Poggio and Bruno Fischer extended the finding to the alert, fixating monkey, describing neurons tuned to zero disparity, to near (crossed) disparities, and to far (uncrossed) disparities in striate and prestriate cortex, and so linking the single-cell responses to behaviour in an animal actually using stereopsis (Poggio & Fischer, 1977).

How such tuning is built was clarified by a quantitative model. Izumi Ohzawa, Gregory DeAngelis, and Ralph Freeman proposed the disparity energy model, in which a complex cell combines the outputs of simple cells whose left-eye and right-eye receptive fields differ in position or in phase, yielding a unit that signals disparity largely independent of the exact stimulus; their recordings matched the model closely, giving stereopsis a concrete circuit (Ohzawa et al., 1990). Bruce Cumming and Gregory DeAngelis, reviewing the physiology, stressed that the disparities encoded by early cortical neurons do not yet correspond to perceived depth, since these neurons respond to the physically impossible matches in anticorrelated stereograms that observers cannot see as depth, so further processing must intervene between the raw disparity signal and the percept (Cumming & DeAngelis, 2001). Andrew Parker traced that further processing through the cortex, arguing that the transformation from disparity signals to depth judgements is distributed across a hierarchy of areas beyond the primary visual cortex, each stage bringing the neural code closer to what the observer actually perceives (Parker, 2007). Recent work has begun to identify what the intervening computation does: Nuno Goncalves and Andrew Welchman found human cortical mechanisms that actively suppress the impossible anticorrelated matches, a set of what-not detectors that discard the false correspondences and so make the depth estimate reliable (Goncalves & Welchman, 2017).

Figure 1

Disparity Tuning of Cortical Neurons

Binocular Rivalry

When the two eyes are shown images that cannot be fused into one, binocular vision does not average them but alternates between them. Present vertical stripes to one eye and horizontal stripes to the other and an observer sees not a plaid but first one grating, then the other, in an irregular, ceaseless exchange that can last for minutes; this is binocular rivalry. Randolph Blake and Nikos Logothetis reviewed the phenomenon as a window onto how the brain resolves conflicting visual signals and, more broadly, onto the mechanisms of visual awareness, since the physical stimulus is constant while the percept changes, dissociating what is seen from what is present (Blake & Logothetis, 2002). Rivalry reveals that fusing the two eyes' views into a single stereoscopic percept is the normal outcome of a system that must actively decide whether the two images belong to one scene or two, and it exposes the suppression that binocular vision applies whenever monocular signals are irreconcilable, the same suppression that, applied chronically in development, can weaken one eye's contribution for good.

Rivalry is only the conflict case of a more general process. When the two eyes' images are compatible, they are not merely fused but combined, so that vision with two eyes is more sensitive than with either alone: fainter patterns are detected and finer contrast differences discriminated binocularly, an advantage known as binocular summation. It was long summarized as an improvement of about a factor of 1.4, the square root of two, the value expected if the two eyes' independently noisy signals were simply pooled; but a meta-analysis of five decades of measurements found the true benefit to be larger than that and to depend on how the monocular signals are weighted and gain-controlled before they are added, so that summation is an active combination rule rather than passive averaging (Baker et al., 2018).

Development and Disorders of Binocular Vision

Binocular vision is assembled in early life and depends on balanced input from the two eyes during a sensitive period. David Hubel and Torsten Wiesel showed that closing one eye of a kitten for a short interval early in development sharply reduces the number of cortical neurons that can be driven by that eye, and that the same deprivation later in life has little effect, defining a critical period during which binocular connections are shaped by experience and after which they are largely fixed (Hubel & Wiesel, 1970). When the two eyes' inputs are misaligned or unequal in childhood, by strabismus, a turned eye, or by unequal refractive error, the visual cortex comes to favour the better input and suppress the other, producing amblyopia, a developmental loss of acuity in an eye that is itself healthy. Dennis Levi, reviewing stereopsis and amblyopia, described how amblyopia degrades or abolishes stereopsis and stressed that the deficit is cortical rather than optical, so that clear retinal images are necessary but not sufficient for normal binocular depth (Levi et al., 2015).

Binocular vision can also be lost later, through damage to the visual brain rather than the eyes. Holly Bridge reviewed how injury to the visual cortex and its pathways disrupts binocular depth perception, showing that lesions beyond the primary visual cortex can selectively impair stereopsis while sparing other aspects of vision, evidence that the cortical machinery for depth is both specialized and vulnerable (Bridge, 2016). According to MeSH, disorders such as amblyopia, strabismus, and stereoblindness are classified as conditions of binocular vision, and their study has repaid the effort twice over: it identifies who will benefit from early intervention, and it reveals, through its failures, how a normal binocular system is built and maintained.

Worked Example

The link between disparity and depth is quantitative, and a short calculation shows how fine the signal is. For two objects near the line of sight, one at distance z and one slightly farther by a small amount delta-z, the binocular disparity between them is given, to a good approximation, by the interocular distance I times delta-z divided by the square of the distance: disparity in radians is approximately I times delta-z over z squared. The inverse-square dependence on distance is the key fact: the same physical depth step yields far less disparity when it is far away.

Take an interocular distance of I = 6.5 cm and fixate at z = 100 cm, with a second object delta-z = 5 cm beyond the first. The disparity is 6.5 times 5 divided by 100 squared, which is 32.5 over 10,000, or 0.00325 radians. Converting to angular measure by multiplying by 3,437.75 arc minutes per radian gives about 11.2 arc minutes, a large and easily seen disparity. Now run the calculation the other way to find the limit of the system. A good stereoacuity threshold is about 10 arc seconds, which is 4.85 times ten to the minus five radians. Solving the same relation for delta-z, the smallest detectable depth step equals the threshold disparity times z squared divided by I, that is 4.85 times ten to the minus five times 10,000 divided by 6.5, which comes to 0.075 cm, or about 0.75 mm. At arm's length an observer can therefore resolve a depth difference smaller than a millimetre from disparity alone. The example makes concrete why stereopsis is prized for fine near work and why its precision falls off with distance: because disparity shrinks with the square of the viewing distance, the same threshold that resolves three quarters of a millimetre at one metre resolves only coarse depth steps across a room (Cumming & DeAngelis, 2001).

Discussion

Binocular vision is a compact example of the constructive character of perception. Wheatstone's stereoscope proved that depth can be built from two flat images that differ only in viewpoint (Wheatstone, 1838), and Julesz's random-dot stereograms proved that the construction runs on disparity alone, before any object is recognized, posing the correspondence problem that Marr and Poggio then solved as an explicit computation (Marr & Poggio, 1976; Marr & Poggio, 1979). The physiology supplied the machinery the computation implied: disparity-tuned neurons in the visual cortex (Barlow et al., 1967; Poggio & Fischer, 1977), a circuit that constructs their tuning (Ohzawa et al., 1990), and a hierarchy that refines the raw signal into perceived depth (Parker, 2007).

The unifying theme of current work is that the disparity measured by early neurons is not yet depth. Those neurons respond to physically impossible matches that observers never see in depth (Cumming & DeAngelis, 2001), and later stages must discard those false matches for the percept to be reliable (Goncalves & Welchman, 2017); the same interpretive stance appears in da Vinci stereopsis, where unmatched, half-occluded points are read as evidence about surface layout rather than discarded (Nakayama & Shimojo, 1990). That the whole system is learned and vulnerable is shown by rivalry, in which the brain visibly arbitrates between irreconcilable images (Blake & Logothetis, 2002), and by development, in which imbalanced early input costs an eye its stereopsis (Hubel & Wiesel, 1970; Levi et al., 2015) and cortical damage can remove binocular depth outright (Bridge, 2016). The distinction worth keeping sharp is that the two retinal images specify only a field of disparities, whereas the stable, single, three-dimensional scene an observer sees is an achievement of a visual system built to recover surfaces from two slightly different views (Read, 2021).

Common Misconceptions

Depth perception requires two eyes.

Two eyes are needed only for stereopsis, one depth cue among many; monocular cues such as perspective, occlusion, motion parallax, and shading let people with one eye judge depth well, and amblyopia removes stereopsis without abolishing depth perception (Levi et al., 2015).

Stereopsis works by the brain simply picking or blending the two eyes' images.

Depth is computed from the disparity between the images, a relation that survives in random-dot stereograms containing no monocular form at all, so it cannot come from choosing or averaging a recognizable picture (Marr & Poggio, 1976).

The disparity signalled by early visual neurons is the depth we see.

Neurons in the primary visual cortex respond to the physically impossible matches of anticorrelated stereograms, which observers do not see as depth, so their raw disparity signal must be refined by later stages before it becomes the percept (Cumming & DeAngelis, 2001).

Glossary

Amblyopia.

A developmental loss of acuity and stereopsis in a structurally normal eye, caused by imbalanced or misaligned binocular input during the critical period.

Binocular disparity.

The small difference in the retinal position of an object between the two eyes, arising from their horizontal separation; the signal from which stereopsis is computed.

Binocular rivalry.

The perceptual alternation that occurs when the two eyes view images too different to fuse, so that awareness switches between them rather than combining them.

Binocular summation.

The improvement in detection and discrimination obtained by combining the two eyes' signals, so that seeing with both eyes outperforms seeing with either alone.

Correspondence problem.

The task of determining which feature in one eye's image is the same scene point as a feature in the other eye's image, a prerequisite for measuring disparity.

Corresponding points.

Paired locations on the two retinas that map to the same visual direction and so, when stimulated by one object, yield a single fused image at zero disparity.

Crossed disparity.

The disparity produced by an object nearer than the fixation point, with the two retinal images displaced toward each eye's nose; a signal for near depth.

Cyclopean perception.

Perception of form defined by binocular disparity alone, as in a random-dot stereogram, where a shape is seen with no counterpart in either monocular image.

Diplopia.

Double vision; the perception of two images of a single object, occurring when its disparity exceeds the fusional range of Panum's area.

Fusion.

The combination of the two eyes' images into a single perceived object, achieved for disparities small enough to fall within Panum's fusional area.

Horopter.

The surface in space whose points project to corresponding retinal points at a given fixation and are therefore seen single and at zero disparity.

Panum's fusional area.

The narrow band of disparities around the horopter within which the two images are fused into one rather than seen as double.

Random-dot stereogram.

A stereo pair of random-dot fields, identical except for a region shifted in one image, in which a shape appears in depth only when the two are fused.

Stereoacuity.

The finest binocular disparity that yields a reliable depth judgement, reaching a few arc seconds in good observers, one of the sharpest of human perceptual thresholds.

Stereopsis.

The perception of depth and solidity derived from binocular disparity, the central achievement of binocular vision.

Strabismus.

A misalignment of the two eyes so that they do not point at the same target, a common cause of disrupted binocular development and amblyopia.

Uncrossed disparity.

The disparity produced by an object beyond the fixation point, with the retinal images displaced toward each eye's temporal side; a signal for far depth.

Vergence.

The opposed rotation of the two eyes that aims them at a common distance, changing the fixation point and thereby the reference from which disparity is measured.

Key Researchers

Horace Barlow (1921-2020). Vision neuroscientist at the University of Cambridge; with Blakemore and Pettigrew he showed that single cortical neurons are tuned to binocular disparity, giving stereoscopic depth a concrete neural detector and grounding the efficient-coding view of sensory processing. Google Scholar - Wikipedia - Wikidata

Randolph Blake. Vision scientist at Vanderbilt University; a leading authority on binocular rivalry, the perceptual alternation that arises when the two eyes view incompatible images, and on how the visual system resolves and suppresses conflicting monocular inputs. ORCID - Faculty Page - Google Scholar - Wikipedia - Wikidata

Béla Julesz (1928-2003). Psychologist and engineer at Bell Laboratories and Rutgers University; he invented the random-dot stereogram, proving that stereoscopic depth can be extracted from disparity alone before any monocular form is recognized, and thereby posing the correspondence problem in its sharpest form. Wikipedia - Wikidata

Ken Nakayama. Vision scientist and emeritus professor at Harvard University; he analysed how the visual system uses half-occluded, unpaired image points at object boundaries, describing da Vinci stereopsis and showing that binocular vision exploits the very regions one eye cannot see to recover depth order. Faculty Page - Google Scholar - Wikipedia - Wikidata

Andrew J. Parker. Physiologist at the University of Oxford; he traced binocular depth perception through the cerebral cortex, linking the responses of disparity-selective neurons to the perceptual judgements they support and clarifying how areas beyond the primary visual cortex build a stable representation of depth. ORCID - Faculty Page - Google Scholar - Wikidata

Andrew E. Welchman. Vision scientist at the University of Birmingham; he uses imaging and modelling to show how the brain reads binocular disparity, including evidence that dedicated cortical mechanisms discount the physically impossible anticorrelated matches to make depth estimates reliable. ORCID - Faculty Page - Google Scholar - Wikidata

Frequently Asked Questions

What is binocular vision?
It is the use of both eyes as a single perceptual system, whose horizontally separated viewpoints let the brain recover the depth and solidity of a scene through stereopsis, a capacity Wheatstone first demonstrated with the stereoscope (Wheatstone, 1838).

What is binocular disparity?
It is the small difference in the retinal position of an object between the two eyes, produced by their horizontal separation; its sign tells the brain whether the object is nearer or farther than the fixation point and its size scales the perceived depth (Read, 2021).

What is stereopsis?
Stereopsis is the perception of depth that the visual system computes from binocular disparity, and it can be experienced even in random-dot stereograms that contain no depth information in either image alone (Marr & Poggio, 1976).

What is the correspondence problem?
It is the problem of deciding which feature seen by one eye matches which feature seen by the other, which the brain solves using constraints such as the uniqueness and smoothness of surfaces before it can measure disparity (Marr & Poggio, 1976).

How does the brain encode depth from the two eyes?
Populations of disparity-selective neurons, first found in the visual cortex, each respond best to a particular disparity, so that near, zero, and far depths are represented across the neural population (Barlow et al., 1967).

What is binocular rivalry?
It is the alternation of awareness between two images too different for the eyes to fuse, so that one is seen and then the other while the physical stimulus stays constant, a much studied window onto visual awareness (Blake & Logothetis, 2002).

Why do some people lose stereopsis?
Imbalanced or misaligned input during a critical period in childhood, as in strabismus or unequal refractive error, leads the cortex to suppress one eye and produces amblyopia, which degrades or abolishes stereopsis even though the eye is healthy (Levi et al., 2015).

Are two eyes needed to see depth?
No; stereopsis is only one depth cue, and monocular cues such as perspective, occlusion, and motion parallax support good depth perception with a single eye, which is why losing stereopsis does not leave a person unable to judge depth (Levi et al., 2015).

References