Pitch Perception

Abstract

Pitch, a dimension of auditory perception, is the perceptual attribute that orders sounds from low to high, the dimension on which melody and intonation are built. For a periodic sound it corresponds to the repetition rate of the waveform, the fundamental frequency, yet the percept is not read off the energy at that frequency: a tone whose fundamental is removed is still heard at its pitch. This article follows pitch from the two codes the ear supplies, place of excitation along the basilar membrane and temporal periodicity of phase-locked firing, through the pattern-recognition and autocorrelation models built to explain the missing fundamental, to the mel scale and the two-dimensional structure of chroma and height. It then turns to absolute and relative pitch, the neural coding of pitch, and congenital amusia. Three interactive demonstrations model the missing fundamental, the mel scale, and pitch chroma.

Keywords: pitch perception, missing fundamental, temporal coding, mel scale, congenital amusia

Pitch is the most musically consequential of the perceptual attributes of sound. It is what allows a listener to say that one note is higher than another, to recognize a melody transposed to a new key, and to hear the rising contour of a question in speech. For a periodic sound its physical correlate is straightforward: the auditory system hears the pitch of the fundamental frequency, the rate at which the waveform repeats. The deep problem of pitch is that this correlate is not the whole story, because the same pitch is heard when the fundamental is physically absent, so the percept must be computed from the pattern of the sound rather than extracted from a single frequency (Oxenham, 2012). How the brain performs that computation, and from which of the two frequency codes it draws, is the central question of pitch perception.

What Pitch Is

Pitch is defined by the American National Standards Institute as the attribute of auditory sensation by which sounds may be ordered on a scale from low to high, a definition that is deliberately perceptual: pitch is a property of the percept, not of the sound wave, and it is measured by what a listener hears rather than by any single number in the signal (Plack et al., 2005). For most sounds that carry a clear pitch the physical correlate is nonetheless simple. The sound is periodic, repeating at some rate, and the pitch corresponds to that repetition rate, the fundamental frequency, expressed in hertz. A sound repeating 220 times a second is heard at the pitch of the note A below middle C; one repeating twice as fast, at 440 hertz, is heard an octave higher (Moore, 2012).

What makes pitch a problem rather than a measurement is that the mapping from repetition rate to percept is not a simple read-out of energy at the fundamental frequency. A complex periodic tone, such as a vowel or a note from an instrument, contains energy at the fundamental and at a series of harmonics, integer multiples of it. The pitch corresponds to the fundamental, but as the demonstrations below make concrete, that pitch persists even when the fundamental itself contains no energy, because the information that specifies it is carried redundantly across the harmonics (Oxenham, 2012). Pitch, then, is a computed property: the auditory system infers the repetition rate of the whole pattern from whatever partial evidence the harmonics provide.

Place and Temporal Coding

The computation begins with the two codes the cochlea supplies. Sound entering the inner ear sets up a travelling wave along the basilar membrane, whose graded stiffness makes each place respond best to a particular frequency, high frequencies at the base and low at the apex. This gives a place code: frequency is mapped onto a position of maximal excitation, and the lower harmonics of a complex tone, being widely spaced relative to the width of the cochlear filters, fall on separate places and are said to be resolved (Moore, 2012). A place theory of pitch derives the fundamental from the spacing of these resolved harmonics along the membrane.

Alongside place, the ear preserves fine timing. Auditory nerve fibres tend to fire at a consistent phase of the stimulating waveform, a property called phase locking that holds up to a few kilohertz, so the pattern of spikes carries the periodicity of the sound directly. This gives a temporal code, and a temporal theory of pitch derives the fundamental from the periodicity in the firing pattern, which repeats at the fundamental rate whether or not any energy is present there (Oxenham, 2018). The two codes are not rivals so much as complementary, and the difficulty of choosing between them is itself telling: pitch is most accurate and most salient for the low harmonics, which are at once the best resolved along the membrane and the best preserved in the timing of firing, so the two codes are confounded precisely where pitch is strongest. The cleanest dissociation lies at the extremes, the dominant, most musically useful pitch arising from the low, resolved harmonics that place coding handles best, while a weaker pitch still persists for tones built only from high, unresolved harmonics, which only timing can explain (Moore, 2012; Oxenham, 2018). Figure 1 sets the two codes side by side.

The Missing Fundamental

The single fact that most constrains any theory of pitch is that the fundamental need not be present for its pitch to be heard. When a complex tone is high-pass filtered so that the fundamental and the lowest harmonics are removed, leaving only the upper harmonics, the pitch does not rise to the lowest remaining component; it stays at the original fundamental, now physically absent from the sound (Schouten et al., 1962). The Dutch physicist Jan Schouten named this residual percept the residue pitch and used it to argue that pitch is extracted from the fine temporal structure of the unresolved upper harmonics rather than from energy at the fundamental. The phenomenon is not a laboratory curiosity: a telephone transmits almost nothing below about 300 hertz, yet a male voice with a fundamental near 120 hertz is heard at its true low pitch, because the harmonics that survive the channel still specify it.

That the missing fundamental is genuinely a central, pattern-based computation rather than a by-product of the two harmonics beating together in the cochlea was shown by presenting the harmonics separately to the two ears. When one harmonic is delivered to the left ear and the next to the right, so that they can never interact mechanically on a single basilar membrane, listeners still hear the low pitch of the fundamental they jointly imply, which means the fundamental is reconstructed centrally, after the two ears' information has been combined (Houtsma & Goldstein, 1972). The first demonstration builds a harmonic complex and lets the reader remove components, showing that the perceived pitch tracks the common spacing of whatever harmonics remain, and holds steady when the fundamental itself is switched off.

Pattern-Recognition and Autocorrelation Models

The missing fundamental forces the question of how the auditory system recovers a frequency that is not there, and two families of model answer it, corresponding to the two peripheral codes. Pattern-recognition, or spectral, models begin from the resolved harmonics that place coding makes available. Because those harmonics are a consecutive set of integer multiples of the fundamental, their frequencies specify it: the system finds the fundamental whose harmonic series best matches the observed set of partials, in effect fitting a harmonic template to the spectrum. Goldstein cast this as an optimum processor that estimates the fundamental most consistent with noisy measurements of the harmonic frequencies (Goldstein, 1973), and Terhardt developed a closely related account in which the pitch of a complex tone is a learned virtual pitch, computed from the subharmonic coincidences of the resolved partials and central to his theory of consonance and harmony (Terhardt, 1974).

Temporal, or autocorrelation, models begin instead from the timing of nerve firing. Licklider proposed that the auditory system performs a running autocorrelation on the phase-locked spike train, detecting the time interval at which the pattern most repeats itself, an interval that equals the period of the fundamental whether or not energy is present there (Licklider, 1951). Meddis and Hewitt implemented this idea as a computational model of the auditory periphery whose summary autocorrelation predicts the pitch and the relative salience of a wide range of complex tones, including missing-fundamental stimuli (Meddis & Hewitt, 1991). The two families are not mutually exclusive, and each fits part of the evidence: the finding that the pitch of a complex tone is carried disproportionately by a few low harmonics, roughly the third to the fifth, the region of dominance, favours spectral models, while the persistence of a pitch for unresolved stimuli favours temporal ones (Moore, 2012; Plack et al., 2005). Table 1 sets out the principal models and the code each exploits.

Scaling Pitch: The Mel Scale

Separate from the question of how pitch is extracted is the question of how the resulting sensation is scaled: by how much does perceived pitch grow as frequency rises? Frequency and pitch are not proportional. Stevens, Volkmann, and Newman had listeners adjust one tone until its pitch seemed half or double that of another, and from these ratio judgements built the mel scale, on which equal intervals correspond to equal perceived distances in pitch, anchored so that a 1000-hertz tone is 1000 mels (Stevens et al., 1937). The scale is compressive: doubling frequency raises pitch by much less than a factor of two at high frequencies, so a listener asked to set a tone an octave above another in sensation does not simply double its frequency. Attneave and Olson later extended the ratio-scaling approach and argued that the perceptually relevant scaling for musical pitch is closer to a logarithmic one, which is what the equal-tempered musical scale already assumes (Attneave & Olson, 1971).

The mel scale matters beyond the psychophysics laboratory: its compressive spacing of frequency is built into the mel-frequency filterbanks that front-end most automatic speech-recognition systems, an engineering acknowledgement that the ear's resolution of pitch is finer at low frequencies than at high. The second demonstration plots the mel scale directly, letting the reader move a frequency and read off its value in mels, and see how far above a note its true octave lies on the sensation scale.

Chroma and Height

Pitch is not a simple line from low to high but has a two-dimensional structure, a fact forced by the special status of the octave. Two tones an octave apart, one at twice the frequency of the other, sound both different, one is clearly higher, and yet somehow the same, sharing a quality that leads every musical tradition to give them the same note name. The standard resolution separates two components: height, the monotonic sense of low-to-high that rises continuously with frequency, and chroma, the circular quality that repeats every octave, so that all the A's share a chroma however far apart in height they lie (Plack et al., 2005). Pitch is thus better pictured as a helix than a line, chroma running round the turns and height climbing the axis.

The clearest evidence that chroma is a genuine perceptual dimension is Shepard's demonstration that pitch can be made to seem endlessly rising. By constructing tones that contain energy only at octave-spaced frequencies, with a fixed spectral envelope, Shepard produced stimuli whose chroma can be stepped round the circle indefinitely while their height stays fixed, so a sequence sounds as though it climbs forever without ever getting higher (Shepard, 1964). The illusion works precisely because chroma and height have been decoupled: the ear tracks the circular chroma round its cycle and, deprived of any consistent height cue, never registers the return to the start. The third demonstration maps a frequency to its musical note, separating the note's chroma, its pitch class, from its octave, and shows that tones an octave apart share a chroma while differing in height.

Absolute and Relative Pitch

Almost all listeners perceive pitch relatively: they recognize a melody by the pattern of intervals between its notes, which is why a tune is instantly identifiable when transposed to a new key, every absolute frequency changed but every interval preserved. Relative pitch is the normal currency of music perception, and it is what lets music cognition treat a melody as a shape rather than a list of frequencies (McDermott & Oxenham, 2008). A minority of listeners additionally possess absolute pitch, the ability to name or produce a specific note with no reference tone, hearing a pitch as a category, F sharp, the way most people hear a colour as a category, red.

Absolute pitch is rare, estimated at well under one percent of the population, and its distribution is informative. It is strongly associated with early musical training, is far more common among speakers of tonal languages, in which pitch carries lexical meaning from infancy, and is very difficult to acquire in adulthood, a profile that points to an early sensitive period for binding pitch to a verbal label (McDermott & Oxenham, 2008). Absolute and relative pitch are not simply more and less of the same skill: many musicians with excellent relative pitch have no absolute pitch at all, and the dissociation suggests that naming a pitch in isolation and computing the relation between two pitches are partly separate abilities.

The Neural Coding of Pitch

The behavioural codes have direct physiological counterparts. In the auditory nerve, the periodicity that temporal models require is plainly present: recordings from single fibres show that the pattern of spike intervals carries the periodicity of a complex tone, and a population measure of the most common interspike interval predicts both the pitch a human hears and its salience, including for missing-fundamental stimuli (Cariani & Delgutte, 1996). At the level of the nerve, then, pitch is legible in the timing of firing, exactly as the autocorrelation models suppose.

Whether that peripheral periodicity is converted into an explicit place code for pitch in the cortex was long uncertain, and a key piece of evidence came from the marmoset. Recording near the low-frequency border of primary auditory cortex, Bendor and Wang found neurons that respond to a given pitch whether it is carried by a pure tone at the fundamental or by a complex tone whose fundamental is missing, that is, neurons tuned to pitch itself rather than to spectral content (Bendor & Wang, 2005). Such pitch-selective neurons are the cortical read-out the models had implied: a small region in which the abstract attribute of pitch, however it is carried in the acoustics, is made explicit. The broader picture across the pathway is one of convergence, the peripheral timing and place codes feeding a cortical representation in which pitch has become a property in its own right (Oxenham, 2018).

Congenital Amusia

That pitch perception can fail selectively, sparing the rest of hearing and cognition, is shown by congenital amusia, a lifelong condition in which a person of normal intelligence, hearing, and language cannot perceive or produce music normally. The core deficit is one of pitch: amusic listeners are markedly impaired at detecting small pitch changes and pitch direction, fail to notice out-of-key or out-of-tune notes that others hear at once, and often cannot tell whether a pitch has gone up or down, despite normal detection of the same frequency differences in non-musical judgements (Peretz & Hyde, 2003). The condition is not a failure of exposure or training but a developmental anomaly, running in families and present from birth, which is why it is called congenital to distinguish it from the acquired amusia that can follow brain damage. The deficit has a neural signature: functional imaging shows that in amusic listeners the right-lateralised pathway linking auditory cortex to the inferior frontal gyrus, which normally supports fine pitch processing and the retention of pitch in memory, responds and connects abnormally, locating the disorder in an atypical fronto-temporal network rather than in the ear (Hyde et al., 2011).

Amusia is theoretically valuable because it dissociates pitch from the faculties it normally travels with. Amusic individuals have normal speech perception even though speech carries pitch in its intonation, which suggests that the fine-grained pitch processing music demands is partly separable from the coarser pitch changes that signal a question or mark emphasis in a sentence (Peretz & Hyde, 2003). The disorder thus supports a view on which pitch is not a single undifferentiated ability but a set of processes, some shared with speech and some specific to the demanding, categorical use of pitch in music, that can be selectively disrupted.

Worked Example

The three demonstrations can be checked against their governing formulas. The first builds a harmonic complex on a fundamental of 200 hertz, so its components lie at 200, 400, 600, 800, 1000, and 1200 hertz. The perceived pitch equals the repetition rate of the whole pattern, which is the greatest common divisor of the components present. With all six present the divisor is 200 hertz. Removing the 200-hertz fundamental leaves 400, 600, 800, 1000, and 1200, whose greatest common divisor is still 200, so the perceived pitch is unchanged at 200 hertz: the missing fundamental. The pitch shifts only when the remaining components no longer share 200 as a divisor; keeping just the 800- and 1200-hertz components, for instance, gives a greatest common divisor of 400 hertz, and the pitch jumps up. The demonstration computes the divisor of whatever harmonics are left in and confirms it holds at 200 across the removal of the fundamental.

The second demonstration plots the mel scale, mel equals 2595 times the base-ten logarithm of the quantity one plus the frequency divided by 700. At 1000 hertz this returns 2595 times the logarithm of 2.4286, which is 2595 times 0.3853, or almost exactly 1000 mels, the scale's anchor point. At 700 hertz it gives 2595 times the logarithm of 2, which is 2595 times 0.3010, or 781 mels; at 500 hertz, 607 mels; and at 2000 hertz, 1521 mels. The compression is the point: doubling the frequency from 1000 to 2000 hertz raises the pitch sensation only from 1000 to 1521 mels, a factor of about 1.5 rather than 2, so a tone an octave higher in frequency is far less than double in mels. The third demonstration maps a frequency to a musical note by the standard formula, the note number equals 69 plus 12 times the base-two logarithm of the frequency divided by 440, with the chroma given by that number modulo 12. At 440 hertz the number is 69, a chroma of 9, the note A. At 220 hertz it is 57, and at 880 hertz it is 81; both reduce modulo 12 to a chroma of 9, so all three are the note A, differing only in octave. This is octave equivalence made arithmetic: multiplying or dividing the frequency by two adds or subtracts 12 from the note number, leaving the chroma, and so the note name, unchanged.

Discussion

Pitch perception is best understood as the inference of a single perceptual attribute, the repetition rate of a periodic sound, from partial and redundant physical evidence. The auditory system approaches the inference with two codes laid down at the cochlea, the place of excitation among the resolved harmonics and the periodicity of phase-locked firing, and the long contest between place and temporal theories has resolved not into a winner but into a division of labour, each code dominating the stimulus regime it is suited to (Oxenham, 2012; Moore, 2012). The missing fundamental, which first made the problem visible, remains the pivot: because it shows pitch surviving the removal of its own frequency, it rules out any theory that reads pitch off energy at the fundamental and demands instead the pattern-recognition and autocorrelation models that compute it from the harmonics or the timing (Schouten et al., 1962; Goldstein, 1973; Meddis & Hewitt, 1991).

Above the extraction of pitch sits its structure and its use. The mel scale captures how the sensation is spaced against frequency, and the decomposition into chroma and height captures the octave-based geometry that music exploits, a geometry laid bare by tones that seem to rise forever (Stevens et al., 1937; Shepard, 1964). The physiology has caught up with the psychophysics: periodicity is explicit in the timing of the auditory nerve, and pitch-selective neurons in cortex make the abstract attribute explicit in a place code, the convergence the models anticipated (Cariani & Delgutte, 1996; Bendor & Wang, 2005). That pitch is a distinct and partly modular achievement is underscored from two directions, by the rarity and early-fixed character of absolute pitch and by congenital amusia, in which pitch fails while hearing and language stand (McDermott & Oxenham, 2008; Peretz & Hyde, 2003). The open questions concern how the two codes are combined and where in the pathway pitch becomes explicit, not whether pitch is a computed attribute, which the missing fundamental settled long ago.

Common Misconceptions

Pitch is just another word for frequency.

Frequency is a physical property of the sound wave; pitch is the perceptual attribute the brain computes from it, and the two come apart. A tone with its fundamental frequency removed keeps the pitch of that fundamental, and equal steps in frequency are not equal steps in pitch, which is why perceived pitch needs its own scale (Schouten et al., 1962; Stevens et al., 1937).

A sound must contain its fundamental frequency to be heard at that pitch.

It need not. When the fundamental is filtered out, the pitch stays put, because the remaining harmonics still specify it by their common spacing and by the repetition rate of the waveform. The belief persists because in most everyday sounds the fundamental is present, so its removal is rarely noticed (Schouten et al., 1962; Oxenham, 2012).

Absolute pitch is the ultimate musical ear, and relative pitch is a lesser version of it.

They are partly separate abilities, not two points on one scale. Music perception runs almost entirely on relative pitch, the pattern of intervals, which is why melodies survive transposition; many fine musicians have superb relative pitch and no absolute pitch. Absolute pitch, naming a note in isolation, is rare and tied to an early sensitive period, and it is not required for musical skill (McDermott & Oxenham, 2008).

Glossary

Absolute pitch.

The rare ability to name or produce a specific musical note without a reference tone, hearing pitch as a fixed category; strongly tied to early musical training.

Autocorrelation model.

A temporal theory of pitch that finds the fundamental by detecting the time interval at which the phase-locked firing pattern most repeats itself.

Basilar membrane.

The structure within the cochlea whose graded stiffness makes each place respond best to a particular frequency, producing the place code for pitch.

Chroma.

The circular quality of pitch that repeats every octave, so that all tones sharing a note name share a chroma however far apart in height they lie.

Congenital amusia.

A lifelong developmental disorder of music perception, centred on a deficit of fine pitch discrimination and direction, in a person of normal hearing, language, and intelligence.

Fundamental frequency.

The repetition rate of a periodic sound, the lowest frequency in its harmonic series and the physical correlate of its pitch.

Harmonic.

A frequency component of a complex tone at an integer multiple of the fundamental; the lower harmonics are resolved by the cochlea, the higher ones unresolved.

Height.

The monotonic dimension of pitch that rises continuously with frequency, the axis of the pitch helix along which chroma cycles.

Mel scale.

A perceptual scale of pitch on which equal distances correspond to equal changes in sensation, anchored so that 1000 hertz equals 1000 mels; compressive relative to frequency.

Missing fundamental.

The phenomenon in which a complex tone whose fundamental frequency has been removed is still heard at the pitch of that fundamental, inferred from the surviving harmonics.

Phase locking.

The tendency of auditory nerve fibres to fire at a consistent phase of the sound waveform, providing the temporal code for pitch up to a few kilohertz.

Pitch.

The perceptual attribute by which sounds are ordered from low to high; for a periodic sound it corresponds to the repetition rate of the waveform.

Place code.

The representation of frequency by which part of the basilar membrane is most excited, from which pattern-recognition models derive the fundamental via the spacing of resolved harmonics.

Relative pitch.

The perception of pitch as a pattern of intervals between notes rather than as absolute values, the basis on which a melody is recognized when transposed.

Residue pitch.

Schouten's term for the pitch of the missing fundamental, argued to be extracted from the fine temporal structure of the unresolved upper harmonics.

Resolved harmonic.

A lower harmonic separated from its neighbours by the cochlear filters so that it falls on a distinct place; the resolved harmonics carry the most salient pitch.

Virtual pitch.

Terhardt's term for the learned pitch of a complex tone, computed from subharmonic coincidences of the resolved partials rather than from energy at the fundamental.

Key Researchers

Ray Meddis (1944-2018). Emeritus Professor of Psychology at the University of Essex; he built an influential computational model of the auditory periphery whose autocorrelation of simulated nerve firing extracts the pitch of complex tones, giving the temporal theory a concrete mechanism. Faculty Tribute - Google Scholar - Wikipedia

Brian C. J. Moore. Emeritus Professor of Auditory Perception at the University of Cambridge; his work on pitch, frequency selectivity, and masking established much of what is known about how the ear resolves the harmonics that carry the pitch of complex tones. Faculty Page - Google Scholar - Wikipedia - ORCID

Andrew J. Oxenham. Distinguished McKnight University Professor at the University of Minnesota; he studies the perceptual and neural coding of pitch, showing how the auditory system extracts pitch from resolved harmonics and temporal fine structure. Faculty Page - Google Scholar - ORCID

Roy D. Patterson. Auditory scientist at the University of Cambridge; he developed the auditory image model, a time-interval representation of neural firing from which the pitch and timbre of complex sounds can be read. Faculty Page - Google Scholar - Wikipedia

Isabelle Peretz. Professor of Psychology at the Université de Montréal and co-director of the BRAMS laboratory; she established congenital amusia as a specific disorder of pitch and mapped its perceptual and neural basis. Faculty Page - Google Scholar - Wikipedia - ORCID

Christopher J. Plack. Ellis Llwyd Jones Professor of Audiology at the University of Manchester; his experimental and modelling work on pitch, cochlear compression, and temporal fine structure shaped the modern understanding of pitch mechanisms, and he is senior editor of the field-defining volume on pitch. Faculty Page - Google Scholar - ORCID

Xiaoqin Wang. Professor of Biomedical Engineering at Johns Hopkins University; he identified pitch-selective neurons at the low-frequency border of primate auditory cortex, giving direct evidence for a cortical representation of pitch, including the missing fundamental. Faculty Page - Google Scholar - ORCID

Frequently Asked Questions

What is pitch in perception?
Pitch is the perceptual attribute by which sounds are ordered from low to high, the dimension that carries melody and intonation; for a periodic sound it corresponds to the repetition rate of the waveform, the fundamental frequency (Plack et al., 2005).

How is pitch different from frequency?
Frequency is a physical property of the sound wave, while pitch is the sensation the brain computes from it; the two diverge, since a tone can be heard at the pitch of a fundamental frequency it does not physically contain (Oxenham, 2012).

What is the missing fundamental?
It is the phenomenon in which a complex tone with its fundamental frequency filtered out is still heard at the pitch of that fundamental, because the surviving harmonics specify it by their common spacing and repetition rate (Schouten et al., 1962).

How does the ear work out the pitch of a complex tone?
Through two routes: pattern-recognition models infer the fundamental from the spacing of the resolved lower harmonics, while autocorrelation models read it from the periodicity in the timing of nerve firing (Goldstein, 1973).

Why is a note an octave up called the same note?
Because pitch has two dimensions, a rising height and a circular chroma that repeats every octave; tones an octave apart share a chroma, and so a note name, while differing in height (Plack et al., 2005).

What is absolute pitch?
It is the rare ability to name or produce a specific note with no reference tone; it is associated with early musical training and tonal-language background, and is distinct from the relative pitch that underlies ordinary music perception (McDermott & Oxenham, 2008).

Is pitch represented in the brain?
Yes; periodicity is explicit in the timing of auditory nerve firing, and neurons near the low-frequency border of auditory cortex respond to a pitch whether or not its fundamental is present, a cortical read-out of pitch itself (Bendor & Wang, 2005).

What is congenital amusia?
It is a lifelong developmental disorder, centred on impaired pitch discrimination and direction, in a person of otherwise normal hearing, language, and intelligence, showing that pitch perception can fail selectively (Peretz & Hyde, 2003).

References