Open Crossmodal perception Spence, 2011

Crossed senses.

You match a sound to one of two shapes, and a taste to a shape or a color. There is no right answer, yet most people give surprisingly similar answers to some of these questions. How closely do your matches resemble the crowd's and the research?

2 minto take part 1responses Anonymousno sign-up needed
which is sweet, which is sour?
EXP. 066

Is a high-pitched sound small or big? Is sweet round or angular? What color is sour? How much do we share the hidden matches between our senses?

Loading the experiment… It needs JavaScript to run; you can still read the science box and the article below.

Take part first.

Reading the science could sway your answer. Everything unlocks once you take part, but you can read it now if you prefer.

Back to the experiment
Science box

Our senses do not work in isolation. People agree to a surprising degree when matching the pitch of a sound to the size of a ball, or a taste to whether a shape is round or angular. Some of these matches appear very early in life; others vary with language and culture.

What we measure

We ask 15 matching questions across six dimensions. In four of them a sound plays: a high or low pure tone goes with a small or big ball, a light or dark disc, a thin or thick line; a soft or harsh timbre goes with a round or spiky shape. Two dimensions have no sound: sweet, sour and bitter each go with a shape; sweet, sour, salty and bitter each go with one of ten colors. For each dimension we compute how often your answers match the direction most commonly reported in research. If you cannot hear sound, there is a silent version with the taste questions only.

What the research says

In his tutorial review, Spence (2011) lists the matches between pitch and elevation, size and brightness among the most robust examples of crossmodal correspondences. Marks (1974) showed that adults without synaesthesia also match higher-pitched sounds with lighter shades and lower-pitched sounds with darker ones. Mondloch and Maurer (2004) showed that even 30–36-month-old children match a higher-pitched sound to a smaller, lighter ball. Adeli, Rouat and Molotchnikoff (2014) found in 119 people that soft timbres go with rounded shapes and harsh timbres with angular ones. In the study by Velasco and colleagues (2016), sweet clustered with round shapes, and salty, sour and bitter with angular shapes. According to the literature summary by Spence and colleagues (2015), sweet is most often matched with red and pink, sour with green and yellow, salty with white and blue, and bitter with black and purple; Woods and Spence (2016) largely replicated this pattern with single colour patches (sweet went most often with pink, sour with green, salty with white and bitter with black).

Why it happens

Spence (2011) discusses three kinds of explanation that are not mutually exclusive. Structural accounts say some matches come from how the senses are coded in the nervous system. Statistical accounts say we learn them from the world: big objects usually make lower sounds. Semantic accounts say shared words and concepts build a bridge: in Turkish a high-pitched sound is 'thin' (ince) and a low one 'thick' (kalın). Dolscheid and colleagues (2020) compared Turkish and Dutch speakers: both groups matched high pitch with thinness, but they differed on height, and when the two conflicted, Turkish speakers mostly chose by thickness.

Limitations

These matches are relative: the same sound counts as 'high' next to a lower one and 'low' next to a higher one; because we play a single sound per trial, your answer may partly depend on the sounds before it. The 'typical direction' is not a right answer, just the direction most often found in research, and cultures differ. Bremner and colleagues (2013) found that the Himba of Namibia make the bouba–kiki match like Westerners but go the opposite way on some taste–shape matches. Phone speakers can weaken low sounds; headphones give cleaner results.

reliable match30–36-month-olds matching a higher sound to a smaller, lighter ballMondloch and Maurer, 2004
strong matchSoft timbre ↔ round, harsh timbre ↔ angular shape (119 people)Adeli et al., 2014
two clustersSweet ↔ round; salty, sour, bitter ↔ angularVelasco et al., 2016
red-pink · green-yellow · white-blue · black-purpleMost common colors for sweet · sour · salty · bitterSpence et al., 2015
mostly thicknessTurkish speakers' choice when thickness and height conflictDolscheid et al., 2020
  1. Spence, C. (2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73(4), 971–995. View source ↗
  2. Mondloch, C. J., & Maurer, D. (2004). Do small white balls squeak? Pitch-object correspondences in young children. Cognitive, Affective, & Behavioral Neuroscience, 4(2), 133–136. View source ↗
  3. Marks, L. E. (1974). On associations of light and sound: The mediation of brightness, pitch, and loudness. The American Journal of Psychology, 87(1/2), 173–188. View source ↗
  4. Dolscheid, S., Shayan, S., Majid, A., & Casasanto, D. (2013). The thickness of musical pitch: Psychophysical evidence for linguistic relativity. Psychological Science, 24(5), 613–621. View source ↗
  5. Dolscheid, S., Çelik, S., Erkan, H., Küntay, A., & Majid, A. (2020). Space-pitch associations differ in their susceptibility to language. Cognition, 196, 104073. View source ↗
  6. Adeli, M., Rouat, J., & Molotchnikoff, S. (2014). Audiovisual correspondence between musical timbre and visual shapes. Frontiers in Human Neuroscience, 8, 352. View source ↗
  7. Velasco, C., Woods, A. T., Marks, L. E., Cheok, A. D., & Spence, C. (2016). The semantic basis of taste-shape associations. PeerJ, 4, e1644. View source ↗
  8. Spence, C., Wan, X., Woods, A., Velasco, C., Deng, J., Youssef, J., & Deroy, O. (2015). On tasty colours and colourful tastes? Assessing, explaining, and utilizing crossmodal correspondences between colours and basic tastes. Flavour, 4, 23. View source ↗
  9. Woods, A. T., & Spence, C. (2016). Using single colors and color pairs to communicate basic tastes. i-Perception, 7(4), 1–15. View source ↗
  10. Bremner, A. J., Caparos, S., Davidoff, J., de Fockert, J., Linnell, K. J., & Spence, C. (2013). “Bouba” and “Kiki” in Namibia? A remote culture make similar shape–sound matches, but different shape–taste matches to Westerners. Cognition, 126(2), 165–172. View source ↗

High sound, small ball

If someone showed you two balls, one small and one big, played a high-pitched sound and asked 'Which one is making this sound?', what would you say? Most people point to the small ball. When a lower sound plays, they point to the big one. Matches like this, between features belonging to different senses and largely shared across people, are called crossmodal correspondences.

Charles Spence's extensive review, published in 2011, is one of the field's key references. Spence lists the matches between the pitch of a sound and visual elevation, size and brightness among the most robust examples. These matches do not show up only in questionnaire answers; responses to congruent pairs can be faster than to incongruent ones.

Do toddlers also say 'high sound, small ball'?

In 2004 Catherine Mondloch and Daphne Maurer showed 30–36-month-old children two bouncing balls and asked which ball a centrally played sound belonged to. The children reliably matched the higher-pitched sound to the smaller, lighter ball. The researchers took this as a hint that some correspondences may be traces of connections present before learning.

But not every correspondence is that early or that universal. Some are learned by observing the world: big animals, big drums and big objects usually sound lower. Others draw on concepts carried by language.

Why is pitch 'thin' and 'thick' in Turkish?

In Turkish a high-pitched sound is called 'thin' (ince) and a low-pitched one 'thick' (kalın). In languages such as Dutch and English, pitch is 'high' and 'low'. In 2013 Sarah Dolscheid and colleagues compared Dutch and Farsi speakers; Farsi, like Turkish, uses thickness words. They found that the two groups behaved differently on simple, non-linguistic pitch tasks, and that once Dutch speakers had been trained to describe pitch with thickness words, they came to resemble Farsi speakers.

In 2020 Dolscheid, Çelik, Erkan, Küntay and Majid tested Turkish and Dutch speakers. Both groups matched high pitch with a thin line, but they differed on height. When a thick line was high and a thin line low, Turkish speakers mostly chose by thickness, while Dutch speakers more often chose by height. That makes the thin–thick question in this experiment especially interesting; in the crowd results we show players on the Turkish and English pages separately.

Harsh timbre, spiky shape

In the famous bouba–kiki experiment (the 'Bouba or kiki?' experiment in this lab), people match made-up words to shapes. Here there are no words: two sounds play at the same pitch, one with a soft, rounded timbre and the other buzzy and harsh. In a 2014 study with 119 people, Mohammad Adeli and colleagues found that soft timbres were strongly matched with rounded shapes and harsh timbres with angular ones.

The same study contains an interesting detail: the fundamental frequency of the sounds, that is, their pitch, was not related to the vertical position participants chose. The match between pitch and elevation has been found to be strong in many studies, but it does not appear with every stimulus and task.

The shape of sweet, the color of sour

Crossmodal correspondences also involve tastes. In the 2016 study by Carlos Velasco and colleagues, participants' judgments of taste words and shapes fell into two clusters: round shapes and 'sweet' on one side; angular shapes with 'salty', 'sour' and 'bitter' on the other. The round cluster was rated more positive, the angular cluster more 'potent'.

Colors follow a similar pattern. According to the 2015 review by Spence and colleagues, sweet goes most often with red and pink, sour with green and yellow, salty with white and blue, and bitter with black and purple. Woods and Spence largely replicated this in 2016: sweet was matched most often with pink, sour with green, salty with white and bitter with black. In our colour questions we name each taste without examples; saying 'like lemon' would suggest yellow, and 'like coffee' brown. Packaging designers are interested in these matches too: the color and shape of a package can shape what we expect its contents to taste like.

Does everyone make the same matches?

No. The field increasingly takes cultural differences seriously. In 2013 Andrew Bremner and colleagues worked with the Himba community in northern Namibia. Himba participants made the bouba–kiki match like Westerners, but they did not match carbonated water to an angular shape as Westerners do, and they tended to match less bitter milk chocolate to angular rather than rounded shapes, the opposite of Westerners.

So the 'typical direction' on the result screen is not a right answer, just the direction most often reported in research. Some of your matches may differ from the crowd; that does not mean your senses are working wrongly, only that they carry traces of your experience, your language and your preferences.

FAQ

What is a crossmodal correspondence?

A match between features of different senses that people largely share: matching high sounds with small, light objects, or sweet with round shapes. It differs from synesthesia; most people show it, and it is a tendency rather than an automatic experience.

Is this synesthesia?

No. In synesthesia one sense automatically triggers an experience in another, for example a letter always appearing in a certain color; that is much rarer. With crossmodal correspondences you do not 'see' anything; you simply find some matches more fitting.

Why do you compare the Turkish and English pages separately?

In Turkish a high sound is 'thin' and a low sound 'thick'. Dolscheid and colleagues (2020) found that Turkish speakers give particular weight to the thickness match. The page language does not show for certain which language someone speaks, but it offers an interesting first comparison.

Can I play without sound?

Yes. If you choose 'I can't hear it' in the sound check, you can switch to the silent version with the taste questions only. Your result will then show the taste dimensions only.

What if my answers are not typical?

Nothing; there is no right answer here. These matches are a matter of probability and vary between cultures and people. The result screen just shows how closely your matches resemble the crowd's and the research.

Discussion 0 comments

Join the discussionReading is open to everyone. Volunteer to comment and vote; it takes 20 seconds.
No comments yet.Be the first to write; a good question gets the discussion going.

Threads about this experiment

Start a new thread →
There’s no separate thread for this experiment yet. Start one to critique the method, share a paper or ask a new question.