Read

my

lips

The McGurk effect

... or why dubbing films should be banned

This is absolutely no joke. Without normally being aware of it, we practise lip-reading in normal conversation. This was proven by Harry McGurk and John MacDonald at the University of Surrey back in 1976 (Nature 264, 746--748). Confronting listeners of different age groups either with a recorded voice only, or with the same voice together with a video showing non-matching lip movements, and asking the subjects to repeat what was said, they found that up to 92 % of the answers were wrong when the conflicting visual input was present, while nearly all of them were correct when it wasn't. For instance, subjects hearing "ba- ba" but seeing a lip movement that corresponds to "ga-ga", reported they heard "da-da", indicating that the perceived speech is a fused signal of the auditory and the visual input.

The McGurk effect, as this phenomenon came to be called, has been quoted more than 80 times in scientific journals ranging from Advanced Robotics to Science. Follow-up studies have addressed all kinds of specific cases, including audiovisual cross-dressing ("... female faces and male voices in the McGurk effect"), intercultural comparisons ("Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects"), and speakers standing on their heads ("Perceiving speech from inverted faces" -- my favourite title in the list). However, a large scale experiment on this effect has been going on in many countries virtually unnoticed -- to my knowledge no-one has even bothered to collect the results. Exposure of millions of individuals to McGurk style experimental conditions is going on in all those countries where foreign films are routinely shown in dubbed versions, as is the practice in Germany and France for instance, but neither in the Netherlands nor in Britain. Amazingly, most German spectators do not find it irritating that the lip movement does not match the sound, suggesting that the McGurk effect can be switched off, if a year-long habit tells the brain that the lip-signals in such cases are nonsensical. In contrast, when found in a situation where the dubbing goes against the habit, e.g. when seeing a German actor dubbed into French, German viewers would strongly object against the "wrong voice".

From my own experience of moving from a dubbing to a non-dubbing country (Germany to Britain), I know that the unconscious lipreading inactivated by habitual viewing of dubbed films comes back after only a few months in a non dubbing country, even with only a moderate exposure to cinema and TV. Nowadays, seeing dubbed material is quite enough to drive me up the wall, no matter from which language into which other provided I know at least one of the two. For instance, watching a Swedish film (with actors I did not know, so I had no preconception as to what language or what voice they should have) dubbed into German was absolutely unbearable, exclusively because of the outrageous mismatch of lip movements. (I do not understand a single word of Swedish, so I didn't mourn the loss of the original language as such in this case.) Other traumatic experiences include the German version of the French film "Cyrano de Bergerac". The story is -- as you may remember -- about a guy with a very long nose winning a woman's heart by the combined use of beautiful language and a man of straw shorter in both nasal protrusion and linguistic ability. The dialogue, which is indeed stunning in the original, turned into unbearably bad verse in the German version, thus making the whole story appear ridiculous.

Even with languages of which I have very little knowledge, such as Italian or Dutch, I feel that something comes across through the spoken language even while I am reading the subtitles -- the Dutch movie "Antonia's line" released last year in the UK was a perfect example for this. I found the original language so poetic I stopped reading the subtitles, relying only on the similarity with German to catch the contents of the dialogue. Similarly, I enjoyed learning some basic Italian from "Il Postino", although it was irritating to see Philippe Noiret dubbed from French into Italian. (Some of the recent films mixed from half a dozen different countries like cheap EU wine, don't actually seem to have such a thing as an undubbed original version.)

Considering, however, that most of the European dubbing business goes on from English into German (and French, Spanish, etc.), while every German moviegoer knows at least as much English as some of Hollywood's highest earning action movie protagonists, and certainly more than I know Dutch, I don't see any point in this whole business. Consumers certainly lose part of the natural audiovisual experience, good movies can be disfigured beyond recognition, and even bad films cannot win. (German TV consumers tend to argue at this point that Starsky and Hutch are more funny in German than in the original version, which I haven't checked. Even if this is true, it is still the exception which proves the rule.)

You may at this point wonder what I am talking about -- if you only know one side of this cultural divide. If you live in a non-dubbing country and have never watched a dubbed film, try getting a Swedish childrens's video like, for instance, "Pippi Longstockings". These should be dubbed in most countries, and the close-ups of children speaking rather loud and with very expressive faces provide a good introduction into the phenomenon. In contrast, if you live in a dubbing country and have never realised that the wrong voices are coming out of your TV set, spend your next holiday in some country where films are dubbed into a different language (France or Spain would do nicely) and watch some ordinary American films with the "wrong" language. Then consider that the voices are exactly as unmatched as the ones emerging from your TV set at home.

In either direction, you will be surprised -- I certainly was. On the scientific side, of course, this mega-McGurk experiment carried out with millions of unaware human guinea pigs provides a unique chance. Experimental psychologists should investigate how viewers manage to switch off the lip-reading without even being aware of what they are missing. For cultural reasons, however, I should like to call for a ban on dubbing, or -- seeing the lack of a suitable authority to enforce this -- for a boycott of dubbed versions.

contact

prose and passion

last update:

24.06.2003