OK, so I should call them 'speech-patterns'... to include the throat? For the time being: mouth-patterns.
For LipRead, I tried to make comprehensive lists of the phonemes and visemes of the project languages: English, German, Dutch, Norwegian and Turkish. For phonemes, that was relatively easy, because I could use Jacques Koreman's L1L2 Map: http://calst.hf.ntnu.no/.
Of course, the L1L2 map doesn't show 'allophones', the different variations of each phoneme, but it's a start.
For visemes, it was much more difficult. Pretty much everyone seems to make his/her own set of visemes for a language. Some based on research, some on articulatory features, some on ... I don't know. There is no 'standardized' set of visemes, within or across languages.
Charlotta Engström looked at Swedish visemes for her Master Degree Project in Speech Communication. One of her conclusions:
"Evidently the classification of phonemes into visemes can be donesee: http://www.speech.kth.se/prod/publications/files/1644.pdf
differently with respect to factors like language, speaker, listener, speech
situation, lighting conditions and, as will be shown below, phonemic
context. The better the viewing conditions are, the more contrastive
categories can be discerned. In other words, visemes are not constant units."
The main thing everyone seems to agree on, is that there are far fewer visemes than phonemes. Many phonemes are invisible, many others are ambiguous: several phonemes share the same mouthpattern. The usual example is 'b-m-p'. Three phonemes - one viseme.
BUT!
Even if we could agree on the number of visemes, teaching single visemes is only a very first step. Visemes change shape, in the context of other letters! A viseme that is perfectly visible when pronounced on its own, may be completely invisible when it's in the company of other letters / phonemes. Or it will look very different. Visible speech is not a sequence of still pictures, it's dynamic. The articulators have to go from one location to the next, and often what you see is not the end-location, but the movement!
Two examples - although I'm not very good at finding examples for English, so please correct me if I'm wrong or if you have better examples:
- 'food' versus 'foot': indistinguishable because both words share the same visemes. But if you say the words side by side, people will see the difference. Because of the end 'd' in food, the 'oo' in 'food' is much longer in duration, than in 'foot'. So we add another viseme: long 'oo' versus short 'oo'?
- 'bend' - 'lend' - 'blend': in many speakers, the 'l' is quite visible. But when it's preceded with a pretty dominant 'b', it may completely disappear from sight!
Step 2 in teaching lipreading, would have to be to teach 'blends', or 'co-articulation'. Of course most tutors will do that, as soon as they move on from simple CVC (consonant - vowel - consonant) words. Then they will have to explain that yes, sorry, there are zillions of exceptions to the visemes that you've just learned.
OR
We teach morphemes, instead of visemes? Morphemes are the next 'basic part' in words. Not single letters / phonemes, but short words, or syllables. The good thing about morphemes is that they are meaningful. The bad news is that there are many more morphemes in a language, than phonemes.
As for the examples above: each of the words is a separate morpheme: food, foot, bend, lend, blend.
What I don't know, but I'm sure that it's been researched: there is more co-articulation within, than between morphemes?
Morphemes show up in research about teaching children to read. E.g. Improving Literacy by Teaching Morphemes, by T. Nunes and P. Bryant, 2006.
The moral of the story: I really really think we can Improve Lipreading by Teaching Morphemes!
Wikipedia on Morphemes: http://en.wikipedia.org/wiki/Morpheme
No comments:
Post a Comment