enginuity

home | contents | previous | next

Sign of the times

Researchers in Roberto Cipolla's Computer Vision and Robotics group are tackling a new and interesting problem: how to teach a computer to recognize 3D sign language (as used by deaf people), and translate it into normal speech or written transcript.

"It may seem like an easy problem, getting a computer to recognize a human hand and determine its 3D shape - but to do this in front of an arbitrary background and in real-time is pretty difficult - the computer has to evaluate thousands of possible interpretations for each view it is shown." explains Cipolla. "In fact this can only be done because computing power is now available relatively cheaply. We are moving away from solving these problems using visual geometry alone, and are into the realms of Bayesian Statistics. The aim is to stabilize the visual interpretation. This uses the concept of 'prior' ( ie expectations about the shape and movements of hands) and 'likelihood' (measurements which takes into account features such as colour, shading and edges in the images) to determine the most probable interpretations (position of hand, shape and movement) of the image being viewed. Each hand has 27 degrees of freedom which result in millions of possible states associated with a given view. Our algorithm quickly determines the most probable."

The Royal Mail are interested in this work, as they already provide an automated booth where typed words can be translated into sign language. They now want to close the loop so that deaf and dumb customers can 'sign' directly to a machine and have their needs recognized.

Other applications of the technique will include intelligent interfaces in cars which will recognise simple driver signals to turn on the radio, the lights, the windscreen wipers etc. An extension of this technique to recognise facial features can also be used to monitor if a driver is falling asleep or not watching the road when a pedestrian is in front of the car. "Cars are pretty lethal machines, really" says Cipolla. "Legislation being introduced in Japan requires that far more use be made of automatic safety features." This work ties in with the interdisciplinary project on 'sentient vehicles' being led in the Department by Professor Nick Collings.

Help is also at hand for parents who worry about their kids slouching in front of screens - the next generation of computer games, using this type of technology will require participants to become actively involved - their movements, such as running, jumping and shooting, being tracked and used as input to the games…

For more details see the website at http://mi.eng.cam.ac.uk/~bdrs2/hand/hand.html