Computers learn to read “body language” up to finger movements
Almost half of all communication between people occurs with non-verbal contacts, often too inaccurately expressed. For a digital system that has become accustomed to accepting specific teams, this is an insurmountable obstacle in establishing a dialogue between the machine and the person. But the Carnegie Mellon University Robotics Institute thinks they have found a way out.
To grab a random gesture with the camera is not difficult, all problems with their accurate and rapid interpretation. The training system consists of three steps. First, the computer does not study the behavior of a particular individual, he selects individual gestures of hands, feet, turns of the head and searches for their analogs on the object under study. For example, anxious waving by hands – first we find a gesture, then we figure out to whom it belongs.
To create a database of gestures and postures, a pair of postgraduate students waved for hours in front of the camera, showing a total of several thousand combinations of body positions. Then came the turn of the third stage – with the help of a Panoptic Studio installation of 500 cameras, each pose and a separate gesture were photographed in detail from multiple angles so that the system learned how to accurately identify them in different situations.
It took 31 ultra-high resolution cameras to capture the movement of the fingers, so they are small and mobile for the electronic eye. But all this machinery and a lot of power are needed only at the learning stage of the system, in the future, to “read” the language of the body will be enough and simple webcams, scientists assure. Now they are working on combining 2D and 3D models and recognizing the gestures of individuals in the crowd.
The scope of the new technology is not really outlined, in view of the fact that it is problematic to prove the reliability of reading a gesture by a computer. It’s one thing to arrange an advertising show with the recognition of the emotion of the crowd and quite another – to entrust the autopilot to drive an electric car, obeying the careless strokes of the palm . And if you let the system work as an assistant referee at sporting events, this may even lead to a scandal. Try, explain to the silicon brain that you did not simulate a collision and pain!