Machines crack jokes

We’ve already become accustomed to communicating with machines – be it talking to automated phone systems or Apple’s Siri software (Speech Interpretation and Recognition Interface), for example. The results can be irritating, if the automated system doesn't precisely understand our input, or amusing if, for example, Siri takes things humorously and tries to crack a joke. Computer specialists, mathematicians, psychologists and many other experts all over the world are working on teaching robots what human emotions are, or more precisely, how they can be evaluated. The goal is to use robots for purposes beyond education, for example, to generate customer loyalty, in the field of medical and health care, and for therapy applications.

All-knowing customer clown

This is where Pepper, Romeo, and NAO come in. The three brothers were “born” at SoftBank Robotics. Pepper was created to support companies with customer service. Not only has he been busy in Nestlé shops in Japan since 2014, but now he has also been put to work at French train stations in the Loire region and in European Carrefour supermarkets. The little humanoid robot has an enormous store of knowledge thanks to a large data memory. He recognizes his counterpart, is able to assess emotional states (“Are you sad?”) and communicates on the basis of existing customer information.

Smart assistants

At 140 cm (4 1/2 feet), Romeo is quite a bit taller than Pepper and was developed to assist people with physical disabilities. With Romeo's help, the autonomy of elderly people can be enhanced. The robot assistant can open doors, climb stairs, and fetch things. With a height of just 58 cm (1 1/4 feet), NAO is a miniature version of Pepper and Romeo, but yet the oldest of SoftBank Robotics’ humanoid robots. Developed back in 2006, and currently in its fifth generation, NAO is used by thousands all over the world.

Robot teacher speaks 20 languages

NAO, the interactive, customizable robot whose name in Japanese means "honesty," recognizes more than twenty languages. NAO can help train on a variety of learning content, so he’s being increasingly used in schools. In Japan - the land of robots - students learn vocabulary, practice mental arithmetic, and do gymnastic exercises as assigned by the robot. The children are enthusiastic and fully involved. A Japanese study (study of the use of robot teachers at the University of Osaka) concerning the use of NAO reveals that the noise level is significantly lower during lessons with a teacher-robot team.

Robot teachers in Germany and Austria

Robots as assistant teachers have arrived in Germany and Austria as well. With his vast knowledge, NAO supplements the teaching of STEM subjects (Science, Technology, Engineering, and Math) in many schools across all age groups. For example, grammar school pupils in Karlsbad, Germany, learn programming with NAO’s assistance, and NAO is also available to pupils for instruction at (vocational high school) HTL Leonding in Upper Austria.

Refugee children learn German from robots

The L2TOR project launched in 2016 (Second Language Tutoring using Social Robots) is designed to teach the new second language to immigrant children from four to six years old using humanoid robots, so that the children can be integrated into the school system as quickly as possible. However, language training with the robots is intended as an individual supplement to, and not a replacement for, existing educational offerings. Clearly, the use of robots creates additional resources, and with a current purchase price of roughly €12,000, NAO will be less expensive than a personal human tutor in the long run.

Poor human-machine interaction

Despite all of this, there are still limits to the quality of human-machine interaction. It functions well enough within the parameters of a child's limited vocabulary. But the artificial intelligence is not capable of understanding the context of a more complex conversation and responding meaningfully and spontaneously. Computer scientists, educators, and linguists are continuously working on the step-by-step improvements toward the perfection of comprehension. NAO should develop quickly in this regard. He’s used in research projects dealing with the issue of human-machine interaction, collecting vast amounts of data. Essential questions are considered, for example: What characterizes human emotions? How can they be recognized with the help of computers, and how can computers themselves learn emotions?

Robot assists autistic children

This topic is part of the EU project known as DE-ENIGMA, which was launched in 2016 and involves the participation of the department for “Complex and Intelligent Systems” at the University of Passau. The goal is to use robots to expand the social imagination of autistic children. “We want to help autistic children improve their emotional reactions and more easily integrate themselves into society. The dialog between the child and the robot is intended as part of the therapeutic process, so that autistic children learn to correctly assess social behavior," says professor Dr. Björn Schuller, director of the Passau team.

Tech-savvy, young patients

The sessions are conducted under the supervision of a therapist. The ZENO R25 robot, which at its current cost of about $5000 is one of the less expensive models, uses software to motivate the child, providing feedback and observing every emotion. A preliminary study showed that the majority of the autistic boys and girls, ranging in age from 5 to 12 years, generally got along very well with their android interlocutor (editor’s comment: android is the designation for a robot which looks and behaves like a human being). This is partly because, as a rule, autistic children tend to be very tech-savvy, and because dealing with rule-based, predictable systems like robots is perceived by them as much less complicated and threatening than social interaction with people.

Recognizing and generating emotions

On the other hand, just how robots are able to adapt to autistic children and their behavior is puzzling for many at first glance. “As a rule, machine behavior is based on demo material – in our case, collected from typically-developed, autistic children. A great deal of preliminary technical work was required in order to link the diagnosis of autism to fixed parameters, but now we have entirely new evaluation methods for very large volumes of data," explains professor Schuller. “These parameters include autistic language, as well as image processing for movement patterns and gestures. We provide the guidelines, and the detection software monitors facial expressions, gestures, and speech characteristics, including fundamental frequency and variability of the voice. This is how emotions can be recognized,” explains Schuller.

Skepticism concerning android competence

German physician, Dr. Christine Preißmann, doubts that android competence will be able to lastingly help the children involved in this project. She’s an autistic specialist and writer who also suffers from Asperger syndrome: “The use of artificial intelligence makes sense on many levels, but whether it can be an asset to emotional development is currently questionable. Every autistic person is an individual with specific needs, and so they require targeted care. I doubt that robots are capable of offering the necessary individuality. I believe that when autistic people learn with robots, they are ultimately limited to only dealing with robots.”

Data deluge becomes robot knowledge

In any case, the evaluation of the information that will be gathered during the 3½ years of the DE-ENIGMA project promises a quantum leap from a data standpoint – not only for autism research, but for the wider scientific community as well. Until now, it has never been possible to collect large volumes of precise information concerning behavior, facial expressions, intonation, volume, and gestures in such a condensed and targeted way.

Uncanny valley phenomenon

While many people around the world are enthusiastic about the large and small android helpers, and the robotics industry is forecasting astronomical sales, it’s becoming more and more likely that the curve toward commercial acceptance could break off along the way to the perfect humanoid. We’re talking about the Uncanny Valley phenomenon which marks a psychological acceptance gap, or valley, as soon as a robot achieves a certain level of anthropomorphism: anything which behaves almost, but not exactly, like a human being can very easily become unacceptable.

The technological singularity

Nevertheless, technological singularity seems to be waiting at the end of the journey – the point at which human beings and machines become equal in intelligence, and machines improve and develop themselves. Fortunately, it won’t be easy, because the world of human feelings will remain inaccessible to robots for the time being. For a closer approximation, information processing would have to at least be coupled to physiology, as is the case with human beings, and that’s not simply a matter of sensor technology. So we shouldn't expect machines to outdo human intellect in the near future, even if Siri already believes this is true, along the lines of Descartes: in response to the question: “What do you think of artificial intelligence?” Siri answers: “I think, therefore I am.”