Earlier we talked about the fact that the problem of the population aging and the increase of the number of patients with Alzheimer’s disease that people will face in the next 20-30 years, makes us think about its solution already now.
Since January 2017, Endurance, along with four US universities, has begun work on a project to create a robot-companion (chatbot) for senior people and patients with Alzheimer’s disease.
The main idea is to create a universal robot-companion for senior people.
The problem we are trying to solve is to give senior people and patients with Alzheimer’s disease the opportunity to communicate: talk, share their experiences, memories.
At the moment, the mechanical part of the solution has been completed. This is a holder for a phone or tablet, which will rotate to follow the speaker. The holder is called SelfieBot and is fully 3D printed. The electronic components, program code, firmware are completely open source and are presented on the site EnduranceRobots.com.
We specially developed a model for 3D printing, so that anybody could create their own robot-companion.
Next, we face the challenge of creating an intelligent companion.
It is assumed that the robot will be able to conduct a conversation on more than 20 topics, such as: weather, nature, news, history, cinema, music, etc.
The chatbot faces the task not only to ask and answer questions but also to memorize the context of the conversation. This is crucial for conducting a cohesive dialogue.
No less important task is to recognize the tone of the user when he/she communicates with the robot. The robot should be able to start and conduct a dialogue at ease. At the moment, more than 10 thousand question and answer templates have been created. Our robot not only asks questions, it is also able to adequately respond to answers.
An interlocutor can give various answers to the question “How are you?” We consider three main scenarios:
In each of the scenarios the robot’s task is to ask the interlocutor to tell about this in more detail.
Here begins the most complicated part. It is almost impossible to determine now how the dialogue will develop further. An elderly person can tell what a wonderful dream he has dreamed, how well he feels, or, on the contrary, can start complaining.
For such cases, the robot must be provided with words of support that will allow a person to speak out.
Dr. Wallace took this into account when created his robot-interlocutor Alice in the 60’s. Now Endurance has its own solutions and those of its partners, with the API of which we can determine the tone and context of the dialogue, find synonyms and much more.
It is also important to keep in mind that senior people often have speech and hearing problems. We are going to use ready-made API for speech recognition and synthesis. At the moment, we use solutions from Microsoft and Google, which we consider to be the best among the available technologies.
The main difficulty now is to understand how to teach the robot to shunt the conversation on another topic if needed. So far, we have not specified a sufficient number of criteria for this.
In general, the set of chatbot’s functions can be quite diverse. In the process of the chatbot developing and testing it will become clear what other tasks require a solution.
The chatbot key function will be its ability to store a communication history, making it available for viewing by the family/friends and a doctor if monitoring is necessary.
We hope that the project will become popular and help any interested person make his/her physical robot-companion, the creation of which will require a SelfieBot rotation mechanism, a phone/tablet and the Internet access.
In the long term, we plan to equip the chatbot with the image recognition function so that it could recognize the interlocutor, read his/her facial expressions and could give its interlocutor simple tasks such as: to draw something or tell.
This function will undoubtedly vastly improve the interactivity of the communication.
Microsoft Cognitive provides an API for emotions evaluation by a facial expression, which can become the first step to implementing this solution.
This project is non-commercial, and all the dialogues corps are open source.
We invite researchers and research centers to cooperate.
The design development of the SelfieBot phone/tablet holder is also open source and posted on EnduranceRobots.com
We believe that this technology will allow senior people and Alzheimer’s disease patients to fully experience the joy of communication.
Full research plan in PDF.
If you are interested in cooperation write to: gf@EnduranceRobots.com
Or call: +7 916 2254302 (в т.ч. WhatsApp, Viber, Telegram)
Facebook messenger: @george.fomitchev