The Program Features
A Robo-Stand or a Registration Robot at Conferences and Exhibitions
A Virtual Robot-Secretary (Robo-Secretary) for Offices and Office Centers
A Robotized Self-Service Terminal. A Robot-Terminal
A Robot-Assistant in a Store or a Salesroom
Robot-waiter SelfieBot in a café/restaurantе
More Variants of the Program Application
The Program Operation Demo
The Recommended Interface
Invitation for Cooperation
The platform software is a set of three modules that form the basis of our solution:
In 2015 we started tо work at a platform to be used as a basis for a simple robotic interface. Our task was to combine face and voice recognition with voice synthesis.
We used the standard Microsoft SAPI as a choice for a speech synthesis module. This product with various language sets is distributed free of charge.
Image recognition is a more complicated task, as it is necessary not just to recognize a face or an object but to recognize it in the streaming video going from the camera. The choice of the resources used by the program is very important. It does not matter for a computer but it does matter for a tablet, as the program should work efficiently and should not lock the system.
For this solution we used the OpenCV library. To speed up the work, any face of a specified size in the camera field vision is searched for at the beginning. The search is done with the help of Haar cascades and the ready trained template from the OpenCV library.
Then the detected face is cut out, normalized (unified) in size and light, and reproduced in black and white.
After that a ready FaceRecognizer algorithm, trained on a number of images of the same face taken from different angles, is used to recognize a certain person.
We use the Google Speech API, which maintains not more than 50 recognitions a day (it’s about 15 minutes of the recognition process). It’s a great solution for open-source.
Google has a very good recognition quality at the distance of a few meters.
The first thing the program does when activated is to detect a face in the field of camera vision. If it sees the face for the first time (it is not found in the database), it asks to enter and save the name. After that the program will always recognize this face.
Then the program switches to the mode of speech recognition: pronounced words, sentences or commands. As soon as the program recognizes a phrase, pronounced be a person, it looks for an adequate answer in the database and voices the answer. It is necessary to put all possible answers into the program beforehand for the program to know what to say. So far, it looks like a simple text file.
(Tim = Good afternoon, Tim. We welcome you at our conference.
Steve = Thank you for coming, Steve. You will be now welcomed. Have a nice day!
35310204 = Your credit repayment is due before the 30th day of the following month.)
This is not about an artificial intellect. This is about a simple interface for image and speech recognition and speech synthesis for educational needs. To have an opportunity to create a new ready solution on the basis of this or that robotized platform.
It is possible to use the program or its separate components for development of a certain robotic platform. This kind of platforms exists nowadays as electronic kiosks.
As we see it, this program will look more elegant if used together with our DIY SelfieBot.
This program can have a great many ways of practical application.
Problem to be solved:
Reduction of the conference/exhibition staff costs.
The Task: Support of conference visitors and guests with a user-friendly interface for automatic registration.
The Solution: A guest comes to the registration counter and tells his name. The system recognizes him, checks him in the database and registers the guest online.
An Example:
— Introduce yourself, please.
— Tim Cook.
— Good afternoon, Tim. You are welcome to our conference.
The problem to be Solved:
Cost reduction for a secretary work in an office center.
The Task: Support of visitors with a user-friendly interface for automatic registration when visiting an office center.
The Solution: A guest comes to the SelfieBot and tells his name. The info goes to the receiving party as a text or voice mail.
An Example:
Robot: — Please, introduce yourself.
Guest: — My name is Sonya.
Robot: Thank you for coming, Sonya. They will meet you now. Have a nice day!
Problem to be solved:
Cost reduction of the customers’ self-service and improvement of the service quality.
The Task
Customer support with a user-friendly interface for interaction with a self-service terminal. Voice communication with the terminal.
The Solution
A guest comes to the terminal and tells what he wants to do. No need to touch the screen.
An Example
Robot: Good afternoon! What would you like to do?
Customer: I’d like to know when my loan payment is due. My account number is 35310204.
Robot: It is due before the 30th day of the following month.
The problem to be Solved
Cost reduction for personnel and virtual information service.
The Task
Customer support with a user-friendly interface for interaction with an electronic shop assistant.
The Solution
A customer approaches the electronic assistant, asks questions and gets all the necessary information.
An Example
Robot: Good afternoon! We are happy to see you in our shop.
Customer: What special discounts do you offer?
Robot: We offer 10% discounts to our regular customers for all goods.
Customer: Do you have new collections?
Robot: We do, you’ll find them in the far right-hand corner.
Problem to be solved: reduction of expenses for personnel; waiting time reduction.
The Task: Customer support with a user-friendly interface for interaction with an electronic waiter.
The Solution: A robot-waiter approaches a customer, asks if he is ready with the order and takes the order.
An Example:
— Good afternoon! We are happy to see you in our café. Are you ready with the order?
— Yes. Please, bring me one cappuccino and a croissant.
— Thank you for the order. You’ll get it in 5 minutes.
The program is ideal for educational process as a separate solution or part of the integrated solution.
The program may be used for:
An example of the program user interface:
This program is ideal for students of high schools, colleges and universities striving to learn to work with the software designed to solve problems of the end users.
Educational Program — «Robotics: Design Engineering and Programming»
