Robo-Erectus @ Home – A Service Robot for
Application in Home Environment
Robotics – Field of the Century
Growing popularity and increasing viable application domains has contributed to greater presence of robots in the commercial marketplace. Among various robotic platforms, especially service robotics research has seen a rapid growth in the recent years due to the ability of service robots to behave and interact like humans. Service robots might provide day-to-day support in the home and the workplace, doing laundry or dishes, assisting in the care of the elderly, or acting as a caretaker for individuals within a home or institution.
What is RoboCup @ Home League?
RoboCup is an international joint research initiative to foster artificial intelligence, robotics, and related research. RoboCup uses home league as a domain of research, aiming at innovations to assist humans in everyday life. RoboCup Home league offers the researchers several challenges in a home environment and an open challenge to exhibit the best capabilities of the participating robot. All the challenges are conducted in a real world living room scenario. RoboCup Home league limits the mode of interaction between human and robot to natural ways (Speech/Gestures).
Robo-Erectus@Home II – An Introduction
Growing popularity and increasing viable application domains has contributed to greater presence of robots in the commercial marketplace. Among various robotics platforms, especially service robotics research has seen a rapid growth in the recent years due to the ability of these robots to behave and interact like humans. Service robots might provide day-to-day support in the home, doing laundry or dishes, assisting in the care of the elderly, or acting as a caretaker for individuals within a home [1]. RoboCup @ home league serves as an excellent platform for researchers around the world to realize their dream of service robots for every home through testing and sharing their research work in robotic home application. Overcoming the challenges set by the RoboCup @home league requires careful investigation and fostering of various technology areas including human robot interaction, electronics, signal processing, mechanics, control engineering and robot localization topics. Robo-Erectus @ Home II (RE@H-II) has gone through many stages of design optimizations for its mechanical structure, electronics and control system. In Robo-Cup 2010, we plan to participate in the RoboCup Home Competition with RE@H-II which was built on top of a pioneer 3 differential drive mobile base, equipped with onboard vision, ultrasonic sensors, touch sensors and electronic compass for perception. In the subsequent sections a brief description on the hardware and software specifications of RE@H-II are presented.
Hardware Specifications
The hardware design of RE@H-II has gone through stages of evolution towards participation in RoboCup @ Home 2010, this section presents the hardware systems of our robot. Figure 1 shows the physical design of RE@H-I.
Figure 1. Physical Design of RE@H-II
Mechanical Designs
RE@H-II was fabricated and developed at Advanced Robotics and Intelligent Control Centre of Singapore Polytechnic. Repeated experiments were performed for optimization of robot’s dimension including height and weight of body, arm and other components. The final optimized dimensions of the robot are 150cm in height and a weight of about 51kg. Most of the frames of the robot were constructed using aluminum. The robot has a size of 45cm x 45cm x 150 cm. RE@H-II is equipped with a Sony VAIO as on board PC. The camera is connected to the VAIO for image processing and high bandwidth communication. A speaker system is located on the front part of the torso and a microphone is placed on the head of the robot are used for interactions. The battery lifetime is approximately 40 minutes. The torque and power requirements of the actuators used in RE@H-II were carefully chosen so as to enable the robot to navigate, turn right/left, and manipulate object [2]. The robot is capable of expressing emotions through the use of colored LEDs, servo motor control of mouth and eye movements. The finger system for the arms of the robot was developed to manipulate objects. A 3D virtual avatar based interaction system was also developed for special performance in open challenge. Figure 2 shows the virtual avatar developed for emotion expression and speech.
Figure 2. Virtual Avatar for Emotion Expressive Speech
Sensors
One of the most important tasks of an autonomous system is to acquire knowledge about its environment. Sensors collect all the information as autonomous system needs to operate and interact with its environment. There are wide varieties of sensors that can be used in mobile robots. RE@H-II is currently equipped with a camera sensor for perception, microphone for speech recognition, an electronic compass to measure the direction towards which the robot is moving, sonar and laser scanner for distance measurement of the objects in the environment. RE@H-II is equipped with a monocular USB camera from Logitech for perception. The camera is a 640 x 480 (VGA) progressive scan CMOS with a standard miniature lenses. The frame rate is 30Hz for 640X 480 with automatic control of exposure, gain and black level. Sick LMS 100 laser scanner provides distance information which is fused with the sonar reading for obstacle avoidance and terrain mapping.
Software Specifications
In this section the software modules including object recognition, control, manipulator and speech processor modules are presented.
Object Recognition
A multiple sensor integration approach was adopted for perception which involved fusing of data from ultrasonic, compass, camera, Speech and tilt sensors. Of all the sensors, especially the vision provides the robot with an enormous amount of information about the environment and enables rich, intelligent interaction in dynamic environment [3]. The camera sensor was selected so as to achieve a wide field of view for the robots to see their own feet and objects above the horizon at the same time. RE@H-II is equipped with one camera mounted on servo that actuates the neck of the robot. The face/object recognition is achieved by using a combination of polygonal matching to shape recognition, use of SIFT features as well as skin color recognition [4] [5]. This algorithm scans the image to find object of interest, once the object is found a window around the object is used to speed-up the tracking.
The system estimates the new position of the object in the image and swifts the window towards that direction, if the object is lost from these window the program searches again in the whole image. The system is reliable to find objects under a clutter environments and lighting conditions. Figure 3 shows case of three faces recognition cases under different condition, Fig. 3(a) shows faces with partial occlusion of one face, Fig. 3(b) presents these faces under a clutter background. Tracking not only involves the next position in the image, but also to face the person that is interacting with the robot. Besides, the mouth provides a clue if the person in the image is talking to the robot or not.
Control Module
This module provides the control and strategy for the robot when undertaking different challenges of RoboCup @ Home league. A framework of hierarchical reactive behaviors is the core of this control module. This structure restricts interactions between the system variables and thus reduces the complexity. The control of the behaviors happens in three layers: skill, reactive, and planning layer. This architecture was initially developed for our Robo-Erectus version of humanoid robots where it yielded excellent results [6]. Figure 4 shows the control architecture of RE@H-I. These layers respond in a different way to sensor data. The interaction of these layers produces the final behavior of the robot. Besides the physical sensor data, the system employs abstract sensors, take decisions.
Figure 3 Faces Recognition Cases under Different Condition
These abstract sensors are built by merging data from different sensors and their history records. The best example of these types of sensors is the map, which is generated with camera information, speech, compass data, and previous positions. The details of the three layers are given in the following part. The skill layer controls the servos, monitors targets, actual positions, and motor duties. It receives actions from the reactive layer and converts them into motor commands. After performing the motor commands, a feedback is sent back to the reactive layer.
The reactive layer implements the robot behaviors like navigation, object manipulation and so forth. This layer determines the parameters for the behavior and these parameters can be adapted on time. This makes possible to correct deviations in the actual data and to account for changes in the environment by using the sensor feedback. Each of these behaviors consists of several actions, which are sent to the skill layer. The selection of the behaviors depends on the desired task of the planning layer. The behaviors in this layer are implemented as subsumption architecture to enable the robot to satisfy the task while it can navigate safely in the environment. The subsumption provides priority to the surviving behaviors. Such surviving behaviors use raw sensorial data to be triggered, and each behavior guarantees that the robot will be save in the environment to carry on its task. Example of surviving behaviors is the object grabbing behavior, which allows the robot to grab the object after recognition process, is complete.
The planning layer used the behaviors of the reactive layer to implement some service skills like approaching the human, emotional expressions, object grabbing and speech synthesis. The planning layer guides the robot to coordinate its efforts with human to achieve the task of interest. The behaviors at the planning layer are abstract goals. These abstract goals are passed to the reactive layer to be sent to the actuators. The response from the planning layer is slower than the reactive layer, the abstract sensors are used by this layer.
Figure 4 Control Architecture of RE@H II
Speech Processor Module
The speech processor module consists of speech recognition and speech synthesis sub modules from Sensory Inc. The speech recognition sub module can be trained for recognizing a set of 50 commands in English language. The recognized words were then used to activate different behaviors of the robot ensuring natural way of communication between the human user and the robot. The speech synthesizer sub module helps the robot to convey its intention in a natural way to the human user through speech by selecting a best suitable sentence from a pre recorded voice storage bank. Lip synchronization has been achieved with the 3D avatar based interaction system. Emotional speech which fuses emotional expressions to speech synthesis has been tested with RE@H-II.
Manipulator Module
We integrated the AL5D robotic arm into RE@H-II towards achieving the manipulation objectives for RoboCup@Home competition. AL5D delivers fast, accurate, and repeatable movements. The arm has 10.25" median reach; 13oz lift capacity, 4 DOF, and all-aluminum construction. The robotic arm features: base rotation, single plane shoulder, elbow, wrist motion, a functional gripper, and optional wrist rotate. Figure 5 shows our arm system of RE@H-II. Inverse kinematics and arm control strategies were handled by the main processor which handles the decision making process. Based on the object of interest information from vision processor, the main processor computes the 3D co-ordinates and sends appropriate motor commands to the actuators fitted to the arm manipulator. The arm manipulator then moves to the 3D co-ordinates received and uses its customized gripper structure to hold the object of interest. A customized two finger gripper system has been built to handle tasks in RoboCup@home competitions.
Figure 5 Robotic Arm System of RE@H-II
Human Robot Interaction
RE@H-II is capable to sensing bio signals including systolic and diastolic blood pressure, temperature, sugar level, muscle contraction and expansion, ECG and EEG signals. With rising elderly population, our robot is expected to work closely taking care of elderly, a personal care box consisting of blood pressure kit, temperature, and diabetes kit was built on the robot. RE@H-II was used for our experiments in human robot interaction [7] [8]. Currently, there are no metric to quantify the erroneous interactions between the human and the robot both of whom we want to interact freely. In most real life applications erroneous interactions between the human and the robot are common due to uncertainties in both human as well as in robots. We defined false alarms in human robot interaction. We also categorized the false alarms into false positives wherein a robot rejects a “correct” interaction and false negatives wherein a robot fails to reject an “incorrect” interaction. False alarms negatively impact the performance and fan out in human robot teams. Most research work assume zero false alarms which would result in an optimistic prediction of performance and fan out, not only leading to operator’s failure in accomplishing the task as scheduled due to higher attention demands in actual situation, but also leads to operator’s inability in handling planned number of robots/task. Performance of robots in human-robot teams is complex and multifaceted reflecting the capabilities of the robots, the operator(s), and the quality of interactions [9]. We extended the neglect tolerance model which is used as a general index for estimating robot performance in relation to autonomy in human robot interaction community by incorporating the demands due to false alarms during human robot interactions. Figure 6 shows the results from our extended neglect tolerance model with RE@H-II for fetch and carry challenge presenting the relationship between performance, operator and time.
Figure 6 Results from Extended Neglect Tolerance Model with RE@H-II for Fetch and Carry Challenge
We also redefined the fan out metric commonly adopted in the human robot interaction community with incorporation of demands due to false alarms towards obtaining a realistic prediction of the maximum number of robots a single operator can handle simultaneously while maintaining performance at acceptable levels.
Conclusion
The RE@H-II project aims to develop a platform for competing in RoboCup @ Home League and for our ongoing research work in human robot interaction, robot localization and navigation areas. An object oriented software frame work is implemented for object recognition, motion control and communication modules to achieve real time capabilities in terms of robot localization and navigation. During the experiments performed, RE@H-II exhibited excellent navigation, object manipulation, human robot interaction and emotion expressive skills. For more detailed information about Robo-Erectus Humanoid robots, please refer to the team’s website www.robo-erectus.org
Team Members
Sunardi
Ziyu Chen
Htin Linn Kyaw
Jia Yin
Yiliang Ang
Maxmilian Baro
Rajesh Elara Mohan
Carlos Antonio Acosta Calderon
Yi Zhou
Changjiu Zhou
Acknowledgements
The authors would like to thank staff and students at the Advanced Robotics and Intelligent Control Centre (ARICC) and higher management of Singapore Polytechnic for their support in the development of our service robots.
References
- Metev, S.M., Veiko, V.P.: Task structure and User Attributes as Elements of Human-Robot Interaction Design. In: IEEE RO-MAN 06 Conference, United Kingdom (2006)
- Acosta Calderon, C.A., Mohan, R.E., Zhou, C.: Robotic Companion Lead the Way!. In: International Conference on Control, Automation, Robotics and Vision ICARCV, Hanoi, Vietnam (2008)
- Mohan, R.E., Acosta Calderon, C.A., Zhou, C., Yue, P.K., Hu, L., Iniya, B.: An embedded vision system for soccer playing humanoid robot: Robo-Erectus Junior. In: IEEE International Conference on Signal Processing, Communications and Networking, India (2008)
- Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 2, pp. 91-110 (2004)
- Mel, B.W.: SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object recognition. Neural Computation, pp. 777-804, (1997)
- Acosta Calderon, C.A., Zhou, C., Yue, P.K., Wong M., Mohan, R.E.: A distributed embedded control architecture for humanoid soccer robots. In: 10th International Conference on Climbing and Walking Robots, Singapore, (2007)
- Mohan, R.E.: Effective performance estimation in human robot teams” Ph.D qualification report. Nanyang Technological University, Singapore (2009)
- Mohan, R.E., Wijesoma, W.S., Acosta Calderon, C.A., Zhou, C.: Redefining Fan Out Metric: Estimating Human Capability in Controlling Multiple Robots, In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, St. Louis, USA (2009).
- Wang, J., Wang, H., Lewis, M., Scerri, P., Velagapudi P., Sycara K.: Experiments in Coordination Demand for MultiRobot Systems. In: IEEE International Conference on Distributed Human-Machine Systems, Greece, (2008).
Video
Publish write-up in file REHome Web Info 2009.doc on the RoboCup @ Home page