For effective human-robot interaction, robots need to understand the current situation context well, but also the robots need to transfer its understanding to the human participant in efficient way. The most convenient way to deliver robot’s understanding to the human participant is that the robot expresses its understanding using voice and natural language. Recently, the artificial intelligence for video understanding and natural language process has been developed very rapidly especially based on deep learning. Thus, this paper proposes robot vision to audio description method using deep learning. The applied deep learning model is a pipeline of two deep learning models for generating natural language sentence from robot vision and generating voice from the generated natural language sentence. Also, we conduct the real robot experiment to show the effectiveness of our method in human-robot interaction.

This paper proposes combination of a cognitive agent architecture named Soar (State, operator, and result) and ROS (Robot Operating System), which can be a basic framework for a robot agent to interact and cope with its environment more intelligently and appropriately. The proposed Soar-ROS human-robot interaction (HRI) agent understands a set of human’s commands by voice recognition and chooses to properly react to the command according to the symbol detected by image recognition, implemented on a humanoid robot. The robotic agent is allowed to refuse to follow an inappropriate command like “go” after it has seen the symbol ‘X’ which represents that an abnormal or immoral situation has occurred. This simple but meaningful HRI task is successfully experimented on the proposed Soar-ROS platform with a small humanoid robot, which implies that extending the present hybrid platform to artificial moral agent is possible.

In this study, we have developed the humanoid joint modules which provide a variety of service while living with people in the future home life. The most important requirement is ensuring the safety for humans of the robot system for collaboration with people and providing physical service in dynamic changing environment. Therefore we should construct the mechanism and control system that each joint of the robot should response sensitively and rapidly to fulfill that. In this study, we have analyzed the characteristic of the joint which based on the target constituting the humanoid motion, developed the optimal actuator system which can be controlled based on each joint characteristic, and developed the control system which can control an multi-joint system at a high speed. In particular, in the design of the joint, we have defined back-drivability at the safety perspective and developed an actuator unit to maximize. Therefore we establish a foundation element technology for future commercialization of intelligent service robots.

This paper presents interaction force control between a balancing robot and a human operator. The balancing robot has two wheels to generate movements on the plane. Since the balancing robot is based on position control, the robot tries to maintain a desired angle to be zero when an external force is applied. This leads to the instability of the system. Thus a hybrid force control method is employed to react the external force from the operator to guide the balancing robot to the desired position by a human operator. Therefore, when an operator applies a force to the robot, desired balancing angles should be modified to maintain stable balance. To maintain stable balance under an external force, suitable desired balancing angles are determined along with force magnitudes applied by the operator through experimental studies. Experimental studies confirm the functionality of the proposed method.

This paper presents a probabilistic head tracking method, mainly applicable to face recognition and human robot interaction, which can robustly track human head against various variations such as pose/scale change, illumination change, and background clutters. Compared to conventional particle filter based approaches, the proposed method can effectively track a human head by regularizing the sample space and sequentially weighting multiple visual cues, in the prediction and observation stages, respectively. Experimental results show the robustness of the proposed method, and it is worthy to be mentioned that some proposed probabilistic framework could be easily applied to other object tracking problems.

For an advanced intelligent service, the need of HRI technology has recently been increasing and the technology has been also improved. However, HRI components have been evaluated under stable and controlled laboratory environments and there are no evaluation results of performance in real environments. Therefore, robot service providers and users have not been getting sufficient information on the level of current HRI technology. In this paper, we provide the evaluation results of the performance of the HRI components on the robot platforms providing actual services in pilot service sites. For the evaluation, we select face detection component, speaker gender classification component and sound localization component as representative HRI components closing to the commercialization. The goal of this paper is to provide valuable information and reference performance on appling the HRI components to real robot environments.