Core Technology

A Brave Attempt by Leaders of the Robot Development Team: Part 2
- In Quest of the Ideal Form of Human-Robot Interaction -

Jan.25,2023

In April 2022, the Technology and Intellectual Property HQ, OMRON’s corporate R&D division, launched a new department named「Voyager PJ」to set out on a new endeavor to create robots of a new dimension. Two development leaders of the project that focuses on cutting-edge R&D in robotics, Yoshiya Shibata and Kazuma Tabuchi, are working to reinvent a robot system architecture from scratch. We invited them for an interview to ask how they have built up their skills through their experiences from past development projects and what led them to aspire to develop brand-new robots that no one has ever seen.

Part 1 of this story will focus on Shibata’s aspiration to realize robots that execute complex movements with simple teaching.

Part 2 of this story will feature Tabuchi’s aspiration to seek the ideal form of human-robot interaction.

Yoshiya Shibata, Leader, Group No. 1, Voyager Project Department, Robotics R&D Center, Technology and Intellectual Property HQ (left)
Kazuma Tabuchi, Leader, Group No. 3, Voyager Project Department, Robotics R&D Center, Technology and Intellectual Property HQ (right)

Brief Profile of Kazuma Tabuchi

2008-2018

Worked on the development of controllers for an industrial robot developer/manufacturer

2018

Worked on the development of control technology and strategic planning for robot arms at OMRON Research Center of America, California

2022

Participated in Voyager Project

Driven to Realize a World Where Robots are Everywhere

At graduate school, I belonged to a laboratory working on a system that would allow humans and machines to coexist harmoniously. To understand how humans and organizations recognize things, make decisions, and behave, I was involved in analytical research of the process where team leaders learn how to appropriately allocate tasks among multiple members within a framework of multi-agent reinforcement learning.
In more concrete terms, I had a computer-simulated team learn “what state quantity a team leader looks at to motivate each member to perform a task” and “how members learn how to achieve a task” simultaneously. Although this research wasn’t directly related to robots, the experiences I had then, plus other research topics at the laboratory, such as human-machine interfaces, taught me the importance of pondering the difficulties of “humans handling machines” and “having machines work like humans.”

After completing my graduate studies, I wanted to move past simulations to something that moves real machines. So I took a position at an industrial robot manufacturer, where I was doing research and development of motor control systems. I deliberately chose the position because I was intrigued by the diverse ways industrial robots move. I didn’t have any preexisting knowledge of robots and controllers, so I found it rather challenging to develop a correct understanding of robots’ kinematics when I was first assigned to a team that was developing technology for dynamically changing robots’ movements depending on the situation. In retrospect, I was fortunate to be able to learn through hands-on experiences that kinematics is the vital technology that realizes the varied movements of robots.
Then I began working on path planning and how to prepare operational procedures. My goal was to develop a system that automatically plans how to allocate tasks most efficiently when there are multiple robots.

Through my experiences of research at the university and previous development, I came to realize that I’d rather pursue the possibility of realizing a system that achieves advanced tasks by optimally combining functions of individual components, including humans, rather than forever seeking high-speed, high-precision control of robots. For instance, I began to find it more interesting to work out “how we should combine individual functions to maximize effects” in light of the original proposition of what we want to achieve by using robots to automate processes, rather than to try to attain the ultimate in individual element technologies that might become necessary in a loosely-conceived imaginary world where robots do everything autonomously. Throughout this, my desire to create a world in which robots support humans and work in collaboration with them on many different occasions became stronger and stronger.

Joining OMRON to Take on a New Development Challenge

I joined OMRON Corporation in 2018, thinking that there, I could have an opportunity to develop “robots that collaborate with people” and take on the challenge of realizing a world where robots play vital roles as the desire of mine, I just described aligned with OMRON’s view expressed in its corporate philosophy: “People should have time to partake in more creative fields of activity, leaving machines to do what they can.” I also expected to be given a chance to try building unconventional robots under a novel design concept at OMRON since the company is involved in a broad spectrum of businesses, including manufacturing, public transportation, and healthcare, and intends to expand its robot business. That’s why I chose OMRON.
In April 2018, OMRON established a new R&D base, OMRON Research Center of America, in California, in a bid to shore up technology for the autonomous mobile robot (AMR) business in the U.S. Soon after joining OMRON, in 2019, I was sent to the U.S. with the mission of integrating R&D strategies between R&D bases in Japan and the new R&D base in the U.S., where I was responsible for the development and strategic planning of control technology for robot arms. Once the direction for development was determined, I handed over my development responsibilities in the U.S. to another resident researcher and came home. Then I participated in the Voyager Project in 2022. During my three years of residence in the U.S., I was fortunate enough to be in an environment where they asked me for suggestions on various matters and expected me to offer my input even though I had just joined the company. I was able to hone my ability to set and express new development targets because of my time in the U.S.

Difficult Yet Important Job of Humans Predicting Robots’ Movements

When humans and robots work collaboratively in the same place, if you are well-versed in how robots move and can program their movements, you can more or less predict a robot’s next move. If you have never programmed anything, however, it is very difficult to predict how a robot will move while collaborating with humans. If it is a robot that simply moves along a predetermined trajectory, you may find it easy to predict its movements. When it comes to complicated sequences associated with collaborative work between humans and robots or movements that are reactionary as the environment changes, however, a human collaborating with a robot may not necessarily be able to predict its next movements. If it is collaborative work among humans, one can predict how the other person will move based on past experiences and anatomical similarities, but humans and robots differ in their shapes and movements, and one cannot anticipate how a robot will move in the way you can with a human partner. Developing a robot that emulates human movements may not be the answer to this, as it may limit some of the advantages of robots, namely, the optimal shape that differs from that of humans and moving with high speed and precision that cannot be expected from humans.
In view of safety, if a human is about to collide with another human, they both can adjust their movements flexibly. Even if they bump into each other, they may not get injured as their bodies are soft. This is not the case for a robot-human collision, which can result in a serious injury on the part of the human. This being the case, I began to think that, for humans and robots to work collaboratively and enhance productivity by utilizing each other’s merits, they need to understand each other’s movements by communicating what they each think through “interaction.”

Realizing a Desirable Interaction and Taking on the Challenging Task of Developing Easy-to-Operate Robots

For humans and robots to work collaboratively, the first key step is how to have humans recognize robots’ movements. Being in widespread use on the manufacturing floor and at distribution warehouses, automated guided vehicles (AGVs) let humans know about the presence of a machine with such things as sound and revolving lights or stop when the laser imaging, detection, and ranging (LiDAR) has detected an obstacle. If its structure is like that of an automobile, which is very familiar to humans, it is relatively easy to predict its movements. In such a case, you only need simple means to communicate with a robot. It is, however, very difficult to let humans see more complicated movements like the ones by articulated robots.
If all collision avoidance actions are to be left to the robot, we need to take safety measures, such as programming for a huge number of cases and limiting movement speed, which makes it difficult to reduce costs while enhancing performance. Besides, there’s a natural limit to variations of movements that may be realized. Suppose two persons pass by each other. If one person avoids the other person who walks in the middle of the corridor, we need a broad corridor. If, however, both are happy to give way to each other, they can pass by in a narrower corridor. I have come to the conclusion that this is what we should be aiming at in the case of humans and robots, too, in the quest for the ideal form of human-robot interaction.

I also think that, just as we need good interaction, it’s important to make robots easy to operate. To realize a world where robots understand vague instructions from their users, we need robots that work normally but also take into account abnormalities. And the task of designing responses to such abnormalities currently rests with the skills of system integrators (SIs), who design robot systems. Yet, for robots to be easily operated by users, they must function without requiring customization. In other words, if robots and humans are to work together efficiently, users must be able to use robots without the help of SIs who are well-versed in robots. We need to understand that no SIs or robot manufacturers have every single piece of hands-on knowledge they need to increase work efficiency. I think users in the field can do a better job of building a productive human-robot system that caters to their needs. To realize a future where users find it easy to manipulate robots without the help of SIs, I am working to give concrete shape to the interface we need for this purpose, giving my all to develop a new robot system.

Robots are now widespread in transfer, picking, and other work processes where single tasks are performed repeatedly with high speed and precision. Because skilled operators are necessary to introduce robots, and robots thus introduced are subject to restrictions in functions/performance and portability for the sake of safety, however, their applications are still limited. Heading toward the goal of creating a world where robots support humans in a greater number of scenarios, Tabuchi remains committed to developing novel robots from his unique perspective of what roles humans and robots each play and how they should interact with each other—Tabuchi’s challenge goes on.

Note: Names of organizations, official positions, etc., are current as of the time of publication (January 2023).



You may also like:

Developing a Novel Robot that Closely Assists Humans
https://www.omron.com/global/en/technology/activities/07/

A Brave Attempt by Leaders of the Robot Development Team: Part 1
- The Challenging Task of Developing a Robot that You Can Teach as You Would a Human
https://www.omron.com/global/en/technology/activities/08/

Recruitment
https://www.omron.com/jp/ja/recruit/