Linking people to the future with OMRON’s perspective on cutting edge technology
In the new normal era caused by COVID-19, automation technology plays a greater role than ever before. There is a growing expectation for automation that allows machines to not only perform tasks to the extent that they are capable of doing on their own, but also to generate a synergy through achieving harmony between humans and machines.
To demonstrate this vision, OMRON organized a table tennis match using the table tennis robot FORPHEUS and the Mobile Manipulator (MoMa), both of which incorporate OMRON's core technology of Sensing & Control + Think, and this table tennis rallies embody the automation of the new normal era which human and machines work together. OMRON engineers challenged to realize this "human-machine doubles" by using two robots.
OMRON's table tennis robot FORPHEUS, a symbol of the "harmony between human and machines", is a robot that consists of control devices such as cameras, sensors, controllers and industrial robots for factory automation, and realizes table tennis with human players. Its development began in 2013 and has evolved significantly over the last seven years. Initially, the robot could only continue the rallying, but in the latest 6th generation, OMRON's unique human measurement technology has evolved to the point where it can measure vital data such as the player's facial expressions, gaze, and hear rate, and select the course, pitch, and speed of the return ball according to the player's skills and emotions. It also equips "AI that boosts motivation" to literally motivate players.
This time, FORPHEUS took on the challenge of rallying with "doubles" for the first time. Its opponent was a mixed team of a human player and a "collaborative robot". This is truly a project where human and machines work together toward a one purpose. What kind of rally did it turn out to be?
A collaborative robot paired with a human player is called a Mobile Manipulator (MoMa). Actually, this is a combination of two robots: an arm-equipped collaborative robot is installed on top of a mobile robot whose sensors detect people and obstacles and can move freely by avoiding them. Conventionally, safety fences have been required to prevent the robots from hitting or injuring people while they work. However, the collaborative robot detects when it comes into contact with a person or an object and make a safety stop. It is also designed with a rounded shape to prevent injury when it comes into contact with people, allowing it to work safely with people and in the same space. MoMa is the culmination of OMRON's unique automation technology, which uses sensors to detect people and objects and develops high-level control of two robots while protecting human safety. Taking advantage of these features, a mixed team of MoMa and human was realized.
This is the first time MoMa, which usually works in a production floor, is challenging to play table tennis. Of course, compared to FORPHEUS, its movement, return range and speed are limited. However, in the demonstration of playing doubles, FORPHEUS showed great coordination, returning the ball to a human player and MoMa depending on their movements, with the human player took charge of the areas where MoMa could not return the ball. FORPHEUS, the player, and MoMa worked in a collaboration and realized a continuous rally.
The doubles, which embodied the three parties with different features playing different roles, complementing each other in their respective areas of expertise, and working together for an one purpose, truly symbolized a scene of the "harmony of human and machines" that OMRON aims for.
Not to mention, there were many technical challenges involved in realizing the human-robot doubles. Kyohei Asai of the Technology and Intellectual Property HQ, who was in charge of FORPHEUS, and Yasuaki Abe of the Product Business Division HQ of the Industrial Automation Company, who was in charge of MoMa, look back on those days and say as follows.
"The major issues were we had to make MoMa hold a racket, perform a motion equivalent to the snap of a human wrist, and vibration during swinging which happens because it is a mobile robot that its feet are not secured to the floor . We added a three-axis motor for the snap action and went through a lot of trial and error in order to combine the snap action with the arm action to make a series of swings without vibration." (Asai)
"After solving each issue, we had to program FORPHEUS and MoMa to function as one interlocking system, and our whole staff worked together in a tight schedule of two months to complete it." (Abe)
This advanced man-machine doubles was demonstrated at an exhibition in China (*). Since it is important to adjust the actual equipment on site to make FORPHEUS work, usually a team of about seven specialists needed to attend to carry out operations from on-site setup to management. This year, however, due to COVID-19 , only one person, Xiaojun Liu, from the Technology and Intellectual Property HQ of the FORPHEUS team, was present at the site. Liu provided on-site direction, while the Japanese team used the same type of FORPHEUS to ensure a successful start-up and doubles with technical support through "remote monitoring" that replicated the local conditions.
Although this was an unprecedented experiment in terms of content and management structure, Liu said, "I was motivated by the chance to show the latest FORPHEUS that I have been developing with my team members at the exhibition in my hometown, and I wanted to make the Chinese people surprise. The exhibition resulted in realizing playing doubles for the first time without any big troubles. The remote monitoring was quite useful for setting up the FORPHEUS in a limited amount of time. Demonstrating that we can provide rapid technical support without being confined to a single location is also a great leering experience. We saw the potential for using it not only for the exhibition, but also for remote support, such as for the start-up and maintenance of production facilities."
In the new normal era, robots that support human-machine interaction in a variety of scenes will become the norm. And as we saw in the table tennis doubles, humans and machines will develop a relationship to expand human abilities and possibilities with the support of machines, and by complementing each other in their respective fields of expertise, we will create an environment in which a diverse range of human resources can play an active role.
OMRON will continue to achieve further growth as an innovative company that uses its core technology of Sensing and Control + Think to solve social issues.
