Boosting Team Performance with a Machine that Connects People:Young Engineers’ Efforts toward Realizing “Harmony between Humans and Machines”

​​​​​To address the multitude of social issues humankind faces today, OMRON has come up with the concept of “harmony between humans and machines,” which envisions the ideal relationship between humans and machines in which machines enhance and maximize human potential, creativity, and ability. A perfect embodiment of “harmony” between humans and machines is demonstrated by FORPHEUS, a table tennis robot that leverages OMRON’s core technology of “Sensing & Control + THINK.” Making a sensational debut at an exhibition OMRON organized overseas in 2013, FORPHEUS continues to evolve with each passing year with input from an R&D team led by young engineers from the Technology & Intellectual Property HQ.

The current seventh-generation FORPHEUS was unveiled at the International Robot Exhibition 2022 (iREX2022) which took place in March 2022. Team members took on many challenges in developing the latest model, and their efforts include the addition of new features inspired by the adverse situation brought about by the COVOD-19 pandemic. We’re thrilled to share some behind-the-scenes stories about the development of the seventh-generation FORPHEUS and the members’ aspirations for the project’s further evolution.

＜Members＞

Members of the Seventh-Generation FORPHEUS Development Team：

	Areas of Expertise
Xiaojun Liu (far right)	Robot control, mechanism design
Takanori Kurisu (far left)	Self-Diagnosis and Fault-Tolerant system , interaction design
Yoshiki Ito (second from left)	Robot control, safety technology
Ryo Mizuyama (second from right)	Interaction design, sensing technology

Members of the Next-Generation FORPHEUS Development Team：

Leader: Yoshitaka Oshio (center)
Members: Yoshiki Ito, Ryo Mizuyama

Realizing the Concept of “Connecting Humans to Bring Out the Potential and Creativity of a Team”

I was told that the seventh-generation FORPHEUS can engage in a table tennis rally with two players on a doubles team. What is the objective of this new function, and what specifically can it do?

Mizuyama: For starters, we had this concept of “harmony between humans and machines,” which not only enhances human potential and creativity, but also brings out innate human abilities. A generation ago, it was machine versus human, but our team decided to focus on “harmony” in the context of a relationship between two or more people. Because of the disruptions caused by the pandemic, we felt that person-to-person communication had become rather superficial, which made it difficult to build a relationship with others. Then we began to wonder if we could perhaps create a “machine that connects people” through “harmony between humans and machines.” This led us to develop the seventh-generation FORPHEUS under the concept of “enhancing performance of a team,” that is, a pair of doubles table tennis players playing with FORPHEUS. To help improve the performance of the team, we added new functions that focus on sharing of emotions and coordination of actions between players. During a rally, FORPHEUS analyzes images of the players’ faces to capture their facial expressions and heart rates and estimates the level of cooperation between the two, namely, how synchronized their emotional responses are, based on the correlation between changes in such parameters. FORPHEUS also assesses the level of their cooperation based on their movements and the tempo of the rally. With FORPHEUS taking the lead in enhancing these parameters, table tennis players can improve their performance as a doubles team.

More Adept at Returning the Ball

You have made major improvements to the composition of the seventh-generation FORPHEUS. What were they exactly?

Liu: Sure. We went back to the drawing board to redo FORPHEUS’s configuration from scratch so that it could respond to a wider range of shots and deliver returns with greater positional accuracy and speed. What makes FORPHEUS unique is that it is composed of a combination of general-purpose equipment that you can find in factories and elsewhere, such as industrial robots and image sensors. For previous models, we used a three-axis*1parallel robot to move the racket. This latest generation FORPHEUS, however, uses a four-axis*2 model to expand the range of motion and improve the robot’s response time. The end result is that FORPHEUS is now capable of responding to a ball that passes just above the net and the range it can cover is 1.5 times larger. We also upgraded drive motors in its elbow joint and wrist joint that manipulate the racket so that it can hit the ball with more powerful spin.
Using the four-axis parallel robot for better manipulation of the racket meant that synchronizing axes was of crucial importance because of its mechanical and structural characteristics, and controlling the robot both accurately and quickly has become far more challenging. Therefore, we redesigned a control algorithm that works on the programmable logic controller (PLC) and is developed exclusively to support table tennis players’ movements. We believe that this improved control algorithm could also be used, say, to speed up the motion generation by industrial robots that move while avoiding obstacles.

*1 Hornet 565 Parallel Robots/Features | OMRON Industrial Automation
*2 iX4 Parallel Robots EtherCAT (NJ501-R) version/Features | OMRON Industrial Automation

Ito: We also wanted FORPHEUS to play a more technical rally with advanced players. What we did was increase the speed of its return by improving how the parallel robot moves. In the past, what FORPHEUS did was to move the racket to the point where it predicted the ball would come and to deliver the return using its elbow and wrist on the arm holding the racket. To deliver a faster return, the seventh-generation model synchronizes its move to the predicted position of the ball with the swing of the racket. This is what us humans do. We can hit the ball faster with greater accuracy if we swing the racket using our entire body rather than just our arms.

Adding a Brand-New Feature for a Trade Show during the Pandemic

I heard that, other than being able to play with a doubles team and having better abilities to return the ball, the seventh-generation FORPHEUS comes with a brand-new feature. What is this brand-new feature?

Kurisu: It’s an autonomous adjustment function which allows FORPHEUS to self-diagnose how well it plays based on whether or not it was able to return the ball where intended. It then shares the data on the status of its cameras and motors with its operator. When the accuracy of the location to which the ball is returned, the robot adjusts the ball-returning movement as necessary to keep the performance. Before, we had to take a break at a regular interval during the day for maintenance. At the iREX2022 this March, where the seventh-generation model gave the public its first glimpse, we were able to finish the four-day show without any downtime for maintenance for the first time.

Liu: For autonomous adjustment, we attached a camera at the tip of the SCARA robot. It tracks the racket and constantly monitors the condition of the racket’s rubber surface while FORPHEUS is engaged in a rally. This allows FORPHEUS to adapt to potential changes of the surface of the racket or tabletop and deliver a return ball accordingly. This SCARA robot also tosses a ball for FORPHEUS to serve.

What made you to decide to add the autonomous adjustment function?

Kurisu: It’s because we wanted to showcase FORPHEUS at a physical trade show held during the pandemic. We have showcased FORPHEUS before at trade shows outside of Japan, including the U.S., Germany, and China. Last year, we put the sixth-generation FORPHEUS on display at a trade show in China, but, due to the pandemic, none of the development team members was allowed to visit the country, which drew much of our attention to the maintenance of the robot. Our solution was to add the self-diagnosis function, so that local operators would know when to do the maintenance. It allows FORPHEUS to detect for itself when its performance lowers and let the operators know when the time comes. For the seventh-generation model, we added this autonomous adjustment function, which, by detecting changes in the racket surface, automatically adjusts the angle and height of the racket. This helps to reduce the frequency of regular maintenance work, such as the cleaning of the racket and applying wax to it.

So, the development team gave yet another interpretation of “harmony between humans and machines,” that is, connecting people to enhance team performance. What responses did you get from the visitors to the show?

Mizuyama: People showed interest in the idea of having FORPHEUS sense the levels of cooperation and empathy and the associated concept of “connecting individuals to enhance team performance.” We were confident that our human-centered concept of “harmony between humans and machines” resonated with them.

Liu: To our great regret, we were not able to let the general public experience a rally with FORPHEUS, for the sake of preventing the further spread of COVID-19. If only they’d had the opportunity, I’m sure they would have seen that FORPHEUS spontaneously supports a rally to connect one individual with another. Based on the responses of the visitors who watched as the development team members put on a demonstration, the members of the general public in attendance were pleasantly surprised to see the accuracy and speed with which the updated FORPHEUS returned the ball when playing with a doubles team.

Another Journey in Pursuit of a New Form of “Harmony”

Someone told me that a new team was formed this past April to work on the concept of the next-generation FORPHEUS. Could you tell me what challenges you foresee as you continue evolving this amazing table tennis robot?

Ito: I think that, with this seventh-generation model, we have reached a certain goal in terms of technology and performance. That raised the bar we are required to clear, which puts us under tremendous pressure. Still, we do have high hopes for what message we can deliver with regards to the relationship between humans and machines. We definitely want to present a concept that transcends any conventional frameworks, so we can make people say, “Wow! What an evolutionary change it has undergone!”

Mizuyama: I presume that, in the future, robots will be intervening in human-to-human communication, just as naturally as we breathe, rather than making their presence felt more. By “naturally,” I mean machines move as we wish without us being conscious of it. At present, things are quite the opposite, and in many cases, humans have to work in tune with machines to obtain efficiency. I believe that this approach should work when we get down to solving specific issues. According to the SINIC Theory*3, however, we have left behind the Industrialized Society, where we pursued material affluence, and then the Optimization Society, where we resolved issues that arose as a result of the Industrialized Society. We are now entering the Autonomous Society, where humans play the leading role, are allowed to be themselves, and demonstrate their potential in pursuit of spiritual affluence. The goal we should be aiming for is to create a robot not only for the sake of increasing efficiency, but also spontaneously supports humans in leading a happy life. We want to make FORPHEUS a robot that gives us a glimpse of such a world of tomorrow.

*3 SINIC Theory | OMRON Principles | OMRON Global

Oshio: I have taken leadership in the FORPHEUS development team since this spring. I have never been involved in the development of this innovative robot, and I have made it a motto not to follow the policy from the previous year. Through the development of generations of FORPHEUS, we have gained a wealth of technology and knowledge. At the same time, such an intangible asset sets the bar higher for the development team that follows, making it difficult to create novel ideas to further evolve FORPHEUS. I may be mistaken, but the presence of the seventh-generation FORPHEUS itself could be too strong to serve as an embodiment of “harmony between humans and machines.” This is not just about a table tennis robot, but it would be exciting if we can realize a world where robots make meaningful contributions in very natural, seamless ways. I’m hoping to build a foundation for generations of FORPHEUS-like robots as I envision how “harmony between humans and machines” should be aligned with the future we wish to create in our ongoing long-term vision, “Shaping the Future 2030*4.”

*4 Shaping the Future 2030 | OMRON Global

Recommended reading:

FORPHEUS | Technology | OMRON Global
Ping-Pong Robot to Control Motivation of a Human Player | OMRON TECHNICS | Technology | OMRON Global