Development of AI-equipped Machine Automation Controller (1)
- Real-time anomaly detection
- Equipment maintenance
- Predictive maintenance
At the manufacturing site, the labor force is chronically short. In the future, productivity is expected to decline due to a decrease in skilled technicians, and there is a demand for labor saving. In view of these backgrounds, we are promoting labor saving by applying AI technology to predictive maintenance in FA field. However, in general, AI technology requires high-performance hardware, it is difficult to introduce it, and maintenance person at the manufacturing site cannot effectively use it.
We developed AI equipped machine automation controller which can realize predictive maintenance using AI technology only with conservation knowledge. In this paper we describe the implementation and verification results of machine automation controller.
The labor shortage in the Japanese manufacturing sector has been worsening over the recent years. The causes include an increasingly rapid decrease in the working-age population owing to a declining birth rate coupled with an aging population. In addition to the issue of a simple labor shortage, another problem is the serious shortage of next-generation labor, which seems too small to take the place of the skilled labor that has supported Japanese manufacturing industries for many years. It takes skills to detect and fix equipment anomalies or product defects at an early stage on the basis of personal experience and intuition. Efforts have been made to hand over such skills to the younger generation, but there has not been much success so far.
At the end of 2017, a staffing situation questionnaire survey was conducted for METIʼs White Paper on Manufacturing Industries1）. The White Paper observes that “...the proportion of respondents answering ʻnot particularly problematicʼ significantly decreased from approximately 19% to 6% on a year-on-year basis from the 2016 year-end survey; meanwhile, that of respondents answering ʻa problem serious enough to affect businessʼ sharply increased from approximately 23% to 32%. The issue of staffing has become more apparent, posing a serious challenge.”
To fight back, manufacturing companies are trying to make manufacturing equipment more labor- and manpower-saving. Manufacturers, large or small, are accelerating to introduce robots and mechanization for purposes such as increased safety, accuracy, and quality stability, as well as reduced labor.
Notably, following the recent developments in science and technology, artificial intelligence (hereafter “AI”) technologies are being put to practical use for the predictive maintenance2） of manufacturing equipment to achieve enhanced labor- and manpower-saving effects3）-8）.
In general, AI technologies face the following problems:
- a) AI technologies involve complicated computational processes and require high-performance CPUs.
- b) AI technologies are difficult to introduce.
We developed an AI technology implementable in controllers and reported the achievements in our preceding papers10）,11）. A preceding paper presented a report on the development of a prototype of our NY5 Series machine automation controller; this prototype was equipped with a machine-learning outlier detection algorithm called Isolation Forest (hereafter “I-Forest”)9）. In addition, we performed a test on I-Forest for its processing time to evaluate its installability in machine automation controllers. Using a packing machine as an example, we demonstrated that “anomalies” in a machine can be detected in real time by the Outlier Detection function installed in a machine automation controller.
AI technologies generally require professional statistical skills, coupled with knowledge specifically required for the field of application. Hence, their introduction is considered difficult. We established product specifications that limit the scope of the application of AI technologies to the predictive maintenance of manufacturing equipment, so that our AI technology can be introduced more easily for use by shop-floor maintenance personnel.
This time, on the basis of the aforementioned technology, we worked on the product development of our AI-equipped machine automation controllers (hereafter “AI Controllers”). Ahead of the rest of the industry, we implemented our AI technology in our NX7 Series and NY5 Series machine automation controllers to install predictive maintenance to practical use.
In our previous technology development10）,11）, we did not examine whether the system equipped with a combination of manufacturing equipment control and our AI technology can be operated in a real control environment. Hence, in our product developments, we must give consideration to the method of ensuring compatibility between manufacturing equipment control and our AI technology in machine automation controllers. What follows presents the design and time behaviors12） required for such compatibility.
2. Targets for this technology
Our purposes are as follows: establishing AI-equipped system that a maintenance personnel can actually operate; achieving implementability of both equipment-control and predictivemaintenance functions in machine automation controllers; and running a verification test on these functions.
2.2 Recognized challenges
The following are the challenges to be addressed to make control processes and AI processes mutually compatible in a machine automation controller:
i）The structure that realizes predictive maintenance established by maintenance personnel
To enable maintenance personnel to establish predictivemaintenance, it takes more than an Outlier Detection function for an AI Controller. The additional necessities are functions provided by: AI Support Tools for configuring the Outlier Detection function and monitoring the equipment condition; and a database for storing real-time data of the manufacturing equipment. We must develop specifications for these functions constituting the AI Controller.
ii）Compatibility of manufacturing equipment control with the predictive-maintenance function
Proper CPU resource allocation must be ensured to implement both manufacturing equipment control processes and AI-based predictive maintenance processes in a single machine automation controller.
Our machine automation controllers guarantee the synchronization of devices, such as sensors and/or servo motors, with a control program to provide accurate control of manufacturing equipment running at high speeds. The AI Controller must retain this advantage and make it compatible with our AI technology.
From a shop-floor perspective, it is not desirable to readjust existing, configured manufacturing equipment for reasons arising from the AI functions of the AI Controller when replacing it with the machine automation controller installed in the equipment. Therefore, it is desirable that the AI functions not affect manufacturing equipment control or that a function be provided to predict likely impacts on manufacturing equipment control.
2.3 Technical challenges
From the above-recognized challenges, the technical challenges for our current product development are set as follows:
i）Development of a structure that realizes predictive maintenance established by maintenance personnel
ii）Verification of the compatibility of manufacturing equipment control with the predictive-maintenance function
3. Technical details
3.1 Development of a structure that realizes predictive maintenance established by maintenance personnel
Generally, the following activities are considered necessary to introduce our AI technology to the predictive maintenance of manufacturing equipment:
- Data collection: collecting data indicative of the condition of the manufacturing equipment from its sensors and/or servo motors and passing these data to Activity 2 “Data analysis.”
- Data analysis: generating rules from the data obtained from Activity 1 to configure the control device(s) according to the rules.
- Data utilization: monitoring shop-floor manufacturing equipment for the purpose of its predictive maintenance by maintenance personnel.
Fig. 1 shows a typical AI Controller configuration that allows these activities. The AI Controller is equipped with an Outlier Detection function that realizes predictive maintenance. Table 1 shows the functions installed in an AI Controller. We collectively refer to the Time Series Database, Feature Extraction, and Outlier Detection functions as the AI functions. Table 2 shows the Windows-compatible support tools for configuring the AI functions. These support tools are called AI Support Tools.
|Time series database||Records variable values, feature quantities, and machine-learning calculation results. Runs on control cycle.|
|Feature Extraction||Determines feature quantities from variable values. Runs on takt cycle.|
|Outlier Detection||Calculates outliers from feature quantities and learning data. The outlier detection algorithm uses I-Forest. Runs on takt cycle.|
|AI Easy Modeler||Creates learning data based on variable values obtained from the Time Series Database.|
|AI Operator/AI Viewer||Configures the AI functions installed in the machine automation controller. Displays outlier detection results on the Windows screen.|
Details are given below on how the AI functions relate to Activity 3 “Data utilization,” the one most relevant to maintenance personnel. The AI Controller collects and uses numerical values from the sensors and/or servo motors on manufacturing equipment to perform outlier detection. The data flow for outlier detection is as follows:
- 1. The AI Controller collects data from the sensors and/or servo motors on the manufacturing equipment.
- 2. The AI Controller determines feature quantities from the values collected in Step 1.
- 3. The AI Controller performs outlier detection, using the feature quantities determined in Step 2 and the learning data obtained by “data analysis.”
- 4. The outlier detection calculation results are written out to AI Viewer.
The maintenance personnel use the calculation results obtained in Step 4 for manufacturing equipment maintenance activities.
3.2 Verification of the compatibility of manufacturing equipment control with the predictive-maintenance function
Details are given below on how to make manufacturing equipment control and our AI technology mutually compatible in a machine automation controller.
Our machine automation controllers have more than one CPU core and assign functions to each of the CPU cores to manage processes therein. This allows processes to take place with the control program in sync with the control device(s).
CPU Core 1 performs high-priority processes, such as I/O refreshing, motion control, and control program runs.
CPU Core 2 performs low-priority processes, such as tag data link, FTP communication, and file operations.
The AI functions in Table 1 are divided according to the above CPU core allocation method and are assigned to the CPU cores as shown in Table 3.
|Function||CPU Core 1||CPU Core 2|
|Time Series Database||Collects and packages variable values.
Packages variable values.
|Writes packaged variable values to storage.|
|Feature Extraction||Calculates feature quantities in sync with the control program.||None applicable|
Hands over feature quantities to CPU Core 2.
Writes I-Forest’s calculation results to variables.
|Calculates outliers from feature quantities.|
Considering the full use of the values of devices, such as sensors and/or servo motors, we assigned the Feature Extraction function to CPU Core 1.
Meanwhile, with consideration for reducing the impact on the control program, we assigned the following to CPU Core 2: processes performed by the Time Series Database to save data in the storage; and time-consuming processes, such as I-Forestʼs outlier detection calculations.
4. Effectiveness verification
4.1 Development of a structure that realizes predictive maintenance established by maintenance personnel
The flow line likely to be followed by maintenance personnel, equipment designers, and our service engineers is presented below on the basis of the structure developed in Chapter 3.
1. “Data collection” activities by equipment designers or our service engineers
The equipment designers or our service engineers operate AI Operator and configure the Time Series Database function. Here, on the basis of their knowledge about the equipment, they select and record variables likely to be relevant to outlier detection.
Among the maintenance personnel, those familiar with the equipment are allowed to operate AI Operator to select and record necessary variables.
2. “Data analysis” activities by our service engineers
Our service engineers operate AI Easy Modeler to create learning data for the Outlier Detection function from the data obtained in Activity 1 “Data collection.” Then, they use AI Operator to transfer the learning data to the machine automation controller to set the Outlier Detection function.
3. “Data utilization” activities by maintenance personnel
The maintenance personnel use AI Viewer to monitor the results of calculations performed by the Outlier Detection function. When any outlier is detected, they take necessary actions including maintenance of the culprit part.
By following the above flow line consisting of Activities “1. Data collection,” “2. Data analysis,” and “3. Data utilization,” we verified that our newly developed technology provides users with functions that suit their needs. We consider with pride that this technology provides a solution to Problem b) “AI technologies are difficult to introduce.”
4.2 Verification of the compatibility of manufacturing equipment control with the predictive-maintenance function
Using our NX7 Series, we performed a two-step evaluation test on the AI functions for their time behaviors, with the first step for function-by-function measurement and the second step for evaluation using a performance evaluation model simulating real manufacturing equipment.
i) Scope of measurement
From Reference 3）, it is known that CPU Core 2ʼs processing time per outlier detection in the equipment under measurement is as short as 0.4 ms. The product specifications newly developed here require that up to 128 outlier detections be performed at 200-ms takt cycles. It follows from Reference 3） that the time required for 128 outlier detections will be 0.4 ms×128 occurrences=51.2 ms. We consider that 51.2 ms is a time sufficiently short for a window of 200 ms.
Thus, CPU Core 2ʼs processing time is unlikely to pose a problem within the product specifications for the AI Controller. Hence, CPU Core 1ʼs processing time is measured on a function-by-function basis.
ii) Measurement conditions
The details are given below on the AI function-related factors likely to affect CPU Core 1ʼs processing time. The processing time required by CPU Core 1 for the AI functions depends on the following: the number of variables recordable in the Time Series Database; the type of statistical quantities used in feature extraction; the number of feature quantities used for outlier detection; and the number of outlier detections.
Table 4 shows the settings used for this measurement. The number of feature quantities per outlier detection is 16, which is the upper limit specified by the product specifications. The number of variables recordable by the Time Series Database is fixed at 512, regardless of the number of outlier detections. From among these 512 variables, 16 variables are selected and used for feature extraction.
The values obtained from real manufacturing equipment would differ from those in Table 4. To make it simple, in this paper, the number of variables recordable in the Time Series Database and the type and number of feature quantities are fixed.
CPU Core 1 usage time trends can be investigated by changing the number of outlier detections in the range from 1 to 128 under the conditions above. During function-by-function measurement, time of CPU Core 1 allocated to the following functions are measured individually: Time Series Database, Feature Extraction, and Outlier Detection.
|Number of outlier detections||n（1≦n≦128）|
|Number of feature quantities per outlier detection||16|
|Type of statistic quantities used for feature extraction||Kurtosis|
|Number of variables recorded by Time Series Database||512|
iii) Function-by-function measurement results
CPU 1 usage time can be approximated by the following equation:
CPU 1 usage time=(number of feature quantities per outlier detection×feature extraction time+outlier detection time)×number of outlier detections＋α
where the outlier detection time is the time used for data transfer to CPU Core 2, as shown in Table 3, excluding I-Forestʼs outlier detection calculation time; and the constant α is the overhead of each function. With these settings, CPU 1 usage time increases in proportion to the number of outlier detections.
Table 5 shows the breakdown of CPU Core 1 usage times allocated to each AI function. Regardless of the number of outlier detections, CPU Core 1 usage time allocated to the Time Series Database is fixed at 62 μs. This is because the number of variables recordable by the Database is fixed at 512 (Table 4).
Table 5 shows that the Feature Extraction function is the dominant function over CPU Core 1 usage time. The Feature Extraction function is processed in CPU Core 1 as follows:
- 1. Variable values are recorded at control cycles in the firmwareʼs built-in workspace.
- 2. On the basis of the trigger detected by the Outlier Detection function, variable values recorded in Step 1 are referenced to determine the feature quantities to be used for outlier detection.
- 3. The determined feature quantities are saved in the firmwareʼs built-in variables.
The number of feature quantities to be calculated is fixed at 16 per outlier detection. Hence, the above Steps 1 to 3 are repeated 2,048 times, respectively, when the number of outlier detections is 128. Owing to this time-consuming repetitive process, the Feature Extraction function accounts for much of CPU Core 1 usage time.
Thus, the outlier detection time is short probably because of a low operational load on CPU Core 1. As shown in Table 3, Outlier Detection function-related processes are assigned to either CPU Core 1 or 2. Outlier detection at CPU Core 1 is dealt with by exchanging variable-values with CPU Core 2.
Note that outlier detection calculations that require considerable processing time are assigned to CPU Core 2. This results in a short outlier detection time in CPU Core 1.
The above information allows equipment designers to factor into equipment design the AI function-induced load on the CPU.
|No. of outlier detections||Time series DB(μs)||Feature extraction(μs)||Outlier detection(μs)|
iv) Conditions for verification using the performance evaluation model
The compatibility of the AI functions with manufacturing equipment control is shown below on the basis of our performance evaluation model.
The performance evaluation model we designed can perform a total of 128 outlier detections, which include one in servo motor data and 127 in virtual device data. Tables 6 and 7 show the outlier detection settings. The term “takt cycle” means the repetitive operation cycle of manufacturing equipment. This cycle is synchronized with the cycle of outlier calculations based on feature quantities. The control cycle is set to 2 ms.
|Variables to be monitored||
Torque value, and
Speed command value
of servo motor
|Feature quantities to be used||
The following 4 types of feature quantities are
determined from the variables being
|Variables to be monitored||4 sign wave variables|
|Feature quantities to be used||
The following 4 types of feature quantities are
determined from the variables being
v) Results of verification using the performance evaluation model
Table 8 shows the measurement results for a machine automation controller equipped with the AI functions and a conventional one not equipped with AI functions. The impact of the implemented AI functions on the control cycle becomes greatest when the AI function-induced load is highest. Hence, a comparison should be made between the measurements for “Max” run time in Table 8. The measured values for the versions with and without AI functions are 1,256 μs and 1,081 μs, respectively. Thus, there is a difference of 175 μs between the two versions. This result is in line with the results shown in Table 5 and hence is considered to provide a satisfactory verification. The impact of the implemented AI functions on the control cycle is small, as is clear from the fact that the value of 175 μs is approximately 16% of the control cycle of the version without the AI functions.
Thus, the technology newly developed this time can reduce the impact of the implemented AI functions on the control cycle. Accordingly, an equipment designer can introduce the AI functions without making significant changes to the hardware of manufacturing equipment, its software configuration, or its control system. This means that our AI functions collectively serve as an effective solution to Problem “a) AI technologies involve complicated computational processes and require highperformance CPUs.”
|Item||Run time (μs)|
|AI functions installed||
|AI functions not installed||
This paper addressed the following: i) Development of a structure that realizes predictive maintenance established by maintenance personnel; and ii) Verification of the compatibility of manufacturing equipment control with a predictivemaintenance function.
Our efforts resulted in the development of the AI Controller and AI Support Tools as AI technological tools that shop-floor maintenance personnel can operate predictive maintenance. We presented a summary of the activities for establishing the specifications for our AI technology for shop-floor application. Thus, we demonstrated that our AI technology is operable by maintenance personnel. Additionally, we examined the loads on the CPU and demonstrated the compatibility of manufacturing equipment control with an outlier detection function to verify the practicability of the predictive maintenance of manufacturing equipment.
In the future, we intend to explore the applicability of our AI technology for more than predictive maintenance and will consider developing a manufacturing equipment control system that uses the results of calculations performed by our AI technological tools.
Lastly, we would like to thank related parties for their cooperation in this development project.
- Ministry of Economy. “White Paper on Manufacturing Industries (Monodzukuri) 2018”. http://www.meti.go.jp/report/whitepaper/mono/2018/honbun_pdf/pdf/honbun01_01_01.pdf, (accessed 2018-11-06).
- JIS Z 8141:2001. 2018-11-06.
- Omron Corp. “Omuron Kusatsu factory example”. http://www.fa.omron.co.jp/solution/sysmac/technology/bigdata/index.html, (accessed 2018-03-01).
- IBM Japan, Ltd. “Predictive Maintenance (PQM) Predicts/prevents asset failures and improves utilization and yield”. https://www-01.ibm.com/software/jp/info/predictivemaintenance/, (accessed 2018-03-01).
- Nippon Electric Co., Ltd. “Invariant analysis that detects “abnormality” and prevents failures and abnormalities”. http://jpn.nec.com/ai/analyze/invariant.html, (accessed 2018-03-01).
- NTT DATA Corporation. “Edge computing initiatives in the IoT & AI era”. http://www.nttdata.com/jp/ja/insights/blog/20170316.html, (accessed 2018-03-29).
- Ministry of Economy. “White Paper on Manufacturing Industries(Monodzukuri) 2017”. http://www.meti.go.jp/report/whitepaper/mono/2017/honbun_pdf/pdf/honbun01_01_02.pdf, (accessed 2018-4-10).
- Nippon Telegraph and Telephone Company. “Formulated an “Edge Computing Concept” to promote the development of new applications that require high response and big data processing”. http://www.ntt.co.jp/news2014/1401/140123a.html, (accessed 2018-03-05).
- Liu, F. T.; Ting, K. M.; Zhou, Z.-H. Isolation-based anomaly detection. ACM Transactions on Knowledge Discovery from Data (TKDD). 2012, Vol.6, No.1, p.3.
- Tsuruta, K.; Minemoto, T.; Hirohashi, Y.; Development of AI Technology for Machine Automation Controller (1). OMRON TECHNICS. 2018, Vol.50, No.1, p.6-11.
- Abe, Y.; Ueyama, Y.; Sakatani, N.; Fujii, T. Development of AI Technology for Machine Automation Controller (2). OMRON TECHNICS. 2018, Vol.50, No.2, p.12-18.
- ISO/IEC 9126.
The names of products in the text may be trademarks of each company.