"IoT x Deep Learning" for Consistent Connection
from Sensing to Action

"Deep Learning" is one of the effective approaches on machine learning, and exhibits its strength when a system "automatically" extracts features such as specified colors and shapes from a massive amount of data consisting of voice, images and other elements. In the past, features have been extracted manually based on a trial-and-error with the help of the knowledge provided by experts in respective fields. Utilization of deep learning allows extraction, classification and inference entirely automatically through the learning features themselves with high accuracy. AI technology has grown phenomenally by deep learning.

Toshiba has also undertaken the development of its AI technology for many years and has been a pioneer in applying this technology into services for industrial IoT. One representative example of such services is Toshiba Communication AI "RECAIUS" that supports human activities by recognizing human´s intentions and situations through complex information such as voice and images. "RECAIUS" is now widely used in various scenes of human life and society.

We are now aiming at expanding the use of deep learning for the industrial IoT region to improve the "speed" and "accuracy" in data analysis utilizing experiences and knowledge cultivated with Toshiba group company. In addition to voice and image data, IoT data corrected by many kinds of sensors which are installed in industrial equipment and products are also analyzed through "IoT x Deep Learning" system which is based on learning, inference and action. We desires to improve the quality of customer products and services, as well as the business operations of its customers, in the hope of contributing to solving social challenges.

Both "Quantity" and "Quality" of Data are
also important factors for resolving critical issues

The data and technology that are required for deep learning in the industrial region differ greatly from those used in the AI services that use data in websites and SNS (Social Networking Service). In general, the data used in deep learning in these services is the very large volume of data on the internet. The approach behind these AI services is to first collect a volume of data as large as possible to improve the accuracy of inference.

Compared with this, what is analyzed in the industrial region is diverse data that is constantly collected in time series from actual equipment, machine and so on through sensors and cameras. Deep learning is used in the industrial region for the optimization of manufacturing lines in the factory, for stable operations of energy and transportation systems, and for system shutdowns in an emergency. In these applications, reliability and urgency will be the critical issues. This means that the both "Quantity" and "Quality" of data are critical requirements in deep learning. The critical elements that are very important in deep learning will be identifying truly necessary data and collecting high-quality and accuracy reliable data. Additionally, there are many issues such as how we can ensure a timely response in an emergency situation and how we can build a database which meets the requirements for individual business fields of customers in energy, transportation, manufacturing and other fields.

Toshiba is undertaking the development of technologies to improve both speed and accuracy for use in deep learning in the industrial IoT region. (See Fig. 1) The values that can be provided to the customers in various business segments through deep learning could be grouped into three categories, namely, "identification," "prediction/inference" and "control." Toshiba has developed innovative technologies to deliver new value for our customer such as neural network parameter optimization, neural network model compression for fast inference process, data augmentation in case of insufficient learning data, distributed parallel learning system to handle tremendous learning data and so on. Toshiba is providing integrated deep learning platform using these advanced technologies, and accumulate experiences and knowledge with customers including Toshiba group companies.

Application Examples of Deep Learning
in the Industrial Region

Toshiba semiconductor factory at Yokkaichi Operations is manufacturing NAND type flash memories and has started to use deep learning to improve their yield and productivity. The deep learning system automatically classifies 300,000 SEM (Scanning Electron Microscope) images per day and more than 200,000 defect data of wafer map images per month with high accuracy from 20 billion daily data group collected from the manufacturing equipment and inspection systems in the factory. It is expected to achieve significant results that will lead to a higher yield and productivity, reducing to one third of the time to infer from wafer map image data those causes that will become defects.

Deep learning is also greatly contributing to our new IoT business creation. A representative example of new IoT business is the monitoring and inspecting system for electric power infrastructure using a drone system collaborated with Alpine Electronics, Inc. and Toshiba. Alpine Electronics, Inc. is in charge of drone navigation control, while Toshiba is in charge of image recognition technology. The image recognition technology that uses deep learning is adopted to find the damage on the power transmission lines. In the past, these inspections have been conducted through visual observations using a helicopter. The new system allows detection of places with a damaged part quickly with high accuracy by examining images photographed by the drone. In the initial stage, there was a problem with this system that images for deep learning required to correctly find anomalies was not enough. Toshiba used deep learning technology to resolve this problem. Deep learning technology automatically created additional abnormal images based on the many normal images and a few abnormal images.

For other application examples, Toshiba start to utilize the deep learning technology to behavior prediction of workers in warehouse through wearable devices and to power generation forecast for solar power system. With the expectation for industrial applications, visualization for sport scenes where the data analytics is highly important, has been undertaken. For example, players and the ball are recognized only from video images of rugby play through image recognition and deep learning without using GPS (Global Positioning System) sensors or other sensing devices attached with players. The movements and motions of players in both teams are recognized. A series of complex plays such as tackles and scrums are recognized for each scene and video data is tagged. These will allow the formulation of a game strategy, as well as suitable play coaching, using video clips during a game. Our current plan calls for future application of this system to production lines to visualize the motions and movements of many workers and things with accuracy, to optimize lines of flow and to develop plans for efficient human resource and assets allocations. The potentials of deep learning are bound to expand.

Leading the Era of "IoT x Deep Learning"
by State-of-the-Art Technology and Strengths of Toshiba

High speed and high-accuracy processing of large volumes of voice, image and sensor data to ensure connection to swift actions will be critical in the future when deep learning is implemented in the industrial region. Toshiba is using its advanced edge computing technology to address these challenges. We are developing "Collaborative Distributed Deep Learning technology". ´Learning process´ which require high performance processing, is conducted on the cloud and ´Inference process´ which require real time processing, is conducted in the edge. (See Fig. 2) This technology enables high accurate algorism and high speed action in the industrial IoT region.

The following three technologies have been established to allow more customers to use deep learning simply with higher accuracy.

(1) Automatic Deep Learning Environment

Toshiba provides environment which enables automated execution of a series of processes from coding, learning and evaluation of the effect f by preparing only data with enough volume and quality for deep learning. Various problems could be resolved such as repeated coding and learning, evaluation of results and optimization of the network configuration. This environment allows the customers to start deep learning smoothly.
* The automatic deep learning environment and the deep learning technology using small volume of learning data are explained in #02.

(2) Parallel Distributed Learning Technology

It takes a very long time to learn from a large volume of data using only one computer. Instead, learning is executed in parallel using plural computers. The overall learning time is shortened by dispersing the loads to the CPUs and GPUs. The status of learning among individual computers is shared and controlled to enhance the learning accuracy.
* Parallel distributed learning technology is described in detail in #03.

(3) Neural Network Optimization Technology

Neural networks that support deep learning are generally built manually by experts who fully utilize their knowledge of deep learning. Toshiba succeeded in automating this process. Various parameters such as the numbers of nodes and layers are automatically searched and algorithms are optimized autonomously and efficiently.

Bringing together these state-of-the-art technologies, Dell EMC, a Dell Technologies Group company, and Toshiba have jointly proposed a platform for verifying the usefulness of deep learning in the IoT field to the Industrial Internet Consortium (IIC). As a result of this proposal, the IIC has approved this platform as the first testbed for deep learning. At present, this platform is installed at the Smart Community Center (LAZONA Kawasaki Toshiba Building in Kawasaki, Kanagawa Prefecture) for verification of analysis of data collected from various sensors in building facilities by deep learning for optimal control and management of facilities and equipment.
Toshiba is additionally tackling the task of ensuring high-reliability security from the edge to the cloud to support the safe use of deep learning.

The 3^rd AI boom is much published in the world. However, in the industrial region, AI is still at the dawn of it. Toshiba Industrial ICT Solutions Company will continuingly develop new technologies and conduct verification experiments to solve problems that confront transformation* undertaken by the customers.

* transformation: The concept of an environment totally surrounding business models, business processes, services and products and other aspects related to enterprises and their customers who undertake a reform through the technology.

Related articlesVol.20: Artificial Intelligence Discovering Values of IoT Data -Analytics towards Deep Learning-

March 13, 2017

Providing a Framework for Automatic Learning Platform Processes from Data Augmentation to Learning are Automated
Challenge: Higher Accuracy Achieved with Shorter Time and Reduced Cost

March 17, 2017

Starting Verification at Smart Community Center New Technical Approach of Parallel Distributed Learning to Practically Apply Deep Learning to IoT

March 24, 2017

Overview and Application Examples of Network Optimization Technology Self-Growing Neural Networks Expand Data Analysis Fields