Active Learning: Integrating Natural Intelligence Into Artificial Intelligence

Today, very few people would likely deny the fact that data can present major added value for companies. But analyzing data from production processes reveals the incompleteness of data collection and the associated reduced potential of the data that can be leveraged. Typical shortcomings include:

• Incomplete representation of processes in the dataspace,
• Inadequate connection of processes and domains in the dataspace,
• Lacking quality assurance during data collection, and
• Insufficient or obsolete interfaces for manual data entry.

The latter especially represents an obstacle not to be underestimated. Often, important process knowledge only exists implicitly in the heads of a few experts, who are only available for limited periods of time. If these people are not available at critical moments, eliminating sudden issues within processes and ensuring process reliability and stability requires much more effort. Over the long term, losing people with relevant knowledge presents significant risks for companies. Companies have various solutions for securing knowledge and sharing it with a wide audience. However, generally, knowledge is lost during transfer, and often no checks are in place to ensure the appropriate knowledge has been transferred, as there is only limited contact between those conveying and those absorbing the information.

This is where using artificial intelligence appears quite appealing. The key here is to avoid using this technology as a stand-alone application, but rather to effectively combine natural and artificial intelligence. Expert systems can be used to achieve this goal. However, it should not be underestimated how much effort it takes to integrate data and formalize natural knowledge. One often seen approach is the absolutely necessary involvement of process experts in developing and interpreting machine learning models (ML models). The specialist knowledge of the process experts is incorporated into the development of features, the selection of methods, as well as the evaluation and interpretation of the output of the ML model. This joint approach of knowledge carriers from process, data, and AI domains is required to successfully develop and implement artificial intelligence.

But this approach brings with it two challenges. On the one hand, in many cases the necessary data is not available during initial development of an ML model due to incomplete data collection. On the other, the involvement of process experts often ends when development of the ML model is completed. The first challenge can be overcome by adapting the data collection process. However, this causes previously collected data to remain incomplete, meaning its full potential cannot be harnessed. This is where the process experts come back into play, as they may be able to provide the missing information based on their implicit knowledge. But if comprehensive data pools need to be examined and missing information must be added (labeling) by process experts at a later time, it causes bottlenecks in the availability of these people, preventing them from effectively performing their regular labeling assignment as well as other day-to-day and project tasks.

Active learning can be applied to ensure effective use of the limited availability of process experts as well as their long-term involvement in the continual improvement process of an existing ML model. The first phase of this approach for training ML models is based on an intelligent selection of a fraction of the available data. This subset of the underlying data is then analyzed and evaluated by the experts using an interface tailored specifically to the application and the information to be collected. The collected information is used to train an ML model, which then attempts to assign the missing information (the label) for data that has not been evaluated yet. This process is not designed to measure the success of the model, but rather its uncertainty. This uncertainty when assigning missing labels forms the foundation for selecting additional subsets of the data pool, which is then once again analyzed and labeled by the process experts.

During initial training of an ML model, the process experts are provided with a minimum volume of data for analysis that promises maximum benefits with regard to developing the ML model. As for monitoring and maintaining ML models used in production, active learning can play a decisive role in assuring the quality of decisions in the ML model. By assessing the uncertainty with which the ML model makes a decision, process experts can be involved automatically to identify any wrong decisions early on and prevent future wrong decisions by correcting the decision of the ML model.