AI Integration
New Feature: AI-Powered Configuration Data Classification
Overview
This new feature introduces an AI-powered system to automatically classify new configuration data within Shopfloor Control. By leveraging machine learning, the system can learn from existing data patterns to predict the appropriate classification for new entries, significantly improving data accuracy and administrative efficiency.
The feature is designed to reduce the manual effort required for data classification while maintaining high data quality. It does this by using a confidence score to determine if a classification can be automatically assigned or if it requires a manual review by an administrator.
Enabling the Feature
To activate the AI-Powered Configuration Data Classification feature, a system administrator must set the following configuration flag:
- Settings_ML__UseClassification: Set this value to True.
How It Works
The system operates in three main stages:
- Model Training: The system uses a machine learning model (ML.NET) that is trained on the ID and Name fields of your existing configuration data. This process allows the model to learn the patterns and associations between the data fields and their classifications.
- Prediction: When a new configuration data entry is created, the system uses the trained model to predict the most likely classification label. It also generates a confidence score for this prediction.
- Automatic vs. Manual Assignment:
- High Confidence: If the prediction's confidence score is above a predefined threshold, the system will automatically assign the predicted classification label. The user can see the assigned label and confidence score.
- Low Confidence: If the confidence score is below the threshold, the system will not assign a classification. Instead, it will prompt the user to manually select the classification label. This ensures that potentially inaccurate predictions are reviewed by a human administrator, preventing errors from being introduced into the system.
Understanding Model Performance Metrics
System administrators can monitor the performance of the machine learning model using a set of key metrics. These metrics provide insight into the model's accuracy and reliability, which is crucial for determining if the model needs to be retrained with new data.
|
Metric |
Description |
What to Look For |
|
Micro-Accuracy |
The fraction of correctly predicted instances across all classes. This is a good measure of overall model performance, especially in datasets with a class imbalance (where some categories have more data than others). |
A value closer to 1.00 indicates better performance. |
|
Macro-Accuracy |
The average of the accuracy for each individual class. This metric gives equal weight to every class, including minority classes, providing a fair assessment of the model's performance across all categories. |
A value closer to 1.00 indicates better performance. |
|
Log-Loss |
Measures the performance of the model's predicted probabilities. A low log-loss means the model's probability predictions are highly accurate. |
A value closer to 0.00 is ideal. A perfect model would have a log-loss of 0.00. |
|
Log-Loss Reduction |
The advantage of the model's predictions over a random guess. A value of 0.20, for example, means the model is 20% better at making a correct prediction than random chance. |
Values range from negative infinity to 1.00, with 1.00 being perfect. |
Maintaining Model Accuracy
To ensure the model remains accurate and effective over time, it is recommended to periodically retrain it with a significant amount of new, labeled configuration data. This allows the model to adapt to new patterns and maintain a high confidence level for its predictions. A scheduled retraining process is a best practice to ensure the system continues to provide reliable automatic classification.
Anomaly Detection for Production Metrics
Overview
This feature provides production managers with automatic notifications of anomalies in key production metrics. By using machine learning, the system can identify unusual patterns in real-time data, allowing managers to quickly pinpoint and address potential issues that could impact production efficiency. This proactive approach helps to minimize downtime and maintain a smooth workflow.
Enabling the Feature
To activate this feature, a system administrator must set the following flag in the Settings table of the SQL database:
Setting Name: Settings_ML__UseAnomalyDetection
Value: True
When this setting is enabled, the system will begin monitoring key production metrics for any unusual deviations from expected behavior.
How It Works
The anomaly detection system uses a machine learning model to analyze historical and real-time data for key production metrics (e.g., cycle time, piece rate, efficiency). The model learns what "normal" behavior looks like and flags any data points that fall outside of the expected range.
Once an anomaly is detected, the system generates a notification that is visible to the production manager. This notification provides a quick alert, allowing for immediate investigation and resolution of the potential issue, such as machine malfunction, a process bottleneck, or a data entry error.
Created with the Personal Edition of HelpNDoc: Streamline Your Documentation Creation with a Help Authoring Tool