Solving this problem is considerably more difficult than it first appears.
The traditional approach would be to collect data directly from robots, PLCs, SCADA systems or industrial control platforms. Unfortunately, that is often not practical. Manufacturing environments frequently contain:
- Multiple robot vendors
- Different controller generations
- Legacy equipment
- Restricted access to proprietary robot data
- Limited integration opportunities
- Security and governance constraints
Even when data is available, the information captured often describes controller states rather than actual physical behavior.
A robot controller may indicate a process step completed successfully, yet the physical movement may have slowed, paused, oscillated or deviated significantly from normal operation. This creates a fundamental visibility gap.
To gain deeper insight, a different form of data must be collected: external observation data.
Video provides a uniquely valuable source of information because it captures what is physically happening regardless of vendor, controller type or machine age. A camera can observe robot motion, workpiece movement, human interaction, material flow, equipment behavior and environmental conditions. However, video introduces its own challenges. Unlike traditional industrial analytics projects, there is usually very little labeled training data and no predefined anomaly catalog. There is also significant variation between "good" cycles, multiple product variants, changing lighting and environmental conditions, and continuous process evolution.
At project initiation, there is rarely an obvious machine learning model suited to the problem. This then becomes both an engineering and research issue: How can a system learn what normal operation looks like when normal itself contains substantial variability?