IoT Real-Time Video Analysis in Industry
The escalating demand for real-time efficiency analysis in industrial settings is driving the need for smart solutions that blend Internet of Things (IoT) devices with Artificial Intelligence (AI). Notably, the majority of workstations in such environments are not stationary, necessitating compact and easily adaptable solutions. Our focus lies on the industrial environment, specifically on workstations, which are mobile in the factory. To address this, we have designed our solution around the widely popular, cost-effective, and compact device—the Raspberry Pi.
Why Raspberry Pi?
We chose to base our solution on Raspberry Pi for several compelling reasons. The Raspberry Pi is not only popular and economical but its small size and affordability make it a perfect fit for our use case. Coupled with its new operating system, Raspbian, Raspberry Pi comes bundled with Libcamera, a software specially crafted to support most common cameras that can be connected to this microcomputer. If you need more computing power to analyze the image, we recommend Jetson Nano.
The Utility of Libcamera
Libcamera, written in the highly efficient C++ language, provides extensive control over various camera parameters such as shutter speed, white balance, among others. This degree of control was critical for our solution, enabling us to tailor the camera's settings according to our specific requirements.
Integration of Machine Learning Model and Libcamera
To further refine our solution, we leveraged technologies like Machine Learning and communication protocols to send image parameters between Libcamera and the application which is using a trained Machine Learning model on Raspberry Pi. In our case, communication protocols was also useful to change internal Libcamera parameters from our web application in real-time.
The Role of a Custom Multiplexing Video Streamer and a Dedicated Android Application
A distinctive feature of our solution is the custom-designed Multiplexing Video Streamer on our server, tasked with grabbing the video stream from the Raspberry Pi and transmitting it to connected multiple Android devices through a dedicated application that we developed. Check RaspPlayer tutorial . These combined technologies make it possible to achieve low-latency video streaming and real-time data from each workstation in the factory.
The Challenge: OpenCV Integration with Libcamera
Despite the advantages of both Libcamera and OpenCV, we encountered a challenge—there was no ready-to-use version of Libcamera integrated with the OpenCV library. OpenCV, one of the most popular computer-vision library, is crucial for image-processing tasks in computer-vision applications.
Our Solution: Ready-To-Use template of Libcamera with integrated OpenCV
In light of this challenge, we have developed a ready-to-use template integrating Libcamera and OpenCV. Accompanied by a step-by-step tutorial, our template makes it simple for you to introduce your computer vision algorithm. In the tutorial, we will guide you on converting your Raspberry Pi into a computer vision powerhouse capable of identifying all black triangles in its field of view. Libcamera Apps Cv Tutorial
Conclusion
The power to analyze work efficiency in real-time can drastically enhance operations in dynamic industrial environments. Through our Raspberry Pi-based solution, leveraging Libcamera, OpenCV, Machine Learning, GRPC, and Custom Multiplexing Video Streamer, we aspire to provide a potent and accessible tool for real-time efficiency analysis.