The Raspberry Pi ecosystem simply bought a large improve with the discharge of the brand new Raspberry Pi AI Digicam, designed in collaboration with Sony.
This isn’t your run-of-the-mill digicam module that simply snaps photos or data video, it’s constructed to carry out AI duties proper on the digicam itself, liberating up the Raspberry Pi’s CPU and GPU to do different work.
Whether or not you are a tech fanatic or an expert developer, this new digicam guarantees to alter the sport for edge AI and laptop imaginative and prescient initiatives.
However is the Raspberry Pi AI Digicam actually price all of the hype?
Let’s dig into the small print and discover out what makes this tiny piece of {hardware} so particular.
What is the hype?
On the coronary heart of the Raspberry Pi AI Digicam is the Sony IMX500 sensor, a 12.3-megapixel beast that doesn’t simply seize pixels it ‘processes’ them.
In contrast to conventional Raspberry Pi digicam modules that depend on the Pi’s CPU or an exterior AI accelerator for picture evaluation, this digicam does all of it onboard.
It options 8MB of devoted reminiscence that’s used solely for operating AI fashions straight on the sensor, permitting it to deal with advanced duties like real-time object detection, picture segmentation, and pose estimation.
Why does this matter? As a result of it helps you to run laptop imaginative and prescient duties with out hogging your Raspberry Pi’s assets.
Whereas the Pi is busy doing different issues like operating your robotic, managing your sensible dwelling, or streaming information to the cloud, the AI digicam handles the heavy lifting of visible evaluation within the background.
Raspberry Pi AI Digicam Specs
Here is what you are working with if you get your arms on this digicam:
Sensor: Sony IMX500 (12.3 megapixels)Decision: 4056×3040 at 10 fps, or 2028×1520 at 40 fpsLens: Guide focus, 4.74mm focal lengthField of View: 66° horizontal, 52.3° verticalShutter: Rolling shutter (watch out for quick movement artifacts)AI Processing Decision: 640×640 (max for AI fashions)Output Codecs: RAW10, Bayer, YUV, RGBMemory: 8MB for AI mannequin storage and processingSize: 25×24×11.9 mmPrice: $70
The AI Digicam’s guide focus is one among its few drawbacks. Whereas we’ve gotten used to the motorized autofocus discovered within the Raspberry Pi Digicam Module 3, right here we’re again to manually adjusting the main target.
It’s not a dealbreaker, nevertheless it’s actually not a function you’d anticipate in a $70 digicam in 2024.
Additionally, it has a rolling shutter, that means fast-moving objects would possibly produce some visible distortion.
What it could do?
Because of the onboard RP2040 microcontroller and the devoted reminiscence, the AI Digicam can run laptop imaginative and prescient fashions straight on the sensor.
Meaning no extra putting in exterior accelerators like Google’s Coral or counting on the Raspberry Pi’s CPU to course of AI fashions every thing occurs on the digicam itself.
For starters, you’ve bought two built-in AI fashions:
And if you wish to transcend these fashions, Raspberry Pi has offered a “mannequin zoo” on GitHub with different choices like YOLOv8n (for high-speed object detection) and ResNet18 (for picture classification).
The method is seamless: you join the AI Digicam to your Pi’s CSI interface, load the AI mannequin into the digicam’s 8MB of reminiscence, and run your Python script.
As soon as the mannequin is loaded, it stays there, always processing photographs in real-time, and spitting out metadata or overlaying it on the video feed with out burning by way of your Pi’s CPU cycles.
Challenge concepts for Raspberry Pi AI digicam
The Raspberry Pi AI Digicam isn’t only a software for capturing moments; it’s a platform for constructing clever programs.
Listed here are some challenge concepts if you happen to’re enthusiastic about this new digicam:
AI-Enhanced Sensible Doorbell
Construct a sensible doorbell that makes use of real-time object detection to distinguish between people, animals, and packages. Ship alerts primarily based on the article detected, and by no means miss a supply once more!
For those who’re on the lookout for some inspiration, take a look at this wonderful information by Tom’s {Hardware}, the place Ryder constructed a sensible doorbell utilizing a Raspberry Pi.
It is a implausible challenge that exhibits you how you can create a person-detecting doorbell system with a little bit of machine studying.
Sensible Child Monitor
With its real-time object detection and pose estimation capabilities, you might create a sensible child monitor that alerts you when the newborn is transferring, waking up, or if one thing uncommon is going on—giving dad and mom peace of thoughts.
For those who’re considering of constructing one your self, make sure to take a look at How you can Make a Sensible Child Monitor, the place Ryan walks you thru how he constructed his personal sensible child monitor.
Interactive Gesture Management for Sensible Properties
Utilizing the PoseNet mannequin, you possibly can create gesture-controlled sensible units.
Think about controlling your lights or thermostat with a easy wave or particular hand gesture, no voice assistants required.
In case you are severe about this, you possibly can see within the picture beneath, the system is performing real-time a number of hand recognition with finger segmentation.
This setup lets you observe hand poses and even use particular gestures to set off instructions.
For those who’d prefer to study extra, take a look at the complete article by Core Electronics.
Wildlife Surveillance
The AI Digicam can act as a sensible wildlife monitor, mechanically recognizing particular animals and logging their exercise.
As an alternative of wading by way of hours of footage, the digicam’s AI capabilities can pinpoint when your goal animal seems.
Within the beneath picture you possibly can see this wonderful Movement-Detecting Wildlife Digicam setup that depends on the Raspberry Pi Digicam Module V2.1 and takes full benefit of the Pi Zero W’s built-in Wi-Fi & Bluetooth to remotely monitor wildlife.
I couldn’t entry the official challenge web site which I consider has gone down, however you possibly can nonetheless see the marvel that this construct grew to become within the article printed by PetaPixel.
Edge AI for Robotics
In robotics, imaginative and prescient is crucial. Equip your robotic with the AI Digicam for duties like object avoidance, face recognition, and even superior object manipulation, all while not having an exterior AI accelerator.
One standout challenge on this area is ZeroBot, featured in a Hackaday article. ZeroBot is a Raspberry Pi Zero W-based robotic that may be managed utilizing any laptop or smartphone by way of an online browser.
I believe its modern and environment friendly, requiring no elaborate exterior setup to get began, although upgrading to a extra highly effective Raspberry Pi might improve its picture processing capabilities.
Safety and Security Monitoring
Deploy the AI Digicam to observe areas for security hazards, like figuring out if a machine is left on, a software is misplaced, or even when somebody has fallen, making it a beneficial software for factories, hospitals, and houses alike.
Take a look at this challenge printed on Hackster.io that demonstrates how you can arrange a sensible safety digicam utilizing a Raspberry Pi Zero W and digicam module.
Outfitted with object detection powered by OpenCV, the digicam can provide you with a warning by way of e mail each time it detects an intruder. Plus, it streams dwell video, permitting you to observe your property in actual time, even if you’re away.
My Ideas
As somebody who’s been following the evolution of Raspberry Pi’s ecosystem for some time, I’m actually excited concerning the Raspberry Pi AI Digicam.
The concept that you may run real-time laptop imaginative and prescient duties straight on the digicam with out taxing your Pi’s CPU is spectacular.
A price contemplating level is the facility consumption. At round 1.87W beneath load, it’s not a deal breaker for many initiatives, however if you happen to’re considering of utilizing this in a conveyable, battery-powered setup, you will must plan rigorously.
Ultimately, the Raspberry Pi AI Digicam looks like a specialised software for a distinct segment viewers. For those who’re simply after a plug-n-play digicam answer for informal use, save your self the additional price and go for the Digicam Module 3.
However if you happen to’re diving deep into machine studying and want real-time picture processing with out additional {hardware}, this AI Digicam is a stable selection.
For those who’d prefer to know how you can set it up and use it along with your Pi, do tell us within the feedback beneath, and we’ll make sure to cowl it within the subsequent article!