Build a Pi NVR with AI Object Detection

Video Analysis with Raspberry Pi and Frigate

A Raspberry Pi, some accessories, open-source software, and an afternoon of tinkering are all you need to build a network video recorder (NVR) with on-device object detection. If you want to monitor your property with cameras but avoid being locked into a proprietary system (such as Ubiquiti’s Unifi Protect), you can assemble your own self-contained solution. Many network-attached storage (NAS) providers like Synology and QNAP, as well as specialised vendors like Genius Vision, offer products with integrated object recognition—but they tend to be costly, especially with multiple cameras.

An alternative is Frigate, an open-source project by Blake Blackshear and supported by a growing community. Frigate integrates seamlessly with the popular Home Assistant platform but also works independently. When combined with a Raspberry Pi and a hardware tensor processing unit (TPU) accelerator, Frigate can deliver reliable object detection locally—no cloud needed.

Overview

Project Info

• Raspberry Pi acting as an NVR server with AI-based object detection
• Linux and networking basics required
• Approx. 2–4 hours setup time
• Estimated cost: around €140–150, depending on parts

Suggested Components

• Raspberry Pi 5 with 4 GB RAM
• Official Raspberry Pi 5 power supply
• Raspberry Pi 5 Active Cooler (recommended)
• Pineboards Hat AI! Bundle (including M.2 Coral Edge TPU)
• MicroSD card (64 GB)
• (Optional) Pineboards universal transparent case

Frigate runs as a Docker container and uses OpenCV and TensorFlow frameworks for object detection. With GPU or TPU support, it performs detection at high speed. The software displays multiple IP cameras on a tidy web interface and manages clip recording and retention. By default, it processes RTSP (Real-Time Streaming Protocol) feeds from cameras by manufacturers like Annke or Ubiquiti. Some cameras from TP-Link and Ring also speak RTSP, though untested here. If a camera outputs only MJPEG or still images, Frigate can use FFmpeg parameters to convert them—be aware, however, that transcoding is CPU-intensive. Frigate is licensed under the permissive MIT license.

In tests, we used Ubiquiti G3, Ubiquiti G4 Bullet, and Annke C800 RTSP streams, plus a Raspberry Pi Zero W-based camera feeding MJPEG. For best results, connect all IP cameras and the Raspberry Pi to the same (wired or wireless) network.

Hardware Choice

NVR with AI
Our server is built around a Raspberry Pi 5 with 4 GB RAM and an expansion board that accommodates an M.2 TPU chip. This TPU chip is connected via PCIe on the Pi 5. The “Hat AI!” bundle from Pineboards includes both the expansion board and the M.2 TPU module. While a Raspberry Pi 4 is feasible, it lacks a built-in PCIe slot, so you’d be forced to rely on a Coral USB accelerator (currently ~€75). We opted for the Pi 5 for a more compact, robust design—no dangling USB TPU needed.

Additionally, we purchased the official Raspberry Pi 5 PSU, the Pi 5 Active Cooler, and a decent 64 GB microSD card. A passive heatsink is also possible if you want a silent setup, but the processor will be under near-constant load when processing video. Active cooling is recommended.

Installing the Software

1. Raspberry Pi OS
   • Flash the 64-bit Raspberry Pi OS Lite onto the microSD card (using Raspberry Pi Imager, for instance).
   • Set up Wi-Fi credentials and SSH access via the “Advanced” settings in the Imager.
   • Insert the card, power on the Pi, and verify that you can SSH in. If all is well, power it back down.
2. Mounting the AI “Hat”
   • Connect the thin copper ribbon from the “Hat AI!” kit to the expansion board. The side labelled “Hat” faces the matching connector.
   • Insert the Coral TPU board into the M.2 slot, using the brass standoff and screws to secure it.
   • Fit the four standoffs to the expansion board, then attach the expansion board to the Pi’s PCIe connector with the ribbon cable. Ensure the cable is not twisted or misaligned.
   • If you’re installing the active cooler, follow the included instructions and plug the fan into the Pi’s four-pin header.
3. System Tweaks
   • Boot the Pi again, log in via SSH, and run: sudo apt update && sudo apt upgrade
   • Edit /boot/firmware/config.txt to add the following: gpu_mem=512 dtparam=pciex1 kernel=kernel8.img dtoverlay=pineboards-hat-ai
   • Reboot. The expansion board is now recognised, but you still need drivers for the Coral TPU. Add Google’s Coral repository: echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list sudo curl https://packages.cloud.google.com/apt/doc/apt-key.gpg -o /etc/apt/trusted.gpg.d/coral.asc sudo apt update
   • Install libraries and build tools for the kernel modules: sudo apt install cmake libedgetpu1-std devscripts debhelper dkms dh-dkms
   • Download and compile the Gasket drivers from GitHub: git clone https://github.com/google/gasket-driver.git cd gasket-driver/ sudo debuild -us -uc -tc -b cd .. sudo dpkg -i gasket-dkms_1.0-18_all.deb
   • Add a udev rule and create the system user and group for the TPU: sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\",GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules" sudo groupadd apex sudo adduser apex
   • Confirm the kernel module is loaded by running sudo lspci -v. You should see Global Unichip Corp. Coral Edge TPU with Kernel driver in use: apex. Also check with lsmod | grep apex.

Docker and Frigate

Frigate runs in a Docker container using hardware acceleration for the TPU. Make sure Docker is installed:

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

File Layout

1. In your home directory (or another folder), create: mkdir -p ~/frigate/docker mkdir ~/frigate/config mkdir ~/frigate/storage
2. In ~/frigate/docker, create a file named docker-compose.yml, referencing the official example from the Frigate repository or a local copy. Adjust paths if needed.
3. The shm_size parameter depends on camera resolution and quantity. Each camera needs a shared-memory buffer for decoding.

version: "3.7"

services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true
    image: blakeblackshear/frigate:stable
    shm_size: "512m"  # Adjust as needed
    volumes:
      - ../config:/config
      - ../storage:/media/frigate
    ports:
      - "8971:443"
    # ...

Configuration File
You also need a Frigate config file (e.g. config.yml) in ~/frigate/config. Provide your camera details (RTSP URLs, roles for detect vs. record). If you’re using the Pi’s camera or an MJPEG-only source, use appropriate FFmpeg parameters.

First Start

1. From ~/frigate/docker, run: sudo docker compose up
2. Watch logs for an admin password. Log in at https://<raspi-ip>:8971/ (accept the self-signed cert).
3. Press Ctrl+C to stop, then: sudo docker compose up -d to start Frigate in the background.

Recording and Object Detection

Frigate’s interface is straightforward: a left sidebar for live camera views, recorded “events,” and general settings. By default, it only displays the live feed. To record and detect movements, edit the config to include:

record:
  enabled: True
  retain:
    days: 0
    mode: all
  events:
    retain:
      default: 30
      mode: motion

Now Frigate stores event clips for 30 days. Each time you change the config, click “Save & Restart” in the top-right corner. Once the service restarts, any motion from your camera triggers an alert, and the matching clip is saved. For various animals, vehicles, etc., you can selectively log or ignore them via object labels if you get too many false alarms.

Multiple Cameras

Add additional cameras by copying the relevant YAML blocks. For example:

cameras:
  living_room:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123/stream1
          roles:
            - detect
            - record
    # ...

Make sure you handle indentation carefully, and consider using a lower-resolution sub-stream for detection to reduce CPU load. The main high-quality stream can be used purely for recording events.

Performance

On a Raspberry Pi 5, Frigate can handle multiple HD camera streams reliably, especially with hardware acceleration via the Coral TPU. If microSD space is limited, you can redirect recordings to a network share by adjusting your Docker volumes. In repeated tests with up to four HD cameras (Ubiquiti G3, G4 Bullet, Annke C800), the system remained stable for weeks.

Future Possibilities

Once stable, you can expand functionality:

• PTZ Support: Frigate can control ONVIF-compliant motorised cameras (pan, tilt, zoom).
• Audio-Triggered Recording: Optionally start recordings via audio events.
• Alerts: Send notifications (e.g., via Telegram) on new events.
• Face Recognition: Add a plug-in for face recognition.

Frigate seamlessly integrates with Home Assistant, allowing motion or object-detection events to trigger other smart-home actions, such as turning on lights. The developer also offers a paid “Frigate+” service—€50 per year—that supports custom training of up to 12 AI models using your own labelled images. The required images are uploaded to Frigate’s servers for training, which might be worthwhile if you need advanced or highly specialised detection scenarios.