Case study · RK3588 edge AI + on-device LLM
A Camera That Thinks Offline
CNN on the RK3588 NPU (ONNX → RKNN) with RAG + TinyLLM on RKLLM.
Edge AI on a Rockchip RK3588 single-board computer, a freelance project in 2026. A convolutional neural network (CNN) classifies camera images on the RK3588's NPU after ONNX-to-RKNN conversion, and a retrieval-augmented generation (RAG) pipeline with a TinyLLM on RKLLM generates context-aware voice and text responses, entirely on-device, with no cloud.
- Org
- Freelance
- Year
- 2026
- Status
- Prototype
- Channels
- CH3CH1
The problem
Vision and language models usually need the cloud. The aim: classify images and explain the result entirely on a small device, offline.
What I built
- A CNN deployed on the RK3588 NPU via ONNX → RKNN conversion for hardware-accelerated inference
- A RAG pipeline with a TinyLLM on RKLLM that generates context-aware voice and text responses fully on-device
Stack
Rockchip RK3588 · CNN · ONNX · RKNN · RKLLM · RAG
Results
- Runs fully on-device, with no cloud calls
- Freelance project (2026)
The vision model is converted from ONNX to RKNN for hardware-accelerated inference on the NPU. For language, RKLLM runs a small language model locally, and the RAG pipeline grounds its answers in local context, so the device works without an internet connection.
I work across NVIDIA Jetson boards (Nano, Orin), the Rockchip family (RK3588, RK3576, RK3566) and Raspberry Pi for on-device AI, embedded Linux and computer vision.
Gallery
Straight from the archive: every photo I could dig up for this build.
