The tiny, ubiquitous, dollar-store champion of always-on AI. If your model is small and your power budget is measured in milliwatts, start here.

The ESP32-S3 is not, in the strict sense, an “AI board” at all — it is a general-purpose Wi-Fi and Bluetooth microcontroller that happens to be very good at running the smallest class of machine-learning models. That framing matters. You do not buy an ESP32-S3 to run a large vision network at high frame rates; you buy it because it costs a few dollars, fits on a coin-sized module, runs from a battery for months, and can still execute a quantized neural network on the same chip that reads your sensors and talks to the cloud. This is the world of TinyML: models measured in kilobytes, doing one focused job extremely efficiently.

Architecture & AI acceleration

At its heart sits a dual-core Xtensa LX7 processor running at up to 240 MHz, paired with 512 KB of on-chip SRAM and, on most modules, several megabytes of external PSRAM and flash. Crucially, the LX7 core includes vector (SIMD) instruction extensions that Espressif’s software libraries exploit to accelerate the multiply-accumulate operations at the core of neural networks. There is no dedicated NPU or GPU — the “acceleration” is these vector instructions plus hand-optimized kernels. That is enough to run keyword spotting, wake-word detection (“Hey device…”), simple gesture and motion classification, vibration and anomaly detection on sensor streams, and even small convolutional networks for low-resolution person or face detection on the camera-equipped ESP32-S3-EYE development board.

Because there is no accelerator to feed with high-precision data, everything runs as INT8 quantized models, and the models must be tiny — typically well under a megabyte, often just tens of kilobytes. The two software paths are Google’s TensorFlow Lite for Microcontrollers (TFLite-Micro) and Espressif’s own ESP-DL / ESP-NN libraries, which provide optimized INT8 kernels for the S3’s vector unit. You train and quantize on your PC, convert to a C array, compile it into your firmware, and flash the whole thing to the chip.

Getting started, in practice

1. Install the ESP-IDF toolchain (or the Arduino core) and confirm you can flash a blink sketch to the board.
2. Train a small classifier on your PC — for example a keyword-spotting model — and quantize it to INT8 TensorFlow Lite.
3. Convert the .tflite file into a C byte-array and drop it into your firmware project.
4. Link against TFLite-Micro or ESP-DL, wire up the input (mic via I2S, or camera on the S3-EYE), and run the interpreter in your main loop.
5. Flash, watch the serial monitor for predictions, then tune quantization and buffer sizes until it fits in SRAM and hits your latency target.

A purpose-built inference machine. When your job is a fixed vision model and every milliwatt counts, the Edge TPU is a scalpel.

The Coral Dev Board answers a very specific question: how do you run a fixed, well-defined vision model as efficiently as physically possible? Google’s answer was to design a dedicated Edge TPU — an application-specific integrated circuit (ASIC) that does one thing, the linear-algebra that underpins neural networks, and does it at roughly 2 TOPS per watt. The result is a small single-board computer that can run a model like MobileNet v2 at close to 400 frames per second while drawing about 2 watts for the accelerator. That efficiency is the entire pitch.

Architecture & AI acceleration

The board is built around a removable system-on-module (SOM) based on the NXP i.MX 8M — a quad-core Arm Cortex-A53 with a Cortex-M4 companion, 1 or 4 GB of LPDDR4, eMMC storage, and on-board Wi-Fi and Bluetooth. It runs Mendel Linux, a Debian derivative, so you get a familiar Linux userland with Python and the PyCoral library. The star is the Edge TPU coprocessor soldered onto the SOM. Because the SOM is detachable and available separately, the Coral platform is explicitly designed to let you prototype on the Dev Board and then move the same module into your own custom hardware for production — a genuinely useful path to manufacturing.

The Edge TPU’s power is also its constraint. It runs only INT8-quantized TensorFlow Lite models, and only after they are passed through the Edge TPU Compiler, which maps supported operations onto the ASIC. Operations the compiler does not support fall back to the CPU, which quickly erases the speed advantage. In practice this means Coral excels at the well-trodden vision architectures — MobileNet, EfficientNet-Lite, SSD detectors, pose estimators — and is a poor fit for exotic layers, transformers, or anything you cannot cleanly quantize to INT8. You choose Coral when your model is a known, quantizable convolutional network, not when you want to experiment with the latest research.

Not a board on its own, but the accelerator that supercharges one. It bolts a data-center-class vision NPU onto a host you already own — and sips power doing it.

The Hailo-8 AI Kit is different in kind from the other entries here: it is not a complete computer but an M.2 accelerator module built around Hailo’s 26-TOPS Hailo-8 processor, designed to plug into a host such as a Raspberry Pi 5 or any Linux machine with an M.2 or PCIe slot. What makes it remarkable is the efficiency. It delivers its full 26 TOPS at roughly 2.5 watts typical — on the order of 10 TOPS per watt, several times more efficient than the general-purpose boards and comfortably ahead of the Coral Edge TPU, while offering far more headroom. If your problem is high-throughput or multi-stream vision and power matters, this is the standout on the whole spectrum.

Architecture & AI acceleration

The Hailo-8 uses a proprietary dataflow architecture: rather than shuttling data back and forth to external memory, the compiler lays the neural network out spatially across the chip, so activations flow from layer to layer on-chip. A practical consequence is that the module needs no external DRAM of its own — a single-chip solution that keeps the board small and the power low. It connects over PCIe Gen 3 in a standard M.2 form factor (keys M, B+M and A+E, 2242 with breakable extensions), and is rated for the industrial −40 °C to 85 °C range, which is why it shows up in real products and not just hobby projects.

Because it is an accelerator, you pair it with a host that runs the operating system, drives the camera, and does pre- and post-processing; the Hailo handles inference. Models come in through the Hailo Dataflow Compiler, which ingests TensorFlow, TensorFlow Lite, ONNX, Keras or PyTorch and emits a compiled HEF file that the HailoRT runtime executes. Like the other fixed-function accelerators it is happiest with INT8-quantized convolutional vision models, and it is scalable — a single device can run multiple streams and multiple models at once, and you can even fit several Hailo-8s to a host. This is the exact silicon that powers the 26-TOPS versions of the Raspberry Pi AI HAT+, so choosing “the Hailo-8” and “a Pi 5 with the 26-TOPS HAT” are two names for much the same capability.

The default first board for a reason. A full Linux computer that scales from “no accelerator” to “40 TOPS” by bolting on a HAT.

If you are learning edge AI, or you want maximum flexibility with minimum friction, the Raspberry Pi 5 is the pragmatic starting point. On its own it is a capable quad-core Linux desktop-in-miniature with no dedicated AI accelerator — you can run TensorFlow Lite, ONNX, or PyTorch models directly on its CPU, but only slowly, which is fine for light or intermittent workloads. What transforms it into a serious inference platform is the AI HAT+ family: add-on boards carrying a Hailo neural processing unit that connect over the Pi 5’s PCIe Gen 3 interface and take on the heavy lifting.

Architecture & AI acceleration

The Pi 5 pairs a Broadcom BCM2712 — quad-core Arm Cortex-A76 at 2.4 GHz — with 2, 4, 8, or 16 GB of LPDDR4X and the full Raspberry Pi OS Debian environment. That means a real Linux userland: apt packages, Python, containers, a desktop, the vast Pi community, and native camera frameworks. The AI acceleration is modular. The AI HAT+ ships in a 13 TOPS version (Hailo-8L) and a 26 TOPS version (Hailo-8), and the newer AI HAT+ 2 offers 40 TOPS on a Hailo-10H with its own 8 GB of memory, explicitly aimed at running larger vision-language and small language models on the edge. The accelerator is auto-detected by the OS, and the built-in rpicam-apps and Picamera2 frameworks can offload compatible post-processing — object detection, segmentation, pose estimation — to the NPU automatically.

The Hailo toolchain follows the familiar pattern: you train in TensorFlow or PyTorch, export to ONNX or TFLite, then use the Hailo Dataflow Compiler to quantize and compile the model into a Hailo Executable Format (.hef) file that runs on the NPU. Hailo publishes a model zoo of pre-compiled networks so you can be running real-time detection within an hour of unboxing. The combination is potent: a friendly, well-documented, endlessly supported computer for the parts of your project that are not AI, plus a genuinely fast NPU for the parts that are.

Prices are approximate and vary by region and retailer. The beauty of the platform is that you can start CPU-only and add a HAT later.

Getting started, in practice

1. Flash Raspberry Pi OS to a microSD or NVMe drive and boot the Pi 5; get a camera working with Picamera2.
2. Fit the AI HAT+ over the PCIe connector; the OS auto-detects the Hailo NPU on next boot.
3. Run a pre-compiled model from the Hailo model zoo to confirm real-time detection end to end.
4. When ready for your own model, train and export to ONNX, then compile to a .hef with the Hailo Dataflow Compiler.
5. Wire the model into rpicam-apps or your own Python app, and iterate on accuracy and frame rate.

The richest software stack in edge AI, and enough raw compute to run transformers, vision-language models, and small LLMs on the device itself.

When your ambitions outgrow fixed convolutional vision — when you want to run modern transformer architectures, vision-language models, or a small large-language model locally — the Jetson Orin Nano is the board that says yes. NVIDIA’s late-2024 “Super” update was unusually generous: an existing 40 TOPS developer kit was boosted to 67 TOPS purely through a software power-mode change, and the price was cut to $249. Owners of the original kit unlocked the full performance by flashing the latest JetPack — no new hardware required. That combination of raw programmable compute, a mature ecosystem, and an aggressive price is what makes the Orin Nano the default choice for edge generative AI and robotics.

Architecture & AI acceleration

The Orin Nano module pairs a 6-core Arm Cortex-A78AE CPU with a 1024-core NVIDIA Ampere GPU that includes 32 dedicated Tensor Cores, backed by 8 GB of LPDDR5 at 102 GB/s of bandwidth in Super mode. Unlike a fixed-function ASIC, the GPU is fully programmable, so it runs essentially any model architecture you can express in a mainstream framework — CNNs, transformers, diffusion models, LLMs — without being limited to a curated operation set. This is the single biggest differentiator: the Jetson does not force you to fit a narrow mold.

The software story is the other half of the value. JetPack provides an Ubuntu-based Linux environment with the full CUDA, cuDNN, and TensorRT stack. You develop in PyTorch or TensorFlow, then optimize with TensorRT — which quantizes (INT8, or FP16 for accuracy-sensitive work), fuses layers, and tunes kernels for the specific GPU — to squeeze out maximum throughput. NVIDIA’s container registry and the Jetson AI Lab supply ready-to-run examples for LLMs, vision-language models, and robotics, so you can stand up a local chatbot or a multi-camera perception pipeline quickly. Because the same CUDA programming model spans the cloud, the workstation, and the edge, skills and code transfer cleanly across all three — a strategic advantage no other board here can match.

Getting started, in practice

1. Flash the JetPack SD-card image (or use SDK Manager) and boot into the Ubuntu desktop.
2. Set the power mode to MAXN / Super to unlock the full 67 TOPS.
3. Pull a ready-made container from the Jetson AI Lab — a local LLM or a detection pipeline — to verify the GPU end to end.
4. Bring your own PyTorch/ONNX model and optimize it with TensorRT (INT8 or FP16) for maximum throughput.
5. Integrate into your application (often with DeepStream for multi-camera vision or a robotics framework), then tune power mode versus performance.

Reconfigurable hardware for people who need deterministic, low-latency, multi-camera vision — and a clean path to industrial production.

The Kria KV260 is the odd one out, and deliberately so. The other four boards run your model on a fixed processor — a CPU, an ASIC, a GPU. The Kria lets you build the datapath itself in reconfigurable logic. At its core is AMD’s Zynq UltraScale+ MPSoC, which combines a conventional Arm processor with a field-programmable gate array (FPGA). On that fabric you instantiate a Deep Learning Processing Unit (DPU) — a configurable inference engine — and, because the pixels can flow from camera to preprocessing to DPU to output entirely inside the hardware, you get something the other boards cannot promise: deterministic, cycle-predictable latency and outstanding performance-per-watt on fixed pipelines. That is exactly what industrial machine vision, smart cameras, and safety-relevant control loops need.

Architecture & AI acceleration

The KV260 pairs the production-grade K26 system-on-module — a quad-core Arm Cortex-A53, dual Cortex-R5F real-time cores, the FPGA programmable logic, a hardened video codec, and 4 GB of DDR4 — with a vision-focused carrier card. That carrier is the point: it exposes multiple camera interfaces (up to eight, via an onsemi IAS MIPI connector with auto-focus after the 2025 refresh, plus a Raspberry Pi camera connector), a dedicated hardware image signal processor (the onsemi AP1302) that offloads debayering and HDR, HDMI and DisplayPort output, gigabit Ethernet, USB, and Pmod expansion. A TPM 2.0 module and secure boot round out a package built for trusted, deployable edge products.

AMD’s Vitis AI environment is how you get a model onto the DPU: you quantize a TensorFlow or PyTorch model to INT8, then compile it to an xmodel targeted at your DPU configuration. AMD promises that, thanks to a library of pre-built accelerated applications and the Kria App Store, you can be running vision applications in under an hour with no FPGA experience — a real change from the historically steep FPGA learning curve. The deeper power, though, is that FPGA engineers can customize everything: instantiate multiple DPUs, add custom preprocessing in logic, or build a bespoke pipeline that a fixed processor could never match on latency or efficiency. And because the K26 SOM is a production part with a multi-year lifecycle, the KV260 is a genuine prototype-to-volume path, not just an evaluation toy.

Getting started, in practice

1. Download the Kria starter SD-card image, flash it, and boot the KV260 to Linux.
2. Run a pre-built accelerated application (e.g. a smart-camera detection demo) from AMD’s library to see the DPU working end to end.
3. Install Vitis AI; quantize your TensorFlow/PyTorch model to INT8 and compile it to an xmodel for the target DPU configuration.
4. Deploy the model into the pipeline, connecting your camera(s) through the IAS or Raspberry Pi interface.
5. If you need more, move into Vitis/Vivado to customize the DPU count or add hardware preprocessing — then migrate to the K26 SOM for production.

Seven mistakes that trip up first-time deployers

Choosing the board is half the battle; the other half is not sabotaging yourself with avoidable errors. These are the traps that catch nearly everyone the first time.

1 · Trusting the TOPS number too much

Marketing leads with peak TOPS because it is the biggest, most impressive figure. But peak TOPS assumes perfect utilization that real models never achieve. Actual throughput depends on your specific architecture, how well it maps to the accelerator, memory bandwidth, and preprocessing overhead. A board rated at 26 TOPS may deliver only a fraction of that on your model. Always benchmark your network, not a vendor’s cherry-picked one.

2 · Forgetting that memory, not compute, is often the wall

People obsess over TOPS and ignore RAM. A model must fit in memory alongside its activation buffers and your application. This is why the ESP32-S3’s 512 KB is such a hard ceiling, and why the Jetson’s 8 GB caps the size of LLM you can run. Check the memory footprint of your quantized model before you fall in love with a board’s compute figure.

3 · Skipping quantization — or doing it carelessly

Most edge accelerators require INT8 models, and even where FP16 is possible, quantization is what unlocks real speed. But naive quantization can crater accuracy. Budget time for quantization-aware training or a good post-training calibration dataset, and always measure accuracy after quantizing, not before. A fast model that gives wrong answers is worthless.

4 · Assuming any model runs on any accelerator

Fixed-function accelerators like the Edge TPU and, to a lesser degree, the Hailo NPU and Kria DPU support a specific set of operations. If your model uses an unsupported layer, it either fails to compile or silently falls back to the CPU, destroying performance. Check the supported-operation list before you design your model, and prefer well-trodden architectures on fixed-function hardware. The GPU-based Jetson is the escape hatch when you need architectural freedom.

5 · Underestimating power and thermals

A Jetson or Kria under sustained load produces real heat and needs active cooling; throttling from an inadequate heatsink will quietly halve your throughput. Conversely, plan the power supply properly — a Raspberry Pi 5 with a HAT and a camera wants a proper 27 W USB-C PD source, not a phone charger. Power and heat are not afterthoughts; they are design constraints.

6 · Ignoring the production path

A developer kit that works beautifully on your desk is not automatically a product. If you intend to manufacture, ask early whether the module is available at volume with a long lifecycle. This is where the Coral SOM and especially the Kria K26 shine — they are designed for the prototype-to-production journey — while a hobbyist board may not be sourceable in the quantities or over the years you need.

7 · Buying the biggest board “to be safe”

Over-provisioning is as costly as under-provisioning. A Jetson running a job an ESP32 could handle wastes money, power, and board space, and adds cooling and complexity you did not need. Match the board to the model. The whole point of the spectrum is that the right-sized board is cheaper, cooler, and more robust than the biggest one.

A universal getting-started checklist

Whichever board you land on, this sequence keeps you out of trouble. It works because it front-loads the questions that are expensive to answer late.

1. Define the job precisely. What is the input (audio, one camera, four cameras, a sensor stream)? What is the required latency and frame rate? What is the power source? Write these down — they select the board.
2. Pick and train the model on a PC or cloud first. Get accuracy where you need it in full precision before worrying about hardware. Note the model’s size and operation types.
3. Check accelerator compatibility. Confirm your model’s operations are supported by the target board’s toolchain, and that the quantized model fits in memory.
4. Quantize and measure. Convert to INT8 (or FP16 on Jetson), then re-measure accuracy. If it drops too far, use quantization-aware training or a better calibration set.
5. Compile with the vendor tool. Edge TPU Compiler (Coral), TensorRT (Jetson), Hailo Dataflow Compiler (Pi HAT), Vitis AI (Kria), or TFLite-Micro/ESP-DL (ESP32).
6. Run a pre-built example first. Every platform ships demos. Get one working end to end so you know the toolchain and hardware are healthy before adding your own model.
7. Deploy your model and benchmark on-device. Measure real latency, throughput, power, and temperature under sustained load — not a single cold-start inference.
8. Plan for production early if this is more than a hobby: sourcing, lifecycle, cooling, enclosure, and updates.

What you’ll need to buy alongside the board

Boards rarely come complete. Budget for the extras so the kit actually boots the first evening.

• POWER A correctly rated supply — often not included (the Kria and many kits sell it separately; the Pi wants a 27 W USB-C PD unit).
• STORAGE A quality microSD card, or better, an NVMe SSD for the Pi 5 and Jetson — cheap cards corrupt and throttle.
• CAMERA A compatible camera module if you are doing vision (CSI/MIPI modules differ per board; the S3-EYE bundles one).
• COOLING An active cooler/heatsink for the Jetson, Kria, and a hard-working Pi 5 with a HAT.
• HAT For the Raspberry Pi, the separate AI HAT+ if you need real-time inference beyond CPU speed.

Frequently asked questions

The sticker price is only the beginning

When people compare these boards on price, they usually compare the headline number for the board itself — five dollars for an ESP32 module, two hundred and forty-nine for a Jetson. That comparison is honest but incomplete, because the real cost of deploying an AI model includes everything you must buy, power, cool, and maintain around the board over its life. Thinking in terms of total cost of ownership rather than sticker price often changes which board actually makes sense.

Start with the accessories. A bare board rarely boots on its own. You will need a power supply sized correctly for the load, and on several of these platforms that supply is sold separately — the Kria and many Jetson-class kits do not include one, and the Raspberry Pi 5 genuinely wants a higher-wattage USB-C source than the chargers most people have lying around. You will need reliable storage; cheap microSD cards are a false economy that corrupt under the constant writes of a running system, so budget for a quality card or, better on the Pi and Jetson, an NVMe SSD. If your project involves vision you need a compatible camera, and camera interfaces differ across boards, so the module you already own may not fit. Add cooling for the higher-powered boards, and possibly an enclosure, and the modest accelerator that looked cheap can quietly double in delivered cost.

Then consider the running costs. Power draw is not just a battery-life question; over months of continuous operation it becomes an electricity bill and a thermal-design problem. A board that draws twenty watts around the clock consumes meaningfully more energy than one that draws two, and it may require ventilation or fans that add noise and failure points. For a single desktop prototype this is negligible; for a fleet of hundreds of always-on devices it dominates the economics, which is precisely why the low-power ESP32 and Coral are so attractive at scale even though their headline capability is modest.

Finally, weigh the human cost. The hours you spend learning a toolchain, debugging a compilation failure, or working around an unsupported operation are real costs even though no invoice captures them. A board with a steep learning curve and a small community can consume weeks that a friendlier platform would have saved — time that is usually far more expensive than the hardware. This is the hidden reason the Raspberry Pi remains so popular: not because its silicon is the best, but because its total cost of ownership, including your time, is often the lowest for anything short of a specialised production deployment.

Plain-language glossary

If any term in this guide sent you searching, here are the essentials in one place.

• INFERENCE Running a trained model to make a prediction on new data — the job all six boards do.
• TRAINING Teaching a model from data. Compute-heavy; done on workstations or in the cloud, not on these boards.
• TOPS Tera-Operations Per Second — a peak measure of an accelerator’s math throughput. A ceiling, not a guarantee.
• QUANTIZATION Converting a model from 32-bit floats to lower precision such as 8-bit integers, to shrink it and speed it up.
• INT8 / FP16 / FP32 Numeric precisions. FP32 is used in training; INT8 is the fast, compact format most edge accelerators want; FP16 is a middle ground the Jetson supports.
• LATENCY The time a single inference takes. Lower is better; for control loops it must also be predictable.
• THROUGHPUT How many inferences per second you can sustain — often expressed as frames per second for vision.
• NPU Neural Processing Unit — a chip dedicated to running neural networks efficiently (e.g. the Hailo on the Pi HAT).
• ASIC Application-Specific Integrated Circuit — fixed-function silicon built for one job, like the Coral Edge TPU.
• FPGA Field-Programmable Gate Array — reconfigurable logic you can shape into a custom datapath, as on the Kria.
• DPU Deep-learning Processing Unit — the configurable inference engine instantiated in the Kria’s FPGA fabric.
• SoC / SOM System-on-Chip / System-on-Module — an integrated processor, and a small board carrying that processor plus memory for slotting into a product.
• MCU / SBC Microcontroller (tiny, no OS, like the ESP32) / Single-Board Computer (runs Linux, like the Pi and Coral).
• ONNX / TFLite Portable model file formats you export to after training, before compiling for a specific board.
• TinyML The practice of running very small machine-learning models on microcontrollers under tight power and memory limits.