Knowledge Vault

Tiny Machine Learning and On-Device Inference: A Survey

2 min read

Overview

Tiny Machine Learning (TinyML) provides a decentralized alternative to cloud-based AI by enabling inference directly on resource-constrained hardware like microcontrollers. This survey examines how TinyML addresses latency, bandwidth, and privacy issues in real-time applications such as autonomous vehicles and healthcare.

Abstract

This systematic review (Sensors, 2025) examines TinyML as an alternative to cloud-based AI inference for applications requiring near-instantaneous response times. Following PRISMA guidelines, the survey covers TinyML use cases, ML techniques employed, and hardware used in experiments. TinyML enables ML tasks on hardware with extreme size and processing constraints, fundamentally changing the approach to edge computing.

Methodology

The review synthesizes experimental studies on TinyML deployment across domains, analyzing:

  • ML techniques adapted for microcontroller-class devices
  • Hardware platforms (microcontrollers with limited clock rates and memory)
  • Performance tradeoffs between model accuracy, latency, and power consumption
  • Comparison with cloud-hosted inference across application domains

Key Contributions

  • Strong real-world usability record — TinyML offers advantages over cloud inference in bandwidth-constrained environments and use cases requiring rapid response (autonomous vehicles, healthcare monitoring)
  • Near-zero latency for ML services by eliminating external communication dependency
  • Privacy enhancement — inference within the device eliminates data transmission risks
  • Critical tradeoff: optimizing for constrained devices involves sacrificing model accuracy or scalability to maintain functionality
  • Key gap identified: the decision between cloud and on-device inference is an architectural choice requiring analysis of latency requirements, privacy concerns, and computational capabilities

Significance

TinyML is shifting the ML deployment paradigm from centralized cloud processing to distributed edge intelligence. As IoT generates increasingly more data outside traditional data centers, on-device inference becomes essential for real-time, privacy-sensitive, and bandwidth-constrained applications.

Graph View

Backlinks