openskills.info
Course catalog
TinyML and Edge Inference logoCourse Preview

TinyML and Edge Inference

TinyML runs machine learning inference on microcontrollers and other tiny, low-power chips. You use it when a device must classify sound, motion, or images on the sensor itself, on milliwatts of power and kilobytes of memory, without sending data to a server.

itComputer architecture and hardware

Don't Panic — TinyML and Edge Inference

TinyML is machine learning that runs on the chip inside a sensor rather than in a data center. Not a smaller cloud. A different address: the same microcontroller that reads your sensor now makes the decision about what the sensor is seeing, on kilobytes of memory and battery power measured in milliwatts.

The problem it exists to solve is the radio. Sending data costs far more energy than computing on it: a CPU operation is picojoules, while even a polite Wi-Fi radio draws tens of milliwatts when active. Before TinyML, an always-on device that needed a decision either streamed raw sensor data to a server and paid for the network trip, or shipped without intelligence at all. Inference on the device reads the sensor, computes for a few milliseconds, and transmits only a summary when something happens. Privacy, latency, and offline survival all improve for the same reason: the network is no longer in the path.

The three ideas everything else hangs off:

The model must fit. Embedded processors trail their mobile cousins by at least 100 to 1,000 times in compute, memory, and power. Training stays on your workstation, where backpropagation is affordable. Only a converted, quantized model ships to the device.

Quantization does the heavy lifting. Converting 32-bit floats to 8-bit integers shrinks the model to roughly a quarter of its size and runs faster on cores without a floating-point unit. The converter learns each tensor's numeric range from a small calibration set, then evaluates accuracy afterward; if the score holds, you ship.

The tensor arena is the budget. The runtime allocates every input, output, and intermediate tensor from one preallocated buffer that you size before building. Too small, allocation fails. Too large, you waste the one resource you cannot afford to waste. Recording builds report the exact usage, so you measure rather than guess.

The surprise: the pipeline is dull on purpose, and that is good news. Train, convert to a compact format, quantize to int8, embed the bytes as a C array in flash, invoke. The reference runtime, LiteRT for Microcontrollers, keeps its core interpreter around 16 KB, needs no operating system, and runs the same four stages on every board. The dullness means one mental model transfers across silicon vendors. What actually eats your schedule is the step nobody markets: collecting representative sensor data. The model is usually the smaller half of the work.

If a week from now you remember nothing else, remember the pattern: tiny model, always-on sensor, no radio beats big model, cloud round trip, battery death. Keyword spotting, motion classification, visual wake words, and anomaly detection all follow it.

What to read next: the Intro gives the full map, from constraints through lifecycle. The Cheatsheet holds the quantization contract and the interpreter call sequence used in real firmware. The practice session walks the hello_world build and the arena measurement. Landscape shows the runtime market if you are deciding what to adopt. Skip the Timeline when making adoption decisions; read it to feel how young the field is, which is young.

Where this skill leads

Relevant careers

See how this topic contributes to broader role-level skill maps.

Sources