Embedded AI · Memory analysis · 24 September 2026
Abstract: weights do not describe peak RAM
A model can fit in a microcontroller’s flash yet fail during tensor allocation. This article asks a precise question: how much does execution order affect peak memory for the same graph and data sizes? We build a two-branch graph, enumerate every valid order, and verify a static allocation without unsafe overlap. Peak memory falls from 192 to 136 KiB without changing tensor sizes. We then introduce scratch and system memory to show where that saving stops being enough.
This is an executed accounting experiment on an educational graph, not a LiteRT benchmark or board test. No convolutions are run, and accuracy, latency, and energy are not measured. The contribution is a verifiable space requirement under explicit assumptions. Prerequisites are arrays, acyclic graphs, and intervals; no particular neural architecture is required. One KiB always means 1024 bytes.
1. Three quantities that must stay distinct
Model file size includes weights, structure, and serialized metadata. Activation memory holds input-dependent data produced during inference. Total RAM additionally includes runtime structures, stack, heap, communication, acquisition, and other firmware functions. Comparing model file size directly with available RAM therefore compares different objects. Weights may be read from flash or copied into RAM; this is a platform choice to verify, not a universal property.
For a dense tensor shaped n₁×…×n_d with b bytes per element, data size is b times the dimension product, before padding and alignment. An INT8 activation of 32×32×96 occupies 98,304 bytes, or 96 KiB, even if its producing layer has few weights. Reducing parameters without changing activation shapes can leave the actual bottleneck untouched. Here all sizes are expressed in KiB and compatible with 16-byte alignment.
2. Defining a tensor lifetime
Assume pure operators without side effects, executed one at a time. Each requires all inputs and a distinct output buffer. In-place computation, recomputation, and data movement during execution are excluded. A tensor is born when its producer’s output is allocated and dies only after its last consumer. If the new tensor’s producer uses the old tensor, both are live during that call; freeing the input first underestimates the peak.
Inclusive endpoints express a precise convention: t identifies an operator execution before releasing its inputs. Final output remains live until the application reads it. In our example the initial input may be released after the first operator. If the runtime or acquisition retains its pointer, that assumption must change. A dependency diagram alone does not contain all these buffer-ownership rules.
3. A small graph large enough to fail
Input I of 16 KiB produces A of 32 KiB. Two branches leave A: B of 96 KiB followed by C of 8 KiB, and D of 64 KiB followed by E of 8 KiB. Finally F combines C and E to produce 16 KiB. Letters denote both an operator and its single output tensor. A possible interpretation is channel expansion, spatial reduction, and final concatenation; we are not defining a trained model.
Order A-B-D-C-E-F is topologically valid: no operator starts before its inputs. During D, however, A needed by D, B waiting for C, and output D itself coexist. Total size is 32+96+64=192 KiB. B’s dependency on A does not force immediate completion of C, but postponing C costs memory. A valid order is therefore not necessarily a good RAM order.
Instead complete branch B-C before starting D: A-B-C-D-E-F. Peak occurs during C, when A must still wait for D and B must still be read: 32+96+8=136 KiB. The saving is 56 KiB, about 29.17% of the previous peak. All tensors sum to 240 KiB: allocating them separately wastes space, while considering only the largest tensor, 96 KiB, underestimates requirements.
| Step | Operator, schedule ABCDEF | Live KiB | Operator, schedule ABDCEF | Live KiB |
|---|---|---|---|---|
| 1 | A | 48 | A | 48 |
| 2 | B | 128 | B | 128 |
| 3 | C | 136 | D | 192 |
| 4 | D | 104 | C | 168 |
| 5 | E | 80 | E | 80 |
| 6 | F | 32 | F | 32 |
4. Enumeration and proving the optimum in this case
A must be first and F last; between them we interleave chains B-C and D-E while preserving internal order. There are six interleavings. The script enumerates permutations, rejects dependency violations, and computes peaks using remaining-use counters. The observed minimum is 136 KiB. For six operators exhaustive search is transparent; it is not a scalable strategy for networks with thousands of nodes.
We can justify the bound without trusting enumeration. When B is born, A and B already require 128 KiB. If the other branch is complete, E adds 8 KiB, reaching 136. If D exists but E does not, cost is greater. If D has not started, executing C requires A+B+C=136, while executing D before C requires A+B+D=192. Every possibility requires at least 136 KiB. An order attaining it proves optimality under the stated assumptions.
5. Live-data sum and arena size
M_peak is a lower bound on allocator space; it does not by itself guarantee contiguous buffers fit without fragmentation. Assign each tensor an offset o_i. If lifetimes overlap, memory intervals [o_i,o_i+size_i) must be disjoint. Required arena size is the largest endpoint o_i+size_i. Optimizing order and finding offsets are related but distinct problems; a heuristic planner can leave holes.
These offsets attain the bound in our example. I and B share a start at 32 KiB because I dies before B is born. D reuses part of B’s space only after C. E reuses A’s start after D finishes reading it. C stays at the arena tail until F. The script checks every live-tensor pair at each operator, not just size sums. Here 136 KiB is therefore a realizable construction with zero scratch and the stated constraints.
6. Scratch: maximum of a sum is not sum of maxima
A kernel may require temporary workspace, for example to transform data blocks before multiplication. Add S KiB of scratch used only during D in order ABCDEF. Tensors occupy 104 KiB during D, while the scratch-free maximum of 136 KiB occurs during C. Hence M_peak(S)=max(136,104+S). Up to S=32 KiB, scratch does not increase the live-data bound; at S=48 the bound becomes 152 KiB.
Crucially, 152 KiB is not a new arena proved by the previous placement. In that placement, the internal contiguous hole during D is only 32 KiB; a contiguous 48 KiB scratch buffer does not fit. A conservative solution reserves another 48 KiB outside the 136 KiB arena, totaling 184 KiB. Achieving less requires a newly verified plan. This separates a theoretical overlap bound from an actually constructed memory layout.
7. From graph to firmware budget
Consider a hypothetical 256 KiB usable-RAM budget. Assume 24 KiB persistent state, 32 KiB for stack and services, and two 16 KiB acquisition buffers, separate from input I through an explicit copy design. Cost outside the arena is 88 KiB. With zero scratch, the 136 KiB plan totals 224 KiB, leaving 32. The order peaking at 192 needs at least 280 KiB overall and cannot fit regardless of fragmentation.
Reserving 48 KiB scratch separately gives a conservative total of 136+48+88=272 KiB, which no longer fits. The live-data bound alone gives 152+88=240 KiB, but we have not constructed an allocator attaining it. A board is not suitable merely because a formula below budget looks promising. Physical RAM, DMA-accessible RAM, and the bank required by an accelerator can also be different sets; budgeting must respect the actual memory map.
8. What to verify in an actual runtime
TensorFlow Lite Micro documentation distinguishes nonpersistent, temporary, and persistent arena areas and describes allocation-recording APIs. Do not automatically transfer those details to every runtime named LiteRT; platforms and execution paths differ. A real test would record model hash, runtime revision, selected kernels, compiler, alignment, input sizes, and linker map, comparing planning with measured peak. Those hardware measurements were not performed here.
Liberis and Lane study operator reordering in their 2020 arXiv v2, including an algorithm and microcontroller experiments. We read methods, experiments, and appendix: their work motivates the problem, but our numbers come from our graph, not a reproduction of their benchmark. Our code deliberately uses exhaustive enumeration rather than presenting a production optimizer. Optimality on six nodes does not demonstrate computational performance on a large network.
Alternatives change different assumptions. Quantization reduces bytes per element but may require conversions and new buffers; fusion avoids materialization if kernels allow it; in-place computation requires proof that overwritten data is no longer needed; recomputation trades memory for extra work. Savings percentages obtained separately cannot simply be added: lifetimes, scratch, and offsets must be recalculated after each change. Less RAM does not automatically mean less energy because accesses and time can increase.
9. Reproducibility and conclusion
The experiment uses Python 3.14.0 and only the standard library, with no random data; plots use Matplotlib 3.11.2. The archive includes the graph, sizes, six valid schedules, per-operator sums, offsets, and collision checks. The following snippet counts the best order; the archive also includes enumeration and scratch sensitivity. This is not pseudocode presented as a result: JSON files come from the saved execution.
sizes = dict(I=16, A=32, B=96, C=8, D=64, E=8, F=16)
live_sets = ['IA', 'AB', 'ABC', 'ACD', 'CDE', 'CEF']
usage = [sum(sizes[t] for t in live) for live in live_sets]
print(usage) # KiB
print(max(usage))For EL-AI, embedded applications are an editorial and technical-exploration area; this example does not demonstrate an available product or company-validated board. Feasibility requires three separate answers: which data must coexist, where they are placed, and how much space remains for the system. Counting only weights or the largest tensors does not fully answer any of them.
Sources and materials
Edgar Liberis, Nicholas D. Lane, Neural networks on microcontrollers: saving memory at inference via operator reordering, arXiv:1910.05110v2 (2020). TensorFlow Lite Micro, Memory Management.
Sources accessed 24 September 2026; main-branch documentation can evolve. Code, results, and instructions. JSON results. Text and experiment prepared with AI assistance, without claiming peer review. Illustrative ImageGen cover: not an EL-AI product.