zephyr-codeplace: caching beyond limits
Two measurements this week had the same cause. On a Raspberry Pi Pico and on a Teensy 4.1, Zephyr’s USB network stack was slow because its code runs from external flash through a small cache, and the code for one packet does not fit in that cache. Both boards became 7–10 times faster when the right code moved into fast on-chip memory. I have put the tools from both studies into one open-source repository: zephyr-codeplace (Apache-2.0).
What is in it
- A profiler for boards without a debug port. A Zephyr module option (
CONFIG_CODEPLACE_PC_SAMPLER, i.MX RT for now) samples the program counter from a timer interrupt that the kernel’s interrupt lock does not delay. Its code and buffer sit in tightly coupled memory, so it does not disturb the caches it measures. A second mode records the caller too, which shows who takes the time in a shared function such as a mutex. On boards with SWD, a script reads the same information over the debug probe. - A small Python package (standard library only) that maps samples to functions and libraries through the linker map, picks the densest functions for a given amount of fast memory and writes them as a Zephyr
zephyr_code_relocate()list, colours the rest for a given cache geometry, and checks that every function landed where it was planned. - A benchmark sample: UDP echo over USB, where the placement is a build option. Whole libraries, kernel and driver files, or a list of function names.
- The procedure as a checklist (
skill/SKILL.md), written for engineers and for AI coding agents alike: what to measure first, when layout alone cannot help, which experiments were dead ends, and the traps that cost me a day each.
Result with the sample
The stock build answers 2,238 echoes per second. A list of 32 KB of hot function names in ITCM gives 16,246; the whole hot path in ITCM with -O2 and data in DTCM gives 22,586, ten times the stock rate. With 1000-byte datagrams the fastest build echoes 10.8 MB/s in each direction. Every build variant is one line in the sample README.
The method in four lines
- Build three images first: stock, the hot libraries in fast memory, and the instruction cache off. If the second is much faster, the path waits for flash.
- Sample under the real workload, on the real board with its real flash chip.
- Move the densest functions by name; a name list survives code and compiler changes, a cache layout does not.
- Only then tune the configuration, and measure throughput, not only latency.
Limits
Prototype. The on-target sampler supports NXP i.MX RT10xx; the SWD route works on any Cortex-M with a DWT unit. Fast memory is shared with the application in a real product, so a name list made from the real workload is the realistic deliverable. The repository might become a Zephyr contribution later.