Methodology

Lemon Metrics wraps Lemonade’s built-in lemonade bench --json with a power-sampling layer. Performance metrics (tokens/sec, TTFT, memory) come from Lemonade itself; we add wall-clock watts sampled at 1–2 Hz for the entire run.

What is recorded per run

Machine fingerprint — CPU, cores, RAM, GPU/NPU list, OS, stable device_id.
Ambient snapshot — power source (AC vs battery), battery %, CPU governor, thermals.
Benchmark — per-scenario TPS, TTFT, duration percentiles, memory peak from lemonade bench --json.
Power samples — raw (timestamp, watts) as JSONL.

Power sources

Platform	Sampler	Source
Linux (AMD Ryzen AI, Radeon iGPU)	`hwmon`	sysfs `power1_average`, summed
Linux (Intel)	`rapl`	`intel-rapl` energy_uj, differentiated
Linux (AMD dGPU)	`rocm-smi`	`rocm-smi --showpower --json`
macOS	`powermetrics`	requires passwordless `sudo -n`
Windows	`null`	no reliable real-time source yet; energy marked unavailable
fallback	`null`	marks `power_available: false`; run still completes

Windows power — call for help

No reliable real-time wattage source exists on Windows today (Win32_PowerMeter is rarely populated; powercfg /energy is a 60 s report, not a per-sample stream). Windows runs record full performance metrics with power_available: false.

Know a reproducible way to sample watts on Windows? Open an issue or PR — we want a first-class windows sampler.

Energy metrics

With P the mean sampled power and T the wall duration:

J/token = P×T / total tokens (input + output)
J/output token = P×T / output tokens — the cost of generating
tokens/kWh = 3.6e6 × total tokens / (P×T)

Per-scenario energy is P × scenario mean duration and is labelled an estimate, because the bench reloads models between scenarios; only wall-level energy is authoritative.

Honesty rules

Never invent power data — missing source means power_available: false.
Battery runs are flagged so readers can discard them.
Raw samples are published alongside every summary so anyone can recompute.

Full details: methodology on GitHub.