Power
— mW
The energy the device draws while the model does its job, above its idle draw.
- Measured with
- [Instrument and tool used]
- Reported as
- [Per request, averaged over the task]
models · how we measure
Every number has a device,a task and a baseline.
We publish what each model costs in power and memory. This page is how those numbers are made, so you can check them, question them or repeat them.
Method version [n]
Last updated [DD Mon YYYY]
[n] models measured
the budget sheet
Four numbers, always in this order.
Power and memory come first because they are what a device runs out of.
Memory
— MB
The most memory the model holds at any moment while running.
- Measured with
- [Tool used]
- Reported as
- [Peak resident memory, model plus runtime]
Disk
— MB
What the model adds to your app's size once installed.
- Measured with
- [How it is counted]
- Reported as
- [Package size on device, after compression]
Latency
— ms
The time from a person's input to a usable result.
- Measured with
- [Tool used]
- Reported as
- [First result, and full result, median]
the rig
The same devices, in the same state, every
time.
Devices
- Apple Watch
- [model] · [chip] · watchOS [n]
- iPhone
- [model] · [chip] · iOS [n]
- Mac
- [model] · [chip] · macOS [n]
- Oldest supported
- [the slowest device we still measure on]
Conditions
- Battery and charging
- [state of charge, unplugged or not]
- Temperature
- [room temperature, device at rest first]
- Radios and screen
- [airplane mode, brightness]
- Warm or cold
- [first run discarded or reported separately]
- Runs
- [n] runs, median reported, range shown
reading a sheet
What each part of a budget sheet tells you.
[Text model 1]
[device] · [OS version] · measured [date]
- Power— mW
- Memory— MB
- Disk— MB
- Latency— ms
Compared against: [baseline]
What it cannot do: [limits]
Method version [n]
One name, one job.
The exact device and OS, and the date. A number without these is not a measurement.
Always in the same order, always with units.
What the same job costs another way, on the same device.
What the model was not built to do.
Which version of this page the numbers follow.
baselines
What we compare against, and what we
don't.
A comparison is only fair if it is the same task on the same device.
We measure it
Apple's built-in model
[How the same task is run through Apple's on-device model, and on which OS version.]
We measure it
A cloud model call
[Which service, and what is counted: radio power, wait time, nothing held in memory.]
We don't quote
Other companies' models
We do not quote figures we did not measure. Where we have run someone else's model ourselves, the sheet says so.
limits
What these numbers do not tell you.
- They are for one device and one OS.
- A different chip or a new OS release changes them. We re-measure when either changes.
- They are not accuracy.
- How well a model does its job is reported separately on its own page.
- They are medians.
- Individual runs vary. The range is shown beside each number.
- They describe our task.
- Your feature may use the model differently. An on-device prototype measures your case.
repeat it yourself
Check our numbers.
- 01[Step 1: get the model package]
- 02[Step 2: run the measurement script on your device]
- 03[Step 3: compare with the sheet]
Re-measurement log
- [DD MON YYYY][All models re-measured on iOS n]
- [DD MON YYYY][Method version n: what changed]
- [DD MON YYYY][New device added]
- [DD MON YYYY][First published]
Want these numbers for your own feature?
An on-device prototype measures your case, on your device.
