gabe /presentato-mf

public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages

acme-camera: flesh out metrics targets and instrumentation

663be22 · Gabriel Morell · 2026-10-13 18:52

3 files +109 -4

Files changed

modified content/presentations/acme-camera.md

+109 −4

@@ -365,14 +365,119 @@ Configures on-device inference using NPU hardware available in modern camera SOC

365	365
366	366	# Metrics
367	367
368		−- Performance given envelope configuration
	368	+- Performance within the configured power envelope
369	369
370		−_leading to insights for better_ <!-- .element: class="fragment" -->
	370	+_leading to insights for better defaults_ <!-- .element: class="fragment" -->
371	371
372		−- Battery Life across a fleet <!-- .element: class="fragment" -->
	372	+- Battery life across the fleet <!-- .element: class="fragment" -->
	373	+
	374	+Note:
	375	+Every config choice (motion mode, light mode, schedules) trades responsiveness for battery,
	376	+so the metrics are about how well a device does inside the envelope its owner picked,
	377	+and fleet data tells us which defaults to ship
	378	+
	379	+--
	380	+
	381	+## Targets
	382	+
	383	+\| Metric \| Target \|
	384	+\|---\|---\|
	385	+\| Battery life, default config \| ≥ 3 months \|
	386	+\| Standby draw, SOC asleep \| ≤ 100 µA \|
	387	+\| Motion → first frame \| < 1 s p95 \|
	388	+\| Motion → notification \| < 5 s p95 \|
	389	+\| False wakes (no person) \| < 20% \|
	390	+\| Crash-free wakes \| > 99.9% \|
	391	+\| OTA success, first try \| > 99.5% \|
	392	+
	393	+Note:
	394	+Assumptions: ~6000 mAh pack, 20 events/day, 10 s clips, ~400 mA while the SOC, Wi-Fi and IR are up
	395	+Each event costs roughly 1.5 mAh, so events dominate and standby has to stay tiny
	396	+Wake latency is the user-visible one: a late first frame means a clip of someone's back
	397	+False wakes are PIR triggers the radar doesn't confirm, they cost battery for nothing
	398	+A/B boot means a failed OTA rolls back rather than bricks, the target is about not needing to
	399	+
	400	+--
	401	+
	402	+## Slicing the Fleet
	403	+
	404	+- Firmware version
	405	+- Motion / light / power mode
	406	+- Battery vs solar vs external
	407	+- Temperature and Wi-Fi signal
	408	+
	409	+Note:
	410	+Averages hide problems: a battery life drop on one firmware, or in cold climates, or on weak Wi-Fi
	411	+These are the dimensions every metric gets broken down by
	412	+Firmware cohorts gate staged OTA rollouts: a regression in one cohort halts the rollout
373	413
374	414	---
375	415
376	416	# Instrumentation
377	417
378		−- Collection of combined environmental, fuel gauge
	418	+What we measure, how it leaves the device, and how often
	419	+
	420	+Note:
	421	+Combines environmental and fuel gauge data with what each component already knows
	422	+
	423	+--
	424	+
	425	+## Signals
	426	+
	427	+\| Source \| Signals \|
	428	+\|---\|---\|
	429	+\| powerctl \| voltage, current, charge \|
	430	+\| mcuctl \| wake reason, radar confirm \|
	431	+\| cameractl \| wake → frame latency \|
	432	+\| network \| connect time, RSSI \|
	433	+\| system \| fw, boot slot, crashes \|
	434	+\| environment \| temperature, light \|
	435	+
	436	+Note:
	437	+powerctl also reports time to empty, mcuctl how long the lights were on, cameractl clip length, network upload time, system the reboot reason
	438	+Each component publishes its own metrics to the broker, so nothing new has to reach into another process
	439	+Energy per wake comes from the fuel gauge before and after, which ties every event to its battery cost
	440	+
	441	+--
	442	+
	443	+## Collection
	444	+
	445	+- Components publish metrics to the broker
	446	+- A telemetry agent aggregates and buffers to flash
	447	+- MCU logs hourly samples while the SOC sleeps
	448	+- Uploads ride along when Wi-Fi is already up
	449	+
	450	+Note:
	451	+Never wake the radio just to report, telemetry should be well under 1% of the energy budget
	452	+The MCU keeps a small ring buffer of sleep samples (battery, temperature, wake counts) that the SOC drains on its next wake
	453	+Buffered to flash so a dead battery or lost network doesn't lose history
	454	+Upload is MQTT over TLS to the backend, the same connection the device already holds for events
	455	+
	456	+--
	457	+
	458	+## Reporting Cadence
	459	+
	460	+\| When \| What \|
	461	+\|---\|---\|
	462	+\| Every event \| wake summary: reason, latency, energy \|
	463	+\| Daily heartbeat \| counters, battery state, sleep samples \|
	464	+\| After a crash or rollback \| coredump and reboot reason \|
	465	+\| On demand \| debug logs, per device \|
	466	+
	467	+Note:
	468	+The heartbeat goes out even with no events, so a silent device is a signal on its own
	469	+Coredumps are symbolicated server side, the device only ships the raw dump
	470	+On demand logs are for support cases, off by default to save power and bandwidth
	471	+
	472	+--
	473	+
	474	+## Fleet Dashboards
	475	+
	476	+- Projected battery life per device vs target
	477	+- Latency and false wake rate per cohort
	478	+- Alerts: projected life below target, missed heartbeat, crash spikes after OTA
	479	+
	480	+Note:
	481	+Projected life comes from the measured drain rate, so we can flag a device months before it dies
	482	+Missed heartbeat for 48 h means dead, offline or stuck, worth an owner notification
	483	+A crash spike in a new firmware cohort pauses the rollout automatically

Parents: 85ae7b2