Measuring Real Sensors for Two Weeks, and the Dedupe Rule It Produced

The goal

Battery-powered fill-level sensors report over a cellular IoT radio every ten to twenty minutes: identity, a frame counter, distance, temperature, battery. The platform stores every frame and feeds alerts from them. Before trusting the pipeline with a fleet, I wanted an empirical reference for what the hardware actually does, not what the vendor spec says it does.

The approach

Run two physical units on a development stage for fourteen days. Configure one through the platform. Leave the other at factory defaults as a control that nothing ever touches, so device behaviour and platform behaviour can be told apart. Count frames at the gateway and rows in the database, and reconcile.

What I built

The measurement. The radio was excellent: the control unit's 789 transmissions had 784 gaps within a second of its interval, and one forty-minute gap was the only genuine loss, roughly one frame in a thousand.

The finding. The gateway had seen 2,155 real frames and the table held 1,235. The cause was exact: the frame counter is a device-side option that ships disabled, so every frame carried zero, and the dedupe key, identity plus counter plus payload hash, had degraded into a lifetime content hash. These sensors repeat payloads constantly, so frames collided with rows from days earlier. The control unit had 789 frames, exactly 390 distinct payloads, and exactly 390 rows.

The fix. Fold the gateway's receive timestamp into the key, document why beside the code, and add a "duplicate dropped" counter to the dashboard so a designed no-op with an unexpected slope becomes an alert.

The reference. The two-week measurement became a standing document: cadence, value ranges, sentinels that are not readings, and every place the hardware disagrees with the spec, each marked as a measurement rather than an intent.

The result

Row counts matched frame counts from the next deploy, and the platform went into its fleet rollout with a measured baseline instead of a documented one. The rule that generalizes: a uniqueness guarantee must include at least one component the platform controls. If someone else can switch off the thing you rely on, it is not a guarantee.

Takeaways

• Measure real hardware before trusting a spec, and keep a control unit nobody configures.
• Every designed no-op needs a metric.
• A uniqueness key must include something you control, such as the time you observed the message.