R.GOLKHANDANI
Home / Blog / Posts / Outbox Over Fire-and-Forget
online--:--:--

Reza Golkhandani // Senior Software Engineer | Backend platforms

Calendar Sync You Can Trust: Building Two-Way Sync on an Outbox

Distributed systems | Integrations | Reliability

01

Calendar Sync You Can Trust: Building Two-Way Sync on an Outbox

The goal

A home-services company with a dozen franchises books estimates all day. Every booking has to appear on the right technician's calendar within seconds, and when the technician moves it on their phone, the office has to see the change. I owned this integration end to end, from the OAuth setup to the sync engine to the technician-facing calendar screens.

The first version, written before I joined, called the calendar provider directly from the request handler. It was a fine prototype, and it had the usual limits of one: a timeout after the database commit left the calendar stale, retries could create an event twice, and once webhooks were added, our own writes came back as changes. My job was to make sync something the field team could stop thinking about.

The approach

The design principle: record the intent, then deliver it. Nothing that crosses a network boundary is called from inside a request. It is written down in the same transaction as the business change and delivered by a worker that can retry, back off, and ask for help.

What I built

An outbox. Every change that must reach the calendar becomes a row in an outbox table, committed together with the booking. A worker drains it with bounded concurrency, exponential backoff, and a dead-letter state that pages a person after the last retry.

BEGIN;
UPDATE bookings SET starts_at = $2 WHERE id = $1;
INSERT INTO outbox (aggregate, aggregate_id, kind, payload, idempotency_key)
VALUES ('booking', $1, 'calendar.upsert', $3, $4);
COMMIT;

Idempotency by version. The key is derived from the booking and its version, so a retried job is a no-op at the provider instead of a duplicate event.

One writer per link. Each attendee link has exactly one owner. Every outgoing write carries a marker of ours; any webhook carrying that marker is dropped as an echo. What remains is a genuine change by a person, which is recorded against the link, including whether they accepted the invite.

Franchise plumbing. Per-franchise service accounts, watch tokens with automatic renewal, a shared sync library the rest of the platform uses, and a verification job that reports calendar state per franchise.

The result

Duplicate and missing events went to zero and stayed there. During a provider outage a few weeks later, events arrived twenty minutes late instead of never, and the worker's queue depth on the dashboard told us about the outage before the provider's status page did. Technicians stopped double-checking their calendars, which was the point.

Takeaways

  • Record intent in the transaction, deliver it with a worker. Reliability becomes a property of the data model, not of the network.
  • Decide who owns each external object before deciding how to sync it.
  • If you read from a system you also write to, design the echo case up front.
02

Connect with Me