Queues

My Daily Bible service sends the day's scripture on WhatsApp or email at 7 AM, in every subscriber's time zone. For a zone with half a million people, a plain loop at 80 messages a second would finish at 8:44. So the work for a 7:00 message starts at 6:00, and the thing that makes it boring is a queue.

The short version

• A queue sits between deciding and doing. The producer drops work in; consumers take it out at their own pace. Neither waits for the other, and bursts wait in the queue instead of failing.
• Freeze before you fire. At 6:00 the zone's list is frozen into shards of 5,000 ready-to-send lines. At 7:00 nothing is computed, only sent, so the database, the AI and payments can all be down.
• Pace, don't burst; add lanes for speed. Providers cap messages per second, so each sender drains through a token bucket just under the limit. Throughput comes from more senders: 40 at 250 a second send half a million in 50 seconds.
• Lanes keep problems apart. WhatsApp and email have their own queues, and verification codes their own lane, so a sign-up rush never waits behind the morning messages. Senders scale up on a schedule at 6:45, not when the queue fills.
• Failures get a place. Retries with backoff, a dead-letter queue for what keeps failing, and a sweeper at 7:10 that resends with the same key, so nobody gets it twice. The catch: a queue makes things steadier, not faster.

The rates and the half-million zone are illustrative, as in my post about the design, which has every diagram and the failure cases.

Further reading

• Seven O'Clock, Every Morning — my full design for Daily Bible: the wave minute by minute, pacing, lanes and failover.
• Amazon SQS dead-letter queues — where messages that keep failing go to wait.
• Timeouts, retries and backoff with jitter — the AWS Builders' Library on retrying without making things worse.
• Token bucket — the pacing algorithm behind "just under the limit".

Next episode: Give Your Agent a Memory. Previously: What an LLM Call Costs. All episodes →