Challenge

Four limiters, one trace

25 minintermediate115 yrs

Edge cases. You have to reason, and two valid fixes differ.

What this teaches

  • The four algorithms differ in memory, burst behaviour and boundary accuracy, not in difficulty
  • An approximation that matches the enforced rate is usually better than an exact one that does not fit in memory
  • Measuring a limiter means replaying one trace through all candidates, not reasoning about them
  • The right choice depends on key count and burst tolerance, and both are numbers you can state

Starter

Starter.java
import java.util.*;

/**
 * Challenge: choose a limiter with numbers instead of adjectives.
 *
 * Four algorithms, one seeded trace, one clock. The deliverable is a table you
 * could put in a design doc — allowed counts, disagreement rates, and memory
 * per key at a realistic key count.
 */
public class Starter {

    static final class Ticker {
        private long millis;
        Ticker(long start) { this.millis = start; }
        long now() { return millis; }
        void advance(long ms) { millis += ms; }
    }

    interface Limiter { boolean allow(); String name(); }

    // TODO 1: implement all four against this interface, each taking the
    // shared Ticker. Keep them single-threaded — this exercise is about
    // behaviour, not concurrency.
    //
    //   FixedWindow(limit, windowMs)     one counter, reset on the boundary
    //   SlidingLog(limit, windowMs)      a timestamp per accepted request
    //   SlidingCounter(limit, windowMs)  current + previous, weighted:
    //                                      estimate = prev * (1 - intoWindow) + curr
    //   TokenBucket(capacity, perSecond) lazy refill from elapsed time

    public static void main(String[] args) {

        // TODO 2: write the trace generator. For each second of a ten-minute
        // run, emit a seeded random number of requests. Take the seed and the
        // mean rate as parameters so you can sweep the load.

        // TODO 3: replay ONE trace through all four simultaneously — call
        // every limiter for every request, in the same order, off the same
        // Ticker. If each limiter gets its own trace you are measuring the
        // random number generator.

        // TODO 4: for each limiter record: requests allowed, requests refused,
        // and the number of decisions where it disagreed with the sliding log
        // (treat the log as ground truth — it is the exact one).

        // TODO 5: sweep the offered rate from about a third of the limit to
        // about three times it. Print a row per load level. Two columns matter
        // and they say different things: how many were allowed, and how often
        // the decision differed from exact.

        // TODO 6: find the offered rate at which the approximations first
        // disagree with the log at all. Explain why it is where it is.

        // TODO 7: memory. For each limiter, count what it holds per key. Then
        // compute the total for 1,000,000 keys with a limit of 10,000/min.
        // One of these numbers is not deployable — say which and why.

        // TODO 8: the boundary test. Send a full limit's worth at the last
        // instant of a window and another full limit at the first instant of
        // the next. Which limiters allow 2x? Which refuse?

        // TODO 9: burst behaviour. After 60 seconds of silence, how many
        // requests will each limiter accept instantly? Say which of these
        // answers is a deliberate parameter and which is an accident.

        // TODO 10: pick one for a public API where clients are allowed short
        // bursts and there are millions of keys. Defend it in two sentences
        // using numbers from your own table.
    }
}

Run it locally:

cd exercises/java/traffic-management/rate-limiting/02-challenge
javac Starter.java -d /tmp/out && java -cp /tmp/out Starter

Hints

  1. Hint 1

    Run every limiter off the same Ticker and the same seeded trace. If the traces differ, you are comparing noise.

  2. Hint 2

    Compare two things separately: how many requests each allowed, and on how many individual decisions they disagreed. They tell different stories.

  3. Hint 3

    Sweep the offered load from well under the limit to well over it. The interesting behaviour is at and above the limit; below it, everything agrees.

  4. Hint 4

    For memory, count what each limiter holds per key, then multiply by a million keys before deciding which is 'accurate enough'.

Done when

  • One seeded trace replayed through all four limiters
  • A table of allowed counts and disagreement rates across at least three load levels
  • You measured per-key memory for each and multiplied it out to a realistic key count
  • You can name the load level at which the approximations start to diverge, and say why
  • You picked one for a public API and defended it in two sentences

← Back to How would you implement a rate limiter?