Challenge

Bound the cache

20 minintermediate210 yrs

Edge cases. You have to reason, and two valid fixes differ.

What this teaches

  • A cache without an eviction policy is a leak with a hit rate
  • LinkedHashMap.removeEldestEntry gives you LRU in the JDK, with no dependency
  • accessOrder=true is what makes it least-recently-USED rather than -inserted
  • WeakHashMap is the wrong reflex: the value usually references the key

Starter

Starter.javaOpen in playground
import java.util.*;

/**
 * CHALLENGE — 20 minutes.
 *
 * A session cache written the way most of them start: a HashMap in a static
 * field, with a comment promising to add eviction later.
 *
 * It is not slow and it does not throw. It simply never gives anything back,
 * and the only thing standing between it and an OutOfMemoryError is how long
 * the process happens to stay up.
 *
 * TASKS
 *   1. Run it. Note the final size against MAX_ENTRIES.
 *   2. Replace the map with a bounded LRU using LinkedHashMap. No libraries.
 *   3. Prove it is least-recently-USED, not least-recently-inserted: touch an
 *      old key, add enough to force eviction, and show the touched key
 *      survived while a newer untouched one did not.
 *   4. Try a WeakHashMap instead, and watch it not help. Session holds its
 *      own id — work out why that defeats weak keys, and write it down.
 *   5. In a comment: this map is static and shared. What did you do about
 *      thread safety, and what does that choice cost?
 */
public class Starter {

    static final int MAX_ENTRIES = 1_000;

    record Session(String id, String tenant, byte[] payload) {}

    /** TODO: this is the leak. It only ever grows. */
    static final Map<String, Session> SESSIONS = new HashMap<>();

    static Session login(String id, String tenant) {
        Session session = new Session(id, tenant, new byte[4 * 1024]);
        SESSIONS.put(id, session);
        return session;
    }

    static Session lookup(String id) {
        return SESSIONS.get(id);
    }

    public static void main(String[] args) {
        for (int i = 0; i < 50_000; i++) {
            login("session-" + i, "tenant-" + (i % 7));
        }

        System.out.println("MAX_ENTRIES      : " + MAX_ENTRIES);
        System.out.println("cache size       : " + SESSIONS.size());
        System.out.println("over budget by   : " + (SESSIONS.size() - MAX_ENTRIES));
        System.out.println("retained roughly : "
                + (SESSIONS.size() * 4L * 1024 / (1024 * 1024)) + " MB of payload");

        System.out.println();
        System.out.println("oldest still present : " + (lookup("session-0") != null));
        System.out.println("newest still present : " + (lookup("session-49999") != null));

        // Question to answer in a comment before you move on:
        // Nothing here is wrong in a way a test would catch. What is the
        // smallest change that would have made this fail in review?
    }
}

Run it locally:

cd exercises/java/jvm/memory-leaks/02-challenge
javac Starter.java -d /tmp/out && java -cp /tmp/out Starter

Hints

  1. Hint 1

    LinkedHashMap has a three-argument constructor. The third one is the interesting one.

  2. Hint 2

    removeEldestEntry is called after every put and you return whether to evict. You do not remove anything yourself.

  3. Hint 3

    Try WeakHashMap first and watch what happens when the Session holds its own key. That failure is the point of the last task.

  4. Hint 4

    For the thread-safety task: what is the smallest change, and what does it cost you under contention?

Done when

  • The cache never exceeds its stated maximum, proven by the output
  • The entry evicted is the least recently ACCESSED, not the oldest inserted
  • A comment says what the WeakHashMap version did and why it did not help
  • A comment names the thread-safety gap you did or did not close, and why

Stretch

Add a time bound as well as a size bound, so an entry expires even in a cache that never fills. Then decide where it is checked — on read, on write, or by a background sweep — and write down the cost of each. This is the point where most people reach for Caffeine, and being able to say exactly what it is buying you is the useful outcome.

← Back to How do you get a memory leak in a garbage-collected language?