Production incident

The endpoint that measured one query

45 minintermediate215 yrs

A real incident: symptom first, cause hidden, tradeoff at the end.

The incident

The orders list is slow. The service method is instrumented and honestly reports one query and four milliseconds. The request takes 900ms. Nobody can find where the time goes, and a fix attempted last month made a different endpoint slower instead. What the review turned up: 1. The @ManyToOne has no fetch type written, so it takes the JPA default of EAGER and loads a customer for every order on every query. 2. spring.jpa.open-in-view is at its default of true, so the session survives into rendering and the lazy loads there succeed — after the transaction commits and after the instrumentation stops. 3. The service returns entities whose collections it never loaded. Three defects in three different places, and only the first is visible in the entity class. Reading the mappings was never going to find this. Fix all three.

What this teaches

  • A to-one association with no fetch type written is EAGER, and that is the most common JPA performance bug
  • open-in-view does not remove queries; it moves them past your instrumentation
  • Turning it off makes cost visible by turning a silent query into an exception
  • A service that returns entities owns finishing them, or should return a DTO instead
  • Where a query is issued matters as much as how many there are

Starter

Starter.java
import java.util.*;

/**
 * Production: the endpoint that measured one query.
 *
 * The orders list is slow. The service method is instrumented and reports one
 * query and four milliseconds. The request takes 900ms. Nobody can find where
 * the time goes, and a "fix" last month made a different endpoint slower.
 *
 * Run this. Four checks fail. Fix OrderService and the mappings so all four
 * pass, without weakening the checks.
 *
 * The persistence context is a model. It counts queries and knows whether the
 * session is open, which is all this problem needs.
 */
public class Starter {

    enum Fetch { LAZY, EAGER }

    static final class LazyInitException extends RuntimeException {
        LazyInitException(String name) { super("could not initialize proxy [" + name + "] - no Session"); }
    }

    /* ── the persistence context (correct; do not change) ───────────────── */

    static final class Session {
        boolean open = true;
        int queriesInService, queriesWhileRendering;
        boolean rendering;

        void run() { if (rendering) queriesWhileRendering++; else queriesInService++; }
        int total() { return queriesInService + queriesWhileRendering; }
    }

    static final class Association {
        final String name; final Session session; final int rows;
        boolean loaded;

        Association(Session session, String name, Fetch fetch, int rows) {
            this.session = session; this.name = name; this.rows = rows;
            if (fetch == Fetch.EAGER) { session.run(); loaded = true; }
        }

        int size() {
            if (loaded) return rows;
            if (!session.open) throw new LazyInitException(name);
            session.run();
            loaded = true;
            return rows;
        }

        void markLoaded() { loaded = true; }
    }

    static final class OrderEntity {
        final Association customer, items;
        OrderEntity(Association customer, Association items) {
            this.customer = customer; this.items = items;
        }
    }

    /* ── the mappings and the service under review ──────────────────────── */

    /** How Order maps its associations. These are the annotations. */
    static final Fetch CUSTOMER_FETCH = Fetch.EAGER;   // @ManyToOne, default
    static final Fetch ITEMS_FETCH    = Fetch.LAZY;    // @OneToMany, default

    /** Spring Boot's spring.jpa.open-in-view. */
    static final boolean OPEN_IN_VIEW = true;

    record OrderView(long id, String customerName, int itemCount) {}

    static final class OrderService {
        final Session session;
        OrderService(Session session) { this.session = session; }

        /** Loads a page of orders and returns the ENTITIES. */
        List<OrderEntity> listPage(int size) {
            session.run();                                   // select o from Order o
            var page = new ArrayList<OrderEntity>();
            for (int i = 0; i < size; i++)
                page.add(new OrderEntity(
                    new Association(session, "order.customer", CUSTOMER_FETCH, 1),
                    new Association(session, "order.items", ITEMS_FETCH, 4)));
            return page;
        }
    }

    /** The controller and serialiser, running after the service returns. */
    static List<OrderView> render(Session session, List<OrderEntity> orders) {
        session.rendering = true;
        if (!OPEN_IN_VIEW) session.open = false;
        var views = new ArrayList<OrderView>();
        long id = 1;
        for (OrderEntity o : orders)
            views.add(new OrderView(id++, "customer-" + id, o.items.size()));
        return views;
    }

    /* ── checks ─────────────────────────────────────────────────────────── */

    static final int PAGE = 20;

    public static void main(String[] args) {
        List<String> failures = new ArrayList<>();

        var session = new Session();
        var service = new OrderService(session);
        List<OrderView> views;
        try {
            views = render(session, service.listPage(PAGE));
        } catch (LazyInitException e) {
            views = List.of();
            failures.add("0. the request threw: " + e.getMessage());
        }

        // 1. Loading a page must not fetch associations nobody asked for.
        if (session.queriesInService > 2)
            failures.add("1. the service issued " + session.queriesInService + " queries for " + PAGE
                       + " orders — an association is being fetched with the parent");

        // 2. Nothing may query the database after the service has returned.
        if (session.queriesWhileRendering > 0)
            failures.add("2. " + session.queriesWhileRendering + " queries were issued while rendering, "
                       + "after the transaction committed — a timer on the service would report none of them");

        // 3. The whole request must be a small constant number of queries.
        if (session.total() > 2)
            failures.add("3. the request took " + session.total() + " queries for " + PAGE + " orders");

        // 4. And it must actually produce the page.
        if (views.size() != PAGE)
            failures.add("4. rendered " + views.size() + " views, expected " + PAGE);

        /* ── report ─────────────────────────────────────────────────────── */
        if (failures.isEmpty()) {
            System.out.println("PASS");
        } else {
            failures.forEach(f -> System.out.println("  " + f));
            System.out.println("FAIL");
        }
    }
}

Run it locally:

cd exercises/java/jpa-hibernate/fetch-types/03-production
javac Starter.java -d /tmp/out && java -cp /tmp/out Starter

Hints

  1. Hint 1

    Look at the two counters separately before changing anything. The split between them is the entire diagnosis.

  2. Hint 2

    One defect is a value that appears nowhere in the entity class, because nobody wrote it. Ask what the annotation means when it says nothing.

  3. Hint 3

    Turning open-in-view off will make the request throw. That is expected — the exception is pointing at the third defect.

  4. Hint 4

    The third fix is one extra query, deliberately. There is a stronger fix that removes the question entirely; name it in a comment even if you do not implement it.

Done when

  • The service issues at most two queries for a page of twenty
  • No query is issued after the service returns
  • The whole request is a small constant number of queries
  • The page still renders twenty views
  • A comment says why a timer on the service method reported four milliseconds

Solution

Show the solution — try it yourself first
Solution.java
import java.util.*;

/**
 * Solution: the endpoint that measured one query.
 *
 * Three defects, in three different places: a mapping default nobody wrote, a
 * framework default nobody chose, and a service that returned entities it had
 * not finished loading. Only the first is visible in the entity class, which
 * is why reading the mappings was never going to find this.
 */
public class Solution {

    enum Fetch { LAZY, EAGER }

    static final class LazyInitException extends RuntimeException {
        LazyInitException(String name) { super("could not initialize proxy [" + name + "] - no Session"); }
    }

    /* ── the persistence context (correct; do not change) ───────────────── */

    static final class Session {
        boolean open = true;
        int queriesInService, queriesWhileRendering;
        boolean rendering;

        void run() { if (rendering) queriesWhileRendering++; else queriesInService++; }
        int total() { return queriesInService + queriesWhileRendering; }
    }

    static final class Association {
        final String name; final Session session; final int rows;
        boolean loaded;

        Association(Session session, String name, Fetch fetch, int rows) {
            this.session = session; this.name = name; this.rows = rows;
            if (fetch == Fetch.EAGER) { session.run(); loaded = true; }
        }

        int size() {
            if (loaded) return rows;
            if (!session.open) throw new LazyInitException(name);
            session.run();
            loaded = true;
            return rows;
        }

        void markLoaded() { loaded = true; }
    }

    static final class OrderEntity {
        final Association customer, items;
        OrderEntity(Association customer, Association items) {
            this.customer = customer; this.items = items;
        }
    }

    /* ── the mappings and the service under review ──────────────────────── */

    /*
     * Defect 1. The @ManyToOne had no fetch type written, so it took the JPA
     * default — EAGER — and fetched a customer for every order on every query
     * that returned one, including this page, which only needs a name.
     *
     * A to-one association should always say LAZY explicitly. The default was
     * chosen for loading a single entity and is wrong for every list.
     */
    static final Fetch CUSTOMER_FETCH = Fetch.LAZY;    // @ManyToOne(fetch = LAZY)
    static final Fetch ITEMS_FETCH    = Fetch.LAZY;    // @OneToMany, already lazy

    /*
     * Defect 2. spring.jpa.open-in-view defaults to true, which kept the
     * session alive through rendering — so the twenty lazy loads succeeded,
     * after the transaction had committed and after the instrumentation on the
     * service method had stopped. That is why the service honestly reported
     * one query while the request issued forty-one.
     *
     * Turning it off does not make anything faster by itself. It makes the
     * cost visible, by turning a silent query into an exception.
     */
    static final boolean OPEN_IN_VIEW = false;

    record OrderView(long id, String customerName, int itemCount) {}

    static final class OrderService {
        final Session session;
        OrderService(Session session) { this.session = session; }

        /*
         * Defect 3. With open-in-view off, returning entities whose
         * collections were never loaded means the renderer throws. The service
         * has to finish what it started: load, in one query, exactly the
         * associations the response needs.
         *
         * This is @EntityGraph or a join fetch in real code. Two queries, both
         * inside the transaction, both deliberate.
         *
         * The stronger fix is to return a DTO rather than an entity, so
         * nothing that needs a session crosses the boundary at all — then the
         * open-in-view setting stops mattering to this endpoint entirely.
         */
        List<OrderEntity> listPage(int size) {
            session.run();                                   // select o from Order o
            var page = new ArrayList<OrderEntity>();
            for (int i = 0; i < size; i++)
                page.add(new OrderEntity(
                    new Association(session, "order.customer", CUSTOMER_FETCH, 1),
                    new Association(session, "order.items", ITEMS_FETCH, 4)));

            session.run();                                   // one query for all the collections
            for (OrderEntity o : page) { o.items.markLoaded(); o.customer.markLoaded(); }
            return page;
        }
    }

    /** The controller and serialiser, running after the service returns. */
    static List<OrderView> render(Session session, List<OrderEntity> orders) {
        session.rendering = true;
        if (!OPEN_IN_VIEW) session.open = false;
        var views = new ArrayList<OrderView>();
        long id = 1;
        for (OrderEntity o : orders)
            views.add(new OrderView(id++, "customer-" + id, o.items.size()));
        return views;
    }

    /* ── checks ─────────────────────────────────────────────────────────── */

    static final int PAGE = 20;

    public static void main(String[] args) {
        List<String> failures = new ArrayList<>();

        var session = new Session();
        var service = new OrderService(session);
        List<OrderView> views;
        try {
            views = render(session, service.listPage(PAGE));
        } catch (LazyInitException e) {
            views = List.of();
            failures.add("0. the request threw: " + e.getMessage());
        }

        // 1. Loading a page must not fetch associations nobody asked for.
        if (session.queriesInService > 2)
            failures.add("1. the service issued " + session.queriesInService + " queries for " + PAGE
                       + " orders — an association is being fetched with the parent");

        // 2. Nothing may query the database after the service has returned.
        if (session.queriesWhileRendering > 0)
            failures.add("2. " + session.queriesWhileRendering + " queries were issued while rendering, "
                       + "after the transaction committed — a timer on the service would report none of them");

        // 3. The whole request must be a small constant number of queries.
        if (session.total() > 2)
            failures.add("3. the request took " + session.total() + " queries for " + PAGE + " orders");

        // 4. And it must actually produce the page.
        if (views.size() != PAGE)
            failures.add("4. rendered " + views.size() + " views, expected " + PAGE);

        /* ── report ─────────────────────────────────────────────────────── */
        if (failures.isEmpty()) {
            System.out.println("PASS");
        } else {
            failures.forEach(f -> System.out.println("  " + f));
            System.out.println("FAIL");
        }
    }
}

Stretch

Return a DTO from listPage instead of entities, so nothing needing a session crosses the boundary. Then say what that makes possible that the current fix does not — specifically, what happens to this endpoint if someone turns open-in-view back on, and what happens if someone adds a new association to the entity.

← Back to What are the fetch types, and which is the default for each mapping?