Warm-up

Count the queries

10 minjunior015 yrs

One concept, guided. Near-impossible to fail.

What this teaches

  • The bug is invisible in the code and obvious in the query count
  • Each loading strategy trades round trips against rows on the wire
  • EAGER moves queries rather than removing them, and applies to every query for that entity
  • A fetch join with a limit reads everything and paginates in memory

Starter

Starter.java
import java.util.*;

/**
 * Warm-up: find a bug you cannot see by reading.
 *
 * No Hibernate and no database. A model of lazy loading with one thing added
 * that a real ORM also has and nobody looks at: a counter.
 */
public class Starter {

    /** Every statement the "database" was asked to run. */
    static final List<String> queries = new ArrayList<>();

    public static void main(String[] args) {

        // TODO 1: build the model. A Map<Long, List<Item>> is the table. Write
        // findOrders(limit), which records ONE query and returns ids, and
        // items(orderId), which records a query and returns that order's items.

        // TODO 2: write the loop every codebase has — for each order, read its
        // items and add up the count. Print the number of orders, the number
        // of lines, and the number of queries.
        //
        // Predict the query count before running it.

        // TODO 3: re-read your loop. Nothing in it says "database". In real
        // code the expensive line is order.getItems(). Write one sentence
        // about why review does not catch this.

        // TODO 4: run it for 10, 100 and 1000 orders. The relationship between
        // rows and queries is the whole bug.

        // TODO 5: add a JOIN_FETCH mode where findOrders also marks every
        // collection as loaded, so items() records nothing. One query. Then
        // count the ROWS it had to return — one per item, not one per order.

        // TODO 6: add a BATCH mode. When items() is called for an order that
        // is not loaded, load that order and the next batchSize-1 unloaded
        // ones in a single query. Confirm the count is 1 + N/batchSize.

        // TODO 7: add a SUBSELECT mode: two queries regardless of N. Say when
        // you would prefer it to the join fetch.

        // TODO 8: now EAGER. Add a customer lookup that fires on every order
        // whether or not the page wants it. Compare the total against lazy.
        // EAGER should make it worse — say why, and why the damage is not
        // limited to this one loop.

        // TODO 9: the trap. With JOIN_FETCH, ask for a page of 10 out of 1000
        // orders. A limit cannot be applied in SQL without cutting collections
        // in half, so the whole result is read and paged in memory. Print the
        // rows read. Then search for "HHH000104" and read what it says.
    }
}

Run it locally:

cd exercises/java/jpa-hibernate/n-plus-one/01-warmup
javac Starter.java -d /tmp/out && java -cp /tmp/out Starter

Done when

  • You produced 1 + N and can state N from the data
  • You compared all four strategies by query count on the same loop
  • You showed EAGER raising the total rather than lowering it
  • You measured rows read for a paged fetch join and can explain the number

← Back to What is the N+1 problem, and how do you detect it?