Production incident

The migration that changed the API

45 minintermediate215 yrs

A real incident: symptom first, cause hidden, tradeoff at the end.

The incident

A security review found a password hash in a public API response. Nobody exposed it. A migration added the column eight months after the endpoint was written, and the response has carried it ever since. What the review turned up on the same controller: 1. Serialising the response recurses — the entity has a back-reference. 2. It publishes five fields nobody agreed to, three of them added by migrations that were reviewed as schema changes. 3. Serialising initialises a lazy association, so a query runs after the transaction has committed. 4. The create endpoint binds the request body onto the entity, so a caller can set the password hash — and setting id turns the create into an update against another user. Four failures with one cause. Fix them.

What this teaches

  • The entity as a boundary is one defect that presents as four
  • Adding a column to a table is an API change nobody reviews as one
  • A serialiser touches every field, including lazy ones and back-references
  • Accepting an entity is mass assignment, and setting id redirects the write
  • Naming what crosses the boundary is what makes the response reviewable

Starter

Starter.java
import java.lang.reflect.*;
import java.util.*;

/**
 * Production: the migration that changed the API.
 *
 * A security review found a password hash in a public API response. Nobody
 * exposed it — a migration added the column eight months after the endpoint
 * was written. While investigating, three more problems turned up on the same
 * controller.
 *
 * Run this. Four checks fail. Fix UserApi so all four pass, without weakening
 * the checks.
 *
 * The serialiser is a model: it reflects over fields the way Jackson reflects
 * over getters. That one behaviour is what all four problems come from.
 */
public class Starter {

    /* ── infrastructure (correct; do not change) ────────────────────────── */

    /** A lazy association that counts how many times it is initialised. */
    static final class Lazy<T> {
        private final java.util.function.Supplier<T> loader;
        private final int[] loads;
        private T value;
        Lazy(java.util.function.Supplier<T> loader, int[] loads) { this.loader = loader; this.loads = loads; }
        T get() { if (value == null) { loads[0]++; value = loader.get(); } return value; }
    }

    /** Emits every field it can reach. It cannot know what you meant to publish. */
    static String serialise(Object v, int depth) {
        if (depth > 6) throw new IllegalStateException("recursion: a cycle in the object graph");
        if (v == null) return "null";
        if (v instanceof Number || v instanceof Boolean) return v.toString();
        if (v instanceof String s) return "\"" + s + "\"";
        if (v instanceof Lazy<?> l) return serialise(l.get(), depth);
        if (v instanceof Collection<?> c) {
            var out = new ArrayList<String>();
            for (Object e : c) out.add(serialise(e, depth + 1));
            return "[" + String.join(",", out) + "]";
        }
        var parts = new ArrayList<String>();
        for (Field f : v.getClass().getDeclaredFields()) {
            if (f.isSynthetic()) continue;
            f.setAccessible(true);
            try { parts.add("\"" + f.getName() + "\":" + serialise(f.get(v), depth + 1)); }
            catch (IllegalAccessException e) { throw new RuntimeException(e); }
        }
        return "{" + String.join(",", parts) + "}";
    }

    static Set<String> keysIn(String json) {
        var keys = new LinkedHashSet<String>();
        var m = java.util.regex.Pattern.compile("\"([A-Za-z][A-Za-z0-9_]*)\"\\s*:").matcher(json);
        while (m.find()) keys.add(m.group(1));
        return keys;
    }

    /* ── the entities ───────────────────────────────────────────────────── */

    static final int[] LOADS = {0};

    static final class Role {
        String name = "USER";
        User user;                                  // back-reference
    }

    static final class User {
        long id = 42;
        String email = "ada@example.com";
        String displayName = "Ada";
        String passwordHash = "$2a$12$Nn0Xy8kQ8Zz";      // added by a migration
        String lastLoginIp = "203.0.113.9";              // added by a migration
        boolean deleted = false;                         // added by a migration
        Lazy<List<String>> permissions = new Lazy<>(() -> List.of("read", "write"), LOADS);
        Role role = new Role();
        User() { role.user = this; }
    }

    /* ── the API under review ───────────────────────────────────────────── */

    /** Fields the product owner agreed this endpoint returns. */
    static final Set<String> AGREED_CONTRACT = new LinkedHashSet<>(List.of("id", "displayName", "email"));

    static final class UserApi {

        /** GET /users/{id} — returns the entity. */
        Object get(long id) {
            return new User();
        }

        /** POST /users — binds the request body onto an entity. */
        User create(Map<String, Object> requestBody) {
            var user = new User();
            for (var e : requestBody.entrySet()) {
                try {
                    Field f = User.class.getDeclaredField(e.getKey());
                    f.setAccessible(true);
                    f.set(user, e.getValue());
                } catch (NoSuchFieldException ignored) {
                    // unknown field, ignored
                } catch (IllegalAccessException ex) {
                    throw new RuntimeException(ex);
                }
            }
            return user;
        }
    }

    /* ── checks ─────────────────────────────────────────────────────────── */

    public static void main(String[] args) {
        List<String> failures = new ArrayList<>();
        var api = new UserApi();

        // 1. The response must be serialisable at all.
        try {
            serialise(api.get(42), 0);
        } catch (IllegalStateException e) {
            failures.add("1. serialising the response threw: " + e.getMessage());
        }

        // 2. And it must contain only the agreed fields. Read them off the
        //    returned type, so this check works even when check 1 is failing.
        var published = new LinkedHashSet<String>();
        for (Field f : api.get(42).getClass().getDeclaredFields())
            if (!f.isSynthetic()) published.add(f.getName());
        var extra = new LinkedHashSet<>(published);
        extra.removeAll(AGREED_CONTRACT);
        if (!extra.isEmpty())
            failures.add("2. the response publishes fields nobody agreed to: " + extra);

        // 3. Serialising must not query the database.
        LOADS[0] = 0;
        try { serialise(api.get(42), 0); } catch (RuntimeException ignored) { }
        if (LOADS[0] > 0)
            failures.add("3. serialising issued " + LOADS[0] + " lazy load(s) — that query happens "
                       + "after the transaction, where nothing is measuring it");

        // 4. A caller must not be able to set fields the API does not offer.
        var hostile = new LinkedHashMap<String, Object>();
        hostile.put("displayName", "Mallory");
        hostile.put("id", 1L);                       // point the write at another row
        hostile.put("passwordHash", "chosen-by-caller");
        var created = api.create(hostile);
        if (created.id != 42 || "chosen-by-caller".equals(created.passwordHash))
            failures.add("4. a request body set id=" + created.id + " and passwordHash directly — "
                       + "the create endpoint can overwrite an arbitrary user");

        /* ── report ─────────────────────────────────────────────────────── */
        if (failures.isEmpty()) {
            System.out.println("PASS");
        } else {
            failures.forEach(f -> System.out.println("  " + f));
            System.out.println("FAIL");
        }
    }
}

Run it locally:

cd exercises/java/jpa-hibernate/entity-at-the-boundary/03-production
javac Starter.java -d /tmp/out && java -cp /tmp/out Starter

Hints

  1. Hint 1

    Do not fix these one at a time. Ask what all four have in common before writing anything.

  2. Hint 2

    Check 2 lists the fields being published. Compare it against AGREED_CONTRACT and ask who would ever have noticed the difference.

  3. Hint 3

    Check 4 is the direction people forget. Work out what a request body containing nothing but an id field would do to a save().

  4. Hint 4

    One record fixes the first three. A second record fixes the fourth. There is no third thing to do.

Done when

  • The response serialises without recursion
  • It contains only the agreed fields
  • Serialising triggers no lazy load
  • A hostile request body cannot set id or passwordHash
  • A comment says why these were four symptoms rather than four bugs

Solution

Show the solution — try it yourself first
Solution.java
import java.lang.reflect.*;
import java.util.*;

/**
 * Solution: the migration that changed the API.
 *
 * Four failures, one cause: the entity was the boundary. Every fix here is the
 * same move made twice — name what crosses, in each direction — and that one
 * change resolves all four rather than four separate patches.
 */
public class Solution {

    /* ── infrastructure (correct; do not change) ────────────────────────── */

    /** A lazy association that counts how many times it is initialised. */
    static final class Lazy<T> {
        private final java.util.function.Supplier<T> loader;
        private final int[] loads;
        private T value;
        Lazy(java.util.function.Supplier<T> loader, int[] loads) { this.loader = loader; this.loads = loads; }
        T get() { if (value == null) { loads[0]++; value = loader.get(); } return value; }
    }

    /** Emits every field it can reach. It cannot know what you meant to publish. */
    static String serialise(Object v, int depth) {
        if (depth > 6) throw new IllegalStateException("recursion: a cycle in the object graph");
        if (v == null) return "null";
        if (v instanceof Number || v instanceof Boolean) return v.toString();
        if (v instanceof String s) return "\"" + s + "\"";
        if (v instanceof Lazy<?> l) return serialise(l.get(), depth);
        if (v instanceof Collection<?> c) {
            var out = new ArrayList<String>();
            for (Object e : c) out.add(serialise(e, depth + 1));
            return "[" + String.join(",", out) + "]";
        }
        var parts = new ArrayList<String>();
        for (Field f : v.getClass().getDeclaredFields()) {
            if (f.isSynthetic()) continue;
            f.setAccessible(true);
            try { parts.add("\"" + f.getName() + "\":" + serialise(f.get(v), depth + 1)); }
            catch (IllegalAccessException e) { throw new RuntimeException(e); }
        }
        return "{" + String.join(",", parts) + "}";
    }

    static Set<String> keysIn(String json) {
        var keys = new LinkedHashSet<String>();
        var m = java.util.regex.Pattern.compile("\"([A-Za-z][A-Za-z0-9_]*)\"\\s*:").matcher(json);
        while (m.find()) keys.add(m.group(1));
        return keys;
    }

    /* ── the entities ───────────────────────────────────────────────────── */

    static final int[] LOADS = {0};

    static final class Role {
        String name = "USER";
        User user;                                  // back-reference
    }

    static final class User {
        long id = 42;
        String email = "ada@example.com";
        String displayName = "Ada";
        String passwordHash = "$2a$12$Nn0Xy8kQ8Zz";      // added by a migration
        String lastLoginIp = "203.0.113.9";              // added by a migration
        boolean deleted = false;                         // added by a migration
        Lazy<List<String>> permissions = new Lazy<>(() -> List.of("read", "write"), LOADS);
        Role role = new Role();
        User() { role.user = this; }
    }

    /* ── the API under review ───────────────────────────────────────────── */

    /** Fields the product owner agreed this endpoint returns. */
    static final Set<String> AGREED_CONTRACT = new LinkedHashSet<>(List.of("id", "displayName", "email"));

    /*
     * The response. Every field here was written down by someone, which is the
     * entire difference. A column added to the table tomorrow cannot appear in
     * it, a rename cannot silently break a client, and there is no association
     * to lazy-load and no back-reference to recurse into.
     *
     * Defects 1, 2 and 3 are all fixed by this record existing — the cycle, the
     * leaked columns and the query during serialisation were three symptoms of
     * returning the entity, not three bugs.
     */
    record UserResponse(long id, String email, String displayName) {}

    /*
     * The request. Binding a body straight onto an entity is mass assignment:
     * the caller could set passwordHash, and setting id turned a create into an
     * update against an arbitrary row.
     *
     * This record is the allowlist. Anything not named here is not settable,
     * and the mapping below chooses which entity fields a caller may influence.
     */
    record CreateUserRequest(String email, String displayName) {}

    static final class UserApi {

        /** GET /users/{id} — returns a response, built while the session is open. */
        Object get(long id) {
            User entity = new User();                 // loaded in the transaction
            return new UserResponse(entity.id, entity.email, entity.displayName);
        }

        /** POST /users — reads only the fields the API offers. */
        User create(Map<String, Object> requestBody) {
            var request = new CreateUserRequest(
                (String) requestBody.get("email"),
                (String) requestBody.get("displayName"));

            var user = new User();
            if (request.displayName() != null) user.displayName = request.displayName();
            if (request.email() != null) user.email = request.email();
            return user;                              // id and passwordHash untouched
        }
    }

    /* ── checks ─────────────────────────────────────────────────────────── */

    public static void main(String[] args) {
        List<String> failures = new ArrayList<>();
        var api = new UserApi();

        // 1. The response must be serialisable at all.
        try {
            serialise(api.get(42), 0);
        } catch (IllegalStateException e) {
            failures.add("1. serialising the response threw: " + e.getMessage());
        }

        // 2. And it must contain only the agreed fields. Read them off the
        //    returned type, so this check works even when check 1 is failing.
        var published = new LinkedHashSet<String>();
        for (Field f : api.get(42).getClass().getDeclaredFields())
            if (!f.isSynthetic()) published.add(f.getName());
        var extra = new LinkedHashSet<>(published);
        extra.removeAll(AGREED_CONTRACT);
        if (!extra.isEmpty())
            failures.add("2. the response publishes fields nobody agreed to: " + extra);

        // 3. Serialising must not query the database.
        LOADS[0] = 0;
        try { serialise(api.get(42), 0); } catch (RuntimeException ignored) { }
        if (LOADS[0] > 0)
            failures.add("3. serialising issued " + LOADS[0] + " lazy load(s) — that query happens "
                       + "after the transaction, where nothing is measuring it");

        // 4. A caller must not be able to set fields the API does not offer.
        var hostile = new LinkedHashMap<String, Object>();
        hostile.put("displayName", "Mallory");
        hostile.put("id", 1L);                       // point the write at another row
        hostile.put("passwordHash", "chosen-by-caller");
        var created = api.create(hostile);
        if (created.id != 42 || "chosen-by-caller".equals(created.passwordHash))
            failures.add("4. a request body set id=" + created.id + " and passwordHash directly — "
                       + "the create endpoint can overwrite an arbitrary user");

        /* ── report ─────────────────────────────────────────────────────── */
        if (failures.isEmpty()) {
            System.out.println("PASS");
        } else {
            failures.forEach(f -> System.out.println("  " + f));
            System.out.println("FAIL");
        }
    }
}

Stretch

The response record is built inside get(), where the entity was loaded. Move the construction to the caller and re-run the checks — check 3 will start failing again. Explain why, then say what that means for the common advice to "map to a DTO in the controller".

← Back to Should you return a JPA entity from a REST controller?