ExerciseChallenge
Challenge
Five deadlock reports, five different fixes
25 minintermediate1–15 yrs
Edge cases. You have to reason, and two valid fixes differ.
What this teaches
- Lock ordering is the fix for some deadlocks and a distraction for others
- A sequential scan inside a write transaction locks rows it does not change
- One of the five is a lock wait timeout, which wants the opposite response
- A single hot row cannot be ordered, so it needs a different shape of fix
Starter
Starter.java
import java.util.*;
/**
* Challenge: five reports from production. Diagnose each and fix it.
*
* They arrive looking the same — an exception, two statements, a rolled-back
* transaction. They have five different causes, and one of them is not a
* deadlock at all.
*
* Reuse the lock manager from the warm-up: heldBy, waitingFor, and a walk of
* the waits-for graph.
*/
public class Starter {
// ── report 1 ──────────────────────────────────────────────────────────
// Two transfers between the same pair of accounts, in opposite directions.
// T1: UPDATE account SET ... WHERE id = 7
// UPDATE account SET ... WHERE id = 3
// T2: UPDATE account SET ... WHERE id = 3
// UPDATE account SET ... WHERE id = 7
// ── report 2 ──────────────────────────────────────────────────────────
// Two nightly batch jobs, each archiving a list of order ids built by a
// different query. The lists overlap. Neither is sorted.
// T1: UPDATE orders SET archived = true WHERE id = ? (ids 4, 1, 9)
// T2: UPDATE orders SET archived = true WHERE id = ? (ids 9, 1, 4)
// ── report 3 ──────────────────────────────────────────────────────────
// Two statements whose matched rows do not overlap at all.
// T1: UPDATE orders SET priority = 1 WHERE id = 500 (primary key)
// T2: UPDATE orders SET status = 'x' WHERE region = 'APAC' (no index on region,
// matches only id 1)
// ── report 4 ──────────────────────────────────────────────────────────
// Reported as a deadlock by the on-call engineer. The log says:
// T1 holds inventory:42 and is running a 4-second call to a payment API
// T2 waits for inventory:42
// T3 waits for inventory:42
// T4 waits for inventory:42
// Error surfaced to users: lock wait timeout exceeded.
// ── report 5 ──────────────────────────────────────────────────────────
// A single counter row, updated by every request during a flash sale.
// T1: SELECT stock FROM inventory WHERE id = 1 FOR UPDATE
// UPDATE inventory SET stock = ? WHERE id = 1
// ...and two hundred other transactions doing exactly the same thing.
public static void main(String[] args) {
// TODO 1: model each report with your lock manager and confirm which
// ones actually produce a cycle. Two of the five do not.
// TODO 2: for each, answer the three diagnostic questions in order:
// a. do the transactions take rows in different orders?
// b. does either statement scan rather than seek?
// c. is either transaction holding locks longer than it needs to?
// Exactly one question explains each report.
// TODO 3: fix report 1. One line. Say why it works without reducing
// how much is locked or for how long.
// TODO 4: fix report 2. The lists come from different queries, so you
// cannot fix it at the call site. Where does the fix belong, and what
// is the smallest change that makes both jobs agree?
// TODO 5: fix report 3. Lock ordering will not help here — show why by
// trying it. Then say what you would have run before touching any code.
// TODO 6: report 4 is not a deadlock. Name what it is, say how you can
// tell from the log alone, and explain why retrying makes it worse.
// TODO 7: report 5 cannot be fixed by ordering, because there is only
// one row to order. Give two fixes of different shapes, and name the
// cost of each. One of them changes what the application can promise
// the user — say which and how.
// TODO 8: a retry policy on deadlocks would hide reports 1, 2 and 3
// from users entirely. Say why you would still alert on the rate, and
// what the rate climbing would be telling you.
}
}Run it locally:
cd exercises/java/transactions/database-deadlocks/02-challenge
javac Starter.java -d /tmp/out && java -cp /tmp/out StarterHints
Hint 1
For each report, ask three questions in order: do they take rows in different orders, does either do a sequential scan, and is either transaction longer than it needs to be.
Hint 2
One report has no cycle in it at all. Read the waits carefully before assuming the error name is accurate.
Hint 3
One is between two statements whose matched rows do not overlap. The answer is in the access path, not the ordering.
Hint 4
One is on a single row that everything updates. Sorting one id does nothing — say what actually helps and what it costs.
Done when
- Each report is classified and has a specific fix
- The lock wait timeout is identified and is not answered with a retry
- The missing-index case is answered with an index, not with lock ordering
- The hot-row case is answered with a different shape of fix, and its cost is named
- You said which of the five would be hidden by a retry policy, and why that is still worth alerting on
← Back to What causes a database deadlock, and how do you prevent it?