ExerciseProduction incident
Production incident
The audit log with missing entries
40 minintermediate2–10 yrs
A real incident: symptom first, cause hidden, tradeoff at the end.
The incident
A nightly job counts orders and records every one it examined, because
compliance requires an audit trail of what was looked at.
It ran correctly for two years. After the platform upgrade from Java 8 to
17, the report is still right and the audit log is empty.
What the team found:
1. No line of the job changed. Only the JDK did.
2. Adding an unrelated filter to the pipeline makes the audit log start
working again. Removing it empties it again.
3. A second report on the same data throws instead of running.
4. Nothing throws or warns in the case that matters — the count is
correct, and the missing audit was noticed by an auditor.
Finding 2 is the diagnosis if you can explain it. Do that first, then fix
both defects.
What this teaches
- Intermediate operations run only if the terminal operation needs them
- count() may compute the size from the source and skip the pipeline entirely
- An effect that depends on an unrelated edit was never guaranteed
- peek is for debugging; an effect that matters needs a terminal operation
- A stream is consumed by its terminal operation and cannot be reused
Starter
Starter.java
import java.util.*;
import java.util.function.*;
import java.util.stream.*;
/**
* Production: the audit log with missing entries.
*
* A nightly job reports on orders and records each one it looked at, for
* compliance. It worked on Java 8. After the upgrade the report is still
* correct and the audit log is empty.
*
* Run it. Three checks, three failures.
*/
public class Starter {
record Order(String id, double amount) { }
/** Stands in for the compliance log. */
static class Audit {
private final List<String> recorded = new ArrayList<>();
void record(Order order) {
recorded.add(order.id());
}
int size() {
return recorded.size();
}
}
static List<Order> orders() {
return List.of(
new Order("A-1", 120.00),
new Order("A-2", 80.50),
new Order("A-3", 12.25),
new Order("A-4", 640.00),
new Order("A-5", 5.75));
}
/**
* DEFECT 1: the audit is a side effect inside peek, and count() may work
* the size out from the source without executing the pipeline at all.
*/
static long countAudited(List<Order> orders, Audit audit, boolean withFilter) {
Stream<Order> stream = orders.stream().peek(audit::record);
if (withFilter) {
stream = stream.filter(o -> o.amount() >= 0);
}
return stream.count();
}
/**
* DEFECT 2: one stream, two terminal operations.
*/
static String summarise(List<Order> orders) {
Stream<Order> stream = orders.stream();
long n = stream.count();
double total = stream.mapToDouble(Order::amount).sum();
return n + " orders totalling " + total;
}
public static void main(String[] args) {
boolean ok = true;
List<Order> orders = orders();
System.out.println("── the nightly count ──");
Audit plain = new Audit();
long counted = countAudited(orders, plain, false);
System.out.println(" counted : " + counted);
System.out.println(" audit entries : " + plain.size() + " of " + orders.size());
ok &= check("every order reached the audit log", plain.size() == orders.size());
System.out.println();
System.out.println("── the same job with an unrelated filter added ──");
Audit filtered = new Audit();
long counted2 = countAudited(orders, filtered, true);
System.out.println(" counted : " + counted2);
System.out.println(" audit entries : " + filtered.size() + " of " + orders.size());
ok &= check("adding a filter does not change what the audit sees",
plain.size() == filtered.size());
System.out.println();
System.out.println("── the summary line ──");
String summary;
try {
summary = summarise(orders);
System.out.println(" " + summary);
} catch (IllegalStateException e) {
summary = null;
System.out.println(" " + e.getMessage());
}
ok &= check("the summary was produced", summary != null);
System.out.println();
System.out.println(ok ? "PASS" : "FAIL");
}
static boolean check(String what, boolean passed) {
System.out.println((passed ? " ok " : " FAIL ") + what);
return passed;
}
}Run it locally:
cd exercises/java/java8/stream-lazy-evaluation/03-production
javac Starter.java -d /tmp/out && java -cp /tmp/out StarterHints
Hint 1
What does count() need to know, and can it get that without running anything?
Hint 2
Why would adding a filter change whether a side effect happens? What does a filter make impossible to know in advance?
Hint 3
Read the javadoc for Stream.count. The behaviour is documented, which is the uncomfortable part.
Hint 4
For the second defect, count the terminal operations applied to one stream instance.
Done when
- Every order reaches the audit log
- The audit sees the same thing with and without the unrelated filter
- Both the count and the summary are produced from the same source
- A comment explains why peek was never a safe place for the audit
Solution
Show the solution — try it yourself first
Solution.java
import java.util.*;
import java.util.function.*;
import java.util.stream.*;
/**
* Solution: the audit log with missing entries.
*
* Two defects, both of them laziness showing through.
*
* Defect 1 — the audit lived in peek. Intermediate operations only run if
* the terminal operation needs their output, and since Java 9 count() can
* determine the size straight from the source when no operation could have
* changed it. The pipeline never executed, so the audit never ran. This is
* documented behaviour, which is why the job worked on 8 and stopped on the
* upgrade without a single line of the job changing.
*
* The tell is the second check: adding an unrelated filter made the audit
* start working again, because now count() cannot know the size in advance.
* Behaviour that depends on an unrelated edit elsewhere in the pipeline is
* the signal that the effect was never guaranteed.
*
* The fix is not a different intermediate operation. An audit IS the work,
* so it belongs in a terminal operation. peek is for debugging.
*
* Defect 2 — one stream, two terminal operations. A stream is consumed by
* its terminal operation and throws on any second use. Where the source can
* be traversed again, hold a Supplier<Stream<T>> and take a fresh stream
* each time; that also documents that several passes are intended.
*/
public class Solution {
record Order(String id, double amount) { }
static class Audit {
private final List<String> recorded = new ArrayList<>();
void record(Order order) {
recorded.add(order.id());
}
int size() {
return recorded.size();
}
}
static List<Order> orders() {
return List.of(
new Order("A-1", 120.00),
new Order("A-2", 80.50),
new Order("A-3", 12.25),
new Order("A-4", 640.00),
new Order("A-5", 5.75));
}
/** FIX 1: forEach is terminal, so the audit always runs. */
static long countAudited(List<Order> orders, Audit audit, boolean withFilter) {
Supplier<Stream<Order>> source = withFilter
? () -> orders.stream().filter(o -> o.amount() >= 0)
: orders::stream;
source.get().forEach(audit::record);
return source.get().count();
}
/** FIX 2: a fresh stream per traversal, from a supplier. */
static String summarise(List<Order> orders) {
Supplier<Stream<Order>> source = orders::stream;
long n = source.get().count();
double total = source.get().mapToDouble(Order::amount).sum();
return n + " orders totalling " + total;
}
public static void main(String[] args) {
boolean ok = true;
List<Order> orders = orders();
System.out.println("── the nightly count ──");
Audit plain = new Audit();
long counted = countAudited(orders, plain, false);
System.out.println(" counted : " + counted);
System.out.println(" audit entries : " + plain.size() + " of " + orders.size());
ok &= check("every order reached the audit log", plain.size() == orders.size());
System.out.println();
System.out.println("── the same job with an unrelated filter added ──");
Audit filtered = new Audit();
long counted2 = countAudited(orders, filtered, true);
System.out.println(" counted : " + counted2);
System.out.println(" audit entries : " + filtered.size() + " of " + orders.size());
ok &= check("adding a filter does not change what the audit sees",
plain.size() == filtered.size());
System.out.println();
System.out.println("── the summary line ──");
String summary;
try {
summary = summarise(orders);
System.out.println(" " + summary);
} catch (IllegalStateException e) {
summary = null;
System.out.println(" " + e.getMessage());
}
ok &= check("the summary was produced", summary != null);
System.out.println();
System.out.println(ok ? "PASS" : "FAIL");
}
static boolean check(String what, boolean passed) {
System.out.println((passed ? " ok " : " FAIL ") + what);
return passed;
}
}Stretch
The fix traverses the source more than once. Argue when that is wrong — a
source that is expensive to produce, or one that can only be read once, such
as a stream over a file or a network response. Then write the version that
makes a single pass and still audits everything, and say what you gave up.
← Back to Why does a stream with no terminal operation do nothing?