ExerciseChallenge
Challenge
Five flaky tests, one cause each
25 minintermediate2–12 yrs
Edge cases. You have to reason, and two valid fixes differ.
What this teaches
- Each cause has a different fix, and the wrong fix hides the symptom
- Asserting an order the code never promised is a test bug, not a code bug
- Unseeded randomness makes a failure unreproducible, which is the real cost
- A sleep is a guess standing in for a guarantee
- One of the five is reporting a genuine defect
Starter
Starter.java
import java.time.*;
import java.util.*;
import java.util.concurrent.*;
/**
* Challenge: five failing tests, five different reasons.
*
* Four are flaky and one is not. For each, name the cause before you change
* anything — the wrong fix makes every one of these go green while leaving
* the problem exactly where it was.
*/
public class Starter {
/* ─────────── 1 ─────────── */
static Set<String> tags = new HashSet<>(List.of("beta", "alpha", "gamma"));
static List<String> tagsInOrder() {
return new ArrayList<>(tags); // a HashSet has no order
}
/* ─────────── 2 ─────────── */
static String pickWinner(List<String> entrants) {
return entrants.get(new Random().nextInt(entrants.size()));
}
/* ─────────── 3 ─────────── */
static LocalDate trialEnds() {
return LocalDate.now().plusDays(14);
}
/* ─────────── 4 ─────────── */
static int slowSum(List<Integer> values) throws Exception {
var pool = Executors.newFixedThreadPool(2);
var total = new java.util.concurrent.atomic.AtomicInteger();
for (int v : values) pool.submit(() -> total.addAndGet(v));
pool.shutdown();
return total.get(); // read before the work finished
}
/* ─────────── 5 ─────────── */
static int applyDiscount(int price, int percent) {
return price - (price * percent / 100);
}
public static void main(String[] args) throws Exception {
System.out.println("1 tagsInOrder : " + tagsInOrder());
System.out.println("2 pickWinner : " + pickWinner(List.of("ana", "bo", "cy")));
System.out.println("3 trialEnds : " + trialEnds());
System.out.println("4 slowSum : " + slowSum(List.of(1, 2, 3, 4, 5)) + " (expected 15)");
System.out.println("5 discount : " + applyDiscount(100, 10) + " (expected 80)");
// TODO 1: for each of the five write the cause in one line.
//
// 1 : ____________________
// 2 : ____________________
// 3 : ____________________
// 4 : ____________________
// 5 : ____________________
// TODO 2: number 1 fails only sometimes and only on some JDKs. Fix
// the ASSERTION rather than the data — say what the test was really
// asserting, and what it should assert instead.
// TODO 3: number 2 cannot be reproduced when it fails. Make it
// reproducible FIRST, before making it pass. That order matters and
// is the point of this one.
// TODO 4: number 3 passes today. Work out the exact date on which it
// will fail, then make the date an input instead of a fact.
// TODO 5: number 4 is a race. Fix it by waiting for the work rather
// than by sleeping — and say why raising a sleep would have been the
// worst available fix.
// TODO 6: number 5 is not flaky. It fails every single time. Say what
// it tells you about a suite where nobody noticed.
}
}Run it locally:
cd exercises/java/test-strategy/flaky-tests/02-challenge
javac Starter.java -d /tmp/out && java -cp /tmp/out StarterHints
Hint 1
For each, ask what differs between two runs. If nothing in the test differs, look at what it shares with the rest of the suite.
Hint 2
One asserts on the order of something whose order was never specified.
Hint 3
One would be reproducible if a single line recorded a number.
Hint 4
One is not flaky at all. It fails for a reason, and the noise hid it.
Done when
- Each of the five has a named cause and a fix that addresses the cause
- The order-dependent assertion is fixed at the assertion, not the data
- The random test is made reproducible before it is made to pass
- The genuine defect is identified and reported rather than patched