Tools & Dataset
Pytest-flakefighters
Pytest-flakefighters is a pytest plugin implementing flaky test failure detection and classification. The plugin is composed of a collection of heuristics that work together to determine whether a test failure is genuine or flaky.
These heuristics come in two flavours: those that execute live after each test, and those that run after the entire test suite completes. All heuristics extend the FlakeFighter base class and implement the flaky_failure method, which returns true when a failure is deemed flaky.
Replication Dataset: Test Flimsiness (ICSE 2026)
This replication package accompanies the Test FLARE research paper “Test Flimsiness: Characterizing Flakiness Induced by Mutation to the Code Under Test” (ICSE 2026).
The dataset contains all experimental artefacts required to reproduce the study, supporting transparency, repeatability, and further investigation into mutation-induced flaky test behaviour.