Lucas Reymond

This project, carried out during my internship in the Biostatistics group at the Nestlé Research Center, focuses on the Fragility Index of clinical trials. The Fragility Index is the minimum number of patients whose outcomes would have to change (from event to nonevent, or the reverse) to flip the statistical significance of a trial. It complements the p-value by measuring the strength of a conclusion in patient units rather than in probability units: a trial where a single patient separates a significant result from a non-significant one is fragile, regardless of how convincing the p-value looks.

The motivation is that p-values are easy to misread. The gap between p = 0.049 and p = 0.051 can come down to one patient, yet one is reported as significant and the other is not. The Fragility Index makes that sensitivity explicit, which is why it has been gaining importance in the medical literature.

I developed a collection of R functions that compute the Fragility Index and several of its variants, designed to work with any statistical test. The user only supplies a function returning the p-value (and, when relevant, the underlying model), and the toolkit handles everything else. This test-agnostic design lets the same code run on t-tests, linear models and GLMs without modification.

Methodology and algorithms

• Exact algorithm: enumerates every admissible outcome modification to return the exact index. Correct, but combinatorially expensive on large trials.
• Randomized bound: for each candidate number of modifications, it samples max(100, log N) admissible reassignments instead of enumerating all of them, giving a fast upper bound.
• Influence-based bound: at each step it refits the regression model and flips the outcome of the most influential patient, ranked by Cook’s distance, leverage or residual.
• Greedy p-value bound: at each step it applies the single modification that pushes the p-value furthest toward (or away from) the significance threshold.

I also implemented the forward, reverse and traditional Fragility Index, as well as the incidence Fragility Index, which only counts modifications that are sufficiently likely given the observed data (following Baer et al., 2021).

The Generalized Fragility Index (GFI)

The main extension of this work is a Generalized Fragility Index. The standard index is defined only for two groups and a binary outcome. The GFI removes both restrictions: it applies to any number of groups and to outcomes that are continuous, discrete or categorical, under any statistical test. To keep modifications realistic, each candidate value is constrained to a highest-density region of the outcome’s distribution with coverage 1−q, so the algorithm only considers changes that are plausible under the data. A greedy procedure then searches for the smallest set of such modifications that reverses significance. The implementation supports normal, exponential, Poisson, negative binomial and finite categorical outcomes.

Findings

• On real clinical-trial datasets, the fast bounding algorithms returned values very close to the exact index, with an error that almost never exceeded 2, and at a small fraction of the runtime.
• The influence-based heuristic performs well when the trial is initially significant, but breaks down when it is not: flipping the most influential points then only reinforces the existing result.
• The toolkit was validated across several test types (t-tests, linear models, GLMs), confirming the test-agnostic design.

Deliverables and outlook

The work produced a reusable R library now available to the Biostatistics team for their own trials, a 20-page technical documentation, and a presentation given at the team’s statistics seminar. I also outlined directions to extend it further: multi-dimensional modifications through box or ball constructions, functional (curve) outcomes approximated with B-splines, and a tabu search to tighten the greedy bounds.