With the number of endpoints in studies increasing and this leading to an increased chance of making incorrect conclusions on a drug’s effectiveness, the US Food and Drug Administration (FDA) released chunky, detailed draft guidance on the analysis and interpretation of study results with multiple endpoints. The guidance seeks to clarify when and how multiplicity due to multiple endpoints should be managed to avoid concluding that a drug is beneficial when it is not, something that is of primary concern to the agency. The guidance presents various strategies for grouping and ordering endpoints for analysis and applying some well-recognised statistical methods for managing multiplicity within a study in order to control the chance of making erroneous conclusions about a drug’s effects.
The guidance describes the three families of endpoints:
- Primary – which consist of the outcome(s) outcomes (based on the drug’s expected effects) that establish the effectiveness, and/or safety features, of the drug.
- Secondary – may be selected to demonstrate additional effects after success on the primary endpoint.
- Exploratory – all other endpoints.
The FDA say that when there is more than one primary endpoint and success on any one alone could be considered sufficient to demonstrate the drug’s effectiveness, the rate of falsely concluding the drug is effective is increased due to multiple comparisons. Demonstration of an effect of a drug is critical to meeting the legal standard for substantial evidence of effectiveness required to support approval of a new drug. The statistical approach often used in the assessment of a treatment effect on a chosen clinical endpoint is a hypothesis test. This test generates three primary measures of interest: a point estimate, a confidence interval and a p-value. FDA points out that “In a clinical trial with a single endpoint tested at α = 0.05, the probability of finding a difference between the treatment group and a control group by chance alone is at most 0.05 (a 5% chance). By contrast, if there are two independent endpoints, each tested at α = 0.05, and if success on either endpoint by itself would lead to a conclusion of a drug effect, there is a multiplicity problem. For each endpoint individually, there is at most a 5% chance of finding a treatment effect when there is no effect on the endpoint, and the chance of erroneously finding a treatment effect on at least one of the endpoints (a false positive finding) is about 10%.”
Type I error probability is widely used to limit the chance of a false positive conclusion about a drug’s effects to less than 2.5% (1 in 40 chance). As the number of endpoints or analyses increases, the probability of making a false positive conclusion can increase beyond the 2.5% level. FDA says that “Multiplicity adjustments, as described in this guidance, provide a means for controlling Type I error when there are multiple analyses of the drug’s effects.”
To read the draft guidance, please click here.