On a consequence of a kind of publication bias: "Why do Phase III Trials of Promising Heart Failure Drugs Often Fail? The Contribution of 'Regression to the Truth'", Krum & Tonkin 2003:
"There has been considerable recent disappointment with the failure of a number of major new pharmacological strategies for the treatment of chronic heart failure. In turn, there has been much speculation as to why trials of these therapies have not shown benefit. Among a number of plausible and scientifically valid reasons, consideration should be afforded to the potential contribution of "regression to the truth." Regression to the truth derives from the biological concept of regression to the mean, whereby random fluctuations in a biological variable occur over time, such that the true value of the variable is approached with repeated measurements. This same concept can be applied to clinical trial programs for new drugs for heart failure. Because only strongly positive trials generally go on to phase III testing, and some of these early phase studies are positive by chance alone, on retesting in phase III the results are very likely not be as strongly positive. Numerous examples of regression to the truth apply for trials of heart failure therapies, as well as in other areas.
To understand regression to the truth we must first consider the concept of regression to the mean. This concept derives from the random fluctuations that can occur in a variable over time. As a consequence, a single measurement of that variable more often yields a value removed from the mean, and the "true" value of the variable is approached with repeated measurements. As a corollary, in population studies, a single measurement of the dependent variable - for example, cholesterol - can lead to an underestimate of the strength of its association with an outcome such as coronary heart disease death ("regression distribution bias"). Consider a theoretical drug (drug x) being studied to determine its benefit in heart failure, as assessed by a surrogate measure, lowering of plasma norepinephrine (Fig. 1). The left panel shows that there is really no difference in plasma norepinephrine levels before and after drug x. However, the investigators went on to perform a subgroup analysis of those patients with norepinephrine levels above the mean (middle panel), and that subgroup demonstrated a significant reduction in norepinephrine levels with drug x. The investigators might therefore claim that drug x is effective in lowering plasma norepinephrine in patients with high norepinephrine levels. Furthermore, it is these patients (ie, patients with high levels) who are those that are particularly in need of a drug that will lower such elevated levels. Although it is possible that drug x does indeed lower elevated plasma norepinephrine levels, it is equally plausible (if not more so) that the high plasma levels were "captured" as being falsely or atypically high (for the individual patient) at baseline and then when the same patients were remeasured at a later time point, levels were not as high (ie, classic regression to the mean).
This concept is well-understood for a biologic variable, but how can this concept be applied to that of a clinical trial program for a new drug for heart failure? This is conceptually illustrated in Fig. 2, which depicts early phase trials conducted in the assessment of a variety of potential new drug therapies for heart failure. Each dot represents a trial of a certain drug. As can be seen, some early phase studies will be strongly negative, some strongly positive, but most will cluster around neutrality and, therefore, one can construct a standard bell-shaped curve. We know that many trials of new chemical entities are conducted in the setting of heart failure. Because of the large number of studies conducted, some will be positive by chance and indeed some will be strongly positive by chance. Does this matter? Yes, it does. It is highly likely that only drugs associated with strongly positive trials (ie, those to the right of the vertical dotted line) will go on to phase III testing. Because some of these studies that are positive by chance alone will be among these, then when retested in phase III trials, the results will no longer be strongly positive. This is analogous to the high plasma cholesterol or norepinephrine being retested in the earlier examples. This concept is true, not just of heart failure trials, but of any drug therapy for any specific indication. What exacerbates the problem in the setting of chronic heart failure is the low percentage strike rate in the development of successful pharmacologic therapies for this condition. Only renin-angiotensin and β-adrenoceptor blocking agents have come to the market over the last 30 years or so.
Therefore, very few promising drugs in early phase would be positive in phase III (if tested) and thus registrable for a heart failure indication. This is illustrated by the open circles below the curved line, interposed on the totality of early phase trials in Fig. 2. This line is curved because, of course, a strongly positive early phase study will make it more likely (but possibly still with low probability) of positive findings in phase III studies. Nevertheless, this still leaves a large number of trials strongly positive in early phase by chance alone (circled cluster) "regressing to the truth.""
"There has been considerable recent disappointment with the failure of a number of major new pharmacological strategies for the treatment of chronic heart failure. In turn, there has been much speculation as to why trials of these therapies have not shown benefit. Among a number of plausible and scientifically valid reasons, consideration should be afforded to the potential contribution of "regression to the truth." Regression to the truth derives from the biological concept of regression to the mean, whereby random fluctuations in a biological variable occur over time, such that the true value of the variable is approached with repeated measurements. This same concept can be applied to clinical trial programs for new drugs for heart failure. Because only strongly positive trials generally go on to phase III testing, and some of these early phase studies are positive by chance alone, on retesting in phase III the results are very likely not be as strongly positive. Numerous examples of regression to the truth apply for trials of heart failure therapies, as well as in other areas.
To understand regression to the truth we must first consider the concept of regression to the mean. This concept derives from the random fluctuations that can occur in a variable over time. As a consequence, a single measurement of that variable more often yields a value removed from the mean, and the "true" value of the variable is approached with repeated measurements. As a corollary, in population studies, a single measurement of the dependent variable - for example, cholesterol - can lead to an underestimate of the strength of its association with an outcome such as coronary heart disease death ("regression distribution bias"). Consider a theoretical drug (drug x) being studied to determine its benefit in heart failure, as assessed by a surrogate measure, lowering of plasma norepinephrine (Fig. 1). The left panel shows that there is really no difference in plasma norepinephrine levels before and after drug x. However, the investigators went on to perform a subgroup analysis of those patients with norepinephrine levels above the mean (middle panel), and that subgroup demonstrated a significant reduction in norepinephrine levels with drug x. The investigators might therefore claim that drug x is effective in lowering plasma norepinephrine in patients with high norepinephrine levels. Furthermore, it is these patients (ie, patients with high levels) who are those that are particularly in need of a drug that will lower such elevated levels. Although it is possible that drug x does indeed lower elevated plasma norepinephrine levels, it is equally plausible (if not more so) that the high plasma levels were "captured" as being falsely or atypically high (for the individual patient) at baseline and then when the same patients were remeasured at a later time point, levels were not as high (ie, classic regression to the mean).
This concept is well-understood for a biologic variable, but how can this concept be applied to that of a clinical trial program for a new drug for heart failure? This is conceptually illustrated in Fig. 2, which depicts early phase trials conducted in the assessment of a variety of potential new drug therapies for heart failure. Each dot represents a trial of a certain drug. As can be seen, some early phase studies will be strongly negative, some strongly positive, but most will cluster around neutrality and, therefore, one can construct a standard bell-shaped curve. We know that many trials of new chemical entities are conducted in the setting of heart failure. Because of the large number of studies conducted, some will be positive by chance and indeed some will be strongly positive by chance. Does this matter? Yes, it does. It is highly likely that only drugs associated with strongly positive trials (ie, those to the right of the vertical dotted line) will go on to phase III testing. Because some of these studies that are positive by chance alone will be among these, then when retested in phase III trials, the results will no longer be strongly positive. This is analogous to the high plasma cholesterol or norepinephrine being retested in the earlier examples. This concept is true, not just of heart failure trials, but of any drug therapy for any specific indication. What exacerbates the problem in the setting of chronic heart failure is the low percentage strike rate in the development of successful pharmacologic therapies for this condition. Only renin-angiotensin and β-adrenoceptor blocking agents have come to the market over the last 30 years or so.
Therefore, very few promising drugs in early phase would be positive in phase III (if tested) and thus registrable for a heart failure indication. This is illustrated by the open circles below the curved line, interposed on the totality of early phase trials in Fig. 2. This line is curved because, of course, a strongly positive early phase study will make it more likely (but possibly still with low probability) of positive findings in phase III studies. Nevertheless, this still leaves a large number of trials strongly positive in early phase by chance alone (circled cluster) "regressing to the truth.""
Shared publicly