r4 - 25 Apr 2008 - 21:29:24 - CtsiBreadYou are here: TWiki > 
CTSpedia Web > Articles > ResultsInterpretation
CTSpedia Web > Articles > ResultsInterpretationConclusions that Match Results
How to provide interpretation that accurately reflects estimates, confidence intervals, and p-valuesIntroduction
The table below provides example phrasing of major conclusions (such as in the first paragraph of a paper's Discussion section) for a study that estimated a relative risk. Phrasing would be similar for studies with other types of outcomes. This is provided to help prevent the problem of interpretation based only on p-values, most notably the fallacy of interpreting p>0.05 as proving or supporting no effect. Conclusions should also reflect the direction and size of the estimated effect, along with the uncertainty around it as shown by the confidence interval.Possible Evolution
Ideally, this could evolve into an interactive applet in which the user specifies the outcome variable, the key predictor variable, the type of analysis, and the obtained estimate, 95% confidence limits, and p-value. The user would then answer some questions requiring clinical or scientific judgment:- If the estimate of <E.EE> turned out to be exactly right, would that be a clinically or scientifically important effect? (Yes, No, or Possibly)
- If the lower confidence bound of <L.LL> turned out to be exactly right, would that be a clinically or scientifically important effect? (Yes, No, or Possibly)
- If the upper confidence bound of <U.UU> turned out to be exactly right, would that be a clinically or scientifically important effect? (Yes, No, or Possibly)
Using the Table
To find the appropriate entry in the table below:- Decide whether your estimate is large enough to be clicially important if you knew that it was exactly correct. (If you find it difficult to decide if it is large enough to be important, then you can use an entry that has "Maybe Important" under the "Estimate" column. Similar "Maybe" possibilities are provided for the confidence limits.)
- Make the same decision for both ends of the confidence interval.
- Use the model text from the row that matches those decisions and the p-value. The text can be used as a starting point for phrasing your key conclusion. It will likely need to be customized for your particular topic area, outcome, predictor, and type of analysis. The phrasing provided is for a study that estimated a relative risk. Phrasing would be similar for studies with other types of outcomes.
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Important Benefit | Important Benefit | Important Benefit | Yes | We found strong evidence for clincially important risk reduction, with the estimated reduction being 0.EE-fold and the confidence interval supporting a substantial reduction in risk by at least a factor of 0.UU. |
| Important Benefit | Important Benefit | Maybe Important Benefit | Yes | We estimated a clinically important and statistically significant 0.EE-fold reduction in risk. The confidence interval supports a reduction in risk by at least a factor of 0.UU, but this could possibly be considered too little risk reduction to be clinically important. |
| Important Benefit | Important Benefit | Small, Unimportant | No | Our results suggest a clinically important estimated risk reduction of 0.EE-fold, but this did not reach statistical significance. The confidence interval does provide strong evidence against any substantial harm, arguing against increases in risk of more than U.UU-fold. |
| Important Benefit | Important Benefit | Small, Unimportant | Yes | We estimated a clinically important and statistically significant 0.EE-fold reduction in risk. Our results therefore suggest a clinically important benefit, although the upper confidence limit of 0.UU is too little risk reduction to be clinically important. |
| Important Benefit | Important Benefit | Maybe Important Harm | No | Our results suggest a clinically important estimated risk reduction of 0.EE-fold, but this did not reach statistical significance. In addition, the confidence interval leaves open the possibility of harm, because it includes increases in risk of up to U.UU-fold, which could possibly be considered clinically important. |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Important Benefit | Important Benefit | Important Harm | No | Our results provide weak evidence for a clinically important 0.EE-fold risk reduction. This did not reach statistical significance, and the confidence interval even leaves open the possibility of harm, because it includes increases in risk up to U.UU, which is large enough to be clinically important. |
| Maybe Important Benefit | Important Benefit | Maybe Important Benefit | Yes | The estimated 0.EE-fold risk reduction was statistically significant and large enough to possibly be clinically important. The confidence interval leaves open the possibility of a clearly clinically important risk reduction (as much as 0.LL) and argues against a clearly unimportant benefit, because the upper bound of 0.UU-fold risk reduction could be considered important. |
| Maybe Important Benefit | Important Benefit | Small, Unimportant | No | The estimated 0.EE-fold risk reduction was enough to possibly be considered clinically important but was not statistically significant. The confidence interval leaves open the possibility of risk reduction large enough to be clearly clinically important (0.LL-fold) and argues against any important harm, because the upper bound of U.UU is too small to be important. |
| Maybe Important Benefit | Important Benefit | Small, Unimportant | Yes | The estimated 0.EE-fold risk reduction was statistically significant and large enough to possibly be considered clinically important. The confidence interval leaves open the possibility of risk reductions clearly large enough (as much as 0.LL) to be clinically important and some small enough (as little as 0.UU) to be clearly unimportant. |
| Maybe Important Benefit | Important Benefit | Maybe Important Harm | No | The estimated 0.EE-fold risk reduction was not statistically significant but was large enough to possibly be considered clinically important. The confidence interval leaves open the possibility of risk reductions (as much as 0.LL) of clear clinical importance but also increases in risk large enough (up to U.UU) to possibly be considered clinically important harm. |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Maybe Important Benefit | Important Benefit | Important Harm | No | The estimated 0.EE-fold risk reduction was not statistically significant but was large enough to possibly be considered clinically important. The study is relatively uninformative, because the confidence interval includes both clinically important risk reductions (as much as 0.LL) and clinically important increases in risk (up to U.UU). |
| Maybe Important Benefit | Maybe Important Benefit | Maybe Important Benefit | Yes | The estimated 0.EE-fold risk reduction was statistically significant and large enough to possibly be considered clinically important. The confidence interval provides strong evidence for a risk reduction in the range 0.LL to 0.UU-fold, but the clinical importance of reductions in this range could be considered unclear. |
| Maybe Important Benefit | Maybe Important Benefit | Small, Unimportant | No | The estimated 0.EE-fold risk reduction was not statistically significant but was large enough to possibly be considered clinically important. The confidence argues against reductions large enough to be clearly important (at most 0.LL-fold) and also against increases large enough to be important (no more than U.UU-fold). |
| Maybe Important Benefit | Maybe Important Benefit | Small, Unimportant | Yes | The estimated 0.EE-fold risk reduction was statistically significant and large enough to possibly be considered clinically important. The confidence argues against reductions large enough to be clearly important (at most 0.LL-fold), but leaves open the possibility of reductions too small to be important (as little as 0.UU-fold). |
| Maybe Important Benefit | Maybe Important Benefit | Maybe Important Harm | No | The estimated 0.EE-fold risk reduction was not statistically significant but was large enough to possibly be considered clinically important. The confidence argues against reductions large enough to be clearly important (at most 0.LL-fold), and it leaves open the possibility of increases large enough to possibly be considered important (as much as U.UU-fold). |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Maybe Important Benefit | Maybe Important Benefit | Important Harm | No | The estimated 0.EE-fold risk reduction was not statistically significant but was large enough to possibly be considered clinically important. The confidence argues against reductions large enough to be clearly important (at most 0.LL-fold), and it leaves open the possibility of increases large enough to be important (as much as U.UU-fold). |
| Small, Unimportant | Important Benefit | Small, Unimportant | No | The estimated relative risk of E.EE suggests that there is no clinically important effect on risk, and it was not statistically significant. The confidence interval argues against any clinically important increase in risk (at most U.UU-fold), but it leaves open the possibility of a clinically important risk reduction (as much as 0.LL-fold). |
| Small, Unimportant | Important Benefit | Small, Unimportant | Yes | The estimated 0.EE-fold risk reduction suggests that there is no clinically important reduction in risk, although it was statistically significant. The confidence interval nevertheless leaves open the possibility of a clinically important risk reduction (as much as 0.LL-fold). |
| Small, Unimportant | Important Benefit | Maybe Important Harm | No | The estimated relative risk of E.EE suggests that there is no clinically important effect on risk, and it was not statistically significant. The confidence interval leaves open the possibility of a clinically important risk reduction (as much as 0.LL-fold) and also the possibility of an increase in risk large enough to possibly be considered important (U.UU-fold). |
| Small, Unimportant | Important Benefit | Important Harm | No | The estimated relative risk of 0.EE was not statistically significant and may provide some weak evidence that the relative risk is too small to be clinically important. Unfortunately, the study was relatively uninformative, because the confidence interval included both clinically important risk reductions (as much as 0.LL) and clinically important increases in risk (up to U.UU). |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Small, Unimportant | Maybe Important Benefit | Small, Unimportant | No | The estimated 0.EE-fold risk reduction suggests that there is no clinically important reduction in risk, although it was statistically significant. The confidence interval nevertheless leaves open the possibility of enough risk reduction (as much as 0.LL-fold) to possibly be considered important. |
| Small, Unimportant | Maybe Important Benefit | Small, Unimportant | Yes | The estimated 0.EE-fold risk reduction suggests that there is no clinically important reduction in risk, although it was statistically significant. The confidence interval leaves open the possibility of enough risk reduction (up to 0.LL-fold) to possibly be considered important. |
| Small, Unimportant | Maybe Important Benefit | Maybe Important Harm | No | The estimated relative risk of E.EE suggests that the effect is too small be clinically important, and it was not statistically significant. The confidence interval, however, leaves open the possibility of enough risk reduction (up to 0.LL-fold) to possibly be considered important and enough risk increase (up to U.UU-fold) to possibly be considered important. |
| Small, Unimportant | Maybe Important Benefit | Important Harm | No | The estimated relative risk of E.EE suggests that the effect is too small be clinically important, and it was not statistically significant. The confidence interval, however, leaves open the possibility of enough risk reduction (up to 0.LL-fold) to possibly be considered important and the possibility of a clearly substantial risk increase (up to U.UU-fold). |
| Small, Unimportant | Small, Unimportant | Small, Unimportant | No | Our results provide strong evidence that the relative risk is too small to be clincially important, because the estimated relative risk of E.EE and the confidence interval from L.LL to U.UU are all small, unimportant effects. |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Small, Unimportant | Small, Unimportant | Small, Unimportant | Yes | Our results provide strong evidence that the relative risk is too small to be clincially important, because the estimated relative risk of E.EE and the confidence interval from L.LL to U.UU are all small, unimportant effects. This remains true even though the results were statistically significant. |
| Small, Unimportant | Small, Unimportant | Maybe Important Harm | No | The estimated relative risk of E.EE suggests that the effect is too small be clinically important, and it was not statistically significant. The confidence interval argues against any substantial risk reduction (at most 0.LL-fold) but leaves open the possibility of enough risk increase (up to U.UU-fold) to possibly be considered important. |
| Small, Unimportant | Small, Unimportant | Maybe Important Harm | Yes | The estimated relative risk of E.EE suggests that the effect is too small be clinically important, although it was statistically significant. The confidence interval leaves open the possibility of enough risk increase (up to U.UU-fold) to possibly be considered important. |
| Small, Unimportant | Small, Unimportant | Important Harm | No | The estimated relative risk of E.EE suggests that the effect is too small be clinically important, and it was not statistically significant. The confidence interval argues against any substantial risk reduction (at most 0.LL-fold) but leaves open the possibility of a substantial risk increase (up to U.UU-fold). |
| Small, Unimportant | Small, Unimportant | Important Harm | Yes | The estimated relative risk of E.EE suggests that the effect is too small be clinically important, although it was statistically significant. The confidence interval, however, leaves open the possibility of a substantial risk increase (up to U.UU-fold). |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Maybe Important Harm | Important Benefit | Maybe Important Harm | No | The estimated E.EE-fold risk increase was not statistically significant but was large enough to possibly be considered clinically important. The confidence interval leaves open the possibility of a clinically important risk reduction (as much as 0.LL-fold) and also the possibility of an increase in risk large enough to possibly be considered important (U.UU-fold). |
| Maybe Important Harm | Important Benefit | Important Harm | No | The estimated E.EE-fold risk increase was not statistically significant but was large enough to possibly be considered clinically important. The study is relatively uninformative, because the confidence interval includes both clinically important risk reductions (as much as 0.LL) and clinically important increases in risk (up to U.UU). |
| Maybe Important Harm | Maybe Important Benefit | Maybe Important Harm | No | The estimated E.EE-fold risk increase was not statistically significant but was large enough to possibly be considered clinically important. The confidence argues against increases large enough to be clearly important (at most U.UU-fold), and it leaves open the possibility of decreases large enough to possibly be considered important (as much as 0.LL-fold). |
| Maybe Important Harm | Maybe Important Benefit | Important Harm | No | The estimated E.EE-fold risk increase was not statistically significant but was large enough to possibly be considered clinically important. The confidence interval leaves open the possibility of enough risk reduction (up to 0.LL-fold) to possibly be considered important and the possibility of a clearly substantial risk increase (up to U.UU-fold). |
| Maybe Important Harm | Small, Unimportant | Maybe Important Harm | No | The estimated E.EE-fold risk increase was not statistically significant but was large enough to possibly be considered clinically important. The confidence interval argues against any substantial risk reduction (at most 0.LL-fold) and against risk increases large enough to clearly be important (at most U.UU-fold). |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Maybe Important Harm | Small, Unimportant | Maybe Important Harm | Yes | The estimated E.EE-fold risk increase was statistically significant and large enough to possibly be considered clinically important. The confidence argues against increases large enough to be clearly important (at most U.UU-fold), but leaves open the possibility of increases too small to be important (as little as L.LL-fold). |
| Maybe Important Harm | Small, Unimportant | Important Harm | No | The estimated E.EE-fold risk increase was not statistically significant but was large enough to possibly be considered clinically important. The confidence interval argues against any substantial risk reduction (at most 0.LL-fold) but leaves open the possibility of a clearly substantial risk increase (up to U.UU-fold). |
| Maybe Important Harm | Small, Unimportant | Important Harm | Yes | The estimated E.EE-fold risk increase was statistically significant and large enough to possibly be considered clinically important. The confidence interval leaves open the possibility of clearly important risk increases (as much as U.UU) and some small enough (as little as L.LL) to be clearly unimportant. |
| Maybe Important Harm | Maybe Important Harm | Maybe Important Harm | Yes | The estimated 0.EE-fold risk increase was statistically significant and large enough to possibly be considered clinically important. The confidence interval provides strong evidence for a risk increase in the range L.LL to U.UU-fold, but the clinical importance of increases in this range could be considered unclear. |
| Maybe Important Harm | Maybe Important Harm | Important Harm | Yes | The estimated E.EE-fold risk increase was statistically significant and large enough to possibly be clinically important. The confidence interval leaves open the possibility of a clearly clinically important risk increase (as much as U.UU) and argues against a clearly unimportant increase, because the lower bound of L.LL-fold risk reduction could be considered important. |
|
|
|
|
|
|
| Estimate | Lower Confidence Bound | Upper Confidence Bound | P<0.05? | Model Text for a Relative Risk |
| Important Harm | Important Benefit | Important Harm | No | Our results provide weak evidence for a clinically important E.EE-fold risk increase This did not reach statistical significance, and the confidence interval even leaves open the possibility of benefit because it includes decreases in risk of as much as 0.LL-fold, which is large enough to be clinically important. |
| Important Harm | Maybe Important Benefit | Important Harm | No | Our results suggest a clinically important estimated risk increase of E.EE-fold, but this did not reach statistical significance. In addition, the confidence interval leaves open the possibility of benefit, because it includes reductions in risk of as much as U.UU-fold, which could possibly be considered clinically important. |
| Important Harm | Small, Unimportant | Important Harm | No | Our results suggest a clinically important estimated risk increase of E.EE-fold, but this did not reach statistical significance. The confidence interval does provide strong evidence against any substantial benefit arguing against reductions in risk of more than 0.LL-fold. |
| Important Harm | Small, Unimportant | Important Harm | Yes | We estimated a clinically important and statistically significant E.EE-fold increase in risk. Our results therefore suggest a clinically important harm although the lower confidence limit of L.LL is too little risk increase to be clinically important. |
| Important Harm | Maybe Important Harm | Important Harm | Yes | We estimated a clinically important and statistically significant E.EE-fold increase in risk. The confidence interval supports an increase in risk by at least a factor of L.LL, but this could possibly be considered too little risk reduction to be clinically important. |
| Important Harm | Important Harm | Important Harm | Yes | We found strong evidence for clincially important risk increase, with the estimated increase being E.EE-fold and the confidence interval supporting a substantial increase in risk by at least a factor of L.LL. |
 |
|
|
|
