Interpretation of results
20 Interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes.
Explanation
It has been argued that the discussion sections of scientific reports are filled with rhetoric supporting the authors' findings (175) and provide little measured argument of the pros and cons of the study and its results. Some journals have attempted to remedy this problem by encouraging more structure to authors' discussion of their results (176, 177). For example, Annals of Internal Medicine (176) recommends that authors structure the discussion section by presenting:
- a brief synopsis of the key findings;
- consideration of possible mechanisms and explanations;
- comparison with relevant findings from other published studies (whenever possible including a systematic review combining the results of the current study with the results of all previous relevant studies);
- limitations of the present study (and methods used to minimize and compensate for those limitations); and
- a brief section that summarizes the clinical and research implications of the work, as appropriate.
We recommend that authors follow these sensible suggestions, perhaps also using suitable subheadings in the discussion section.
Although discussion of limitations is frequently omitted from reports of original clinical research (178), identification and discussion of the weaknesses of a study have particular importance. For example, a surgical group recently reported that laparoscopic cholecystectomy, a technically difficult procedure, had significantly lower rates of complications (primary outcome) than the more traditional open cholecystectomy for the management of acute cholecystitis (179). The authors failed to discuss the potential bias of their results: namely, that the study investigators had completed all the laparoscopic cholecystectomy operations whereas 80% of the open cholecystectomy procedures had been completed by trainees. The positive results observed for laparoscopic cholecystectomy may have been merely a function of surgical experience, thus biasing the results. Evaluation of the results in light of this methodological weakness would have been helpful to readers.
Authors should also discuss any imprecision of the results, perhaps when discussing study weaknesses. Imprecision may arise in connection with several aspects of a study, including the measurement of a primary outcome (see item 6) or diagnosis (see item 3a). Perhaps the scale used was validated on an adult population but used in a pediatric one, or the assessor was not trained in how to administer the instrument. Issues such as these can lead to imprecise results and should be discussed by the authors.
The difference between statistical significance and clinical importance should always be borne in mind. Authors should particularly avoid the common error of interpreting a nonsignificant result as indicating equivalence of interventions. The confidence interval (see item 17) provides valuable insight into whether the trial result is compatible with a clinically important effect, regardless of the P value (94).
Authors should exercise special care when evaluating the results of trials with multiple comparisons. Such multiplicity arises from several interventions, outcome measures, time points, subgroup analyses, and other factors. In such circumstances, some statistically significant findings are likely to result from chance alone.
Page last edited: 17 July 2007
