You are here: CTSPedia>CTSpedia Web>CRCaseStudies>DiagnosticsComparison (13 Mar 2011, ErinEsp)EditAttach

The two new diagnostics, through separate mechanisms, have the ability to subtype Influenza. Subtyping has become a crucial part of flu diagnosis largely due to advances in preventive medicine (i.e., vaccinations). Because of this the researcher is willing to lose a bit in both sensitivity and specificity to gain accuracy in subtyping. Suppose that it is known that the two tests are equal in their ability to subtype. Joel is interested in determining which of the two tests is the optimal test with regard to sensitivity and specificity in diagnosing Influenza.

The accuracy of a test can be measured by the area under the ROC curve. Thus we can compare the accuracy of the two new diagnostic tests by comparing the areas under the ROC curves. This can be done by

- Applying the diagnostic tests to two independent random samples
- Applying the diagnostic tests to the same random sample.

We know that the test with the largest area under the ROC curve is the preferred diagnostic test. It is clear from the above plot that the area under the ROC curve for test A is larger than the area for test B. Now we need to determine whether or not this difference is statistically significant.

We calculate the area under the curves using the trapezoidal rule. The area under the ROC curve for test A was found to be 0.89 and the area under the ROC curve for test B was found to be 0.81. Following methods described by Hanley and McNeil? we can calculate the test statistic and p-value. Vassar College has created a calculator to assess the significance of the difference between the areas under two independent ROC curves. A picture of the calculator is provided below.

From this output we see that there doesn't appear to be a significant difference between the efficacy of the two new diagnostic tests.

Again we calculate the area under the curves using the trapezoidal rule. The area under the ROC curve for test A was found to be 0.89 and the area under the ROC curve for test B was found to be 0.81. Since these tests were carried out on the same individuals the ROC curves are correlated. Hence we need to take this correlation into account and therefore we cannot perform the same procedure as above. When the ROC curves are correlated the procedure becomes more complicated and interested parties should see DeLong? et al. for further details.

We note that in the above examples the ROC curve for test A is almost always above the ROC curve for test B. Thus even without a formal statistical test it is clear that test A is the preferred diagnostic. In some situations this may not be the case.

DeLong? ER, DeLong? DM, Clark-Pearson DL. Comparing the Areas Under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics, 1988, 44, 837-845.

http://faculty.vassar.edu/lowry/roc_comp.html

-- ErinEsp - 16 Jan 2011

Topic revision: r9 - 13 Mar 2011 - 19:11:11 - ErinEsp

Copyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Ideas, requests, problems regarding CTSPedia? Send feedback

Ideas, requests, problems regarding CTSPedia? Send feedback