Comparison of Two or More Correlated AUCs in Paired Sample Design

Abstract

Purpose of study

Methods of comparing the accuracy of diagnostic tests are of increasing necessity in biomedical science. When a test result is measured on a continuous scale, an assessment of the performance of the overall value of the test can be made using the Receiver Operating Characteristic (ROC) curve. This curve describes the discrimination ability of a diagnosis test in terms of diseased subjects from non-diseased subjects. The area under the ROC curve (AUC) describes the probability that a randomly chosen diseased subject will have higher probability of having disease than a randomly chosen non-diseased subject. For comparing two or more diagnostic test results, the difference between AUCs is often used. This paper proposes a non-parametric alternative method of comparing two or more correlated area under the curve (AUCs) of diagnostic tests for paired sample data. This method is based on Chi-square test statistic.

Methods

This paper investigated both parametric and non-parametric methods of comparing the equality of two AUCs and proposed a Chi-square test for the comparison of two or more diagnostic test processes. The proposed method does not require the knowledge of true status of subjects or gold standard in evaluating the accuracy of tests unlike other existing methods. The proposed method is most suitable for paired sample design. It also offers reliable statistical inferences even in small sample problems and circumvent the difficulties of deriving the statistical moments of complex summary statistics as seen in the Delong method. The proposed method provides for further analysis to determine the possible reason for rejecting the null hypothesis of equality of AUCs.

Results

The proposed method when applied on real data, avoids the lengthy and more difficult procedures of estimating the variances of two AUCs as a way of determining if two AUCs differ significantly. The method is validated using the Cochran Q test and was shown to compare favourably. The proposed method recommended for comparing two or more correlated AUCs when the data is paired. It is simple and does not require prior knowledge of true status of subjects unlike other existing methods.

Keywords: Chi-square test, Cochran Q test, cut-off value, area under the curve, receiver operating characteristic, Dichotomous data

DOI: 10.7176/JNSR/9-9-06

Publication date:May 31^st 2019