Package Inserts Overstate Diagnostic TB Tests' Accuracy

Clinicians and laboratory professionals often rely on manufacturers' package inserts to assess the accuracy of diagnostic medical tests. However, package inserts frequently greatly overstate such tests' accuracy, according to a case study of package inserts for tuberculosis (TB), which is published in the July 2012 Journal of Clinical Microbiology.

"It is likely that these issues are not restricted to TB, and they should be explored for diagnostics for other diseases as well," says first author Claudia M. Denkinger of Beth Israel Deaconess Medical Center in Boston.

The study grew out of an understanding that the market for TB tests includes many diagnostics which have not been recommended by independent organizations such as the World Health Organization (WHO) and the Food and Drug Administration (FDA).
 
"These diagnostics have significant market share in much of the world where the market is not regulated, notably in developing nations, despite questionable performance," says Denkinger. "Given that oftentimes end-users have only the package insert to assess a test, we wondered how well package inserts actually reflected a tests performance."

In the study, Denkinger et al. compared package inserts' evaluations to systematic reviews of 19 diagnostic tests. The package inserts tended to overrate tests' accuracies by 20-30 percent as compared with systematic reviews of their performance, although within the package inserts, tests that were recommended by independent organizations were likely to report more realistic performance. "Our study suggests that independent assessments of diagnostic tests are helpful to provide reliable data to decision-makers and clinicians on the performance of diagnostic tests," says Denkinger.

"We propose that countries create systems for in-country validation of TB tests or that international organizations such as the WHO expand their evaluation programs in order to help countries procure quality-assured TB tests."

Such studies "must go beyond accuracy and assess clinical impact of tests on decision-making and patient outcomes and collect operational and cost-effectiveness data in programmatic settings," conclude the investigators, who were led by Madhukar Pai of McGill University, Montreal, Canada.

Reference: Denkinger CM, et al. 2012. Promise versus reality: optimism bias in package inserts for tuberculosis diagnostics. J. Clin. Microbiol. 50:2455-2461.)