Failing the stress test: Nuclear imaging AUC trip up some raters

As currently written, appropriate use criteria (AUC) for cardiac nuclear stress tests may not be as clear they need to be, according to a study published online Jan. 6 in Circulation: Cardiovascular Quality and Outcomes.

Researchers uncovered a twofold problem with the AUC when reviewing inter-rater reliability in applying 2009 AUC for radionuclide imaging. In a sample of 400 nuclear stress tests from 2006, raters identified only 61 to 70 percent of tests as appropriate. Rater identification of appropriate, uncertain or inappropriate tests also varied greatly across raters, despite orientation and training.

Siqin Ye, MD, MS, of Columbia University Medical Center in New York City, and colleagues enrolled two board-certified cardiologists not affiliated with their nuclear cardiology laboratory, two internal medicine hospitalists, two first-year cardiology fellows and two internal medicine interns. They used 40 separate randomly chosen patients as the training sample.

The cardiologists were asked to review the training sample separately; meanwhile the other six reviewers were oriented to the 2009 AUC and then completed the training sample for themselves before all were asked to review the main body of tests.

Rater agreement for the six noncardiologist raters was 0.74; agreement between those of the same training level ranged from 0.66 to 0.79.

Ye et al suggested the variability in identifying inappropriate nuclear stress tests lay in rater’s understanding and judgment based on experiences, suggesting “a potential explanation for why AUC interventions that relied on judgment of appropriateness by individual providers did not reduce inappropriate nuclear stress testing.”

“The study highlights the difficulty in applying the AUC, as they are written, to clinical practice,” wrote Grace A. Lin, MD, MAS, and Ian S. Harris, MD, of the University of California, San Francisco, in an editorial. They noted that writing AUC that allow for broad application to complex modern practice and yet are specific enough to provide meaningful guidance is difficult. “The results suggest that the AUC are failing in at least one of these regards, resulting in variation of interpretation of the criteria.”

Ye et al suggested that as future iterations of radionuclide imaging AUC classifications are developed, experts consider addressing complexity and improve decision support.