Risk model performs poorly at predicting atrial fibrillation

An analysis of electronic medical records (EMRs) found that a previously validated risk model for predicting atrial fibrillation did a poor job predicting atrial fibrillation.

The model underpredicted atrial fibrillation in low-risk individuals and overpredicted atrial fibrillation in high-risk individuals. The model was developed in 2012 in the Cohorts for Heart and Aging Research in Genomic Epidemiology-Atrial Fibrillation (CHARGE-AF) trial.

Lead researcher Matthew J. Kolek, MD, of Vanderbilt University in Nashville, Tennessee, and colleagues published their findings online in JAMA Cardiology on Oct. 12.

The researchers mentioned that atrial fibrillation is the most common sustained cardiac arrhythmia. By 2050, an estimated 12 million to 16 million people in the U.S. are expected to have atrial fibrillation, which is nearly double of the current incidence.

In this study, the researchers more than 33,000 deidentified EMRs of people who were at least 40 years old and had no history of atrial fibrillation. They then followed up the participants for incident atrial fibrillation from Dec. 31, 2005, through Dec. 31, 2010.

The median age was 57 years old, while 57 percent of participants were women, 85.7 percent were white and 14.3 percent were African American. The participants visited a Vanderbilt University internal medicine clinic at least three times within a 24-month period.

For each participant, the researchers also obtained CHARGE-AF model predictors, including sex, race, age, weight, height, body mass index, systolic and diastolic blood pressure, history of MI, heart failure and type 2 diabetes and treatment for hypertension.

During a mean follow-up period of 4.8 years, 7.3 percent of participants developed atrial fibrillation.

The CHARGE-AF model had a C index of 0.708, while a simple model that did not include electrocardiogram predictors had a C index of 0.709.

“Calibration for both models was poor in our cohort, indicating a failure of validation,” the researchers wrote. “Our study represents a novel use of an EMR repository to evaluate an existing [atrial fibrillation] risk model and illustrates the limitations of applying a model developed in prospective cohort studies to a real-world EMR context.”

The researchers used an automated algorithm to measure incident atrial fibrillation during the follow-up period. They also conducted a manual review of 200 random EMRs to assess the accuracy of the algorithm. The review found that the sensitivity of the algorithm was 96.5 percent, the specificity was 94.8 percent, the positive predictive value was 93.2 percent and the negative predictive value was 97.3 percent.

The researchers mentioned the study had a few limitations, including that multiple people entered data into the EMRs, which could have led to inaccuracies. In addition, although they excluded participants with atrial fibrillation in 2005, they might not have excluded all adults with atrial fibrillation at baseline. Further, they noted that the analysis was prone to indication bias, which they defined as participants developing atrial fibrillation could have had more clinical encounters than those who did not develop atrial fibrillation.

“Risk models for the development of [atrial fibrillation] or other complex disorders are unlikely to be widely used in clinical care unless they can be incorporated into EMR systems,” the researchers wrote. “Risk models, therefore, should be derived from and validated in different EMR cohorts, with the goal of prospectively and automatically identifying individuals at high risk for [atrial fibrillation] and implementing personalized strategies for primary prevention.”