ACC Feature: Use of SYNTAX scores in daily practice reveals operator bias
Inter- and intra-observer reliabilities show disparities when SYNTAX scores are calculated by individual operators no matter their training and background. Researchers reported that multidisciplinary approaches should be considered when deciphering between CABG and PCI for patients with multivessil or left main coronary artery disease (CAD), according to a poster presentation featured during the 59th annual American College of Cardiology (ACC) scientific session yesterday in Atlanta.
Stéphane Zuily, MD, of the Cochin Hospital in Paris, and colleagues selected 20 coronary angiograms of patients with left main or multivessel CAD at random to evaluate the inter- and intra-observer variability of SYNTAX scores assessed by an individual observer.
“In daily practice, coronary angiograms are analyzed by individual operators and discrepancies are common,” the researchers wrote.
During the study two cardiac surgeons and six interventional cardiologists were asked to spontaneously estimate a subjective SYNTAX score as one of three classes—low, intermediate or high.
The inter-observer reliability scores were calculated using the online calculator. To investigate intra-observer reliability, the same eight observers estimated the SYNTAX score six weeks after patient angiograms and the numbers were calculated using intra-class correlation coefficients and Cohen’s Kappa.
The mean age of the patients evaluated was 64.5. Eighteen men and two women were included in the study and 16 were diagnosed with multivessel CAD, one had left main CAD and three patients showed signs of both.
Researchers recorded the mean SYNTAX score to be 24 and found that differentiations between observers were significant.
“The range of results varied widely between observers coding exactly the same coronary angiograms in patients,” Zuily told Cardiovascular Business News. “We found huge discrepancies and it’s quite dangerous because from the results of the [SYNTAX] score we will have a different strategy for revascularization--either PCI or CABG,” he said.
When comparing the results of the patient angiograms, Zuily said that the researchers noted a “lack of inter-observer variability.”
Additionally, during the study, the researchers looked at differences in reporting based on experience and background of the individual observer.
The researchers found that concordance was better in cardiologists and interventional cardiologists than cardiac surgeons. The research also showed that fellows and residents fared better than senior staff members whose inter-observer variability was often too low.
In addition, the researchers had observers estimate the severity of lesions by subjective score before calculating SYNTAX score by placing patients in categories—low, intermediate and high.
“In daily practice syntax scores are provided to help physicians and to guide between PCI or CABG in all revascularization patients,” said Zuily.
Low and intermediate risk is assisted with PCI, while high syntax score associated with CABG.
“We have seen that there was a significant differentiation between the type of revascularization performed and the one which would have been chosen if the strategy based on the SYNTAX score was applied. So, in daily practice there are discrepancies," said Zuily.
The researchers concluded that when having observers estimate, “the subjective score is not a good way to assess the severity of lesions.”
“In conclusion, our study showed that inter- and intra-observer variability was noted when the SYNTAX score was calculated by individual operators, therefore we discourage individual operators to use the SYNTAX score and we recommend a multidisciplinary approach when choosing between PCI and CABG in those patients.”
According to Zuily, sharing results, comparing results and holding staff meetings could potentially reduce the variability of syntax score results.
Stéphane Zuily, MD, of the Cochin Hospital in Paris, and colleagues selected 20 coronary angiograms of patients with left main or multivessel CAD at random to evaluate the inter- and intra-observer variability of SYNTAX scores assessed by an individual observer.
“In daily practice, coronary angiograms are analyzed by individual operators and discrepancies are common,” the researchers wrote.
During the study two cardiac surgeons and six interventional cardiologists were asked to spontaneously estimate a subjective SYNTAX score as one of three classes—low, intermediate or high.
The inter-observer reliability scores were calculated using the online calculator. To investigate intra-observer reliability, the same eight observers estimated the SYNTAX score six weeks after patient angiograms and the numbers were calculated using intra-class correlation coefficients and Cohen’s Kappa.
The mean age of the patients evaluated was 64.5. Eighteen men and two women were included in the study and 16 were diagnosed with multivessel CAD, one had left main CAD and three patients showed signs of both.
Researchers recorded the mean SYNTAX score to be 24 and found that differentiations between observers were significant.
“The range of results varied widely between observers coding exactly the same coronary angiograms in patients,” Zuily told Cardiovascular Business News. “We found huge discrepancies and it’s quite dangerous because from the results of the [SYNTAX] score we will have a different strategy for revascularization--either PCI or CABG,” he said.
When comparing the results of the patient angiograms, Zuily said that the researchers noted a “lack of inter-observer variability.”
Additionally, during the study, the researchers looked at differences in reporting based on experience and background of the individual observer.
The researchers found that concordance was better in cardiologists and interventional cardiologists than cardiac surgeons. The research also showed that fellows and residents fared better than senior staff members whose inter-observer variability was often too low.
In addition, the researchers had observers estimate the severity of lesions by subjective score before calculating SYNTAX score by placing patients in categories—low, intermediate and high.
“In daily practice syntax scores are provided to help physicians and to guide between PCI or CABG in all revascularization patients,” said Zuily.
Low and intermediate risk is assisted with PCI, while high syntax score associated with CABG.
“We have seen that there was a significant differentiation between the type of revascularization performed and the one which would have been chosen if the strategy based on the SYNTAX score was applied. So, in daily practice there are discrepancies," said Zuily.
The researchers concluded that when having observers estimate, “the subjective score is not a good way to assess the severity of lesions.”
“In conclusion, our study showed that inter- and intra-observer variability was noted when the SYNTAX score was calculated by individual operators, therefore we discourage individual operators to use the SYNTAX score and we recommend a multidisciplinary approach when choosing between PCI and CABG in those patients.”
According to Zuily, sharing results, comparing results and holding staff meetings could potentially reduce the variability of syntax score results.