JAMA: Many FDA-approved cardiac devices need better studies
Pre-market approval (PMA) by the FDA of cardiovascular devices is often based on studies that lack adequate strength or may have been prone to bias, according to a study in the Dec. 23/30 issue of the Journal of the American Medical Association. The researchers found that of nearly 80 high-risk devices, the majority received approval based on data from a single study.
"In 2008, at least 350,000 pacemakers, 140,000 implantable cardioverter-defibrillators, and 1,230,000 stents were implanted. Although there has been recent scrutiny of evidence used in the FDA drug approval process, less attention has been paid to the approval process for medical devices," the authors wrote. They added that study data on which FDA approval is based should be of high quality.
"Ideally, this evidence should consist of randomized, double-blinded studies with adequate controls, sufficient duration, and thorough follow-up on prespecified primary end points without bias,” wrote Sanket S. Dhruva, MD, of the University of California, San Francisco, and colleagues.
The researchers analyzed the type and quality of study evidence used by the FDA for the PMA of cardiovascular devices. These types of devices were included in the study because it was expected they would undergo the most stringent approval process, given their increasing usage and potential impact on illness and risk of death.
The authors conducted a systematic review of 123 summaries of safety and effectiveness data (SSEDs) for 78 PMAs for high-risk cardiovascular devices that received PMA between January 2000 and December 2007, examining the methodological characteristics and primary end points. SSEDs are intended to present a reasoned, objective and balanced critique of the scientific evidence, which served as the basis of the decision to approve or deny the PMA.
The researchers found that of the 78 PMAs, 65 percent were supported by a single study. Of the 123 studies, 80 percent reported the number of participants enrolled. Of 123 studies in SSEDs, 27 percent were randomized and 14 percent were blinded. Fourteen percent of the studies reported did not have a primary end point stated. Of the 213 primary endpoints, 52 percent were compared with controls.
"In the SSEDs, there were 157 primary end points for which both the number enrolled and analyzed were stated. Of these, 122 (78 percent) had a discrepancy between the number enrolled and those analyzed," the authors wrote. One hundred thirteen discrepancies (93 percent) were that more patients were enrolled than analyzed. They added that the discrepancies between the number of enrolled patients and the number analyzed for primary end points may introduce bias because patients with less favorable outcomes may be lost to follow-up and safety concerns may underlie this missing data.
The researchers also found that of the 213 primary end points reported in the SSEDs, the results of 15 percent were non-interpretable. The most common reason was that no target goal for device performance was stated in 25 end points (78 percent), and in one instance the results were not stated. The authors stated, "In some instances, end points were interpreted to meet their targets when they may have met only a part of them."
The authors wrote that there are several possible reasons why the criteria on which FDA device approval is based appear to be less rigorous than those for drug approvals.
"First, device approvals are a more recent activity for the FDA, having begun in 1976 with the FDA Device Amendment, so the agency has less experience with devices than it does with drugs. Further, the last decade has brought a significant increase in the number and complexity of devices," Dhruva and colleagues wrote. They added that new surgical operations do not require FDA approval, and new devices, which are nearly always implanted, are between surgical operations and drugs on the FDA approval continuum.
The authors acknowledged that their study was based on the information presented in the SSEDs and some study information may be missing. However, they argued that SSEDs "should be a thorough and accurate compilation of the FDA's critique of evidence."
"The emphasis at the FDA in the last 17 years since the Prescription Drug User Fee Act has been rapid approval of new drugs,” they concluded. “This study suggests that the emphasis for the FDA in 2009 and beyond must be approvals based on research that meets rigorous scientific standards for evidence of benefit and lack of harm to patients. To uphold the FDA's mission of ensuring 'safe and effective' medical devices, it is essential that high-quality studies and data are available."
"In 2008, at least 350,000 pacemakers, 140,000 implantable cardioverter-defibrillators, and 1,230,000 stents were implanted. Although there has been recent scrutiny of evidence used in the FDA drug approval process, less attention has been paid to the approval process for medical devices," the authors wrote. They added that study data on which FDA approval is based should be of high quality.
"Ideally, this evidence should consist of randomized, double-blinded studies with adequate controls, sufficient duration, and thorough follow-up on prespecified primary end points without bias,” wrote Sanket S. Dhruva, MD, of the University of California, San Francisco, and colleagues.
The researchers analyzed the type and quality of study evidence used by the FDA for the PMA of cardiovascular devices. These types of devices were included in the study because it was expected they would undergo the most stringent approval process, given their increasing usage and potential impact on illness and risk of death.
The authors conducted a systematic review of 123 summaries of safety and effectiveness data (SSEDs) for 78 PMAs for high-risk cardiovascular devices that received PMA between January 2000 and December 2007, examining the methodological characteristics and primary end points. SSEDs are intended to present a reasoned, objective and balanced critique of the scientific evidence, which served as the basis of the decision to approve or deny the PMA.
The researchers found that of the 78 PMAs, 65 percent were supported by a single study. Of the 123 studies, 80 percent reported the number of participants enrolled. Of 123 studies in SSEDs, 27 percent were randomized and 14 percent were blinded. Fourteen percent of the studies reported did not have a primary end point stated. Of the 213 primary endpoints, 52 percent were compared with controls.
"In the SSEDs, there were 157 primary end points for which both the number enrolled and analyzed were stated. Of these, 122 (78 percent) had a discrepancy between the number enrolled and those analyzed," the authors wrote. One hundred thirteen discrepancies (93 percent) were that more patients were enrolled than analyzed. They added that the discrepancies between the number of enrolled patients and the number analyzed for primary end points may introduce bias because patients with less favorable outcomes may be lost to follow-up and safety concerns may underlie this missing data.
The researchers also found that of the 213 primary end points reported in the SSEDs, the results of 15 percent were non-interpretable. The most common reason was that no target goal for device performance was stated in 25 end points (78 percent), and in one instance the results were not stated. The authors stated, "In some instances, end points were interpreted to meet their targets when they may have met only a part of them."
The authors wrote that there are several possible reasons why the criteria on which FDA device approval is based appear to be less rigorous than those for drug approvals.
"First, device approvals are a more recent activity for the FDA, having begun in 1976 with the FDA Device Amendment, so the agency has less experience with devices than it does with drugs. Further, the last decade has brought a significant increase in the number and complexity of devices," Dhruva and colleagues wrote. They added that new surgical operations do not require FDA approval, and new devices, which are nearly always implanted, are between surgical operations and drugs on the FDA approval continuum.
The authors acknowledged that their study was based on the information presented in the SSEDs and some study information may be missing. However, they argued that SSEDs "should be a thorough and accurate compilation of the FDA's critique of evidence."
"The emphasis at the FDA in the last 17 years since the Prescription Drug User Fee Act has been rapid approval of new drugs,” they concluded. “This study suggests that the emphasis for the FDA in 2009 and beyond must be approvals based on research that meets rigorous scientific standards for evidence of benefit and lack of harm to patients. To uphold the FDA's mission of ensuring 'safe and effective' medical devices, it is essential that high-quality studies and data are available."