ASC NSQIP Data is More Accurate Than Administrative Data for Surgical Patient Outcomes

As patient safety and quality improvement efforts continue to gain momentum throughout healthcare, the need for accurate sources of information is crucial, yet the question remains: Is one resource better than another? According to two new studies presented today by researchers at the 2015 American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) National Conference in Chicago, ACS NSQIP provides more accurate data than administrative data for driving surgical quality improvement in hospitals.

Currently, researchers across the country typically use several types of databases to evaluate patient outcomes. However, outcomes often vary from database to database, making it more difficult for many clinicians and researchers to know with certainty which source of information is best for developing new strategies to improve patient care.

In the first study, investigators from Inova Health System in Virginia sought to determine the validity of two sources of information for quality metrics--ACS NSQIP clinical data and administrative claims data used for billing purposes. ACS NSQIP is the leading nationally validated, risk-adjusted, outcomes-based program to measure and improve the quality of surgical care in hospitals.

"We wanted to better understand how NSQIP could be used to drive quality improvement efforts in our hospital," says lead study author Amber Trickey, MSc, PhD, a surgery epidemiologist and biostatistician at Inova Fairfax Hospital.

The researchers looked at data on 157 hospital readmissions in the Inova Fairfax Hospital ACS NSQIP database. The patients had undergone general, endovascular and colorectal surgical procedures between January and December 2013 and were readmitted within 30 days of their operations.

Medical records of all readmitted patients were reviewed by three experienced surgeons to determine the causes for readmission. The results from that review became the gold standard for hospital readmissions, with the top two reasons being surgical site infections (SSIs) and intestinal obstruction.

The researchers then assessed the accuracy of both the ACS NSQIP clinical data and the administrative claims data by comparing them both to the gold standard. The results showed that ACS NSQIP readmission reasons had a 71 percent agreement rate with the gold standard while administrative claims data had a 61 percent agreement rate. More significantly, the researchers estimated that the NSQIP analysis showed that approximately 60 percent of the hospital readmissions are potentially preventable.

"There are various reasons for the discrepancies, but one main cause is the lack of specificity within the diagnosis codes of administrative claims data," Trickey explains. In contrast, ACS NSQIP codes are standardized, very specific and can identify the primary causes for readmissions.

From here, Trickey says her team will focus on the top two reasons for hospital readmissions: SSIs and intestinal obstruction. "Our study shows that NSQIP remains a vital tool for supporting quality improvement efforts, and we see a lot of potential going forward," she said.

In another study, researchers from three institutions--the University of California San Diego, University of California Davis, and Massachusetts General Hospital, Boston-- worked together to find out if ACS NSQIP is superior to National Inpatient Sample (NIS), the largest U.S. administrative database of inpatient hospital stays. The NIS includes data on patients covered by Medicare, Medicaid, private insurance and the uninsured and contains information on patients discharged from about 1,000 community hospitals across the United States.

For the study, lead study author Anna Weiss, MD, a surgical resident at UC San Diego and her colleagues pooled data from ACS NSQIP and NIS databases for the year 2010. Their analysis involved approximately 300,000 patients who had undergone 11 major surgical procedures, including coronary artery bypass grafting, aortic valve replacement and appendectomy.

The researchers compared models built on ACS NSQIP data versus NIS data using the c-statistic. The c-statistic is a measure of model discrimination where a value of 1 indicates that the model is perfect in discrimination between cases that experienced and did not experience the adverse event, and a value of 0.5 indicates that discrimination is at the level of chance.

When the researchers compared the c-statistic for complications and mortality models built from the two datasets, they found that they were consistently higher (closer to one) for ACS NSQIP data, compared with the NIS data.

The study results showed that unadjusted complication rates were higher in hospitals in the NIS for 7 out of the 11 operations, but unadjusted mortality rates in every procedure were lower in ACS NSQIP hospitals, in comparison with NIS. The study authors concluded that patients at NSQIP hospitals seemed to have lower inpatient mortality rates than patients' hospitals in the NIS database.

"I am hopeful that we are instilling a program and a mentality of examining our own surgical outcomes in hospitals that are using NSQIP," Weiss says. "We are actually improving our own outcomes by studying them. That is what I think it might be. That is what I hope it is."

What is unique to ACS NSQIP is the detail of preoperative risk variables collected by the database, allowing for a more robust risk-adjusted analysis compared with NIS, as evidenced by higher c-statistic values, Weiss explains. In addition, the immediate feedback loop, including the ability to get resident specific outcomes, is another advantage of the ACS NSQIP database.

"If a greater number of hospitals participated in NSQIP, then those hospitals could participate in more robust surgical outcomes research, which in the end really only benefits the patient," Dr. Weiss said. "It would allow more hospitals to have more accurate risk-adjusted analyses."

Other researchers involved in the Inova Health System study include Jeffrey Wright, MPH; Jean Donovan, MSN, RN; H. David Reines, MD, FACS; Jonathan M. Dort, MD, FACS; Heather A. Prentice, MPH, PhD; Paula Graling, CNOR, DNP, RN; John Moynihan, MD, FACS.

Weiss's research colleagues for the study comparing NSQIP with NIS are Jamie E. Anderson, Massachusetts General Hospital, Boston; and David C. Chang, UC Davis, Sacramento, Calif.

Source: American College of Surgeons