Billions of Dollars on AIDS Prevention: Did Any of It Work?

In January 2003, President George W. Bush asked Congress to back the President's Emergency Plan for AIDS Relief (PEPFAR) with a $15 billion commitment "to turn the tide against AIDS." Congress agreed, and the program--which was continued by the Obama and Trump administrations--has, as noted in The New England Journal of Medicine, "had an unprecedented impact on the pandemic of HIV and AIDS."

But how do we really know that? Yes, HIV rates have--with the exception of a few countries--declined, but can we say the programs PEPFAR funded in countries with the highest prevalence of HIV and AIDS were responsible for those declines? After all, childhood mortality rates from AIDS dropped in sub-Saharan countries that did not receive PEPFAR funding.

Unfortunately, PEPFAR didn't create a statistical plan at the outset that could easily answer these questions. A statistical plan is a bit like asking your accountant what data you should record before you start your business--so you'll know whether you're making or losing money. The question facing PEPFAR was whether they had collected enough data over the years to evaluate their program interventions.

These kinds of after-the-fact analyses are fraught with difficulty. "Planned evaluations are preferable, of course, but we can still learn a lot from post-hoc evaluations," says Donna Spiegelman, the newly appointed Susan Dwight Bliss Professor of Biostatistics at the Yale School of Public Health. "I would never let the perfect be the enemy of the good."

Few were as qualified as Spiegelman to take on the challenge of figuring out whether PEPFAR had worked. With a joint doctorate from Harvard in biostatistics and epidemiology and rigorous training in mathematical statistics, Spiegelman had become an expert in developing methods to evaluate public health prevention science. In 2014, she was the first (and as yet only) biostatistician to receive the National Institutes of Health Director's Pioneer Award, a sort of MacArthur genius grant for biomedical researchers that frees and funds them--$500,000 per year for five years--to pursue bold, risky, innovative research.

At around the same time as Spiegelman received the Pioneer Award, she met the newly appointed U.S. Global AIDS Coordinator and Ambassador-at-Large Deborah Birx. Birx, a physician and former colonel in the U.S. Army, had been director of the Centers for Disease Control and Prevention's Division of Global HIV/AIDS and worked as a physician in Kenya. As Kenya was one of the original recipients of PEPFAR funding, Birx thought it would be a good candidate for Spiegelman to look at; it would be the most data-rich environment among PEPFAR's recipients.

With doctoral student Dale Barnhart, Spiegelman focused on 10,000 clinics running what they call PMTCT--prevention of mother-to-child transmission--of AIDS. PEPFAR had spent $248 million supporting PMTCT in Kenya between 2004 and 2014, and child mortality for those aged under five years had halved during this period. "The basic idea," says Spiegelman, "was to see if child mortality went down as funding went up."

The rationale behind the intervention is that, in order to prevent the transmission of HIV to children, pregnant mothers have to be tested. "If they are tested," says Spiegelman, "they can be treated--and that treatment will drastically lower the transmission rate from 30 or 40 percent to below 1 percent."

That was the basic idea, but teasing out causal inferences was anything but simple. Thankfully, as PEPFAR is a U.S. government program, it has to report to Congress how much it spends each quarter. From this data, Barnhart extracted a decade's worth of spending on PMTCT. They then obtained neonatal and infant mortality data from the Kenyan government and other agencies.

By doing a "difference in difference" analysis, they were able to look at the effects of varying levels of funding for PMTCT among the Kenyan provinces over time, along with trends in local infant mortality, and then compare them to other provinces with similar demographics that did not have PMTCT programs--or had them but with less financial support. The team did a wide range of sensitivity analyses to model missing data, but these had little impact on the outcome.

"The results were quite dramatic," says Spiegelman. Provinces in the top quartile of PEPFAR spending in any given year over the study period experienced more than 30 percent lower infant mortality than those in the bottom quartile. "Cumulative PMTCT spending significantly reduced infant mortality, saving the lives of many Kenyan children," she says. "We've shown, and rigorously, that the U.S. investment in PEPFAR delivered on its promise."

All of this research was carried out while Spiegelman was professor of epidemiologic methods in the departments of epidemiology, biostatistics, nutrition and global health at the Harvard T.H. Chan School of Public Health. Since July 1, she has been director of the new Center for Methods in Implementation and Prevention Science (CMIPS) at Yale University's School of Public Health. "We're hoping to be the go-to place for implementation and prevention science methods in the world. It is my dream job."

Source: American Statistical Association