Pollsters and physicists are now turning to a new optimization technique that speeds up Bayesian inference without requiring additional work from scientists. This new method, called deterministic ADVI (DADVI), offers faster and more accurate results compared to traditional approaches.
"When you actually dig into what people are doing in the social sciences, physics, chemistry, or biology, they are often using a lot of the same tools under the hood. There are so many Bayesian analyses out there. If we can build a really great tool that makes these researchers lives easier, then we can really make a difference to a lot of people in many different research areas," said senior author Tamara Broderick, an associate professor in MIT’s Department of Electrical Engineering and Computer Science.
DADVI defies conventional wisdom by using an efficient approximation method, sample average approximation, to estimate unknown quantities with better accuracy and speed. This technique provides clear indications of when the optimization process is complete and offers more reliable uncertainty estimates than other methods like ADVI.
"We wanted to see if we could live up to the promise of black-box inference in the sense of, once the user makes their model, they can just run Bayesian inference and don’t have to derive everything by hand, they don’t need to figure out when to stop their algorithm, and they have a sense of how accurate their approximate solution is," Broderick explained.
The researchers tested DADVI on various real-world models and datasets, demonstrating its ability to estimate unknown parameters faster and more reliably than other methods while maintaining accuracy. This breakthrough could provide a significant boost to scientists in a wide range of fields using Bayesian inference.
"In applied statistics, we often have to use approximate algorithms for problems that are too complex or high-dimensional to allow exact solutions to be computed in reasonable time. This new paper offers an interesting set of theory and empirical results that point to an improvement in a popular existing approximate algorithm for Bayesian inference," said Andrew Gelman, a professor of statistics and political science at Columbia University.
This research, recently published in the Journal of Machine Learning Research, was supported by a National Science Foundation CAREER Award and the U.S. Office of Naval Research.
