Saving the World with Bayesian Modeling(pymc-labs.io) |
Saving the World with Bayesian Modeling(pymc-labs.io) |
Another question, what are the practical limits of MCMC with lot? I asked a question on statsexchange a year ago [1] that had ~2000 data points, but it is analogous to a research problem I have that would have more like ~100 million data points. Would a model like the one I suggested there ever finish running?
[1] https://stats.stackexchange.com/questions/441250/pope-effect...
Julia also does better with differential equations in MCMC in general, in my experience compared to pymc3, but it can still be problematic. In some cases, it is most ideal to use the wrapper DiffEqBayes, but it has been somewhat abandoned in support for a neural differential equations package, DiffEqFlux. In some cases if you have a lot of data input, it simply is not possible to use DiffEqBayes. Stan allows for a lot of data input too alongside the system of differential equations, without any issue at all.
Not only that, Statistical Rethinking is based on Stan, first and foremost. Everything else (and there are a lot of renditions in other MCMC packages) is reprogrammed and reformatted to perform the same tasks as Stan does, and in a lot of cases, it just is not as nice.
As of 2019 you should probably always be using direct Turing.jl: https://turing.ml/dev/tutorials/10-bayesiandiffeq/ . If you slap the differential equation solver into a Turing model it just works, so the wrapper was basically obsoleted when that came into fruition which had nothing to do with DiffEqFlux.jl (on the surface at least: DiffEqFlux.jl led to internal changes that made it very easy). You're then free to write any Julia code you want for handling data, so you can minibatch or whatever you want.
DiffEqBayes.jl still exists because there's no reason to break old code (I am a fan of stability in package ecosystems), and it gives a way to automatically generate Stan differential equation code which is nice for benchmarking. But since the benchmarks show Turing.jl as faster and more robust than Stan (in this context), and the wrapper is less flexible than directly using Turing.jl, at this point the documentation is very clearly pointing people to just use Turing.jl directly, like: https://diffeq.sciml.ai/stable/analysis/parameter_estimation... or https://diffeqflux.sciml.ai/dev/examples/turing_bayesian/ . So if you follow what the current documentation says to do (as of the last few years), then what is quoted is a non-issue.