Based my interests, I have elaborated or summarized four areas from Gelman20.

Bootstrapping and simulation-based inference


We can enhance our sufficient statistics (Xbar) and sampling distribution (normal) by resampling; their are approximated with estimates and bootstrap distribution. Like the following, population, statistics, and sampling distribution are pairs. Bootstrap allows us to investigate the sampling variability of a statistics of interest (e.g. mean, variance, skewness) without making any distributional assumptions about the population. Traditionally, we analytically elicited distribution (\bar{X} - \mu / s \sim t_{n-1}) of sufficient statistic(t) to infer population parameter (\mu) based on the its assumed distribution (N). Now we could simply compute the mean of the resamples to know its distribution. For example, if you compute the skewness of a univariate sample, you get an estimate for the skewness of the population. Bootstrapping could be used for bias correction and variance estimation where analytical calculations are not possible.

the above traditional pair (population, ss, sd) are approximated with (population, estimate, bootstrap distribution)

Unlike bootstrap where data was the only source of sampling, we could get the help of a model or data generating process in simulation-based inference. Sampling from a know data generating mechanism could complement or replace mathematical theory when analyzing complex models or algorithms. In this setting, to validate the model, we believe it to be a true data generation process and generate multiple datasets which are then compared with the observed values. As comparing the true system and its approximated version, model, is too abstract and high dimensional, we are using their outputs or summary to measure their closeness. Approximate bayesian computation where reject the prior parameters with the probability proportional to the distance of their summary statistics and the true summary statistics shares its philosophy in that known dgm is used to reproduce multiple datasets.

given p(\theta), p(y|\tilde{\theta})
repeat
\tilde{\theta} \sim p(\theta)
\tilde{y} \sim dgm(y|\tilde{\theta}) # unkonwn p(y|\tilde{\theta})
\tilde{s} <- \tilde{y}
if d(s,\tilde{s}) \leq \epsilon:
store \tilde{\theta}

resulting {\tilde{\theta}} is an approximation of the posterior p(\theta|y)

Note that even though we know dgm, we cannot express the likelihood which is needed to use the Bayes formula.

Generic computation algorithms

Generic automatic inference algorithms allowed decoupling the development of the models so that changing the model did not require changes to the algorithm implementation. They include EM algorithm, Gibbs sampler, variational inference, expectation propagation, Metropolis algorithm, and Hamiltonian Monte Carlo.

Adaptive decision analysis

Important developments in statistical decision analysis involve Bayesian optimization and reinforcement learning are related to a renaissance in experimental design for A/B testing in industry and online learning in many engineering applications. Using richly parametrized models such as Gaussian process or neural network as priors for functions in adaptive decision analysis, massive simulation are carried out in a reinforcement learning way to create artificial intelligence.

Robust inference

Robustness thrive based on the acute awareness of the imperfections of statistical models and econometricians’ interest in “as if” analysis which requires the model to perform well in a range of assumptions. We hope our models to give useful results even when their assumptions are sometimes violated.