What? is simulation

The following is table of contents from Regression and other stories (Gelman, et al.) and Stochastic models (Ross).

  • Simulation of discrete probability models
  • continuous and mixed discrete/continuous models (inverse transformation, rejection, hazard rate + normal, gamma, chi-sq, beta, exponential, nonhomogeneous or 2d poisson, Generating from the Stationary Distribution of a Markov Chain (crtp),
  • Summarizing a set of simulation using median and median absolute deviation
  • Bootstrapping to simulate a sampling distribution, Fake data simulation as a way of life.

(+Ross): variance reduction techniques using antithetic variables, conditioning, control variates, Determining the Number of Runs, sample generation from Markov stationary distribution.

When? are simulation necessary

Uncertainty quantification, intractable likelihood, blackbox model are keywords. Just as peice-wise linear function has fine enough resolution for convex function, clever point-wise matching is enough. Information bottleneck theory explains this cleverness by formulating the weighted mutual information $I(x,t) – beta I(t,y)$ as the objective function. The underlying truth is represented with approximation that is efficient and retain enough information. The following are from Princeton engineering mathematical model book.

  • 계산모델+관측데이터+통계이론을 기반으로 numerical 분야에서 발전
  • 다음 하위분야 포함: 불확실성 전파, 민감도분석, 추론과 calibration, 불확실성하 의사결정, 실험설계, 모델검정
  • 주로 simulation만으로 접근가능한 복잡모델(intractable likelihood, NN)들의 예측신뢰구간설정에 응용됨
  • 불확실성의 종류는 다음과 같음
  1. aleatory: irreducible uncertainty from inherent variability
  2. epistemic: feducible uncertainty that reflects a lack of knowledge
  3. param. uc: 불확실한 Input제공에서 오는 불확실성
  4. param. variability: 통제불가한 모수의 불확실성
  5. residual: 주어진 모델 실행 중 코드에서 오는 불확실성
  6. obs err: error from actual observation and measurement
  7. model disc: model limitaion, assumption