| Abstract | U ovom radu govorimo o MCMC algoritmima i ilustriramo njihovu primjenu u bayesovskoj statistici. MCMC algoritmi služe za simuliranje uzorka iz distribucije s gustoćom f koju moramo znati samo do na konstantu. Oni konstruiraju Markovljev lanac čija se putanja uzima kao uzorak iz distribucije s gustoćom f, što nam omogućuje ergodski teorem. Nakon motivacijskog uvoda, u prvom poglavlju obradujemo nužnu teoriju koja je u pozadini MCMC algoritama, a to je teorija Markovljevih lanaca na općenitom skupu stanja. Prvo definiramo osnovne pojmove poput prijelazne jezgre i Markovljevog lanca, a zatim neka svojstva Markovljevih lanaca poput ireducibilnosti, povratnosti, Harrisove povratnosti i stacionarnu distribuciju Markovljevog lanca. Iskazujemo teorem koji nam daje dovoljne uvjete da marginalne distribucije lanca konvergiraju prema stacionarnoj distribuciji f i ergodski teorem koji nam omogućuje da putanju lanca koristimo kao uzorak dobiven iz stacionarne distribucije f. Nakon toga obradujemo Metropolis-Hastings algoritam, jedan od najvažnijih MCMC algoritama. Dokazujemo da putanju lanca dobivenog ovim algoritmom možemo koristiti kao uzorak iz zadane ciljne distribucije f. Uzorak iz distribucije f služi nam kako bismo procijenili karakteristike distribucije f pa smo uveli efektivnu veličinu uzorka i Monte Carlo standardnu grešku kao mjere preciznosti dobivenih procjena. Kao primjere Metropolis-Hastings algoritma obradili smo Nezavisni Metropolis-Hastings algoritam i Metropolis-Hastings algoritam sa slučajnom šetnjom. Na primjeru smo vidjeli da je kod Nezavisnog Metropolis-Hastings algoritma najbolje odabrati proposal distribuciju što sličniju ciljnoj distribuciji, ukoliko je to moguće. Kod Metropolis-Hastings algoritma sa slučajnom šetnjom uočili smo da za efikasnost algoritma moramo odgovarajuće skalirati proposal distribuciju. Na kraju smo spomenuli i Gibbsovo uzorkovanje koji je vrlo važan i u primjenama čest algoritam. U zadnjem dijelu ovoga rada objašnjavamo glave ideje bayesovske statistike - pojmove poput apriori i aposteriori distribucije. Na primjeru bacanja novčića ilustriramo kako se aposteriori distribucija ponekad može izračunati egzaktno, no uočavamo i da se to ponekad neće moći zbog čega onda koristimo MCMC algoritme. Nakon toga ukratko objašnjavamo generalizirane linearne modele kako bismo mogli napraviti model linearne regresije s teškim repovima na bayesovski način. Kao primjer uzeli smo određivanje potrošnje goriva automobila. Nakon definiranja modela komentiramo izbor apriori distribucija te koristimo MCMC algoritme kako bismo dobili uzorak iz aposteriori distribucije. Prije nego što analiziramo rezultate, radimo dijagnostiku kako bismo provjerili da dobiveni uzorak doista predstavlja uzorak iz aposteriori distribucije. U dodatku objašnjavamo kako smo tehnički proveli ranije opisan postupak pomoću programskog jezika R i programa JAGS. |
| Abstract (english) | In this paper we talk about Markov chains Monte Carlo (MCMC) algorithms, and we illustrate their application in Bayesian statistics. MCMC algorithms are used to generate sample from distribution with density f, which is known up to a normalising constant only. They produce Markov chains whose trajectory represents a sample from distribution with the density f, which is justified by Ergodic theorem. After motivational introduction, in chapter one, we introduce necessary background theory of MCMC algorithms; that is the theory of Markov chains on general state spaces. First, we define basic terms such as transition kernels and Markov chains, and after that we define some properties of Markov chains such as irreducibility, recurrence, Harris recurrence and stationary distribution of Markov chain. We state a theorem which gives us necessary conditions on the chain, so that its marginal distributions converge to its stationary distribution f, and Ergodic theorem. Secondly, we define Metropolis-Hastings algorithm, one of the most important MCMC algorithms. We prove that we can use a trajectory generated with Metropolis-Hastings algorithm as sample from target distribution f. Sample from distribution f is used to approximate distribution f and its characteristics, so we introduced effective sample size and Monte Carlo standard error as measures of accuracy. Furthermore, we give examples of Metropolis-Hastings algorithm: the Independent Metropolis-Hastings and Random walk Metropolis-Hastings. On a simple example we demonstrate that in the Independent Metropolis-Hastings algorithm optimal proposal distribution should be as much proportional to target distribution as possible. In the Random walk Metropolis-Hastings algorithm we noticed that we need to scale proposal distribution for effectiveness. Finally, we mention Gibbs sampler which is very important and widely used algorithm. In the last part of this paper, we explain the main ideas of Bayesian statistics. We introduce terms like prior and posterior distribution. On a simple coin tossing example we illustrate that sometimes one can exactly calculate posterior distribution, but we notice that in cases when that is not possible, one can use MCMC algorithms. After that, we briefly explain generalised linear models, so that we can introduce heavy-tailed linear regression model from Bayesian perspective. As an example we are estimating car fuel consumption. After defining the model, we comment on choosing prior distributions, and we use MCMC algorithms to get sample from posterior distribution. Before we analize posterior distribution, we check if our sample is representative of prior distribution and accurate enough. In the appendix, we explain how we used programming language R and JAGS to give Bayesian analysis of the heavy-tailed linear regression model. |
