Introduction to Bayesian Data Analysis

1. 00:00Alright.
2. 00:01So now let's take a look at how we can evaluate this log-normal model with these relatively informative priors.
3. 00:09So what we fit in the previous lecture was a simple linear model with reaction time as a function of some intercept, which
4. 00:20is the mu parameter.
5. 00:21We had the space bar data which recalls reaction times in a button pressing task and we have a log-normal likelihood with
6. 00:29these priors.
7. 00:30Normal distribution mean 6,
8. 00:33Standard deviation 1.5 for the mu parameter for the intercept and a truncated normal with mean 0 and standard deviation 1
9. 00:42for sigma.
10. 00:43The truncation is done by brms internally because brms
11. 00:47as a package knows that sigma cannot have negative values.
12. 00:50So the truncation is done internally.
13. 00:53So once you have the posterior distribution for the mu parameter for example, typically in a scientific research problem
14. 01:01this is the parameter that we are primarily interested in.
15. 01:04The sigma is usually a nuisance variable for us.
16. 01:07So often there is not much focus on the sigma variable.
17. 01:11But what we're interested in is the posterior distribution of the mu parameter given the data.
18. 01:16So how would I summarize it?
19. 01:18Well, of course you could summarize the posterior in terms of the log on the log scale but it makes more sense to actually
20. 01:27look at the posterior on the millisecond scale here.
21. 01:30So you can interpret it in terms that you understand.
22. 01:32And so one way that you can do this is to basically just take the exponent of the posterior distribution.
23. 01:44This entire distribution just exponentiated. And this gives you the median, you know, of the posterior.
24. 01:52And and so what you can then do is you can calculate statistics on this distribution and report the mean and
25. 02:02the 95% credible interval.
26. 02:04So this information I would report in the paper, you know, if I'm summarizing the results of this analysis and this tells
27. 02:10me pretty much everything I need to know that, given the data and given my priors, my predicted or expected reading time is
28. 02:18167 milliseconds.
29. 02:20And I'm 95% sure given these data, given these particular data that this range is between 164 and 169. Of course,
30. 02:31this credible interval does not mean that these are the true values of this mu parameter.
31. 02:36Because these values are conditioned on the data that we happen to get.
32. 02:40Maybe we've got skewed data, maybe we've got biased data from a subject who was particularly tired or not representative
33. 02:48of their normal behavior on some other day.
34. 02:52They might not have slept the previous night for example.
35. 02:55So it could be biased data.
36. 02:57And so if the data are biased then your estimates are not gonna represent the reality.
37. 03:02So you should treat all these credible intervals
38. 03:05and this means as you should be very clear about the fact that they're conditioned on the data that you have.
39. 03:12And whether those data reflect reality or not, who knows?
40. 03:16So it's tempting to generalize from these data, from these posteriors to reveal the unknown reality out there.
41. 03:24But the reality is by definition unknown.
42. 03:26We just don't know what the true mu is.
43. 03:29We're trying to get at some estimates of these given some priors and given the data that we happen to have.
44. 03:36Okay, alright.
45. 03:37But we can still look at the posterior distributions, the posterior predictive distributions to see if the future data produced
46. 03:45by this model is reasonable given the data that we actually have.
47. 03:49So this is just a sanity check which tells us that yes, the model is producing reasonably, you know, distributed data given
48. 03:58what we've seen in this particular data set.
49. 04:01One other question that one often asks is, well, I had this normal likelihood earlier and then I switched to the log-normal
50. 04:09likelihood.
51. 04:10Did I get an improvement in the fit in some way?
52. 04:14So you could ask whether the likelihoods are, you know, one likelihood is better than the other likelihood.
53. 04:20So, that's an interesting question to ask to.
54. 04:22And you can use the posterior predictive data to check that.
55. 04:25Okay, so here is the distribution of the minimum reading times produced by the normal likelihood model that we have
56. 04:35fit long ago and this vertical bar is the observed minimum reading time.
57. 04:40That's of course a point value in the data.
58. 04:42And so what we notice is that the normal model is actually generating two small minimum reading times here.
59. 04:54The distribution is too far away from the vertical bar that we've got here. In the log-normal model, we see somewhat
60. 05:01better distribution of minimum reading times.
61. 05:05Compared to the observed data.
62. 05:06So in that sense, subjectively just looking at these figures.
63. 05:10Intuitively, I don't have any quantified statistics or anything here.
64. 05:13I'm just using a graphical check to decide whether the log-normal model does better than this normal model.
65. 05:21And the answer seems to be yes.
66. 05:23I'm producing minimum reading times that are pretty close to the actual observed minimum reading time.
67. 05:29Now one can also check whether the two models, the normal likelihood model and the log-normal likelihood model, whether
68. 05:37they reflect the maximum observed reading time in the data.
69. 05:41So the maximum reading time observed is about 400 and something milliseconds.
70. 05:47And but the normal model, the maximum reading times are very short.
71. 05:52So it's kind of missing something important.
72. 05:55The normal model is missing something important about the observed data.
73. 06:01And interestingly, the log-normal model is also underestimating the maximum reading time.
74. 06:07So if this data is representative, you know of some kind of systematic behavior of this subject, then both models actually
75. 06:15failing to capture that.
76. 06:17So often models will be imperfect in this sense.
77. 06:20Even the log-normal model does not capture or capture all aspects of the data.
78. 06:26Often that's okay. Models are always imperfect.
79. 06:29That's why it's called a model.
80. 06:31It's not the actual reality.
81. 06:32So they will miss some aspects of reality.
82. 06:34You just need to know what those aspects are and what's missing in the data.
83. 06:39So in this particular case, you know, what could be happening
84. 06:42is that what we might be looking at in the data is not just one distribution, but a mixture of distributions.
85. 06:53So what could be happening is that when we look at data in, we see the skew in the data.
86. 07:02What could lie behind that single distribution that we're seeing is a mixture of two distributions that look like one skewed
87. 07:10distribution.
88. 07:10So there could be a mixture process that's producing this data.
89. 07:14As you will see in the later lectures in the textbook, you can actually define a generative process where you assume that
90. 07:23there are a few rare slow reading times that could model this long tail here of you know, very rare but very slow reading
91. 07:33times.
92. 07:33This could be attentional timeouts that the subject is experiencing getting bored pressing the button.
93. 07:39You know, they could be in rare occasions producing very slow reading times, but most of the data could be coming from a
94. 07:45different distribution.
95. 07:46So you can actually define this kind of finite mixture models that can model this kind of data.
96. 07:52So this is an elaboration of the linear model.
97. 07:55And this also illustrates the great flexibility of the Bayesian approach.
98. 07:58You can really build elaborate process models that reflect the underlying generative process and it doesn't have to be a
99. 08:06simple generative process of the type that we are studying here.
100. 08:09That's one of the beauties of the framework that we're looking at here.
101. 08:14All right.
102. 08:15So what's happened so far, we've looked at two simple examples of a simple linear model using two different likelihoods the
103. 08:22normal and the log-normal.
104. 08:24And what we learned to do was to generate prior predictive and posterior predictive data using the
105. 08:30brms package.
106. 08:31We can do that or we can do it in R ourselves.
107. 08:34And so these two approaches to understanding the model are very important.
108. 08:39They will teach us about what the model predicts and what the underlying assumptions of the model are.
109. 08:45Whether they produced reasonable data.
110. 08:50And so what we tend to say is that these are telling us about the descriptive adequacy of the model.
111. 08:57And so we can ask both what happens before we've seen the data.
112. 09:01That's the prior predictive distribution.
113. 09:03And what happens after we've seen the data.
114. 09:05That's the posterior predictive distribution.
115. 09:07And this is usually part of the workflow of doing a Bayesian data analysis.
116. 09:11You do this prior and posterior predictive checks.
117. 09:14You do a sensitivity analysis and that constitutes a complete analysis of data set.
118. 09:20Usually you may only report the final analysis in the paper but underlying all this investigation, one should
119. 09:28do to check whether the model makes any sense at all.
120. 09:32Alright.
121. 09:33So what's going to happen next is that we are going to now elaborate on this simple linear model in several ways.
122. 09:40First we're going to add a predictor.
123. 09:42So instead of just having an intercept, we're going to have an intercept and slope.
124. 09:46And this becomes a regression model, you can also have multiple regression models now.
125. 09:51So this improves the flexibility of this linear modeling approach and allows us to ask all kinds of questions about complex
126. 09:58data sets.
127. 10:00Another example I will show you is of using logistic regression.
128. 10:04This will be familiar to people doing machine learning where you've got a 0, 1 response.
129. 10:10Where we're trying to model this kind of Bernoulli process,
130. 10:15using a linear modeling framework.
131. 10:17So we're going to use logistic regression for that.
132. 10:19And finally, I'm going to show you just a glimpse of what a linear mixed model or a hierarchical model looks like
133. 10:26where you start adding information about individual participants,
134. 10:31the variability due to individual participants into the model.
135. 10:35This is a very sophisticated framework that allows you to study individual differences for example, and it allows you to
136. 10:43analyze repeated measures of dependent data.
137. 10:46This is a very important framework and it's a bread and butter framework for many fields like psychology and linguistics.
138. 10:53So that's coming up next.

To enable the transcript, please select a language in the video player settings menu.