Introduction to Bayesian Data Analysis

1. 00:00In the last lecture I gave you an example of the beta binomial conjugate case in which I showed you that you can easily derive
2. 00:08the posterior by just multiplying out likelihood and the prior.
3. 00:12And you can choose different prior specifications for θ.
4. 00:15So today in this lecture I'm going to talk about another conjugate case which is the Poisson-Gamma case.
5. 00:23Okay.
6. 00:23So the example that I want to work with is some real data that it comes from one of my old papers.
7. 00:28What we have is from an eye tracking study.
8. 00:30So we are looking at reading times when people are reading sentences on the computer screen, you can record the number of
9. 00:37leftward eye movements that you make from a particular word when you're reading a sentence.
10. 00:42So that's what the data looks like.
11. 00:44You would get some Discrete values.
12. 00:4701234.
13. 00:49That would tell you the number of regressions that you're making at a particular word in some particular condition and for
14. 00:55some particular subject or item and so on.
15. 00:57Okay.
16. 00:58So how would I analyze this data?
17. 01:00Well, first of all, if you take a look at the data, if I look at the summary, you know, I see that the data
18. 01:05has a maximum value of 21, minimum of zero, lots of missing data, we can ignore that for now.
19. 01:11And although I'm getting a mean of 1.165.
20. 01:15The actual numbers are discrete numbers here.
21. 01:18Okay.
22. 01:18You can't get 1.5 regressive eye movement in a particular word.
23. 01:23So because the outcomes are discrete.
24. 01:27We're going to model it using some kind of discrete random variable and one useful, you know, random variable you can use
25. 01:35for this particular type of data is the Poisson distribution.
26. 01:40So the Poisson distribution is defined as shown in equation 1.
27. 01:44It's called a parameter λ which is the rate parameter, the rate at which you do those regressive eye movements.
28. 01:51And of course we don't know what this λ is.
29. 01:54It's always a question of estimating it from the data and access the data here.
30. 01:58Okay.
31. 01:59And x! here is I assume that you know what this is.
32. 02:03You x! is a mathematical term here.
33. 02:06That should be familiar to everyone.
34. 02:09Okay.
35. 02:10So how do I go about doing a Bayesian analysis given some data of this type?
36. 02:16So we're going to just simulate some data because I don't want to work with the complexity of the real data right now, just
37. 02:23to illustrate the point, suppose I have 10 data points that are coming from some λ distribution with some unknown λ
38. 02:31parameter.
39. 02:32So these are the data points that are generated randomly.
40. 02:36This this will serve as our data now.
41. 02:38Okay, so that's an example.
42. 02:39So
43. 02:43These are the 10 data points.
44. 02:44Again, you can see that they have this discrete distribution.
45. 02:48So they put the points here because there's nothing between zero and one for example.
46. 02:53That's why this is a discrete distribution.
47. 02:55Okay, so now let's assume for now that prior research on the topic of leftward eye movements or maybe expert knowledge.
48. 03:06There are many eye tracking experts in the world, in psychology, in other areas suggests that the prior mean of λ
49. 03:13So the rate of regressions is 3 and the prior variance is 1.5.
50. 03:18Okay, So this could be a possible prior knowledge that we have from previous work.
51. 03:23So the first step that we will take it, we're going to have to define a prior distribution
52. 03:28PDF for the rate parameter, λ in our Poisson distribution.
53. 03:33Because we're going to carry out that multiplication likelihood times prior.
54. 03:37The parameter is now not θ at λ.
55. 03:39So, I'm going to have to define a prior for λ.
56. 03:43And this prior has to reflect or should ideally reflect our prior belief or knowledge about λ before we've
57. 03:51seen any new data.
58. 03:53So one it turns out, okay, it just turns out that one good choice for a prior for the λ parameter is the Gamma distribution
59. 04:02which I will explain in a minute.
60. 04:04The Gamma distribution is continuous distribution which is again defined in terms of two parameters a and b.
61. 04:13Okay, so they have different names in R but in statistics, you will see that's generally defined in terms of some parameters
62. 04:20called a and b.
63. 04:22Now.
64. 04:23At this point, Usually people start getting nervous about prior specification and people keep asking me, my students always
65. 04:29ask me, how do I decide what the prior is going to be?
66. 04:32I'm going to talk about that later.
67. 04:34Right now, we're going to rely on our prior knowledge from some expert, you know, these prior mean and standard prior variants
68. 04:42tells us what possible values λ can have.
69. 04:44And we're going to use that to define the prior for now.
70. 04:47Okay.
71. 04:48So what does the Gamma PDF look like?
72. 04:50But this is what it looks like?
73. 04:51For any random variable x.
74. 04:54With parameters a and b.
75. 04:56This is the definition.
76. 04:57Notice that x has to be larger than zero.
77. 04:59So the rate cannot be less than zero.
78. 05:01So we're gonna be modeling the rate parameter λ using this Gamma.
79. 05:06So, instead of x will be using λ here.
80. 05:08Because we're modeling the λ parameter and it has the right property.
81. 05:13This Gamma distribution, that rate cannot be less than zero.
82. 05:16Okay, so this is the form of the distribution and the details actually don't matter to us right now.
83. 05:22Okay.
84. 05:23As you will see, it all simplifies and everything works out very nicely in
85. 05:28these kinds of analysis.
86. 05:29Okay.
87. 05:30So in R as I mentioned, a and b have different names.
88. 05:34They're called shape and scale, respectively.
89. 05:37So, you could actually simulate data, for example, this is rgamma
90. 05:41I'm using remember the DPQR family of functions for Gamma.
91. 05:45I have it too.
92. 05:46So I've generated 10 data points which are producing simulated data from 3 and 1.
93. 05:55a and b
94. 05:55are 3 and 1 distribution.
95. 05:57And so you can see that this is the type of data that I would get.
96. 06:01So the reason I'm showing you this is that when I when I'm actually doing data analysis, I actually generate some simulated
97. 06:08data from my prior to see what whether they reflect my prior belief adequately or not.
98. 06:14So just drawing out the prior distributions graphically is a good way to understand what you're actually asserting about
99. 06:22plausible values about the prior about the parameter.
100. 06:26Okay, so this is what it looks like the (3, 1) parameter.
101. 06:29So in a practical setting it was actually analyzing data with regressive eye movements.
102. 06:34I would look at this and check whether this reflects my prior knowledge about leftward eye movements, you know?
103. 06:40So I've been doing eye tracking work for 20 years so I have a reasonable idea of what the range of variability might be in
104. 06:47leftward eye movements and experts in eye tracking generally know this too or
105. 06:52can tell you about this or you can use prior data to work this out.
106. 06:55Okay.
107. 06:56Anyway, so we are now going to decide on the a
108. 06:59and b
109. 06:59priors.
110. 07:00So we need to first need to figure out what these parameters.
111. 07:03a and b are gonna be right.
112. 07:05We have to decide on this now, we already know something.
113. 07:08We know that the mean and the variance of λ from prior research, the mean is 3 and the variance is 1.5.
114. 07:16What we also know?
115. 07:18Well, I didn't tell you this, but we know this from statistical theory,
116. 07:21that in the Gamma distribution, the mean of the Gamma is a/b.
117. 07:27And the variance of the Gamma distribution is (a/b)^2.
118. 07:32So this is very cool because I have two variables a and b and I have two equations now that I can write out.
119. 07:39So all I'm gonna do in the next lecture is that I'm going to simply carry out this calculation and figure out what a and
120. 07:47b is.
121. 07:47Given these two equations and given these two values for those two equations.
122. 07:53So we're just going to solve for a and b.
123. 07:55Simple algebra.
124. 07:56And this will give us the parameters on the Gamma prior
125. 07:59that we will plug into the Bayes' rule to get the posterior on the λ parameter.
126. 08:05Okay, so that's what will happen in the next lecture.

Щоб увімкнути запис, виберіть мову в меню налаштувань відео.