Introduction to Bayesian Data Analysis

1. 00:00It's time now to look at a very interesting class of regression models.
2. 00:04These are called generalized linear models and a very important case that we use a lot in science is logistic regression
3. 00:13And this also has a hierarchical version called generalized linear mixed model.
4. 00:17But today I will only introduce the generalized linear model with the special case of the logistic regression example.
5. 00:26So let's start with an example as usual.
6. 00:30So imagine doing an experiment like the following.
7. 00:33What we want to do is we show our participants different words, one after another in a sequence.
8. 00:41So for example dog had what's that third word run and hug.
9. 00:46And so then they see a blank screen and then they are prompted to respond to what was the third word that you saw in
10. 00:56that list.
11. 00:57So they have to give a response.
12. 00:58Now that could be either correct or incorrect.
13. 01:02So they have to type in that response that's what that box is therefore.
14. 01:06So that's what the experiment is about.
15. 01:09And so what we're going to do is we're going to increase the set size.
16. 01:12The number of words that people have to see on the screen.
17. 01:15And we want to know whether their accuracy of recall is going to be affected by the number of words that they're gonna see
18. 01:22You can imagine what's going to happen.
19. 01:24As you increase the number of words accuracy is gonna tank.
20. 01:28They're gonna go down.
21. 01:29Okay.
22. 01:30So let's take a look at this data.
23. 01:31This is also in our beak oxide package.
24. 01:33This DF recall data set.
25. 01:36So if you look at this data set, it's important to note that I have repeated measurements from each subject here.
26. 01:43Okay, So again, I'm ignoring that detail, I'm gonna treat these data as independent and identically distributed.
27. 01:50I'm going to fix that problem at the end of this set of lectures but for now I'm ignoring the the independence assumption
28. 01:58here and my response is you know, the dependent variable now is no longer a millisecond reading times or pupil size or anything
29. 02:06like that.
30. 02:07These are 01 responses.
31. 02:10What distribution corresponds to a 01 response?
32. 02:13It's the Bernoulli.
33. 02:15We studied that at the beginning.
34. 02:17So here we have the 01 response to an obvious thing to use here would be the Bernoulli likelihood.
35. 02:24And the generalization of that is of course the binomial.
36. 02:27So that's why you might sometimes see in software either Bernoulli or binomial.
37. 02:32So don't be confused by that.
38. 02:33It just refers to the number of trials.
39. 02:35Right, okay, so we have some other information in this data frame that tells us which session it was which block and so on
40. 02:43and what the set size was.
41. 02:45This is the interesting thing for us.
42. 02:46So there are different numbers of words that we are exposing our participants to write and so again as before.
43. 02:54What we're gonna do is we're going to center are set size.
44. 02:57This is a vector that contains numbers like 2,4,6 and 8.
45. 03:01We're going to center it.
46. 03:03So that's either negative or positive.
47. 03:04But with a mean of zero.
48. 03:06That's what centering is doing for us.
49. 03:07And the reason we're doing this is that we want the intercept to have the interpretation that it represents the grand mean
50. 03:16the grand mean of what?
51. 03:17I will just explain that in a minute.
52. 03:19But in this model it will again represent the grand mean that's an interesting and useful thing to do statistically.
53. 03:27Okay, so let's look at what we have in this dataset.
54. 03:31So just to quickly confirm that we indeed have set sizes 2,4,6 and 8.
55. 03:37So we're going to look at the accuracy change as set size increases, theoretically what I expect from common sense.
56. 03:45Is that my accuracy will go down as the set size increases.
57. 03:49Alright.
58. 03:50So
59. 03:53yeah, Notice also that I have multiple measurements for each set size.
60. 03:59I've got 23 measurements here for each of these set sizes.
61. 04:02So I've repeated measurements on this.
62. 04:05So a simple model that we can start with which is technically not exactly right for this situation, but it's almost right
63. 04:13is the generalized linear model with a so called link function.
64. 04:20Okay, so the way this works is this follows we're going to model the 01 responses.
65. 04:25The correct incorrect responses rather than incorrect correct responses as a Bernoulli distribution.
66. 04:31Remember that?
67. 04:32The Bernoulli distribution has only one trial and there's some probability of success represented by this data parameter.
68. 04:39So we are getting these ones and zeros from subjects coming from some Bernoulli distribution.
69. 04:45And this index end that I've put under data represents the row id so every zero and one in the data frame, let's go back
70. 04:54every zero and one in the data frame is being described by some Bernoulli distribution and there is some data associated
71. 05:02with that row that is generating that data.
72. 05:06There's a generative process that I'm describing.
73. 05:09You know when I write down this model here.
74. 05:11Okay, so this model looks pretty straightforward but what we're gonna do is we're gonna put a little twist to this situation
75. 05:18The twist is that we're gonna take those theta sub ends for each of those roads and take the odds With those thetas and then
76. 05:29take the log of that.
77. 05:30So this is called this term inside the brackets.
78. 05:33It's called the odds right?
79. 05:34So if the probability of success is 50% for example the odds are 1-1 right?
80. 05:43That's how we speak so often in day to day life.
81. 05:46People will say what are the odds of it's raining today, what they actually mean is what are the what is the probability
82. 05:51of it's raining today In statistics odds has a different meaning than probability.
83. 05:55Okay, so it takes the probability value theater and converts it into a ratio in this way the ratio of probability of success
84. 06:03divided by the probability of failure.
85. 06:05Okay, so we're going to work on this log odds scale and we're going to fit a linear model on this scale or the reason
86. 06:15is that this we want to stay with the linear modeling framework because it's so convenient.
87. 06:20It has such generalize ability.
88. 06:22If I have multiple predictors, I can plug them in and I can do a lot of things now.
89. 06:27But with this Bernoulli generative process, I'm kind of limited by the nature of the 01 outputs, right?
90. 06:34The discrete outcomes.
91. 06:35So I want to convert it to some kind of continuous outcome and this is the transformation that I will do to achieve that
92. 06:44to achieve a continuous outcome.
93. 06:45So my dependent variable will now become the the log odds.
94. 06:53Okay, And I'm going to fit a model that's going to let me just write it out.
95. 07:01What's going to happen is that I'm gonna have log I'm just going to drop the subscript for just for convenience now and I'm
96. 07:10going to say that I'm going to fit a model not on 01 responses, but I'm going to fit the model on the log odds
97. 07:18scale.
98. 07:20So I will have some predictor.
99. 07:21The story remains exactly the same.
100. 07:24That's the beauty of the gen linear model that we stay with the regression modeling framework.
101. 07:28We don't have to depart from this general framework that is so well established, But now we've got our predictor.
102. 07:34So call it I don't know set size in this case let's say it was centered set size.
103. 07:41We have centered set size.
104. 07:45And so basically this is the standard predictor as before.
105. 07:50Some vector of values.
106. 07:51And we have an intercept and slope now but the intercept and slope is going to be on the log odds scale.
107. 08:00So I will unpack the implications of all this in a minute.
108. 08:04Okay, so that's the model.
109. 08:07And so first of all, let's try to understand what it means to convert a value of probability from the probability scale to
110. 08:17the log out scale, graphical visualizing visualization is the key to everything.
111. 08:22Whenever you're trying to understand statistical concepts, try to visualize the ideas graphically.
112. 08:27So that's what I'm going to do here.
113. 08:29I have created a vector here going from close to zero to close to one.
114. 08:33So this is all the possible values, discrete values of probabilities.
115. 08:38This X.
116. 08:39Then what I did is I converted it to a log odd scale.
117. 08:43Okay.
118. 08:43And so then I just created a data frame that has the probabilities theta
119. 08:48and what I'm calling eta which is the log odds transformation of the probabilities.
120. 08:55Okay, so you can plot all this.
121. 08:57I'm going to show you the plots.
122. 08:58I just wanted to provide the figures so that you can see how it's produced.
123. 09:03It's basically very straightforward.
124. 09:04I'm just using the q plot function G plot to to generate on the X axis side the thetas and on the y axis I
125. 09:12have the right And so what this gives me right, I'll show you is this function here?
126. 09:20So I put in the thetas here on the X axis, on the y axis, I get the log odds equivalent.
127. 09:25And you can see that Now I've got a continuous distribution here.
128. 09:30That I can now use in my generalized linear model.
129. 09:35So this is a very convenient approach that I can take to model this.
130. 09:39And what's even more interesting is, so this is called the logic link function.
131. 09:46It's a function that takes us input the probability and returns the log odds.
132. 09:50Okay, that's called the logic link.
133. 09:53The inverse of the largest link is called the logistic function and it takes us input the data values.
134. 10:00That means these log odds and gives you back the probabilities.
135. 10:05So that's actually pretty easy to do if you can imagine how this would work.
136. 10:09So let me show you how can I figure out what the probabilities would be in this kind of model that I've written out here
137. 10:17Let me write the model out again, just to be clear.
138. 10:21So the model I actually talked about was that I've got log of theater one minus theater
139. 10:30is equal to on the right hand side I have some terms alpha plus beta, blah blah blah
140. 10:36which I will just write as mu, I could just expand that to alpha plus beta, whatever centered set size.
141. 10:46So this is just just an abbreviation just to make my life easier as I write out the story.
142. 10:51Okay, so now how would I figure out the problem to be solved is solved for theater?
143. 10:59How do I do that algebra to the rescue?
144. 11:02Right, exponentially both sides get rid of the log if I exponentially ate the log, right, I get theater over one minus theta
145. 11:12right bracket missing here.
146. 11:14And then I exponential at the right hand side as well.
147. 11:18So then I get exponent of mu, so this guy disappears.
148. 11:22So on the left hand side I've got theta over one minus theta.
149. 11:27And on the right hand side I've got exponent of mu right, what do I do now I'm trying to solve for theta.
150. 11:33Okay, so let me multiply both sides with one minus theta to get rid of that annoying denominator.
151. 11:38And so I end up with data on this side and I end up with one minus theta times exponent of mu on the right
152. 11:46hand side.
153. 11:47So how do I proceed?
154. 11:48Now I'm still trying to solve for theta.
155. 11:50Okay, so let me solve, let me open up the brackets on the right hand side.
156. 11:54So I got exponent of mu right?
157. 11:57They're just simple math.
158. 11:59Now it's not much.
159. 12:01Not very exciting, right?
160. 12:04But the result might surprise you.
161. 12:06So I think I did this correctly.
162. 12:08Yes.
163. 12:08And so on the left hand side, I've got theta.
164. 12:10On the right hand side, I've got exponent of mu minus theta.
165. 12:13Exponent of mu, I want to get all the theta terms on one side.
166. 12:18So how do I do that?
167. 12:20I add theta exponent of mu two both sides.
168. 12:24So I get theta plus theta.
169. 12:27Exponent of mu you can see where this is going now, right?
170. 12:31Because if I end up here now, what do I do?
171. 12:35I still want to solve for theta.
172. 12:36So what do I do I take out the common term theta out of this picture
173. 12:40got theta
174. 12:41one plus exponent of mu, and the bracket missing again.
175. 12:47And then I've got exponent of mu here.
176. 12:50And lo and behold, I can solve for theta by just saying exponent of theta.
177. 12:57Oh sorry, exponent of mu I'm sorry about that, confused this.
178. 13:03Exponent of mu and divided by one plus exponent of mu, so what this means?
179. 13:10The practical implications of this are huge.
180. 13:14Because once I have estimated my parameters on the right hand side of the linear model.
181. 13:20This A and B parameters.
182. 13:21Once I've estimated these guys right, I have new for every possible set size.
183. 13:27I can compute mu and therefore I can go back to the probability space and investigate the implications of my model predictions
184. 13:35at in the probability space by using this equation.
185. 13:39So this is just the inverse, you know, of the link function.
186. 13:44And that's this guy here, right?
187. 13:46You plug in the theta the eight hours and you get back to the right.
188. 13:51That's the basic story here, right.
189. 13:52And this code is only there for your convenience to reproduce the plots.
190. 13:57If you want to understand these, how to produce these plots.
191. 14:00Okay, Alright.
192. 14:02So now that I've explained the basic idea behind this logistic regression model, I'm going to now decide on price and then
193. 14:10start fitting the model.

To enable the transcript, please select a language in the video player settings menu.