Künstliche Intelligenz und Maschinelles Lernen in der Praxis

1. 00:00Welcome to Unit 2.7, Result and evaluation.
2. 00:04In this unit, we want to look at the content-based Take a closer look at filtering and collaborative filtering.
3. 00:12analyze our film forecasts in detail.
4. 00:16Let's start with the content-based filtering system
5. 00:21Once again, it should be emphasized that the Content-based filtering system, as we have presented it,
6. 00:28An unsupervised machine Learning method.
7. 00:32We do not have any labels, if a film is similar to another film.
8. 00:38The content similarity is very difficult to check.
9. 00:42This can usually only be done by domain experts or by domain experts.
10. 00:48This means that these people are assessing the quality of the Results usually with a quantitative metric.
11. 00:54And yes, that's how we want to do it.
12. 00:57That means we look at the quality of the results and see if they are coherent and meaningful for us.
13. 01:04Let's start again with the old familiar Example of a movie Golden Eye, a James Bond movie.
14. 01:12Let's get some suggestions from this Golden Eye movie, we see that
15. 01:19that a lot of James Bond movies in the first proposals.
16. 01:23Casino Royale, Never Say Never Again, You Only Live Twice, Octopussy, Live And Let Die, License To Kill for example also.
17. 01:33However, it is noticeable that, for example, on place 1, The Way Of The Dragon, a movie is listed that is not a James Bond movie.
18. 01:43Now we can let ourselves be output, What are the short descriptions of these films?
19. 01:47the list, which are not James Bond movies.
20. 01:50Maybe that gives us a little hint of why these movies but still similar in content or whether our model
21. 01:57is just not good in quotation marks.
22. 02:03When we look at the short descriptions once in the And then we see a lot of words.
23. 02:11for example in Johnny Stool Pidgeon useful or at least in content similar to James Bond films.
24. 02:19A federal agent infiltrates a crime sydnicate.
25. 02:22Also with Doctor X, scientist, moon killer, a quest, are certainly words that James Bond films or
26. 02:32Short descriptions of James Bond movies can occur.
27. 02:36Also in The Way Of The Dragon here is also called a syndicate again.
28. 02:41Pressured, syndicate, defeat, all these words are similar to content in James Bond films.
29. 02:53Only the movie Dreamwork falls out a bit here and that's really not a good prediction.
30. 02:58This is an experimental film.
31. 03:03Of course, we should not only look at the individual forecast or a single proposal.
32. 03:12That means we give ourselves more Suggestions for known films from.
33. 03:17For example, on Batman Begins.
34. 03:21And you can see here very well that all of these Suggestions are roughly in the universe Batman.
35. 03:27Batman, Batman. Bill, Batman Forever.
36. 03:32If you're familiar with these movies, or if you're familiar with the movies, , you will find a colorful mix of comic films here
37. 03:41documentaries on this film series or on the Directors, as well as new and old films from this film series.
38. 03:52So as a final experiment, we see We also have another classic film.
39. 03:57This means that we want to make proposals which are similar in content to a Star Wars movie.
40. 04:03We also see here that very often among the proposals Other Star Wars movies are listed, Return Of The Jedi,
41. 04:12The Empire Strikes Back, but of course also dive here one or two movies on,
42. 04:18which may not be at first glance here , for example, fanboys.
43. 04:25In conclusion, we should first of all,
44. 04:29the quality of unsupervised models is without Domain knowledge very up to claim not well assessable.
45. 04:37That means every time you give a unsupervised approach or plan to take,
46. 04:42You should always be thinking about it. how to subsequently and, if appropriate, with
47. 04:47which domain experts you want to design this evaluation.
48. 04:53Because that's the central part of knowing if the unsupervised technology is or are not appropriate.
49. 05:03Let's now come to a collaborative filtering approach
50. 05:07and here we look at each other once that, in contrast to the
51. 05:14Content-based approach to a supervised technique.
52. 05:19We have as in section or Unit 2.6 already mentioned
53. 05:24we have for this Record a Ground Truth.
54. 05:28We know what real users are users as rating for movies.
55. 05:34We also split our data set training data and test data.
56. 05:44To get the model and the model training a bit more precise Let's look at the loss curve.
57. 05:51Over the training.
58. 05:52Loss is quoted to indicate the The error that a model still has, i.e.
59. 05:59the deviation from true value predicted value during training.
60. 06:05You should not finish the training until the Value only very minimal or no longer changed.
61. 06:13So once we plot the curve here, you see very strong that at the beginning of the training the progress or the
62. 06:21What we learned is very strong and towards the end we only marginal improvements have been made in the predictions.
63. 06:29And yes, how we get there roughly on a plateau.
64. 06:32That is, it only changes over the Time and over the individual epochs very, very little.
65. 06:36This means that it was a good time for us to to stop the training after epoch 8 or 10.
66. 06:48We'll give ourselves again the root-mean-squared error.
67. 06:54If you look very carefully, you give way to this root-mean-squared error is slightly off the
68. 06:59that we showed last time.
69. 07:02This is simply because they models in quotation marks.
70. 07:09That is, if you train models multiple times, run multiple times and do not consciously pay attention to it, then can be quite
71. 07:18marginally different Results, yes, result.
72. 07:28Now we want to take a look at one of these predicted View and compare values with the true value
73. 07:35That is, the true value instead three is 3.5, and the prediction is 2.9.
74. 07:41That means we have about a deviation of 0,6.
75. 07:46Because we have a matrix that is over has 100,000 different elements, it's not very
76. 07:54catchy and very directly visible because we are not very just like the content-based filtering system, the films
77. 08:04The films about our domain knowledge.
78. 08:06However, the collaborative filtering approach as shown here, of course
79. 08:13to be able to evaluate it in a clearly objective manner.
80. 08:15That is, we have the real ratings and we can define a quantitative metric and say exactly how
81. 08:23the derogation from before this true value.
82. 08:28And that is what our unity already decides on content-based and collaborative filtering.

To enable the transcript, please select a language in the video player settings menu.