6.6 Exploratory Search and Recommender Systems

Este vídeo pertenece al curso Knowledge Graphs de openHPI. ¿Quiere ver más?

6.6 Exploratory Search and Recommender Systems

Duración: aproximada 21 minutes

An error occurred while loading the video player, or it takes a long time to initialize. You can try clearing your browser cache. Please try again later and contact the helpdesk if the problem persists.

Scroll to current position

00:00This is knowledge graphs lecture number six "Advanced Knowledge
00:04Graph Applications. This is the very last part of the lecture
00:08and here we are going to talk about exploratory search and
00:11recommender systems, of course based on knowledge graphs.
00:16Ok you might remember from the last part of the lecture that there is an
00:21extreme difference between retrieval and exploration.
00:26What again was that kind of difference?
00:29Retrieval - in retrieval usually you are looking for something
00:33that you already know or more or less like to know, or you know
00:38how to describe the stuff. It means you are looking for something familiar.
00:42You know that for example in a library when you're looking for a book
00:46what you have often in mind is the title of the book or the
00:49name of the author. So like for example if you look for that
00:52book that you see here in the right corner,
00:55the author is Jule Verne and the title is "From the Earth to the
00:59Moon", which means you are often using kind of unique identifiers like
01:03name of the author plus name of the title, or you use descriptive
01:08metadata to describe what exactly you are looking for because
01:11you know that already. You are familiar with the domain,
01:14you know how to describe this stuff. So then retrieval is no problem.
01:20However there might be other situations. What do you do in the
01:23library? To stay exactly at that example,
01:27in older times what you did is of course you went to the library catalogue
01:32and within the catalogue you were looking for exactly the right
01:35card, the catalog card, for exactly the book you were looking for
01:40and there you found itJules Verne "From the Earth to the Moon"
01:43and you saw also exactly the title in reality is even a bit longer
01:48and it has been here released in, sorry, eighteen seventy three.
01:53And in the end you see a few numbers and stuff like that and
01:56this in the end showed you the directions to the shelf where
02:00to find the book in the end.
02:03This was the the solution to your retrieval problem.
02:08However you read the book and then of course what you want
02:12to do you want to find another comparable book, because you
02:15should find something that also interests you in reading but
02:18now you have no idea what exactly to look for, because
02:21what could be in similar interesting, what could be somehow
02:25related? So you will find books of the same or of related topics.
02:30And for example also you want to know how did the author then develop
02:34over the timer or how did the topic travelling to the moon develop over time?
02:39So what else should I like to read? No idea.
02:43In the end you should come up with something like this, which
02:47means meaningful recommendations of books which are likewise
02:52of interest, if you have already an interest in, for example, that book.
02:57That might be rather similar book, that might also be related
03:01books and they might be related directly or directly similar or
03:06indirectly. And this is something a problem that is addressed for example
03:12besides other problems by exploratory search. So you want to
03:16explore what's in there in your library, which means in your information system.
03:22And of course in the library this was much more easier than it is nowadays
03:26on the web, because in the library you might remember
03:30there are shelves and of course these shelves usually are organized
03:34according to specific ordering principles. So
03:37simply traditional libraries enable exploratory search by providing you shelves which
03:43contain books ordered according to a specific system. Mostly
03:46in one shelf they might be ordered at alphabetically or chronologically.
03:51However if you go to a public library there is a section of
03:55classic science fiction novels, for example, where you find exactly the book
03:59that we were looking for and also in the same shelf you might find
04:03books of the same category and there of course there might be something
04:07of your interest.
04:10And if you don't know how to proceed? What you do then in a traditional library?
04:14Of course you are going to ask your friendly librarian because
04:19he can give you intelligent recommendations.
04:23And exactly that exploratory search as well as intelligent recommendations
04:29are two applications that are supported by knowledge graphs and
04:34knowledge graph based technologies. And this wewill take a look into it today.
04:40So first of all let us define what is exploratory search.
04:44Exploratory search represents all the activities that are carried out
04:50by a searcher or by searchers who are in the first place unfamiliar with the domain
04:56of their goal. It means they need to learn about the topic in
04:59order to understand how to achieve that goal.
05:03They are often unsure about the ways how to achieve their goals, so
05:07what should I use as a search string? And they are either unsure about the technology
05:12or of the process how to do that. And sometimes even they are
05:16unsure about the goals in the first place which means
05:20I look for something interesting but how should I phrase it? So this is
05:23kind of different. What you want to do there?
05:27You want to go for more or less like browsing instead of searching.
05:32In the library or in a bookshop it's quite easy. So you simply
05:35look at the shelves and you browse through the shelves. So this
05:38is kind of exploring of course the content.
05:42How do you do that with the internet or the web?
05:47There are billions of documents and they are not ordered in
05:51any sense. So this is really really difficult because what you
05:54want to do, you want to find you want discover something by chance that
05:59you did not know that exists, that you did not know you were
06:01looking for, but it's there and of course you are interested
06:05in it. Hurray! So this is called serendipity.
06:09Sometimes also you want to get an overview ,so what kind of
06:11other books did that guy read or write and sometimes you want to
06:16enabled content-based navigation which means you want to create somehow
06:20paths through the entire search space that guides you through collections like that.
06:26And of course every user might have different needs to follow
06:31exactly his own path through the search space. And this of course will be or
06:37is enabled and supported by exploratory search.
06:43You might think, oh yeah, this is quite of course science fiction too.
06:46However, it is not. There is an example. You already know the
06:50scihi blog that we were talking about, this blog about daily
06:53history of science and arts and philosophy and that stuff. And this
06:59besides the semantic annotation that I have already shown you
07:03also provides means of visualization that enable exploratory search.
07:08So this was part of our researc. Sso for example here if we go to
07:12the page in scihi blog of
07:16Jules Verne there is an article about him "Around the World
07:19in Eighty Days", also another famous book.
07:21And here if you click on Jules Verne what happens is
07:25tada, so you see here another visualization which provides you
07:29interesting information about Jules Verne. So you see here for example
07:34people in green that are somehow related with Jules Verne.
07:38You see here in blue places, locations associated with Jules Verne.
07:43You see in yellow points in time which means events and here
07:48in purple all of the rest of things that are somehow related
07:51with them and these are all things which are somehow also reflected
07:54in the blocks, so in the information system we are talking about.
07:58Now I'm going to show you something interesting, so
08:01as soon as I move or hover over one of these persons, so this
08:06is another author Italian author called Emilio Salgari.
08:10This was also an author of adventure and fantastic novels and
08:14you see here then automatically displayed all the potential connections
08:19that are there between Jules Verne and Emilio Salgari. So for example
08:24you might see that here
08:28Emilio Salgari has been influenced by Jules Verne and overall you see also that
08:33Emilio Salgari for example is an Italian, he wrote a notable
08:36work that is "The Black Corsair" and he was born in Verona.
08:41We could look at other people Phileas Fogg. That's of course a
08:45a fictional character created here by Jules Verne in his famous
08:50novel "in eighty days, around the world". Let's see what's else there.
08:54So I don't know that guy.
08:56Norbert Casteret seems also to be an author, Jean-Michel Cousteau;
08:59so what's the connection there? Also is an influence connection.
09:04Konstantin Tsiokovsky, so this is one of the fathers of space travel.
09:08So he also has been influenced as you see here by Jules Verne. And these are
09:12of course connections and if you are then interested in one
09:15of these other guys you simply click on that and then you have
09:18this guy Constantine Tsiokovsky, in the middle of the attention
09:23and of course you see also hear how that guy might connected
09:27to Jules Verne or to other things. And of course you get recommendations
09:31based on the content. So for Constantine Tsiokovsky for example
09:35you see here as a second suggestion Valentin Glushko.
09:40Who was valentin glushko? Also some engineer that you see here
09:45who was one of the main participants or main drivers of
09:50the space race between the US and USSR in the nineteen
09:54sixty, seventy, eighty 's and so on.
09:57So this only as an example for exploratory search because this enables you
10:02to explore the content of an information system that you saw
10:06on the other side. And this is based of course on semantic annotation
10:11and there is in the background of course a knowledge graph.
10:16So let's see how this works in principle.
10:20We have again our example from the earth to the moon and of
10:23course we could use here now one of the knowledge graphs that
10:26we already have used. So here for example we will use DBpedia simply
10:29because you already know the self speaking names and understand
10:32better the context that we show.
10:35So what information is there for the novel from the earth of
10:39the moon which is part of the knowledge graph DBpedia?
10:43On the one hand you of course find out that this is of RDF
10:47type book so this is a book. Okay.
10:49You find out more interesting information, for example, the previous work of
10:54whoever was here exactly "In Search of the Castaways", another book of Jules Verne.
10:59So we here we have already in relation which is interesting
11:02that can be used for a recommendation. And of course you find
11:06other things like for example the category, the wikipedia category
11:10in which exactly this subject has been collected. So
11:15from the earth to the moon is in the category eighteen sixty
11:18five novels or French science fiction novels, novels by Jules Verne,
11:22moon in fiction, fictional rivalries, novels set in Florida or eighteen
11:27sixties science fiction novels. So lots of interesting categories that
11:32of course this book is in and it might be interesting to see
11:34what other books are in exactly these categories because this might lead you
11:39to your next favourite book that you want to read.
11:43Let's have a look of course we also know who is the author here and
11:47we get even more information you know if you further follow
11:50you know the paths leading from the author to other authors.
11:53You might find authors that have been influenced by Jules Verne
11:56like for example HG Wells, another science fiction writer, who
12:00wrote novels that might be of interest for you.
12:04Okay we have to do this a bit more formally. You see the principle already behind that.
12:09Here for example if I want to give a recommendation which comes pretty close
12:13or is rather similar to the things I have in mind, so I start
12:17always here with from the earth to the moon, I look for things
12:21of the same type, which means I look for other books. It doesn't
12:24make sense if I now look let's say for a grocery store as a
12:27recommendation. I want to look for other books. So the books
12:31I recommend should have the same type and then I look for things
12:34which have the same type and are somehow also via other links
12:38connected to exactly my original item. And there you find the
12:42subsequent work and the previous work of Jules Verne and this is
12:45a Journey to the Center of the Earth or in Search of the Castaways
12:49which would be a recommendation for you if you like From the
12:52Earth to the Moon.
12:54Now you might say yeah that's quite nice but
12:57we have to relax this rule a bit because I want to have more recommendations. Okay.
13:02We can also go the following way. So you start from the book.
13:05You go to the author and then you look to the author what other
13:09things of the same type this author is connected with. So for
13:13example the author Jules Verne of course is author of many more books
13:17and all these books of course are of type books and are connected
13:21to Jules Verne via the property author. So we have two
13:25equivalent property connections here that might wait and the more there are
13:30the more similar are the things you are looking for. So here for example
13:34now I could extend my recommendations from these two I had
13:38in the beginning to also other books like for example Matthias Sandorf
13:42Master of the World or The Mysterious Island.
13:46Now you might say ok this is quite easy. I only get other books
13:49of the same authors. So this is something I could have done
13:52easily in another way. Okay, so let's go one step further. You have
13:56already seen that for example we can go one step further from our
13:59our author Jules Verne than to another author which is here hg wells
14:00author Jules Verne to another author which is here HG Wells
14:03and we go the same principle that we did in the beginning. We
14:06look for something you know of the same type, so both are authors
14:10and they are connected via a property which is your dbo:influenced.
14:15So therefore we think they are connected, they are of the same
14:18type. Therefore they possess sufficient similarity.
14:22And now I look of course again for things of type book
14:26that are connected to now this new item, which means here HG Wells
14:32and I find out again via simply looking that there are similarities
14:36that for example HG Wells has also the few books that might
14:40also be interesting for me. Like for example The Invisible Man,
14:45The Island of Doctor Moreau or the War of the Worlds.
14:49And this might be already interesting recommendations. And exactly this
14:54you might use for writing your own recommender system or for guiding the user
14:59through all the books that are represented in your library or here
15:03in DBpedia. So we know now what exploratory searches and recommender
15:09systems, what data is also clear they seek to predict preferences of a user
15:14would give to an item that we can recommend. So
15:18for example of course this means personalized recommendation
15:22but there is also something if you know nothing about the user
15:25if you have something like a cold start problem then of course
15:28you go simply content-based and look for similar or related things.
15:33We have now mostly looked for similarities but also you can take into account
15:37relationships. We did this here by looking you know not
15:41for books but first we went for authors and then we went for
15:45related authors and then we went for similar items for that related thing.
15:49So we combine here relationships and similarities to generate recommendations
15:56from knowledge graphs.
15:59We have a very simple and very easy recommender for you that
16:02is based on DBpedia. So there for example I created a very short
16:06SPARQL query that you see here.
16:09What I want to have is of course for my book From the Earth to the Moon
16:13I want to have recommendations what should I read according to DBpedia.
16:18And what I do is I simply look for the most similar books
16:22in the sense here I have only made the restriction because
16:25I otherwise might easily run into a time out. There are so many
16:28books you know and SPARQL is sometimes rather slow. So I simply
16:32look for the categories that we had in the beginning because the categories rather
16:37well are able to represent the content of a novel here.
16:43So I'll simply look for other novels that share as many categories as possible
16:50with my book From the Earth to the Moon. So what I do here in
16:54this SPARQL, query just have a look at it I have prepared it here
16:57in the browser for you. So we have to go back, then you see it here.
17:01So this is the query. We select of course here this is the book
17:05we are looking for and then we have here a score and what we
17:09are looking for patterns is first of all we want to select
17:12all the categories in which From the Earth to the Moon is in and then
17:16we say ok we select some recommendation s that should share
17:20the same category and of course I do not want to have the same
17:23book recommended that I already read, so I simply filter out
17:27the book From the Earth to the Moon from my results. I group
17:30all these results by the recommendation and for that I count
17:34simply the number of distinct categories they share with my original book.
17:40And then I order the thing in a descending way by the
17:44score I achieve and then I run the query and you see exactly
17:48what is the outcome. The book that shares most categories with
17:52my book is Around the Moon and I think
17:56this is, yeah, interesting. Of course it's from Jule Verne and it's the sequel
18:02to From the Earth to the Moon. So this is not bad you know.
18:06If i look for what to read next then of course I should probably
18:10read the sequel. But as you see there are also other recommendations
18:14Off on a Comet, for example, or Apollo 23. So this is
18:18definitely not a book by Jules Verne. Let's have a look what is apollo 23.
18:22That's a book from the Doctor Who series.
18:26Quite interesting. So you see you might play around with that
18:29and of course you might produce a much more sophisticated SPARQL
18:33query which produces an even better recommender system.
18:38Ok with that I leave you alone now and let you play around.
18:43The point now is of course you could of course extend this
18:46also to other items and what beside book recommendation is interesting
18:50you have movie recommendations. And now the question is
18:54which movie should I watch next.
18:57And of course never make boring suggestions because in that case
19:01your recommender system is busted.
19:06Ok that's it with the entire lecture. I hope you have enjoyed these lectures.
19:13You know now much, a lot, more about knowledge graphs and knowledge graph technology.
19:19We tried to give you a six week program with everything which
19:24is interesting to know as a starter with that kind of technology
19:28and of course now it's up to you to deepen that knowledge. And we give you
19:32lots of pointers where you can connect to if you, for example,
19:36want to broaden or deepen your knowledge toward, let's say, knowledge graph programming,
19:41developing information systems based on knowledge graphs or improving search systems
19:47or improving generating visualizations and stuff like that
19:50all based on knowledge graphs which are an emergent
19:53and by that time rather popular technology. So thank
19:59you very much for your attention and hope to see you again
20:02soon in some of my next courses.