Inside the Belly of a Search Engine

They have made encyclopedias obsolete: Web search engines like Google or Bing provide us with the information we are looking for at lightning speed. Surveys have shown that every Internet user searches via Google 3 to 4 times a day on average. In total, estimates assume more than 2 trillion search queries* per year, and the trend is rising.

If you want to understand how these search engines work, this course is for you.

Self-paced since March 20, 2023
Language: English
Beginner, Fundamentals

Course information

Are you curious about how web search engines provide you with information daily? Are you surprised at the speed at which web search engines are able to respond to your complex and sometimes incomplete queries? Are you wondering how to build your website so that it can be easily found by others through the search engine (search engine optimization)?

This two-week workshop aims to provide an introduction to how search engines operate in terms of how web pages are discovered, indexed, and retrieved to display information in a structured form to respond to queries (words used to search for information). We discuss how documents are clustered based on similarity and ranked based on query relevance to generate responses to user queries. As a concluding highlight, we consider various forms of visualizing search engine results and why search engines like Google remain popular.

*source: Wikipedia

What you'll learn

  • understand how search engines structure queries
  • understand how search engines discover and index web pages
  • To make the best use of search engines
  • Structure web pages so that they can be found easily

Who this course is for

  • people who use search engines, and who are concerned about privacy issues or tracking
  • Students who conduct research on the Internet using search engines as part of their studies
  • Auditors who use search engines to obtain information about companies
  • General public

Course contents

  • Intro

  • Week 1:

    In this week we cover introductory material into the general field of web-based search engines, specifically ones that are index based. We study the basic architecture of search engines, in terms of its components and how they work together to respond to user queries. As a side effect, we discuss the implications for search engine optimisation and the reasons why highly linked web pages are generally more discoverable by search engines.
  • Week 2:

    We build on the learnings in Week 1, to proceed to cover material on the information gathering procedure that search engines employ to analyse documents on the web to determine relevance to queries. Data preparation and analysis are also studied as a means of understanding how search engines match document content to queries based on a relevance model guided primarily by keyword occurrences. As an example, we delve into the mechanisms used to evaluate or compute document relevance, discover documents and rank documents in terms of relative importance with respect to search engine queries. We conclude with some further considerations to take into account with respect to how search engines operate not just in terms of textual data, but also multimedia data such as images, and videos.
  • Final Exam:

    Final Exam
  • I like, I wish:

    Please provide your feedback on the course.

Enroll me for this course

The course is free. Just register for an account on openHPI and take the course!
Enroll me now

Learners

Current
Today
3,377
Course End
Mar 20, 2023
3,005
Course Start
Feb 22, 2023
1,530

Rating

This course was rated with 4.25 stars in average from 4 votes.

Certificate Requirements

  • Gain a Record of Achievement by earning at least 50% of the maximum number of points from all graded assignments.
  • Gain a Confirmation of Participation by completing at least 50% of the course material.

Find out more in the certificate guidelines.

This course is offered by

Dr. Anne Kayem

Dr. Anne Kayem is a senior researcher at the Hasso-Plattner-Institute (Digital Engineering Faculty, University of Potsdam, Germany). She leads the Data Privacy team embedded within the Internet Technologies and Systems Chair. Her research interests range from designing and analysing secure and privacy preserving data sharing algorithms, to evaluating their usability. She is a senior member of the ACM and IEEE. For more details about her research, please visit: https://hpi.de/sprite/home.html