Computational Learning Theory and Beyond

In this course you will be introduced to computational learning theory and get a glimpse of other research towards a theory of artificial intelligence.

Our starting point will be a hands-on binary classification task. Basically, this is the challenge of classifying the elements of a given set into two groups (predicting which group each one belongs to) on the basis of given labeled data. Thus the goal of the supervised machine learning algorithms is to derive a correct classification rule. Our interest lies in strategies that work not only for one specific classification task but more universally for a pre-specified set of such. You will get to know a formalization of the aforementioned notions and see illustrating examples. In the main part, you will get to know different learning models which are all based on a modular design. By investigating the learning power of these models and the learnability of the prominent set of half-spaces, we also give arguments for how to choose an appropriate one.

In this T-shaped course you will be introduced to computational learning theory and get a glimpse of other research towards a theory of artificial intelligence. "T-shaped" means that on the one hand we will concentrate on different learning models in depth, on the other hand we want to give a broad overview and invite experts from other AI projects to show what else can be done in AI and theory of computer science research.

The focus is on learning from informant, a formal model for binary classification, for example by a support vector machine or perceptron. Illustrating examples are linear separators and other uniformly decidable sets of formal languages. Due to the learning by enumeration technique by Gold the learning process can be assumed consistent when full-information is available.

After the proofs of the latter observations, the model is adjusted towards the setting of deep learning. We first investigate the learnability of the set of half-spaces by this incremental learners and then show that they have less learning power than the full-information variant by a fundamental proof technique due to Blum and Blum. Finally, we will apply this technique to also separate consistency.

Beyond these models, we present and visualize applied binary classifiers and you will get concentrated insights into other approaches towards a theory of AI and important topics in theoretical computer science. These include evolutionary algorithms, fair clustering, game theory, low-dimensional embeddings, stable matchings, 3-satisfiability and submodular optimization.

Further, more models in Gold-Style computational learning theory are being discussed by experts in the second week.

Course characteristics:

Language: English
Starting from: October 6, 2020
Course end: October 27, 2020
Duration: 2 weeks + 1 examination week
Course requirements: familiarity with mathematical notation (basic studies at the university)

Target audience

everyone who is interested in models of AI and like mathematical accuracy/evidence
students who follow mathematics lectures (e.g. mathematics, physics, computer science, cognitive science/systems, computational linguistics)
(hobby) scientists who have already come into contact with formal mathematics (definitions and proofs)

Attention: This course is currently in self-study mode, in which you do not have access to graded assignments/exams. Therefore, we can only issue you a certificate of participation.

Our teachers

Karen Seidel

Karen Seidel is a PhD student in the Research Group for Algorithm Engineering at the Hasso-Plattner-Institute (HPI) in Potsdam, Germany. Her previous research in Artificial Intelligence focuses on modelling learning with Automata and Turing Machines. She graduated from the University of Bonn with a master in mathematics and worked in Mathematical Logic and Cognitive Mathematics at the Universities of Münster, Osnabrück and Cologne. She has wide-ranging experience in teaching and works at the HPI since 2017.