Reinforcement Learning (DAT605)

Content

NB! This is an elective course and may be cancelled if fewer than 10 students are enrolled by August 20th.

Artificial Intelligence in this era has become synonymous with Supervised and Unsupervised Learning. Supervised learning is best suited for cases that have a vast set of examples of inputs and desired outputs and the objective is to learn based on such examples in order to generate output from some future, currently unseen input. Text classification, Image Classification, Object location, Regression problems, and Sentiment analysis are areas where supervised learning is extensively used. Whereas, Unsupervised learning aims to discover some hidden structure of the data without the need to have a specific distinction in the input and output values. Such learning techniques are commonly used for clustering of data that tries to combine data items into a set of clusters revealing relationships in data.

Reinforcement learning lies somewhere in between supervised and unsupervised forms of learning techniques. On one hand, it builds on established methods of supervised learning for function approximation, stochastic gradient descent, and backpropagation to learn data representation, however, on the other hand, it does not require supervision in order to discover hidden patterns and relationships in data. Reinforcement learning primarily focuses on the problem of automatic learning of optimal decisions over time in a complex environment by building on advances in computer science, behavioral psychology, and neuroscience. Due to its flexibility and generality, the field of RL is developing very quickly and attracting lots of attention, both from researchers who are trying to improve existing methods or create new methods and from practitioners interested in solving their problems in the most efficient way.

The target group for this course would be professionals and students working or interested in areas of artificial intelligence, machine learning, game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, and statistics.

Learning outcome

Concepts covered in this course would provide relevant theoretical and hands-on programming knowledge. Every topic is demonstrated using easy-to-understand real-world examples. The following topics would be covered during the course duration:

Knowledge:

• Topic 1: Reinforcement Learning - an introduction
• Topic 2: Course Materials, Supplementary Resources, and Development Environment
• Topic 3: Tabular Methods
  • Multi-Armed Bandit
  • Markov Decision Processes
  • Cross-Entropy Method
• Topic 4: Dynamic Programming
• Topic 5: Monte-Carlo & Temporal Difference and Q-Learning
• Topic 6: Policy Gradients
• Topic 7: The Actor-Critic Method
• Topic 8: Deep Q-Network - an Overview
• Topic 9: Further Exploration

Skills:

• Good programming knowledge
• Knowledge of basic algebra, probability, and statistics
• Python Programming Knowledge
• Understanding of Numpy, Matplotlib

General competence:

• Artificial Neural Networks
• Deep Learning
• Convolutional Neural Networks

Required prerequisite knowledge

Recommended prerequisites

Exam

Coursework requirements

Throughout the semester students work individually on 2 assignments. These assignments are mandatory and must be passed within the announced deadline so that the student has the right to take the exam.

Absence due to illness or for other reasons must be communicated to the laboratory personnel as soon as possible. One cannot expect that provisions for completion of lab assignments at other times are made unless prior arrangements with the laboratory personnel have been agreed upon.

Course teacher(s)

Course coordinator:

Head of Department:

Method of work

Open for

Course assessment

Literature