Seminar on Reinforcement Learning for Education
Saarland University — Summer Semester 2021
The course will provide an overview of the research opportunities and challenges in applying reinforcement learning methods for improving education.

Organizers

Timeline and updates

  • 24 April 2021: We have a new mailing list that includes all organizers/tutors. To reach out to us, you should send an email to rl4ed-s21-tutors@mpi-sws.org (instead of contacting individuals).
  • Until 15 May 2021: After you have been allocated a slot in the seminar, you need to register for the seminar course examination at Saarland University (UdS). This registration must happen within three weeks after the seminar topics have been announced. This is also the deadline to withdraw.
  • 25 May 2021: Reports for the first two papers are due.
  • 15 June 2021: Reports for the next three papers are due.
  • 15 June 2021: Project details will be announced by this date.
  • 15 July 2021: Report and executable code for the project are due.
  • 20 July 2021: Assignment for presentations will be finalized. Prior to the assignment, you will be asked for a choice to present either (a) the project or (b) a paper. If you choose (b), then one of the five papers will be assigned.
  • 10 Aug 2021: Presentation slides are due.
  • Between 11 Aug to 10 Sep 2021: Final presentations will take place. The exact dates will be finalized in discussion with enrolled students.

Course structure

The course consists of three main components: (i) reading research papers, (ii) a project, and (ii) final presentation. There will be no weekly classes. To resolve doubts in the assigned papers and receive feedback on the project or presentation slides, the tutors will arrange specific meeting times after each deadline — further information will be communicated to students via emails as we move along in the semester. If needed, you can reach out to us anytime by sending an email to rl4ed-s21-tutors@mpi-sws.org.

Reading research papers

  • The list of papers is provided below. For each of the papers, you will have to write a two-page report.
  • Each report should be submitted as a PDF file via sending an email to rl4ed-s21-tutors@mpi-sws.org. You should name your PDF files as lastname_#.pdf (i.e., lastname_1.pdf, lastname_2.pdf, lastname_3.pdf, lastname_4.pdf, and lastname_5.pdf).
  • Reports should be written in latex using NeurIPS style files.
  • Structure the report as an extended review, e.g.,
    • Summarize the paper.
    • Write down the main strengths of the paper.
    • Write down the main weaknesses of the paper.
    • Write down ways in which this paper could be improved.
    • Write down ideas in which this paper could be extended.
  • These reports will correspond to one-third of the final score.

Project

  • There will be a project on implementing RL methods for educational settings. Project details will be announced by 15 June 2021.
  • You will have to submit a two-page report and executable code for the project. Each student will work on the project separately (no teams).
  • The project will correspond to one-third of the final score.

Presentations

  • You will have to prepare a presentation of 25 mins. You will be asked for a choice to present either (a) the project or (b) a paper. If you choose (b), then one of the five papers will be assigned.
  • At the end of the semester, you will give a final presentation. We will block about 6 hours of time for the presentations. The exact dates will be finalized in discussion with enrolled students. Attendance at the final presentations will be mandatory.
  • The slides and presentation will correspond to one-third of the final score.

Reading material

RL methods

The reports for these two papers (#1, #2) are due on 25 May 2021.
  1. Playing Atari with Deep Reinforcement Learning
    by V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller. 2013.
  2. Deterministic Policy Gradient Algorithms
    by D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller. ICML 2014.

RL applied to educational settings

The reports for these three papers (#3, #4, #5) are due on 15 June 2021.
  1. Reinforcement Learning for the Adaptive Scheduling of Educational Activities
    by J. Bassen, B. Balaji, M. Schaarschmidt, J. Painter, D. Zimmaro, A. Games, E. Fast, C. Thille, J. Mitchell. CHI 2020.
  2. Zero-shot Learning of Hint Policy via Reinforcement Learning and Program Synthesis
    by A. Efremov, A. Ghosh, A. Singla. EDM 2020.
  3. Offline Policy Evaluation Across Representations with Applications to Educational Games
    by T. Mandel, Y. Liu, S. Levine, E. Brunskill, Z. Popovic. AAMAS 2014.

Background

As additional background, it is recommended to read the following material from the book Reinforcement Learning: An Introduction by R. S. Sutton and A. G. Barto.
  • Chapter 3: Finite Markov Decision Processes — Sections 3.1 to 3.6.
  • Chapter 4: Dynamic Programming — Sections 4.1 to 4.6.
  • Chapter 6: Temporal-Difference Learning — Sections 6.1 to 6.5.
  • Chapter 13: Policy Gradient Methods — Sections 13.1 to 13.5.


Imprint / Data Protection