Rupam Mahmood

About

Assistant Professor

Canada CIFAR AI Chair

Director of RLAI; Fellow at Amii

Department of Computing Science

University of Alberta

Personal website: https://armahmood.github.io/

Email: armahmood@ualberta.ca

Research

I develop reinforcement learning algorithms and real-time learning systems for controlling physical robots. My research objective is developing general and constructive mechanisms for continually improving robot minds. Currently, I am working on two long-term programs consisting of several short-term projects.

A Simple and General Reinforcement Learning System for Robot Control

In this program, we develop an RL system that can be easily deployed in many different robots for solving various tasks. Our system compliments the current robot learning systems based on learning from simulations or human demonstrations by solely training with real-time interactions under self-propelled behavior. Our system is general in the sense that the same system is expected to learn to control various different robots. On the other hand, we develop the system to be simple and accessible to enable community-wide scientific understanding and large-scale industrial adoption.

Core Constructive Mechanisms for Continually Learning Agents

In this program, we develop and analyze algorithms for learning policies and representations in a continual learning setup, where the agent is expected to go through a series of changes in the environment and tasks. Therefore, the agent has a vested interest in not only learning the specific task of the moment but also acquiring and accumulating knowledge and skills that are generally pertinent to its environment and likely applicable to an unseen future task in the same environment. We analyze the shortcomings of current policy and representation learning methods based on deep learning in a continual learning setting and address them by developing new methods for self-propelled behavior, off-policy policy gradient updates, and curation of useful state features.

Teaching

CMPUT 652: Reinforcement Learning with Robots (Fall 2019)

In this course, we will study the foundations of RL to be able to develop policy learning methods and learn about systematic ways of studying a real-time system to reveal the uncertainties involved in real-world tasks. This investigation will allow us to understand the differences between real-world and standard simulated tasks so that we can adapt task setups and algorithmic implementations to the real world as well as enhance the simulated tasks to incorporate the additional challenges in real-time systems. En route, we will learn about other promising approaches to learning in robotics that are not performed in real-time, such as learning from demonstration and simulation-to-reality transfer.

CMPUT 397: Reinforcement Learning (Winter 2020)

In this course, we study the design, analysis, and applications of reinforcement learning agents that interact with a complex, uncertain world to achieve a goal. We will emphasize agents that can make near-optimal decisions in a timely manner with incomplete information and limited computational resources. The course will cover Markov decision processes, reinforcement learning, planning, function approximation (online supervised learning) and real-world applications. The course will take an information-processing approach to the concept of mind and briefly touch on perspectives from artificial intelligence, psychology, neuroscience, philosophy, and robotics.

The course will use a recently created MOOC on Reinforcement Learning, created by the Instructors of this course. Much of the lecture material and assignments will come from the MOOC. In-class time will be largely spent on discussion and thinking about the material, with some supplementary lectures.

Pre-requisites: The course will use Python 3. You should either know Python 3 or be sufficiently experienced with programming in other languages that you can learn Python 3 quickly. We will use elementary ideas of probability, calculus, and linear algebra, such as expectations of random variables, conditional expectations, partial derivatives, vectors, and matrices. Students should either be familiar with these topics or be ready to pick them up quickly as needed by consulting outside resources or the teaching assistants.

CMPUT 340 - Introduction to Numerical Methods

Computer arithmetic and errors. The study of computational methods for solving problems in linear algebra, non-linear equations, optimization, interpolation and approximation, and integration. This course will provide a basic foundation in numerical methods that supports further study in machine learning; computer graphics, vision and multimedia; robotics; and other topics in Science and Engineering. Prerequisites: CMPUT 204 or 275; MATH 125, 214; one of STAT 141, 151, 235 or 265 or SCI 151.

CMPUT 653 - Topics in Artificial Intelligence

Browse more courses taught by Rupam Mahmood