Amir-massoud Farahmand



Research Goal

I help machines learn how to live a rewarding life! A machine can be an industrial controller whose job is to optimize the production of a chemical factory, or it might be a recommendation app that suggests products on the web. If the suggestions are appreciated, people will continue using the service. This would be a rewarding life for this app.

These are two examples of sequential decision-making problems under uncertainty (also known as reinforcement learning (RL) problems). The goal of an RL agent is finding a policy based on interaction with the environment such that the long-term performance is maximized. This abstract framework includes a wide-range of real-world applications such as problems in robotics, healthcare decision support systems (e.g., adaptive treatment strategies for chronic diseases, epilepsy management and suppression), decision making in financial markets, smart video games, etc.

My research goal is to develop and analyze algorithms to solve high-dimensional reinforcement learning problems based on solid mathematical principles. I also work on other areas of machine learning. See my publications for more information.

Research Interests

  1. BulletMachine Learning and Statistics: statistical learning theory, nonparametric algorithms, regularization, manifold learning, non-i.i.d. processes, online learning

  2. BulletReinforcement Learning, Sequential Decision Making, and Optimal Control: high-dimensional problems, regularized algorithms, inverse optimal control

  3. BulletRobotics: uncalibrated visual servoing, learning from demonstration, behavior-based architecture for robot control

  4. BulletLarge-scale Optimization

  5. BulletEvolutionary Computation: cooperative co-evolution, interaction of evolution and learning


  1. BulletSample-based Approximate Regularization is accepted at the International Conference on Machine Learning (ICML), 2014. Joint work with Philip Bachman and Doina. Extended version with proofs is here. Code on GitHub.

  2. BulletUploaded Classification-based Approximate Policy Iteration: Experiments and Extended Discussions on arXiv. A shorter version is submitted to IEEE Trans. on Automatic Control. Joint work with Doina, André, and Mohammad.

  3. BulletI left McGill University after more than two years of working on many interesting projects with Doina Precup, Joelle Pineau and other wonderful members of the Reasoning and Learning Lab. The good news is that I am joining the Robotics Institute, Carnegie Mellon University to work with Drew Bagnell.

  4. BulletWe are organizing Sequential Decision-Making with Big Data workshop at AAAI-2014.

  5. BulletTwo papers at the Neural Information Processing Systems (NIPS) conference, 2013. Learning from Limited Demonstrations and Bellman Error Based Feature Generation using Random Projections on Sparse Spaces. Joint work with Beomjoon, Doina, Joelle, Mahdi, and Yuri.

  6. BulletPhD Outstanding Thesis Award for the period of 2011-2012, Department of Computing Science, University of Alberta, 2013.

Academic blog: ThesiLog

Persian blog: ضدخاطرات

Academic and Non-academic Tweets