Research Goal
Understanding the computational and statistical mechanisms required to design efficient AI agents that interact with their environment and adaptively improve their long-term performance.
It is useful to expand on what I expect from this agent. This is the guiding map for much of my research activities:
Desiderata: This social agent should be able to control its stream of experience, by learning how the outside and inside worlds work while focusing on aspects that are most relevant to its decision making. It should sample-efficiently solve problems that (1) have large state and action spaces, (2) require decisions to be made at varying temporal granularities, and (3) require risk-awareness.
My research team, Adaptive Agents (Adage) Lab, approaches this goal often through the lens of reinforcement learning (RL). We use a diverse set of research methodologies, ranging from theoretical/mathematical analysis to empirical studies to solving novel and challenging applications.
Here I provide a brief summary of some of my research projects, divided to two broad categories of Theoretical Push and Application Pull, with pointers to some relevant publications. This is not a comprehensive list. See my research statement, Rethinking Reinforcement Learning (2024), for more detailed explanation and Publications for an almost complete list of papers.
Theoretical Push
- Reinforcement Learning
- value function regularities [JMLR2016] [TAC2016] [NeurIPS2012] [NeurIPS2011] [NeurIPS2010] [ACC2009]
- policy regularities [TAC2016][NeurIPS2011]
- model regularities [NeurIPS2023][ICLR2022][NeurIPS2018][AISTATS2017]
- learning from demonstration [NeurIPS2013]
- RL acceleration [RLC2024][ICML2021][AAAI2016]
- inverse optimal control [AAAI2015]
- model selection [MLJ2011]
- distributional RL [NeurIPS2023][NeurIPS2019]
- search-control [UAI2022][ICLR2020][IJCAI2019]
- Machine Learning
- adversarial robustness [TMLR2023][arXiv2020]
- non-i.i.d. processes and time series [NeurIPS2017] [JSPI2012]
- regularizer design [ICML2014]
- manifold learning [ICML2007]
- Intersection of Control Engineering and Machine Learning [ACC2016]
- Interaction of Learning and Evolution [TEC2010]
Application Pull
- PDE control (smart air conditioning systems) [ICML2018] [ACC2017] [CDC2016]
- Fault detection/prognostics for time series [NeurIPS2017] [PHM2017]
- Robotics (uncalibrated visual-servoing, behavior-based architectures) [ICRA2010] [ICRA2009] [IROS2007]
- Hybrid vehicles energy management system [AAAI2016]