Most engineering applications have multiple design objectives. In this talk, we will consider the problem of building a Reinforcement Learning (RL) framework for jointly optimizing multiple objectives, which can be used in multiple scheduling applications. An example is maximization of fairness among multiple agents, which requires balancing the cumulative rewards received by individual agents, with an optimization objective that is often non-linear across the agents. With such objective functions, Bellman Optimality no longer holds. Thus, existing RL algorithms aiming at optimizing the (discounted) cumulative reward of all agents fail to address this issue. We formalize the problem of optimizing a non-linear function of multiple long term average rewards, to explicitly ensure multi-objective optimization in RL algorithms. We then propose model-based and model-free algorithms to learn the optimal policy and discuss regret guarantees. Further, we will discuss the implementation of our algorithms on scheduling problems and demonstrate that the proposed RL framework can enable multi-objective optimization in these applications with significant improvement as compared to standard RL algorithms. Finally, we will also discuss the impact of constraints in multi-objective reinforcement learning.
Vaneet Aggarwal received the B.Tech. degree from the Indian Institute of Technology, Kanpur, India in 2005, and the M.A. and Ph.D. degrees in 2007 and 2010, respectively from Princeton University, Princeton, NJ, USA, all in Electrical Engineering. He is currently an Associate Professor at Purdue University, West Lafayette, IN, where he has been since Jan 2015. He was a Senior Member of Technical Staff Research at AT&T Labs-Research, NJ (2010-2014), Adjunct Assistant Professor at Columbia University, NY (2013-2014), and VAJRA Adjunct Professor at IISc Bangalore (2018-2019). His current research interests are in machine learning and networking areas.
Dr. Aggarwal received Princeton University's Porter Ogden Jacobus Honorific Fellowship in 2009, the AT&T Vice President Excellence Award in 2012, the AT&T Key Contributor Award in 2013, the AT&T Senior Vice President Excellence Award in 2014, and the Purdue Most Impactful Faculty Innovator in 2020. He received the 2017 Jack Neubauer Memorial Award recognizing the Best Systems Paper published in the IEEE Transactions on Vehicular Technology, and the 2018 Infocom Workshop HotPOST Best Paper Award. He was on the Editorial Board of IEEE Transactions on Green Communications and Networking, and is currently on the Editorial Board of the IEEE Transactions on Communications and the IEEE/ACM Transactions on Networking.