Introduction to Online Control

This tutorial guide introduces online nonstochastic control, an emerging paradigm in control of dynamical systems and differentiable reinforcement learning that applies techniques from online convex optimization and convex relaxations to obtain new methods with provable guarantees for classical settings in optimal and robust control. In optimal control, robust control, and other control methodologies that assume stochastic noise, the goal is to perform comparably to an offline optimal strategy. In online control, both cost functions and perturbations from the assumed dynamical model are chosen by an adversary. Thus, the optimal policy is not defined a priori and the goal is to attain low regret against the best policy in hindsight from a benchmark class of policies. The resulting methods are based on iterative mathematical optimization algorithms and are accompanied by finite-time regret and computational complexity guarantees. This book is ideal for graduate students and researchers interested in bridging classical control theory and modern machine learning.

Author Biography

Elad Hazan is Professor of Computer Science at Princeton University. His research focuses on the design and analysis of algorithms for basic problems in machine learning and optimization. He is a pioneer of online nonstochastic control theory. Karan Singh is Assistant Professor of Operations Research at Carnegie Mellon University, and has previously worked at Google Brain and Microsoft Research. He works on the foundations of machine learning, control, and reinforcement learning.