Online Control via Counterfactual Tracking

We develop a method for online control that competes with general classes of causal policies, beyond the linear-controller classes used by most existing algorithms. Over a horizon of (T) rounds, we consider a known linear dynamical system subject to adversarial disturbances and convex costs revealed after each action. The method simulates the benchmark policies on the revealed history, uses their counterfactual state--input pairs to form a moving reference, and applies a fixed stabilizing controller to track that reference on the physical system. We call this method \emph{counterfactual tracking}. Counterfactual tracking applies to any measurable class of causal policies that can be simulated from the revealed history and whose counterfactual state--input pairs have bounded diameter at every round. The policies may be nonlinear or dynamic and need not share a parameterization. We establish PAC-Bayes regret guarantees that hold for every posterior over policies and depend on its relative entropy to a chosen prior. On a fixed plant with a tracker of bounded impulse-response gain, a finite class of (N) policies admits the minimax-optimal (\sqrt{T\log N}) dependence on (T) and (N) when (\log N=O(T)). As a central application, we compete with a system-level response ball of stabilizing linear dynamical controllers. The ball bounds the summed impulse-response deviation from the fixed tracker, but imposes no common decay envelope, memory length, or controller-order bound. To our knowledge, this is the first online-control guarantee uniform over such a class. A matching lower bound shows that our guarantee is tight up to constants.

Source: arXiv:2607.13029v1 - http://arxiv.org/abs/2607.13029v1 PDF: https://arxiv.org/pdf/2607.13029v1 Original Link: http://arxiv.org/abs/2607.13029v1