Elad Hazan, Satyen Kale, and Manfred K. Warmuth
In Machine Learning Journal (MLJ), (2016). Also in proceedings of 23rd Conference on Learning Theory (COLT), 2010. Corrigendum to conference version, 2010

We describe online algorithms for learning a rotation from pairs of unit vectors in $$\mathbb{R}^n$$. We show that the expected regret of our online algorithm compared to the best fixed rotation chosen offline is $$O(\sqrt{nL})$$, where $$L$$ is the loss of the best rotation. We also give a lower bound that proves that this expected regret bound is optimal within a constant factor. This resolves an open problem posed in COLT 2008. Our online algorithm for choosing a rotation matrix in each trial is based on the Follow-The-Perturbed-Leader paradigm. It adds a random spectral perturbation to the matrix characterizing the loss incurred so far and then chooses the best rotation matrix for that loss. We also show that any deterministic algorithm for learning rotations has $$\Omega(T)$$ regret in the worst case.