The Polytope of Optimal Subgradient Methods

N:
Rotation
Speed 1.0×

Every minimax-optimal fixed-step method for Lipschitz convex minimization corresponds to a unique set of dual multipliers \(\lambda_{ij} \geq 0\), \(i < j\), sitting inside the polytope

\[ \mathcal{P}^N = \left\{\, \lambda \mid \lambda \geq 0,\ \ \sum_{i=j+1}^{N} \lambda_{j,i} - \sum_{i=0}^{j-1} \lambda_{i,j} = \frac{1}{N+1}, \ \ j = 0, \dots, N-1 \,\right\}. \]

There are exactly N! vertices on this polytope, and each can be characterized by a particular “arc diagram.” These diagrams can be understood as a selection of \(N\) dual multipliers, one in each row of the upper triangular \(\Lambda\) matrix: a choice of \(\lambda_{j,s(j)}\) where the map \(s(j) > j\) for each \(j = 0, 1, \dots, N-1\).

The SSEP method constructed by Drori and Taylor corresponds to \(s(j) = j+1\), and the averaged subgradient method corresponds to the constant \(s(j) = N\). The last-iterate subgradient method (without averaging) of Zamani and Glineur is the barycenter of all \(N!\) vertices.