FISTA: Achieving a rate of convergence proportional to k⁻³ for small/medium values of K

The fast iterative shrinkage-thresholding algorithm (FISTA) is a widely used procedure for minimizing the sum of two convex functions, such that one has a L-Lipschitz continuous gradient and the other is possible nonsmooth. While FISTA's theoretical rate of convergence (RoC) is pro-1 portional to _αkt2k , and it is related to (i) its extragradient rule / inertial sequence, which depends on sequence t_k, and (ii) the stepsize αk, which estimates L, its worst-case complexity results in O(k⁻²) since, originally, (i) by construction t_k ≥ ^k+1₂ , and (ii) the condition αk ≥ αk₊₁ was imposed. Attempts to improve FISTA's RoC include alternative inertial sequences, and intertwining the selection of the inertial sequence and the step-size. In this paper, we show that if a bounded and non-decreasing step-size sequence (α_k ≤ αk+1, decoupled from the inertial sequence) can be generated via some adaptive scheme, then FISTA can achieve a RoC proportional to k⁻³ for the indexes where the step-size exhibits an approximate linear growth, with the default O(k⁻²) behavior when the step-size's bound is reached. Furthermore, such exceptional step-size sequence can be easily generated, and it indeed boots FISTA's practical performance.