Energy Hessian Reuse — Stationarity with Gradient-Dependent Third Derivatives
Descriptions may be stale. Consult the linked Lean sources.
Stationarity with Gradient-Dependent Third Derivatives · Lean proof
Stationarity
with Gradient-Dependent Third Derivatives
We consider a possibly nonconvex objective on a finite-dimensional
real inner product space . The
function is three times continuously differentiable, with its third
derivative bounded by at each ,
where and . A given starting point has objective gap at most above the infimum and
gradient norm at most . Two
exact oracles return the gradient and Hessian. For , the goal is an output
satisfying
almost surely under , the law of
the algorithm’s randomness.
The underlying optimization problem is The norm is induced by the inner product
on , and derivative norms are the induced
operator norms. The algorithm receives , , , , , , , and an integer .
Our assumptions:
Assumption 1 (Third-derivative bound). The function
belongs to and satisfies Here denotes the trilinear third derivative of , equivalently the second derivative of
its gradient under the inner product identification.
Assumption 2 (Initial bounds). The given point satisfies
Queries to either oracle may be made at any point of . The gradient oracle returns at . The Hessian oracle returns at , as a linear operator from to . Gradient and Hessian calls are counted
separately.
After the allotted calls to each oracle, the output must satisfy
Proved result
OracleTemplate.Convergence LongTails.Setups.GradientDependentHessian.spec LongTails.Theorems.EnergyHessianReuse.budget
Relevant previous results
No qualifying prior reference verified; this is not a proof of novelty.