Nonnegative Sample Stationarity — Stationarity for Nonnegative Smooth Sample Losses
Descriptions may be stale. Consult the linked Lean sources.
Stationarity for Nonnegative Smooth Sample Losses · Lean proof
Stationarity for
Nonnegative Smooth Sample Losses
We consider minimizing a possibly nonconvex expected loss on a separable real
Hilbert space . Every sample loss
is nonnegative and has an -Lipschitz gradient, where . A given starting point satisfies for a bound . Each oracle call returns
the sample gradients at two queried points using a single fresh
independent sample. For a tolerance , the goal is a
measurable output
satisfying , with expectation over the
algorithm and oracle randomness.
Let be the
sample space, a probability law
on it, and the loss
at for sample . The optimization problem
is The
algorithm receives with its Borel
structure, ,
, , , and .
Our assumptions:
Assumption 1 (Sample losses). For every , the function is -integrable. For every , the function is differentiable on
. Its gradient is
jointly measurable in .
For all and ,
Assumption 2 (Paired gradient queries). The oracle
samples are
independent random variables with common law . The sample sequence is independent of
the algorithm’s randomness. On call , a query receives the reply
After the allotted oracle calls, the output must be measurable, its gradient
norm must be integrable, and
Proved result
OracleTemplate.Convergence LongTails.Setups.NonnegativeSmoothSampleLoss.spec
LongTails.Theorems.NonnegativeSampleStationarity.budget
Relevant previous results
No qualifying prior reference verified; this is not a proof of novelty.