• Zero-order oracle: returns the function value f (x).
• First-order oracle: returns the function value f (x) and the gradient ∇f (x).
• Second-order oracle: returns f (x), ∇f (x), and the Hessian ∇ 2 f (x).
1.2 Local methods, unconstrained
1.3 first order methods
gradient and newton methods are first and second order methods:
different convergence rate: newton(quadratic) vs gradient(linear)
conjugate gradient:= (BFGS most stable)
constrained optimization,
- Lagrange relaxation, supinf <= infsup,
- penalty function
- barrier function, which goes to infinity at the bound
Dual
convex optimization only when f is concave
Comment is the energy for a writer, thanks!