Optimization Guide#

This guide covers how to run an optimization: which method to pick, how to call it, how to follow and stop a run, how to keep it fast, and what to expect when something goes wrong. How to build the problem (variables, operands, merit functions, constraints) is in Default Merit and Constraints and in the tutorials starting at Tutorial 3a.

Note

minimize(), the native "dls" optimizer, observers, hard constraints and the default merit are experimental: they are public and tested, but their details may still change. The legacy optimizer classes are unchanged.

The problem in four lines#

Every optimizer works on an OptimizationProblem: variables to change, and operands (or residual blocks) that define the merit function. The merit is the sum of squared residuals, sum(r ** 2), with each operand residual r = effective_weight * delta.

from optiland.optimization import OptimizationProblem, minimize
from optiland.samples.objectives import CookeTriplet

lens = CookeTriplet()
problem = OptimizationProblem()
problem.add_operand("f2", target=50.0, weight=10.0, input_data={"optic": lens})
problem.add_default_merit(lens)                      # see the merit page
problem.add_variable(lens, "radius", surface_number=1)
problem.add_variable(lens, "thickness", surface_number=1, min_val=0.5)

result = minimize(problem)       # the lens now holds the best design found
problem.info()

effective_weight is the operand weight times the field and wavelength weights of the optic, so those weights count in every optimizer.

Choosing a method#

Situation

Use

Notes

A normal lens design: continuous variables, residual-style merit

minimize(problem) ("auto" picks "dls")

Native damped least squares. Fastest on small and medium problems.

Hard limits on thicknesses or other operands

minimize(problem, "dls") after add_constraint or add_physical_constraints

Only "dls" enforces hard constraints; every other method raises ConfigurationError. Penalty operands (min_val/max_val) work everywhere.

Same, from the legacy API

LeastSquares(problem).optimize(method_choice="dls")

Also honors variable bounds. "lm", "trf" and "dogbox" still mean the SciPy solvers.

Merit is a scalar, not a residual vector, or you want a SciPy solver

minimize(problem, "l-bfgs-b") (also "bfgs", "slsqp", "trust-constr", "cobyla", "nelder-mead", "powell", …)

Minimize sum(r ** 2). "least_squares" calls scipy.optimize.least_squares on the residuals.

Torch backend, gradients through the whole model

minimize(problem, "adam") or "sgd"; or Tutorial 7b

Options lr and gamma; maxiter is the step count.

Stuck in a local minimum

DualAnnealing, DifferentialEvolution, BasinHopping, SHGO, ParticleSwarm

Legacy classes only. See Global Optimization (Differential Evolution).

Pick glasses from a catalog

GlassExpert

See Tutorial 3e.

Rule of thumb: start with minimize(problem). If your variables include categorical ones (glass names), "auto" falls back to "l-bfgs-b"; use GlassExpert for those.

minimize()#

result = minimize(
    problem,
    method="auto",       # "dls", a SciPy name, "least_squares", "adam", "sgd"
    maxiter=None,        # residual evaluations for "dls" and "least_squares"
    tol=1e-3,            # ftol for "dls"; SciPy tol for the local solvers
    stop=None,           # extra stopping criterion
    observers=None,      # list of observers
    cancel_token=None,   # cancel the run from another thread
    disp=False,          # print progress
    **options,           # per-method options (below)
)

Things worth knowing:

  • The optic is changed in place. On return it holds the best accepted design, result.x. Copy the optic first (copy.deepcopy(lens)) if you need the starting point back.

  • ``maxiter`` counts residual evaluations, Jacobian columns included. One Jacobian of a 12-variable problem costs 12 evaluations, so the default budgets are not “iterations”.

  • Unknown options raise. minimize(problem, "dls", sheme="central") raises ConfigurationError listing the valid names, instead of being silently ignored.

  • The result is a SciPy OptimizeResult with extra fields: stop_reason, n_evals, history (with a HistoryObserver), method, and for "dls" also multipliers, active_set, max_constraint_violation and constraint_report. result.fun is the final merit.

Options of "dls":

Option

Values

Meaning

scheme

"forward" (default), "central"

Finite-difference scheme. "central" costs twice as many evaluations per Jacobian; use it for tight convergence on smooth merits.

rel_step

float

Relative finite-difference step.

aim_policy

"exact" (default), "frozen"

Whether ray aiming is recomputed for every Jacobian column or reused from the baseline design. "frozen" is faster for systems with expensive aiming (wide field, telecentric); try "exact" first.

jacobian_update

"exact" (default), "auto", "broyden"

Recompute the Jacobian after every step, or reuse it with rank-one updates after good steps. Cuts evaluations on large problems, at some risk to robustness.

on_failure

"reject" (default), "raise"

See When a ray trace fails.

minimize() and LeastSquares#

The classes you already use keep their signatures, and no deprecation warnings are issued. LeastSquares(problem).optimize(method_choice="dls") runs the same native core as minimize(problem, "dls"); callback, plot and disp work as before. Use minimize() when you want observers, a stopping criterion, a cancel token or hard constraints on a typical run.

Stopping, observers and cancellation#

Stopping criteria decide when a run ends. "dls" always stops on tol, on a vanishing step or gradient, on a collapsed trust region and on maxiter; stop= adds your own tests, checked after every accepted step. Combine criteria with | (any) and & (all, each latched once true):

from optiland.optimization import CostTolerance, MaxEvals, StepStall

stop = MaxEvals(2000) | (CostTolerance(1e-6) & StepStall(1e-6))
result = minimize(problem, "dls", stop=stop)
print(result.stop_reason)

Also available: MaxIter (accepted steps) and GradNormTolerance.

Observers watch a run without changing it. They receive on_start, on_step (after each accepted step) and on_end, and the current OptimizationState (x, value, iteration, n_evals, …), which they must treat as read-only.

from optiland.optimization import ConsoleObserver, HistoryObserver, CheckpointObserver

history = HistoryObserver()
result = minimize(
    problem,
    "dls",
    observers=[history, ConsoleObserver(log_every=5), CheckpointObserver("runs", every=10)],
)
merit = [r["value"] for r in history.records]    # also result.history

disp=True is shorthand for adding a ConsoleObserver. Any object with the three methods is an observer; see Observer. SciPy-based methods call observers at the start and end of the run, and per step where SciPy offers a callback (the local solvers do, "least_squares" does not).

Cancellation is cooperative. Set a CancelToken from any thread (a GUI button, a timer); the run ends cleanly after the current step, the optic holds the best design so far, and result.stop_reason ends with "cancelled by user".

import threading
from optiland.optimization import CancelToken

token = CancelToken()
threading.Timer(30.0, token.cancel).start()      # give up after 30 s
result = minimize(problem, "dls", cancel_token=token)

Making it fast#

Most of the cost of an optimization is ray tracing, and in a finite-difference Jacobian, tracing the same rays again and again. Optiland removes much of that for you; these are the levers that remain.

  • Use ``”dls”`` on the NumPy backend. It reuses the ray traces of the baseline design for every Jacobian column (only surfaces at or after the changed one are re-traced), and per-evaluation work such as material indices, paraxial data and ray aiming is computed once per design. On the benchmark set of python -m benchmarks.optimization.run it converges two to ten times faster than SciPy "lm" on a Cooke triplet, a Double Gauss and a ten-variable scaling probe, and sometimes ends at a slightly higher merit, so compare the final problem.sum_squared() for your own design.

  • Keep batching on (the default). Ray operands that share a field, wavelength and pupil are traced together. Call problem.disable_batching() only when debugging.

  • Trace fewer rays. Cost scales with ray count. The default merit needs rings * arms rays per field and wavelength (18 for the default 3 × 6); rms_spot_size operands with num_rays=100 trace 30 301. Start coarse, finish fine.

  • Use bounds instead of penalty operands where you can; they cost nothing extra.

  • Reduce Jacobian cost on large problems with jacobian_update="auto", or aim_policy="frozen" when aiming dominates.

  • Do not tighten ``tol`` blindly. The default 1e-3 is a relative merit change; much smaller values mostly add evaluations.

  • Torch backend: correctness is guaranteed, but the speed work targets NumPy. Prefer "adam" / "sgd" there, with gradients from autograd.

To see where time goes, print result.n_evals and time problem.sum_squared() once: runtime is roughly n_evals times that.

When a ray trace fails#

During an optimization a trial design can be untraceable: a ray misses a surface, hits total internal reflection, or a thickness goes negative. Optiland treats this the same way everywhere:

  • A failed trace gives NaN in the affected residual rows. It never raises out of an evaluation, and one bad operand does not poison the others.

  • Native ``”dls”``: a trial with any non-finite residual is rejected; the optimizer shrinks its step and tries again from the last good design. The set of rays stays fixed during an iteration, and rays that keep failing are dropped from the merit between iterations (their rows read zero) and restored at the end of the run.

  • SciPy-based methods cannot digest NaN, so failed rows are replaced by a penalty that scales with the last finite residual norm, which keeps their finite-difference Jacobians usable. The best finite design seen is restored if the final point is non-finite.

  • The starting design must be traceable. A non-finite initial residual raises ConfigurationError naming the failing operands or blocks.

  • on_failure="raise" ("dls", SciPy local solvers) raises OptimizationFailure at the first failure instead. Use it to find out which operand fails.

If a run keeps rejecting steps and ends with success=False: loosen variable bounds that allow impossible geometry, check the starting design with lens.info() and lens.draw(), add thickness limits (physical constraints), and read result.stop_reason.

Migration notes#

Weights are no longer dropped in ``LeastSquares``. The legacy LeastSquares optimizer used to build residuals from weight * delta and so ignored the field and wavelength weights, while every other optimizer and problem.sum_squared() used effective_weight * delta. They now agree. If you set non-uniform field or wavelength weights, a LeastSquares run converges to a different (now consistent) design than before. Equal weights give the same results.

SciPy ``LeastSquares`` no longer sees ``NaN``. A failed trace used to feed NaN into SciPy’s Jacobian and could end the run at a non-finite point. It now uses the penalty described above and restores the best finite design.

The default of ``method_choice`` is still ``”lm”``. The native "dls" is opt-in: pass method_choice="dls" (or use minimize). The default will change to "dls" only after it is at least as fast and as good as SciPy on the whole benchmark set; until then, nothing changes for existing scripts. Pin method_choice="lm" if you want to be unaffected when it does.

``maxiter`` for ``”dls”`` counts residual evaluations, Jacobian columns included, the same as maxiter of "lm" and "trf".

Hard constraints are new and ``”dls”``-only. Existing penalty operands (min_val / max_val) behave as before on every optimizer.