NSQ Limitations & Roadmap#

This page is the canonical, single source of truth for the non-sequential (NSQ) engine’s pre-release status, its capability envelope, its known limitations, and the development roadmap. The package __init__ docstring, the gallery overview, and the developer guide all link here rather than keeping their own copies. See NSQ Validation Report for what the engine is actually checked against in CI (closed-form benchmarks and invariants).

Pre-release status#

NSQ has never shipped in a tagged Optiland release — it exists on master as an actively-developed feature. The public API may therefore still change without a deprecation cycle; once NSQ ships in a tagged release, Optiland’s usual API-stability guarantee applies. Differentiability is interior-correct for refractive and reflective surfaces; visibility gradients are not yet supported (see Limitations).

Capability envelope#

NSQ ships two engines for two jobs:

  • Forward mode (NumPy backend). 1 × 107+ rays at max_depth 16, fully batched with live ray compaction and hard pixel binning. This is the production path for illumination and stray-light analysis, and the only backend that supports bounded ghost-path splitting.

  • Gradient mode (PyTorch backend). ~1 × 105 rays at max_depth 16 on a single GPU. A full autograd graph is built through the Monte Carlo loop, so memory scales as O(num_rays × max_depth) (compaction is disabled to keep fixed tensor shapes). This is the path for optimization and ML layers.

Switch engines through optiland.backend:

import optiland.backend as be
be.set_backend("torch")   # enables gradient mode

Physics coverage, as of this page:

Capability

Status

Fresnel reflection/refraction

Differentiable, both backends.

AR / thin-film coatings

Implemented via optiland.coatings (unpolarized only); NSQ and the sequential engine agree on R by construction.

Mirror reflectance

Required, explicit (constant, wavelength-dependent, or coating). No implicit perfect-mirror default.

Bulk (Beer-Lambert) absorption

Implemented, differentiable w.r.t. k and path length. Volumetric scattering (not just absorption) is still roadmap item 5.

Diffuse/scatter BSDFs

Lambertian (with a differentiable reflect/transmit split via transmissive_fraction), Harvey-Shack, tabulated.

Medium sidedness

Geometric (n_geom-based), correct for index-matched/cemented interfaces. Volume adds construction-time watertightness validation for compound components.

Detectors

Unified intersection pass, absorb flag for transmissive (non-terminating) detectors, attached total_flux, true differentiable Gaussian splat.

Rare-path sampling

Importance biasing (both backends), bounded splitting (NumPy only), unbiased Russian roulette.

Reproducibility

Counter-based PCG32, bit-identical random decisions across batch_size/compaction/backend (see the developer guide).

Photometric units

Radiometric core (W) with a lumen/lux conversion layer (optiland.nonsequential.units).

Diagnostics

Self-diagnosing SimulationResult.report(): depth truncation, roulette loss, unreached geometry, per-detector undersampling.

Volumetric scattering

Not implemented (roadmap item 5).

Polarization

Not implemented (roadmap item 6).

Visibility gradients

Zero (roadmap item 1) — see Limitations below.

Limitations#

  1. Visibility gradients are zero — the biggest physics gap. When a ray silhouette moves across a surface boundary (vignetting, occlusion, which-surface-hit), that boundary contributes no gradient. The detached-sample / attached-weight estimator differentiates interior interactions (Fresnel, refraction, dispersion, BSDF lobes, absorption) but not the discrete visibility decision. Reparameterization (roadmap #1) closes this gap. A dedicated test (tests/nonsequential/test_nsq_geometric_gradients.py) explicitly asserts this behaviour.

  2. Mesh geometry is forward-only. Analytic conics (ConicGeometry, SphereGeometry, ParaboloidGeometry) are differentiable; MeshGeometry is not — calling backward() through a mesh interaction will raise.

  3. Source geometry is not differentiable. Source sampling (aperture position, emission angle, emitter extent) runs in NumPy, so aperture_radius, half_angle_deg and the extended-source dimensions cannot carry gradients. Passing a requires_grad tensor for one of these raises NotImplementedError rather than detaching silently, so a dead design variable is never mistaken for a live one. Source total_flux is differentiable. SpectralDetector extents are likewise detached (the detector accumulates into a NumPy histogram); IrradianceDetector extents and total_flux are differentiable.

  4. No polarization. Stokes tracking is not present yet (roadmap #6), so polarization-sensitive coatings (BaseCoatingPolarized) raise rather than being silently averaged to a scalar.

  5. No volumetric scattering. Bulk absorption (Beer-Lambert) is implemented; a photon being scattered within a volume, rather than only attenuated, is not (roadmap #5).

  6. Gradient-mode memory cap. The ~1 × 105-ray envelope is a hard constraint of the naive autograd strategy. Path Replay Backpropagation (roadmap #3) lifts this cap.

  7. No acceleration structure. Intersection is O(rays × surfaces) per bounce on both backends — fine for the scene sizes NSQ targets today, but a scaling limit for very high surface-count scenes. A benchmarks/nonsequential/ harness measures this rather than estimating it, so a future BVH/batched-traversal effort has a baseline (roadmap #7).

Roadmap#

Ordered by priority. Each item names the seam or reference paper.

  1. Reparameterization for visibility gradients — warped-area reparameterization (Loubet et al. 2019; Bangaru et al. 2020) so silhouette / vignetting / which-surface-hit gradients become correct. Closes the single biggest physics limitation.

  2. Optimization integration — wire NSQ into Optiland’s Variable / operand system and optimizers for end-to-end illumination and stray-light design.

  3. Path Replay Backpropagation (PRB) — constant-memory, unbiased gradients via the gradient_mode seam already present in TorchBackend (Vicini et al. 2021); lifts the ~1 × 105-ray cap.

  4. GUI integration — NSQ scene building and visualization in optiland_gui.

  5. Volumetric scattering — beyond the Beer-Lambert absorption already implemented, actual in-volume scatter events.

  6. Polarization — Stokes tracking through Fresnel coatings, done properly.

  7. Acceleration structure — a BVH or batched-traversal scheme, sized against the benchmarks/nonsequential/ measurements rather than guessed.

  8. Dr.Jit / Mitsuba 3 / OptiX backend — high-performance TracerBackend plugin consuming the same SceneIR the reference backends interpret.

Get involved#

Try it and tell us what you build. If you use NSQ for illumination design, stray-light analysis, or differentiable optics, open a GitHub issue and describe your use case — your feedback directly shapes the roadmap.

Contribute. The roadmap items above (especially reparameterization, PRB, and GUI integration) are open for contributors. Development happens on master; PRs target master directly. The canonical engine reference is the developer guide.

References#

  • Loubet, Holzschuch & Jakob, Reparameterizing Discontinuous Integrands for Differentiable Rendering, SIGGRAPH Asia 2019.

  • Bangaru, Li & Durand, Unbiased Warped-Area Sampling for Differentiable Rendering, SIGGRAPH Asia 2020.

  • Vicini, Speierer & Jakob, Path Replay Backpropagation of Light Paths, SIGGRAPH 2021.