Skip to content

feat: opt-in gradient-safe log-det via Settings (default unchanged) - #392

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/gradient-safe-logdet
Jul 17, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/gradient-safe-logdet

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Closes #391.

Adds an opt-in, non-default gradient-safe log-determinant method to inversions, selected via Settings. The default evidence path is unchanged and byte-identical.

Why

The two Bayesian-evidence log-det terms compute 2*sum(log(diag(cholesky(M)))). On the JAX backend the Cholesky returns NaN (numpy raises) when M is not numerically positive-definite — which stalls gradient-based searches on ill-conditioned regularization matrices (autolens_workspace_developer#104: the pixelized likelihood's NaN wall localised to exactly this term).

Why opt-in and not a default change

An independent adversarial review of the reg-logdet investigation established that the current evidence is scientifically correct — d log det(H) / d log λ = 1798 to machine precision, so the coefficient-dependence that drives inference is right, and the 1e-8 conditioning term is only a constant offset. A relative lift would be a regression; a pseudo-determinant would break archived-evidence comparability. So the default is deliberately preserved and this adds an alternative only, for gradient work and for comparison.

Change

Settings.log_det_method, following the existing Optional[…] = None → conf general.yaml pattern (mirrors use_positive_only_solver):

  • "cholesky" (default) — the historical computation, byte-for-byte unchanged including the test-mode LinAlgError guard. Both terms now route through a shared _log_det_symmetric_from helper; the default branch is the identical expression.
  • "slogdet" — logabsdet of xp.linalg.slogdet(M): identical where M is positive-definite, finite and differentiable where the Cholesky would NaN. General (serves the adaptive schemes, unlike an analytic precomputed-eigenvalue route). Opt-in via al.SettingsInversion(log_det_method="slogdet").

Applies to both evidence log-det terms (log_det_curvature_reg_matrix_term and log_det_regularization_matrix_term) — a gradient search needs both finite.

API Changes

Additive only. New optional Settings(log_det_method=...) field, default None → config "cholesky" → current behaviour. No removed/renamed/changed symbols; no downstream migration.

Detail
  • Added: Settings.log_det_method (optional kwarg + property); AbstractInversion._log_det_symmetric_from (private helper); inversion.log_det_method: cholesky in packaged general.yaml.
  • Removed / Renamed / Changed Signature: none.
  • Changed Behaviour: none at default — the two log-det properties produce the identical value; only an explicit log_det_method="slogdet" opt-in differs.
  • Migration: n/a. No workspace config mirroring required — packaged-only keys layer through production config (verified against the existing packaged-only nnls_target_kappa).

Testing

  • Full test_autoarray/ suite: 926 passed. Every existing log_evidence test (fit_imaging / fit_interferometer / fit_dataset) unchanged — this is the byte-identical-default gate.
  • New unit tests: default log_det_method == "cholesky" and reproduces numpy log det; slogdet == cholesky on a positive-definite matrix (~1e-8); slogdet finite (== np.linalg.slogdet) on a non-PD matrix where Cholesky fails.
  • Note: on ill-conditioned PD matrices slogdet matches the true log det more closely than Cholesky (~2.5e-7 on a 25-pixel Constant reg matrix, slogdet being the accurate one) — a further reason it is non-default and for comparison rather than a silent swap.

Follow-ups (not in this PR)

  • JAX gradient-finiteness assertion → autogalaxy_workspace_test (mirrors autofit_workspace_test/scripts/jax_assertions/fitness_nan_gradient_contract.py).
  • Two independent bugs this investigation surfaced, filed as separate PyAutoMind prompts: ConstantZeroth is dead code (shape + missing-arg), and Adapt's double-square 4th-power coefficient.

Gate

  • Tests — 926 passed in the task worktree.
  • Smoke — n/a at library level; downstream JAX assertion follows to autogalaxy_workspace_test.
  • Heart — RED, acknowledged contemporaneously by the human on 5 pre-existing unrelated reasons; RED reason verbatim: PyAutoLens: 1 uncommitted source change(s) (a stray paper_jax/paper.md edit in the main checkout, untouched by this branch).

…lt unchanged)

The two Bayesian-evidence log-det terms (log_det_curvature_reg_matrix_term and
log_det_regularization_matrix_term) use 2*sum(log(diag(cholesky(M)))), which
returns NaN on the JAX backend (raises on numpy) when M is not numerically
positive-definite -- stalling gradient-based searches on ill-conditioned
regularization matrices (autolens_workspace_developer#104).

Add Settings.log_det_method (Optional -> conf general.yaml fallback, mirroring
the use_positive_only_solver pattern):

- "cholesky" (default) -- the historical computation, byte-for-byte unchanged
  including the test-mode LinAlgError guard. Both terms route through a shared
  _log_det_symmetric_from helper; the default branch is the identical expression.
- "slogdet" -- logabsdet of xp.linalg.slogdet(M): identical where M is
  positive-definite, finite and differentiable where the Cholesky would NaN.
  Opt-in, non-default, general (serves the adaptive schemes too); intended for
  gradient-based work and for comparison against the Cholesky evidence.

Scope decided by an independent adversarial review of the reg-logdet
investigation: the current evidence's coefficient-dependence is correct to
machine precision, so the DEFAULT is deliberately not changed. This adds an
opt-in alternative only.

Full test_autoarray suite 926 passed (every existing log_evidence test
unchanged = byte-identical default). New tests: default==cholesky==numpy log det;
slogdet==cholesky on a PD matrix; slogdet finite where cholesky fails on a
non-PD matrix. No workspace config mirroring needed -- packaged-only keys layer
through production config (verified, like nnls_target_kappa).

Closes #391.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Jul 17, 2026
@Jammy2211
Jammy2211 merged commit a240782 into main Jul 17, 2026
4 checks passed
@Jammy2211
Jammy2211 deleted the feature/gradient-safe-logdet branch July 17, 2026 16:50
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: opt-in gradient-safe log-det via Settings (default unchanged)

1 participant