Natural-gradient parameter updates

Explains the information-geometric inner update, the paper’s diagonal Fisher approximation, and tests required before claiming convergence or numerical stability.

Research
Last verified
2026-06-25 00:00 UTC
Updated
Reading time
1 minutes

Explains the information-geometric inner update, the paper’s diagonal Fisher approximation, and tests required before claiming convergence or numerical stability.

Research documentation: this page interprets a cited research source and defines evidence requirements. It does not claim a released Teleodynamic AI implementation.

Information geometry

Natural gradients precondition an ordinary gradient with the inverse Fisher information metric so a step is interpreted relative to the parameter manifold rather than raw coordinates.

Reported approximation

DE11 uses a diagonal Fisher approximation. The paper acknowledges that ignoring parameter correlations can be suboptimal and lists richer approximations as future work.

Verification

  • Compare ordinary and natural-gradient traces.
  • Check Fisher positivity, floors, and overflow behavior.
  • Record convergence criterion and uncertainty.
  • Test correlated features where the diagonal approximation should be stressed.

Scope

This starter page defines the questions, boundaries, evidence, and failure modes that should be recorded before a capability is presented as supported.

Engineering considerations

  • Identify the source, version, target environment, and owner.
  • Separate observed values from estimates and externally reported values.
  • Record trade-offs, unsupported cases, and fallback behavior.
  • Link performance statements to a compatible benchmark methodology.

Verification questions

  • What exact artifact, revision, backend, and environment were reviewed?
  • Which assumptions could change the result?
  • Which data should be retained so another engineer can reproduce the conclusion?