Next: Initializing the Nonlinear Least-Squares Solver, Previous: Overview of Weighted Nonlinear Least-Squares Fitting, Up: Nonlinear Least-Squares Fitting [Index]
In cases where the Jacobian J is rank-deficient or singular,
standard nonlinear least squares can sometimes produce
undesirable and unstable solutions. In these cases, it can
help to regularize the problem using ridge or Tikhonov regularization.
In this method, we introduce a term in our minimization function
which is designed to damp the solution vector x, or give preference
to solutions with smaller norms.
Here, the regularization matrix L is often set
as L = \lambda I, for a positive scalar \lambda, but can
also be a general m-by-p (where m is any number
of rows) matrix depending on the
structure of the problem to be solved. If we define a new
(n+m)-by-1 vector
or, in the weighted case,
then
which is in the same form as the standard nonlinear least squares
problem. The corresponding (n+m)-by-p Jacobian matrix is
or for weighted systems
While the user could explicitly form the \tilde{f}(x) vector
and \tilde{J} matrix, the fdfridge
interface
described below allows the user to specify the original data vector
f(x), Jacobian J, regularization matrix
L, and optional weighting matrix W, and automatically
forms \tilde{f}(x) and
\tilde{J} to solve the system. This allows switching
between regularized and non-regularized solutions with minimal
code changes.