Why Nonlinear Regression? Reference Page

Author

DOFPro group

  • Video links go directly to the YouTube video.
  • JTF (Just the Facts) videos are the streamlined versions: greenscreen LaTeX equations, clean graphics, minimal narrative. Think efficient and to the point. Each has a companion TFS video.
  • TFS (The Full Story) videos include interviews, additional explanation, and equations written on whiteboards. Same math, more context, more personality. Each has a matching JTF version.
  • Info Page links lead to definitions, expanded explanations, and related material—because sometimes you really do need to explain it.
  • Visuals links contain the greenscreen or whiteboard materials used in the video, for those who like to see the scaffolding.
  • Wondering about the titles? See Appendix B: If you have to explain it, it’s no longer funny.
  • Videos marked This is NOT a DOFPro video were not produced by DOFPro but are included because they are relevant to the topic. They are shown in red so no one calls the academic integrity police.

Intro to Why Nonlinear Regression?

Many engineering relationships are nonlinear. While linear regression can often be applied after transforming variables (for example using logarithms), some models cannot be easily converted into a linear form.

Nonlinear regression provides a way to estimate parameters for models that are nonlinear in the fitting parameters. These techniques are widely used in chemical and thermal process engineering for modeling reaction rates, thermodynamic relationships, transport phenomena, and other complex systems.

The video on this page introduces the basic idea of nonlinear regression and explains how numerical optimization is used to determine the best-fit parameters.

Nonlinear Regression

ImportantNot a DOFPro video

This is NOT a DOFPro video. It may contain copyrighted material and references to proprietary software.

This video introduces the basic concepts behind nonlinear regression and explains how numerical optimization methods are used to determine the parameters of nonlinear models that best fit experimental data.

Examples and Definitions

Definitions

Nonlinear Regression
A regression method used when the model relating the dependent variable to the independent variables is nonlinear in its parameters.

A nonlinear model might take the form

\[ y = a e^{bx} \]

or

\[ y = \frac{a}{b + x} \]

where the parameters must be estimated using iterative numerical methods.

Sum of Squared Residuals
A measure of the discrepancy between observed data and the values predicted by a regression model:

\[ \mathrm{SSR} = \sum_{i=1}^{N} (y_i - \hat{y}_i)^2 \]

Nonlinear regression algorithms attempt to minimize this quantity.

Numerical Optimization
A class of mathematical techniques used to find parameter values that minimize or maximize a function. In nonlinear regression these methods are used to minimize the sum of squared residuals.

Common optimization algorithms include:

  • gradient descent
  • Newton’s method
  • Gauss-Newton method
  • Levenberg–Marquardt algorithm