Linear Regression Reference Page

Author

DOFPro group

  • Video links go directly to the YouTube video.
  • JTF (Just the Facts) videos are the streamlined versions: greenscreen LaTeX equations, clean graphics, minimal narrative. Think efficient and to the point. Each has a companion TFS video.
  • TFS (The Full Story) videos include interviews, additional explanation, and equations written on whiteboards. Same math, more context, more personality. Each has a matching JTF version.
  • Info Page links lead to definitions, expanded explanations, and related material—because sometimes you really do need to explain it.
  • Visuals links contain the greenscreen or whiteboard materials used in the video, for those who like to see the scaffolding.
  • Wondering about the titles? See Appendix B: If you have to explain it, it’s no longer funny.
  • Videos marked This is NOT a DOFPro video were not produced by DOFPro but are included because they are relevant to the topic. They are shown in red so no one calls the academic integrity police.

Intro to Linear Regression

Linear regression is one of the most widely used tools for analyzing engineering data. It is used to determine the best-fitting straight-line relationship between variables and to estimate parameters such as slope and intercept from experimental measurements.

In chemical and thermal process engineering, linear regression is commonly used for model fitting, calibration, parameter estimation, and for extracting physical meaning from noisy data. The videos on this page introduce the basic ideas behind linear regression and show how to perform the calculations with several different software tools.

Linear Regression

ImportantNot a DOFPro video

This is NOT a DOFPro video. It may contain copyrighted material and references to proprietary software.

This video introduces the basic ideas of linear regression and derives the equations for the slope and intercept of the best-fit line. It also explains the least-squares criterion and shows how regression can be used to analyze experimental data.

Linear Regression Spreadsheet

ImportantNot a DOFPro video

This is NOT a DOFPro video. It may contain copyrighted material and references to proprietary software.

This video demonstrates how to perform linear regression with a spreadsheet, including calculating the regression parameters and using the fitted line to interpret data.

Linear Regression MATLAB

ImportantNot a DOFPro video

This is NOT a DOFPro video. It may contain copyrighted material and references to proprietary software.

This video shows how linear regression can be carried out in MATLAB, including both the underlying calculations and the use of MATLAB tools to automate the fitting process.

Linear Regression in R

This video demonstrates how to perform linear regression in R, including both manual calculation approaches and the use of built-in regression functions.

Examples and Definitions

Definitions

Linear Regression
A statistical method for modeling the relationship between a dependent variable and one or more independent variables by fitting a linear equation to observed data.

For the simplest case of one independent variable, the fitted model is

\[ y = mx + b \]

where \(m\) is the slope and \(b\) is the intercept.

Least Squares Fit
A method for determining the best-fit line by minimizing the sum of the squared residuals between the observed data points and the predicted values on the fitted line.
Slope
The parameter that describes how much the dependent variable changes for a unit change in the independent variable. In the equation \(y = mx + b\), the slope is \(m\).
Intercept
The value of the dependent variable predicted when the independent variable is zero. In the equation \(y = mx + b\), the intercept is \(b\).
SSE (Sum of Squared Residuals)
The quantity

\[ \mathrm{SSE} = \sum_{i=1}^{N} (y_i - \hat{y}_i)^2 \]

which measures the total squared deviation between the observed values \(y_i\) and the predicted values \(\hat{y}_i\). Smaller values of SSE indicate a better fit.

It is also commonly called the residual sum of squares (RSS).

Root Mean Squared Residual (RMSE)
A measure of the typical size of the residuals in a regression model. For a straight-line fit with two fitted parameters, it is often estimated as

\[ \mathrm{RMSE} = \sqrt{\frac{\mathrm{SSE}}{N-2}} \]

A smaller RMSE indicates that the fitted line more closely matches the observed data.

Functional Bounds
Bounds on the fitted regression function itself. They describe the uncertainty in the estimated best-fit relationship and are analogous to confidence intervals for the fitted model parameters or fitted curve.
Observational Bounds
Bounds on the expected observed data values about the fitted regression line. They account for both uncertainty in the fitted model and the scatter of the data, and are analogous to prediction intervals.