Simple Linear Regression
Model the straight-line relationship between one predictor and a continuous outcome. Fit the least-squares line, read the slope, intercept, R-squared and p-values, and validate the fit with residual diagnostics.
Run Simple Regression →What is Simple Linear Regression?
Simple linear regression models the relationship between a single independent variable (x) and a continuous dependent variable (y) by fitting a straight line. The line is chosen by ordinary least squares (OLS), which minimizes the sum of squared vertical distances between the observed points and the fitted line.
The fitted equation, y = b₀ + b₁x, gives an intercept b₀ (the predicted y when x is zero) and a slope b₁ (the average change in y for a one-unit increase in x). These coefficients quantify both the direction and the strength of the linear relationship.
Regression goes beyond correlation by producing a predictive equation and by testing, through a t-test on the slope, whether the relationship is statistically significant. R-squared then reports the proportion of variance in y explained by x.
In plain terms: You have two measurements that seem related, and you want the best straight line through the cloud of points. That line lets you predict one value from the other and tells you how tight the relationship is. R-squared near 1 means the line explains most of the pattern; near 0 means it barely helps.
Key Outputs
Slope & Intercept
The slope b₁ is the average change in y per unit of x; the intercept b₀ is the fitted value at x = 0. Together they define the prediction line.
R-squared
The coefficient of determination, between 0 and 1, is the fraction of variance in y explained by x. It measures fit quality, not correctness of the model.
Significance & Intervals
A t-test on the slope tests whether the relationship differs from zero. Confidence intervals bound the true slope and predicted means.
Key Formulas
Interpreting the Results
A significant slope (small p-value) means the linear relationship is unlikely to be due to chance. The slope's sign gives the direction and its magnitude the practical effect size.
R-squared measures how much variance is explained, but a high R-squared does not prove the model is correct, and a significant slope does not prove causation. Always inspect residual plots: a good simple linear model shows residuals scattered randomly around zero with constant spread.
Assumptions & Validation
Linearity
The average relationship between x and y is a straight line; check with a residuals-versus-fitted plot.
If violated: Transform x or y, or use polynomial or nonlinear regression.
Independence
Observations and their errors are independent; violated by time-series autocorrelation or repeated measures.
If violated: Use time-series methods or models with clustered standard errors.
Homoscedasticity
Error variance is constant across x (equal residual spread); a fanning residual plot indicates a violation.
If violated: Apply a variance-stabilizing transformation or use weighted least squares.
Normality of Residuals
Residuals are approximately normal, needed for valid t-tests and intervals; check with a Q-Q plot.
If violated: Transform the response or rely on large-sample robustness.
⚠️ Check assumptions first
A high R-squared does not validate a model. Regression can fit a straight line to curved data and still report a large R-squared while making biased predictions. Always examine residual plots for curvature, funnel shapes, and outliers before trusting the slope, the p-value, or any prediction from the line.
When NOT to Use Simple Linear Regression
Curved Relationship
If the true pattern bends, a straight line is biased. Use polynomial or nonlinear regression, or transform the variables.
Multiple Predictors
When the outcome depends on several factors at once, one predictor causes omitted-variable bias. Use multiple linear regression.
Binary or Categorical Outcome
For yes/no or pass/fail outcomes, linear regression can predict impossible probabilities. Use logistic regression instead.
Industry Applications
Forecasting
Predict sales from advertising spend, or demand from a single leading indicator, and quantify the strength of the link.
Process Engineering
Relate a process parameter such as temperature to a quality metric to find operating relationships for control.
Calibration
Build a calibration line between an instrument reading and a known reference value.
Cost & Estimation
Model cost as a function of a single driver such as volume or size for quick estimates.
Frequently Asked Questions
What does R-squared actually tell me?
R-squared is the proportion of variance in the outcome explained by the predictor, ranging from 0 to 1. An R-squared of 0.7 means 70 percent of the variation in y is accounted for by x. It measures how tightly the points cluster around the line, but it does not prove the model form is correct or that x causes y; a curved relationship can still yield a high R-squared.
What is the difference between correlation and regression?
Correlation summarizes the strength and direction of a linear association in a single number between minus one and one. Regression goes further by fitting a predictive equation, estimating the slope and intercept, testing significance, and enabling prediction of y from x. Correlation describes; regression models and predicts.
Does a significant slope prove causation?
No. A significant slope shows that x and y are linearly associated beyond chance, but association is not causation. A lurking confounder can drive both variables, as when ice-cream sales and drowning both rise with temperature. Establishing causation requires controlled experiments or careful adjustment for confounders, not regression alone.
How do I check whether simple linear regression is appropriate?
Fit the line, then examine the residuals. Residuals plotted against fitted values should scatter randomly around zero with constant spread and no curvature. A Q-Q plot should show approximate normality. Systematic patterns signal that a transformation, additional predictors, or a different model is needed.
What if my relationship looks curved?
A straight line will give biased predictions for curved data. Options include transforming the predictor or response (for example a log transform), fitting a polynomial term, or using a nonlinear model. Residual curvature is the clearest signal that the linear form is inadequate.
Can I use simple linear regression for forecasting?
Yes, for short-range forecasts driven by a single predictor, provided the linear relationship is stable and the assumptions hold. Be cautious extrapolating far beyond the range of the observed x values, where the linear relationship may no longer apply and prediction intervals widen considerably.
Model a Linear Relationship in Seconds
Fit the least-squares line, test significance and check diagnostics. Free during Beta.
Run Simple Regression →