Logistic Regression
Model a binary outcome and predict the probability of an event from one or more predictors. Estimate coefficients and odds ratios by maximum likelihood, and evaluate classification quality with the ROC curve and AUC.
Run Logistic Regression →What is Logistic Regression?
Logistic regression models the probability of a binary outcome (event or no event, success or failure) as a function of one or more predictors. Instead of fitting a straight line, it fits an S-shaped logistic (sigmoid) curve that always produces predicted probabilities between 0 and 1.
It works by modeling the log-odds (the logit) of the outcome as a linear function of the predictors. Exponentiating a coefficient gives an odds ratio, the multiplicative change in the odds of the event for a one-unit increase in that predictor, which is the natural way to interpret the results.
Because the outcome is not continuous, coefficients are estimated by maximum likelihood rather than least squares, and model fit is judged by likelihood-based measures and by classification performance such as the ROC curve and its area under the curve (AUC).
In plain terms: When the answer is yes or no, a straight line is the wrong tool, because it can predict probabilities below 0 or above 1. Logistic regression fits an S-curve that stays between 0 and 1, and reports odds ratios, so you can say something like each extra year of age raises the odds of the event by 8 percent.
Key Concepts
Log-Odds & Logit
The model is linear in the log-odds of the event. The logit link keeps predicted probabilities inside the 0 to 1 range.
Odds Ratios
Exp(coefficient) is the odds ratio: greater than 1 means the predictor raises the odds of the event, less than 1 means it lowers them.
ROC & AUC
The ROC curve plots sensitivity against 1 minus specificity across thresholds; AUC summarizes classification ability, with 0.5 being chance and 1.0 perfect.
Key Formulas
Interpreting the Output
Interpret coefficients as odds ratios. An odds ratio of 1.08 for age means each additional year multiplies the odds of the event by 1.08, an 8 percent increase in odds, holding other predictors constant.
Assess classification with the ROC curve and AUC rather than a single accuracy number, especially when the classes are imbalanced. A high AUC means the model ranks positive cases above negative ones well; the operating threshold can then be chosen to balance sensitivity and specificity for your context.
Assumptions & Validation
Binary Outcome
The dependent variable has two categories (standard logistic regression).
If violated: For more than two categories use multinomial or ordinal logistic regression.
Independence
Observations are independent of one another.
If violated: Use mixed-effects or clustered logistic models for grouped data.
Linearity in the Logit
Continuous predictors are linearly related to the log-odds, not to the probability itself.
If violated: Add polynomial terms, spline, or categorize the predictor.
No Severe Multicollinearity
Predictors are not near-perfectly correlated; check VIF.
If violated: Remove or combine correlated predictors.
Adequate Events
Enough events per predictor (a common guideline is 10 or more) for stable estimates.
If violated: Reduce predictors or use penalized logistic regression.
⚠️ Check assumptions first
Logistic regression does not assume normally distributed residuals or a linear relationship with the probability; that is a frequent misconception carried over from linear regression. It does assume linearity in the log-odds. Also beware of separation, where a predictor perfectly splits the outcome: it makes coefficients blow up and requires penalized estimation to handle.
When NOT to Use Logistic Regression
Continuous Outcome
For a continuous response, use linear or multiple regression; logistic regression models probabilities, not amounts.
More Than Two Categories
For three or more unordered outcomes use multinomial logistic regression; for ordered categories use ordinal logistic regression.
Count Outcomes
For counts of events, Poisson or negative-binomial regression fits better than forcing a binary model.
Industry Applications
Quality Pass/Fail
Predict the probability that a unit fails inspection from process variables, prioritizing high-risk conditions.
Customer Churn
Estimate the probability a customer churns from usage and account features, and rank customers by risk.
Medical & Risk Screening
Model the probability of an outcome from risk factors, reporting odds ratios clinicians can interpret.
Credit & Fraud
Classify applications or transactions as high or low risk and tune the threshold to the cost of errors.
Frequently Asked Questions
When should I use logistic regression instead of linear regression?
Use logistic regression whenever the outcome is binary, such as pass or fail, churn or retain, event or no event. Linear regression can predict values below 0 or above 1 for such outcomes, which are impossible as probabilities. Logistic regression fits an S-shaped curve that always yields probabilities between 0 and 1 and reports interpretable odds ratios.
What is an odds ratio?
An odds ratio is the exponential of a logistic regression coefficient. It is the multiplicative change in the odds of the event for a one-unit increase in the predictor, holding the others constant. An odds ratio above 1 means the predictor increases the odds of the event, below 1 means it decreases them, and exactly 1 means no effect.
Does logistic regression assume normally distributed residuals?
No. That assumption belongs to linear regression. Logistic regression assumes independent observations and linearity between the predictors and the log-odds of the outcome, and it is fit by maximum likelihood rather than least squares. Expecting normal residuals here is a common mistake carried over from linear models.
How do I evaluate a logistic regression model?
Evaluate classification performance rather than R-squared. The ROC curve plots the true-positive rate against the false-positive rate across thresholds, and the area under it (AUC) summarizes how well the model separates the two classes, where 0.5 is chance and 1.0 is perfect. A confusion matrix at your chosen threshold shows the concrete trade-off between sensitivity and specificity.
What is the logit or log-odds?
The logit is the natural logarithm of the odds of the event, ln of p divided by one minus p. Logistic regression models this logit as a linear function of the predictors. Working on the log-odds scale is what keeps predicted probabilities bounded between 0 and 1 once you transform back through the logistic function.
What is separation and why does it cause problems?
Separation occurs when a predictor or combination of predictors perfectly, or almost perfectly, distinguishes the two outcome classes. Maximum likelihood estimation then pushes coefficients toward infinity and standard errors become unusable. Penalized approaches such as Firth's method or ridge-style regularization provide stable estimates when separation is present.
Model Binary Outcomes and Probabilities
Estimate odds ratios, predict probabilities, and evaluate with ROC and AUC. Free during Beta.
Run Logistic Regression →