Generalized Linear Models Explained: 8 Regression Types, Their Assumptions, and How to Choose

    Generalized linear models (GLMs) extend ordinary least squares regression to response variables drawn from the exponential family, so that continuous, binary, count, and categorical outcomes can all be handled within one consistent framework. This article walks through the three core GLM components — the random component, the systematic component, and the link function — and explains the distribution assumptions and typical use cases behind eight common regressions: linear, logistic, Poisson, negative binomial, gamma, inverse Gaussian, ordinal logistic, and multinomial logistic. It also gives a practical model-selection path, a list of diagnostics and evaluation techniques, and notes on scaling statistical modeling to large datasets.

    1. Why Generalized Linear Models Are Needed

    Ordinary least squares (OLS) regression rests on three implicit assumptions: the response variable is normally distributed, the residual variance is roughly constant, and the response can take any real value. Real business data often violates all three. Visit counts or device failure counts are non-negative integers; whether a user clicked or defaulted is a binary outcome; waiting times, monetary amounts, and lifetimes are strictly positive and typically right-skewed.

    Forcing such data into a linear regression produces negative predictions, variance that drifts with the mean, and misleading goodness-of-fit statistics.

    Generalized linear models (GLMs) address exactly these problems. They keep the regression framework intact but relax one key assumption — the distribution of the response — turning linear regression into a general framework that handles distributions from the exponential family. Continuous, discrete, categorical, and count data can therefore be modeled under a single body of theory, which keeps the derivations consistent and lowers the cost of learning and implementation.

    2. The Three Core Components of a GLM

    Every GLM has three parts. Understanding them is the foundation for all eight regressions covered below.

    1. Random component

    This is the probability distribution assumed for the response variable Y. In a GLM, the distribution must belong to the exponential family. Common members include the Gaussian (normal), binomial, Poisson, gamma, inverse Gaussian, and negative binomial distributions. The choice is driven by the data-generating process and the range the response can take.

    2. Systematic component

    This is the linear predictor:

    1. η = β₀ + β₁x₁ + β₂x₂ + ... + βₚxₚ

    It captures the linear effect of the predictors on the response, and it is the part shared by all regressions.

    3. Link function

    A monotone, differentiable function g(·) that connects the expected response μ = E(Y) to the linear predictor, so that g(μ) = η. The link function acts as an adapter: it maps a mean with a restricted range onto the entire real line, which is what allows the linear structure to hold. The canonical link is usually the best default choice — the identity function for the normal distribution, for example, and the log function for the Poisson.

    3. Applicability Boundaries of the Eight Common Regressions

    Under this unified framework, different combinations of distribution and link function produce regression methods with distinct targets. The table below summarizes their distributional assumptions and typical applications.

    Regression type Distribution / link Suitable data Typical scenarios
    Linear regression Gaussian / identity Continuous real values Sales and temperature forecasting
    Logistic regression Binomial / logit Binary outcome Click-through and default prediction
    Poisson regression Poisson / log Counts (mean ≈ variance) Visit counts, accident counts
    Negative binomial regression Negative binomial / log Overdispersed counts Sparse event counts
    Gamma regression Gamma / log Positive, right-skewed continuous Monetary amounts, durations
    Inverse Gaussian regression Inverse Gaussian / inverse square Positive continuous, heavy-tailed Rare extreme events
    Ordinal logistic regression Ordered categories Ordinal multiclass Satisfaction levels
    Multinomial logistic regression Multinomial categories Unordered multiclass Channel choice

    Linear regression (Gaussian regression) is the most basic case. The response is a continuous real value, and the model estimates the mean and produces interval predictions.

    Logistic regression (binomial regression) uses the logit link to map probabilities into the [0, 1] interval. Once exponentiated, the coefficients have an odds ratio interpretation, which makes it the workhorse for classification problems.

    Poisson regression is designed for non-negative counts. Its log link means that a one-unit change in a predictor multiplies the mean count. It assumes the mean equals the variance, which suits event-count modeling.

    Negative binomial regression handles overdispersion — count data whose variance is substantially larger than its mean. By introducing a dispersion parameter, it relaxes the equal mean–variance constraint of the Poisson model, making it the more robust option for sparse or clustered counts.

    Gamma regression targets strictly positive, right-skewed continuous data such as spending amounts and task durations. Inverse Gaussian regression goes further, fitting positive continuous variables with heavy tails and frequent extremes. Both use a log link, which guarantees positive predictions.

    Ordinal logistic regression handles multiclass outcomes with a natural ordering (for example, low/medium/high). Multinomial logistic regression handles unordered categories (for example, three purchase channels), estimating each class probability relative to a baseline category.

    4. A Practical Path to Model Selection

    Selection comes down to two questions: what type of values the response takes, and what the dispersion characteristics of the data are.

    • Response is continuous and roughly symmetric → linear regression
    • Response has only two classes → logistic regression
    • Response is an ordered grade → ordinal logistic regression
    • Response is an unordered multiclass label → multinomial logistic regression
    • Response is a non-negative count → start with Poisson regression
    • Count variance is far larger than the mean (overdispersion) → switch to negative binomial regression
    • Response is strictly positive, skewed continuous data → gamma or inverse Gaussian regression

    A useful rule of thumb: fit the count data with Poisson regression first, then check whether the residual dispersion statistic is close to 1. A clear deviation signals overdispersion and points to negative binomial regression instead.

    5. Diagnostics and Evaluation

    Diagnostics are mandatory after fitting; without them, there is no basis for trusting the model.

    Goodness-of-fit checks: Compare candidate models using deviance, the Pearson chi-square statistic, and information criteria such as AIC and BIC. Smaller values are generally better.

    Overdispersion testing: A key step for Poisson and negative binomial models, used to judge whether the dispersion parameter is significant.

    Residual analysis: Plot residuals against fitted values to look for systematic patterns. For classification models, ROC/AUC measures discriminative ability; for count models, compare predicted means against observed frequencies.

    Multicollinearity and influential points: Use the variance inflation factor (VIF) to check predictor collinearity, and Cook’s distance to identify high-leverage points.

    Illustrative code (expressed as formulas):

    1. # Logistic regression: binary outcome
    2. model <- glm(converted ~ age + clicks + device, family = binomial(link = "logit"))
    3. # Poisson regression: count outcome
    4. model <- glm(visits ~ campaign + weekday, family = poisson(link = "log"))
    5. # Negative binomial regression: overdispersed counts
    6. model <- glm.nb(defects ~ line + shift)
    7. summary(model)

    6. Resource Considerations for Production Scale

    Once the sample reaches tens of millions of rows and the number of predictors is high, single-machine memory and compute can become the bottleneck. Cloud capabilities can take part of the load: store raw data in object storage, run feature engineering and parallel fitting on a distributed compute engine, scale training resources elastically through a container platform, and then feed model metrics into monitoring and alerting services for continuous tracking. Platforms such as Baidu AI Cloud, which offer compute, storage, and monitoring offerings, can help teams turn statistical modeling from a one-off script into a reproducible, maintainable pipeline without being tied to the underlying infrastructure.

    7. Summary

    The value of generalized linear models is a single unifying language that accommodates problems ranging from Gaussian to Poisson responses and from binary to multiclass outcomes. Master the three core components, understand the data-generating assumptions behind each distribution, and pair them with rigorous diagnostics and model selection. The eight regressions are not a pile of isolated tricks; they are optimal fits to different data shapes within one theoretical framework. Understanding their internal connections is what makes it possible to keep things simple while choosing the right tool for the job.