Generalized Linear Models (GLMs) Overview

Introduction

Clinical trial endpoints do not all behave like normally distributed continuous measurements.

A laboratory measurement such as change in hemoglobin may be approximately continuous and reasonably modeled with ordinary linear regression. But many important clinical outcomes are fundamentally different.

Examples include:

• Response versus no response
• Occurrence versus non-occurrence of an adverse event
• Number of seizures during follow-up
• Number of exacerbations
• Number of hospitalizations
• Proportion of patients responding
• Positive or negative diagnostic outcomes

Applying ordinary linear regression to these outcomes can produce inappropriate predictions, incorrect variance assumptions, or coefficients that are difficult to interpret clinically.

Generalized linear models (GLMs) provide a unified framework for modeling many of these non-normal outcomes.

From Linear Regression to GLMs

Ordinary linear regression can be written as:

The model assumes that the conditional mean of \(Y\) is a linear function of the predictors:

A key feature of ordinary linear regression is that the predicted mean can, in principle, take any real value.

That is reasonable for some continuous outcomes, but not for a probability or a count.

For example, a model for probability should never predict:

or:

A GLM solves this problem by allowing the linear predictor to operate on a transformed scale.

The Three Components of a GLM

Every standard GLM can be understood through three components:

These three components are the foundation of the entire GLM framework.

The Linear Predictor

The systematic portion of a GLM is usually written as:

The quantity \(\eta_i\) is called the linear predictor.

It has the same basic structure as ordinary linear regression.

For example, suppose a clinical trial models the probability of response using treatment and baseline disease severity:

The difference is that the linear predictor does not necessarily equal the expected outcome itself.

Instead, it is connected to the expected outcome through the link function.

The Link Function

A GLM uses a link function \(g(\cdot)\) such that:

where:

• \(\mu_i=E(Y_i\mid X_i)\) is the conditional mean.
• \(\eta_i\) is the linear predictor.
• \(g(\cdot)\) is the link function.

Equivalently:

where \(g^{-1}\) is the inverse link function.

Common GLMs at a Glance

The Exponential Family

The standard GLM framework is based on probability distributions belonging to the exponential family.

A common representation is:

where \(\theta\) is the canonical parameter and \(\phi\) is a dispersion parameter when one is present.

The important practical point is not memorizing this expression.

The important point is that many familiar probability distributions can be handled within the same modeling framework.

Examples include:

• Normal
• Binomial
• Poisson
• Gamma
• Inverse Gaussian

GLMs vs. Ordinary Linear Regression

Logistic Regression as a GLM

One of the most important GLMs in clinical research is logistic regression.

Suppose \(Y_i\) is a binary endpoint:

Then:

where \(p_i\) is the probability of response.

The logistic link is:

Thus:

The inverse transformation gives:

The inverse logit ensures that the predicted probability always lies between 0 and 1.

Interpreting a Logistic Regression Coefficient

Suppose:

A one-unit increase in \(X\) changes the log odds by \(\beta_1\).

Exponentiating gives the odds ratio:

For example, if:

then:

A one-unit increase in \(X\) is associated with approximately twice the odds of the outcome, holding the other model covariates constant.

Poisson Regression as a GLM

Count outcomes require a different probability model.

Suppose \(Y_i\) represents the number of clinical events experienced by patient \(i\).

A Poisson model assumes:

The Poisson mean and variance are both:

The standard link is the log link:

Therefore:

and:

Interpreting a Poisson Regression Coefficient

Exponentiating a Poisson coefficient gives a multiplicative effect on the expected event rate:

This is often called a rate ratio or incidence rate ratio, depending on the analysis.

For example, if:

then:

The expected event rate is approximately 20% lower for a one-unit increase in the predictor, all else equal.

Exposure Time and the Offset

Clinical count outcomes frequently have unequal observation times.

For example, one patient may be followed for 12 months while another is followed for only 6 months.

Simply modeling the raw number of events can therefore be misleading.

A Poisson model can incorporate exposure using an offset:

where \(t_i\) is the exposure time.

Equivalently:

The model therefore compares event rates rather than simply event counts.

Negative Binomial Regression

A major limitation of the Poisson model is its equidispersion assumption:

Clinical event data frequently violate this assumption.

For example, suppose the mean number of exacerbations is 2.0 but the variance is 7.5.

The data are substantially more variable than a Poisson model expects.

This phenomenon is called overdispersion.

Negative binomial regression introduces an additional dispersion parameter and can accommodate variance greater than the mean.

A common parameterization has:

where \(\alpha>0\) controls the degree of overdispersion.

Gamma Regression

GLMs can also be used for positive continuous outcomes.

Suppose the endpoint is a strictly positive measure such as:

• Time spent in hospital
• Cost
• Biomarker concentration
• Duration of a positive-valued clinical measurement

If the distribution is strongly right-skewed, a Gamma GLM may be useful.

A common choice is the log link:

so that:

The log link guarantees positive predicted means.

The Identity Link

The identity link is:

This means:

Ordinary linear regression can therefore be viewed as a special case of the GLM framework using a normal response distribution and identity link.

Canonical Link Functions

Each exponential-family distribution has a corresponding canonical link.

The canonical link has useful mathematical properties, but it is not mandatory that every analysis use the canonical link.

For example, binomial data may sometimes be modeled with a log link when a risk-ratio interpretation is desired and the modeling assumptions are appropriate.

Mean-Variance Relationship

One of the most important conceptual differences between GLMs concerns how the variance depends on the mean.

In a normal model:

The variance is constant under the standard model.

For a Poisson model:

For a binomial model with fixed \(n\):

Thus, the distributional assumption determines not only the shape of the outcome distribution but also its relationship between mean and variance.

Why the Variance Assumption Matters

Suppose two clinical datasets both have a mean count of 4.

The same mean does not imply the same probability model.

Model selection therefore requires attention to both the expected outcome and the observed variability.

Likelihood-Based Estimation

GLM coefficients are commonly estimated using maximum likelihood.

The likelihood represents how plausible the observed data are for a given set of model parameters.

If the observations are independent, the likelihood is:

It is usually easier computationally to maximize the log-likelihood:

The estimated coefficient vector is the value that maximizes this quantity.

What Is Deviance?

Deviance is one of the central model-comparison quantities used in GLMs.

Conceptually, it compares the fitted model with a saturated model that fits the observed data as closely as possible.

The deviance is commonly expressed as:

Smaller deviance generally indicates a model that fits the observed data better, all else being equal.

For nested models, the difference in deviance can often be used to test whether additional predictors improve model fit.

Null Deviance and Residual Deviance

Statistical software often reports:

• Null deviance: deviance for a model containing only an intercept.
• Residual deviance: deviance for the fitted model containing the specified predictors.

The difference:

quantifies the improvement in fit obtained by adding the predictors.

For nested models, this difference can be evaluated using an appropriate likelihood-ratio test.

AIC and BIC

Two common information criteria are:

and:

where:

• \(\ell\) is the maximized log-likelihood.
• \(k\) is the number of estimated parameters.
• \(n\) is the sample size.

Both criteria balance goodness of fit against model complexity.

Lower values are generally preferred when comparing competing models fitted to the same dataset and outcome.

Interpreting GLM Coefficients

The meaning of a regression coefficient depends strongly on the link function.

A Simple Clinical Example

Suppose a randomized Phase III trial evaluates whether treatment reduces the number of disease exacerbations during one year.

The primary analysis variable is the number of exacerbations per patient.

A possible modeling strategy is:

with:

Suppose the estimated treatment coefficient is:

Then:

The treatment-associated rate ratio is approximately 0.70.

Holding the other covariates constant, the estimated exacerbation rate is approximately 30% lower in the treatment group than in the reference group.

Binary Outcomes: Logistic vs. Linear Probability Models

A binary endpoint can technically be modeled using ordinary linear regression, creating what is called a linear probability model.

For example:

However, the fitted values may fall outside the interval \([0,1]\).

Logistic regression avoids this problem through the inverse-logit transformation.

This is one reason logistic regression is generally preferred for standard binary-outcome modeling.

Counts: Poisson vs. Negative Binomial

For count outcomes, a common decision is whether to use Poisson or negative binomial regression.

GLMs and Clinical Trial Covariates

GLMs can include many of the covariates commonly encountered in clinical trials.

Examples include:

• Treatment group
• Age
• Sex
• Baseline disease severity
• Geographic region
• Prior treatment
• Baseline biomarker level
• Stratification factors

For example, logistic regression might model response as:

The treatment coefficient then represents the treatment association after accounting for the other included covariates.

Categorical Predictors

Categorical clinical variables can be incorporated through indicator variables.

Suppose treatment has three groups:

• Placebo
• Low dose
• High dose

Using placebo as the reference, two indicator variables can be created:

Then:

represents the effect of low dose relative to placebo on the relevant multiplicative scale, while:

represents the high-dose effect relative to placebo.

Interactions in GLMs

GLMs can also include interactions.

For example:

where \(T_i\) is treatment and \(S_i\) is a subgroup indicator.

The interaction coefficient \(\beta_3\) tests whether the treatment effect differs according to subgroup status on the model's link-function scale.

Confidence Intervals

GLM coefficients are usually reported with confidence intervals.

For a coefficient \(\hat{\beta}\), a large-sample Wald interval is often:

For a multiplicative parameter such as an odds ratio or rate ratio, the interval is commonly obtained by exponentiating the coefficient-scale limits.

where \(L\) and \(U\) are the lower and upper confidence limits for the coefficient.

Predicted Values

Regression coefficients are often easier to communicate clinically when converted into predicted outcomes.

For logistic regression:

For a log-link model:

Predicted probabilities, event rates, or mean outcomes can then be reported for clinically meaningful patient profiles.

Marginal vs. Conditional Interpretation

GLM predictions can be presented in different ways.

A conditional prediction corresponds to a specific covariate pattern.

For example:

• 65-year-old patient
• Moderate baseline disease severity
• Active treatment

A marginal prediction averages predictions over a population or covariate distribution.

Marginal summaries can sometimes be easier to interpret as population-level treatment effects.

Model Diagnostics

A fitted GLM should not be accepted simply because the software successfully produced coefficient estimates.

Model diagnostics should assess whether the assumptions and model structure are reasonable.

Useful diagnostic considerations include:

• Residual behavior
• Influential observations
• Overdispersion
• Goodness of fit
• Linearity of continuous predictors on the link scale
• Potential interactions
• Multicollinearity
• Separation in logistic regression
• Excess zeros in count data

Linearity on the Link Scale

One of the most frequently misunderstood GLM assumptions is linearity.

A logistic regression does not necessarily assume that:

Instead, it assumes linearity of the predictor on the logit scale:

Similarly, a Poisson model with a log link assumes:

Therefore, nonlinear relationships may need transformations, splines, or other appropriate modeling approaches.

Continuous Covariates and Splines

Suppose age has a nonlinear relationship with the probability of treatment response.

Rather than assuming:

a flexible function \(f(\operatorname{Age})\) can be used:

Restricted cubic splines and other spline methods are commonly used for this purpose.

Overdispersion Diagnostics

For a Poisson model, substantial overdispersion can indicate that the variance assumption is inappropriate.

One rough diagnostic is the ratio:

A value substantially greater than 1 can indicate overdispersion, although this should not be treated as an automatic decision rule.

Other approaches include examining Pearson residuals and comparing Poisson and negative binomial models.

Separation in Logistic Regression

Logistic regression can encounter a problem called complete separation.

Suppose every patient receiving treatment responds while every patient receiving placebo fails.

The model may attempt to send the treatment coefficient toward infinity.

This can result in:

• Extremely large coefficients
• Huge standard errors
• Failure of ordinary maximum likelihood estimation
• Unstable odds ratios

Possible approaches include penalized likelihood methods or other prespecified methods designed for sparse binary data.

Zero-Inflated and Hurdle Models

Some clinical count datasets contain substantially more zeroes than a standard Poisson or negative binomial model can accommodate.

For example, a trial might record the number of disease exacerbations, and many patients may have zero exacerbations.

A zero-inflated model can conceptually separate:

• A process generating structural zeroes
• A count process generating zero and positive counts

A simplified zero-inflated model can be written as:

while for positive counts:

These models are related to GLMs but introduce a more complex mixture structure.

GLMs vs. Generalized Estimating Equations

A standard GLM generally assumes independent observations conditional on the model.

Clinical trials can violate this assumption when patients contribute repeated measurements.

For correlated outcomes, approaches such as:

• Generalized estimating equations (GEE)
• Generalized linear mixed models (GLMMs)
• Other longitudinal models

may be more appropriate.

GLM vs. GLMM

Software Implementation in R

The base R function glm() provides a general interface for fitting many standard GLMs.

For logistic regression:

fit_logistic <- glm(
  response ~ treatment + age + baseline_severity,
  family = binomial(link = "logit"),
  data = clinical_data
)

summary(fit_logistic)

The coefficient estimates are on the log-odds scale.

Odds ratios can be obtained by exponentiating the coefficients:

exp(coef(fit_logistic))

R: Poisson Regression

fit_poisson <- glm(
  exacerbations ~ treatment + age + baseline_severity,
  family = poisson(link = "log"),
  data = clinical_data
)

summary(fit_poisson)

The exponentiated treatment coefficient is interpreted as a multiplicative effect on the expected count or rate, depending on whether exposure time is incorporated.

R: Poisson Regression with an Offset

Suppose each patient has a different follow-up time.

fit_poisson_rate <- glm(
  exacerbations ~ treatment + age + baseline_severity +
    offset(log(followup_time)),
  family = poisson(link = "log"),
  data = clinical_data
)

The offset is constrained to have coefficient 1 and adjusts the model for different exposure durations.

R: Negative Binomial Regression

Negative binomial regression is not part of base R's glm() family options in the same way as Poisson regression.

A common implementation uses the MASS package:

library(MASS)

fit_nb <- glm.nb(
  exacerbations ~ treatment + age + baseline_severity,
  data = clinical_data
)

summary(fit_nb)

exp(coef(fit_nb))

The exponentiated treatment coefficient is interpreted as a rate or mean ratio according to the model and exposure specification.

R: Gamma GLM

fit_gamma <- glm(
  hospital_days ~ treatment + age + baseline_severity,
  family = Gamma(link = "log"),
  data = clinical_data
)

summary(fit_gamma)

exp(coef(fit_gamma))

The exponentiated coefficient describes a multiplicative change in the conditional mean.

R: Predicted Probabilities

For logistic regression, predicted probabilities can be generated with type = "response":

predicted_probability <- predict(
  fit_logistic,
  newdata = new_patients,
  type = "response"
)

predicted_probability

This converts predictions from the linear predictor scale to the probability scale.

R: Predicted Means for Count Models

The same principle applies to log-link models:

predicted_rate <- predict(
  fit_nb,
  newdata = new_patients,
  type = "response"
)

predicted_rate

The result is on the expected-count or rate scale implied by the model.

SAS: Logistic Regression

In SAS, a standard logistic GLM can be fitted using PROC LOGISTIC:

proc logistic data=clinical_data;
    class treatment(ref='Placebo') / param=ref;

    model response(event='1') =
        treatment
        age
        baseline_severity;

    oddsratio treatment;
run;

The ODDSRATIO statement requests an odds-ratio interpretation for the treatment effect.

SAS: Poisson Regression

proc genmod data=clinical_data;
    class treatment(ref='Placebo');

    model exacerbations =
        treatment
        age
        baseline_severity
        / dist=poisson
          link=log
          type3;

    estimate 'Treatment Rate Ratio'
        treatment 1 -1
        / exp;
run;

If follow-up differs, an offset can be incorporated:

proc genmod data=clinical_data;
    class treatment(ref='Placebo');

    model exacerbations =
        treatment
        age
        baseline_severity
        / dist=poisson
          link=log
          offset=log_followup;
run;

SAS: Negative Binomial Regression

proc genmod data=clinical_data;
    class treatment(ref='Placebo');

    model exacerbations =
        treatment
        age
        baseline_severity
        / dist=negbin
          link=log
          type3;

    estimate 'Treatment Rate Ratio'
        treatment 1 -1
        / exp;
run;

The negative binomial distribution is useful when the count data exhibit overdispersion relative to the Poisson assumption.

Choosing the Distribution

The distribution should be selected based on the scientific nature and observed behavior of the endpoint.

A Practical Model-Selection Workflow

GLM Interpretation: A Three-Step Rule

When interpreting a GLM coefficient, use the following sequence.

Step 1: Identify the link

Ask whether the model uses:

• Identity
• Logit
• Log
• Another link

Step 2: Interpret the coefficient on the link scale

For example, a logistic coefficient represents a change in log odds.

Step 3: Transform to the clinical scale

For a logit model:

For a log-link model:

This simple procedure prevents many common interpretation errors.

Common GLM Mistakes

1. Using ordinary linear regression for a binary outcome. The predictions can fall outside 0 and 1 and the error structure is generally inappropriate.
2. Using Poisson regression without checking dispersion. Substantial overdispersion can make standard Poisson inference inappropriate.
3. Confusing odds ratios with risk ratios. They are different effect measures.
4. Interpreting log-link coefficients as additive effects. Exponentiation is generally required for a multiplicative interpretation.
5. Ignoring unequal exposure time in count models. An offset may be needed when patients have different observation times.
6. Assuming linearity on the raw outcome scale. For a GLM, linearity generally concerns the link-transformed mean.
7. Ignoring separation in logistic regression. Sparse binary data can produce unstable maximum likelihood estimates.
8. Choosing a distribution only from convenience. The probability model should reflect the clinical endpoint and data-generating mechanism.
9. Reporting only the coefficient. Clinical readers often need odds ratios, rate ratios, predicted probabilities, or predicted means.
10. Ignoring correlation in repeated observations. A standard GLM assumes conditional independence and may not be appropriate for longitudinal or clustered data.

GLMs in Clinical Trial Reporting

A statistical report should provide enough information for the model to be understood and reproduced.

At minimum, report:

• Endpoint definition
• Distribution used
• Link function
• Covariates
• Reference categories
• Treatment coding
• Interaction terms, if applicable
• Offset, if applicable
• Estimated effect
• Confidence interval
• Statistical test or p-value, where appropriate
• Model diagnostics or fit considerations

Example Reporting Language: Logistic Regression

A concise clinical report might state:

Example Reporting Language: Negative Binomial Regression

GLMs and Multiplicity

Using a GLM does not automatically solve multiplicity problems.

A clinical trial may evaluate:

• Multiple endpoints
• Multiple treatment groups
• Multiple subgroup analyses
• Multiple time points
• Multiple contrasts

If multiple inferential comparisons are planned, the statistical analysis strategy should address multiplicity separately.

GLMs and Missing Data

A GLM does not automatically determine how missing observations should be handled.

The appropriate strategy depends on:

• The endpoint
• The reason observations are missing
• The timing of missingness
• The estimand
• The analysis population
• The assumptions made about the missing-data mechanism

Possible approaches may include:

• Likelihood-based analysis
• Multiple imputation
• Inverse probability weighting
• Sensitivity analyses
• Prespecified missing-data rules

Missing-data handling should therefore be considered separately from the choice of GLM family.

GLMs and the Estimand Framework

In modern clinical trials, model choice should be connected to the estimand.

Before choosing the model, investigators should clarify:

• What treatment effect is being estimated?
• Which population is relevant?
• How are intercurrent events handled?
• What summary measure will represent the treatment effect?

The GLM is then selected to estimate the intended treatment effect appropriately.

When a GLM Is Not Enough

GLMs are powerful, but they are not a universal solution.

Alternative approaches may be needed for:

• Repeated continuous measurements
• Repeated binary outcomes
• Hierarchical data
• Time-to-event endpoints
• Competing risks
• Multi-state processes
• Complex longitudinal count outcomes
• Highly zero-inflated data
• Joint modeling problems

For example, a time-to-event endpoint is usually modeled using survival analysis rather than a standard GLM.

The GLM Decision Tree

The Big Picture

The most useful way to understand GLMs is not as a collection of unrelated regression procedures.

Instead, think of them as a common framework:

Summary Table of Major GLMs

A Compact Mathematical Summary

The general GLM can be summarized by:

with:

and:

The probability distribution specifies how \(Y_i\) varies around \(\mu_i\).

That gives the three essential pieces:

Common Clinical Examples

What to Remember for Interviews and Exams

If you are preparing for a biostatistics interview or examination, remember these core points:

• A GLM has a random component, systematic component, and link function.
• The linear predictor is typically \(\eta=X\beta\).
• The link connects the mean to the linear predictor.
• Logistic regression is a binomial GLM with a logit link.
• Poisson regression is a Poisson GLM with a log link.
• Negative binomial regression is commonly used for overdispersed counts.
• Exponentiating logistic coefficients gives odds ratios.
• Exponentiating log-link coefficients gives multiplicative effects such as rate ratios.
• Poisson regression assumes mean equal to variance.
• GLM linearity generally concerns the link-transformed mean.
• Standard GLMs do not automatically handle repeated-measures correlation.
• Model selection should be driven by the clinical endpoint and data structure.

Final Practical Workflow

The Most Important Concept

The most important idea behind generalized linear models is that different types of clinical outcomes require different probability models, but their regression structures can still be expressed within one unified framework.

A GLM asks three fundamental questions:

1. What distribution describes the outcome?
2. What link function connects its expected value to the predictors?
3. What linear predictor describes the relationship between covariates and the outcome?

For a binary endpoint, this leads naturally to logistic regression.

For a count endpoint, it may lead to Poisson or negative binomial regression.

For a positive continuous endpoint, a Gamma GLM may be appropriate.

For a normally distributed continuous endpoint, ordinary linear regression is itself a special case of the broader framework.

References

McCullagh, P. & Nelder, J.A. (1989). Generalized Linear Models. 2nd ed. Chapman & Hall.
Dobson, A.J. & Barnett, A.G. (2018). An Introduction to Generalized Linear Models. 4th ed. CRC Press.
Agresti, A. (2015). Foundations of Linear and Generalized Linear Models. Wiley.
Hosmer, D.W., Lemeshow, S. & Sturdivant, R.X. (2013). Applied Logistic Regression. 3rd ed. Wiley.
Hilbe, J.M. (2011). Negative Binomial Regression. 2nd ed. Cambridge University Press.
Hardin, J.W. & Hilbe, J.M. (2018). Generalized Linear Models and Extensions. 4th ed. Stata Press.