Automating TLF Production Pipelines in R

Introduction

Producing a single clinical-trial table in R is relatively straightforward. Producing 300 tables, listings, and figures repeatedly and reliably is a very different problem.

A clinical study may contain dozens of demographic tables, adverse-event summaries, laboratory listings, vital-sign tables, exposure summaries, efficacy tables, Kaplan-Meier figures, subgroup analyses, and patient-level listings.

If every output is programmed independently, the project quickly becomes difficult to maintain.

A small change to the treatment-group structure may require modifications to dozens of programs. A change to a footnote may require editing dozens of files. A new data snapshot may require manually rerunning an inconsistent collection of scripts.

This is where a TLF production pipeline becomes valuable.

What Does TLF Mean?

TLF stands for:

• Tables
• Listings
• Figures

These outputs are central to clinical-trial reporting.

Why Automate TLF Production?

The most obvious benefit is speed.

But speed is not the primary reason to automate.

The more important benefits are:

• Consistency
• Reproducibility
• Traceability
• Standardization
• Reduced manual intervention
• Centralized formatting
• More efficient QC
• Reliable reruns
• Better change management
• Scalability

A well-designed pipeline should make the second production run nearly identical to the first except for changes caused by the underlying data, specifications, or controlled configuration.

The Core Production Concept

A robust TLF pipeline can be represented as:

The Difference Between a Script and a Pipeline

A script generally performs a sequence of operations.

A pipeline defines a system of controlled transformations.

For example, a simple script might say:

data <- read.csv("adsl.csv")

summary <- data |>
  dplyr::count(TRT01P)

write.csv(summary, "table.csv")

A pipeline asks much larger questions:

• Which version of the input data was used?
• Which specification defined the output?
• Which analysis population was used?
• Which program generated the output?
• Did validation pass?
• What warnings occurred?
• Which other TLFs depend on the same input?
• Can the entire package be regenerated from a clean environment?

A Typical Clinical Trial TLF Ecosystem

A production reporting environment may contain:

Why ADaM Should Usually Be the TLF Input

TLFs should generally be built from analysis-ready datasets rather than repeatedly deriving clinical analysis variables directly from raw or SDTM data.

For example, an adverse-event table may depend on:

• Treatment assignment
• Treatment-emergent flags
• Analysis population flags
• Preferred terms
• System organ classes
• Severity
• Relationship
• Maximum severity

Those analysis concepts should already be represented in the appropriate analysis dataset whenever possible.

Specification-Driven Programming

One of the most important improvements a programming team can make is to move TLF definitions out of individual programs and into structured specifications.

Instead of hard-coding:

table_014 <- make_ae_table(
  data = adae,
  population = "SAFETY",
  treatment = "TRT01P"
)

a specification can describe:

tlf_id: T14
type: table
dataset: ADAE
population: SAFETY
treatment_variable: TRT01P
analysis: treatment_emergent_ae
statistics:
  - n
  - percent
  - total
footnotes:
  - "Subjects may be counted once within each preferred term."

The exact implementation can vary, but the principle is powerful: metadata describes what should be produced; reusable code determines how.

A TLF Specification Table

A practical specification dataset might contain:

Why Metadata Matters

Suppose a study contains 80 adverse-event tables.

If the treatment variable changes from:

TRT01P

to:

TRT01A

a specification-driven system can update the configuration rather than requiring the programmer to edit dozens of independent programs.

This is one of the central ideas behind scalable TLF automation.

Reusable R Functions

A production pipeline should contain reusable functions rather than one-off scripts whenever the same operation appears repeatedly.

For example, a simple treatment-group summary function might be:

summarize_n <- function(data, treatment_var) {

  data |>
    dplyr::count(
      {{ treatment_var }},
      name = "N"
    )

}

The function can then be reused:

summarize_n(
  adsl,
  TRT01P
)

The objective is not to eliminate TLF-specific programs.

The objective is to move repeated logic into tested components.

Build Functions Around Clinical Concepts

Functions should generally represent meaningful reporting operations.

Examples include:

make_population_summary()

make_ae_summary()

make_lab_summary()

make_demographic_table()

make_time_to_event_summary()

make_listing()

make_km_plot()

make_forest_plot()

format_tlf()

validate_tlf()

This makes the code easier for another clinical programmer to understand.

A Simple TLF Function Architecture

A useful conceptual architecture is:

Keep Analysis and Formatting Separate

One of the most common mistakes in TLF programming is mixing statistical calculation with presentation code.

For example, this is difficult to maintain:

data |>
  filter(SAFFL == "Y") |>
  group_by(TRT01P) |>
  summarise(
    n = n()
  ) |>
  gt() |>
  tab_header(
    title = "Table 14.1.1"
  ) |>
  gtsave("table14.html")

The analysis and presentation are tightly coupled.

A more maintainable design is:

analysis_results <- summarize_population(
  data = adsl,
  population = "SAFFL",
  treatment = "TRT01P"
)

formatted_table <- format_tlf(
  analysis_results,
  specification = spec
)

write_tlf(
  formatted_table,
  output = "table14.html"
)

This separation makes testing substantially easier.

Example: Automated Demographics Table

Consider a standard demographics table containing:

• Age
• Sex
• Race
• Ethnicity
• Weight
• Height

A reusable function might begin with:

make_demographics <- function(
  adsl,
  treatment = TRT01P,
  population = SAFFL
) {

  analysis_data <- adsl |>
    dplyr::filter({{ population }} == "Y")

  # Statistical summaries would be generated here.

  analysis_data

}

In a production environment, the function would then generate the required summary statistics and preserve the treatment-group structure defined in the specification.

Analysis Population Logic

Population selection should be centralized wherever possible.

For example:

get_population <- function(
  data,
  flag,
  value = "Y"
) {

  data |>
    dplyr::filter(
      .data[[flag]] == value
    )

}

Then:

safety <- get_population(
  adsl,
  "SAFFL"
)

efficacy <- get_population(
  adsl,
  "ITTFL"
)

Centralizing population logic reduces the risk of inconsistent filtering across TLFs.

Denominators Are a Critical Automation Issue

Counts are usually straightforward. Percentages are more complicated.

For a categorical endpoint:

The difficult question is: What is \(N\)?

The denominator may be:

• Total randomized subjects
• Total treated subjects
• Subjects with available data
• Subjects within a treatment group
• Subjects with a particular baseline measurement
• Subjects at risk at a particular time point

An automated system must therefore encode denominator rules explicitly.

Never Hide Denominator Logic Inside Formatting

A dangerous design is:

paste0(
  n,
  " (",
  round(n / nrow(data) * 100, 1),
  "%)"
)

The calculation may look harmless, but the denominator is implicit.

A more robust design is:

calculate_percent <- function(
  numerator,
  denominator,
  digits = 1
) {

  ifelse(
    denominator > 0,
    round(100 * numerator / denominator, digits),
    NA_real_
  )

}

Now the denominator is explicit and testable.

Standardized Formatting Functions

Clinical TLFs often require highly consistent formatting.

Common formatting rules include:

• Number of decimal places
• Percentage formatting
• Missing-value representation
• Negative-number formatting
• Thousands separators
• Confidence interval formatting
• p-value formatting
• Significant digits
• Column alignment

These should be implemented centrally.

fmt_pct <- function(
  x,
  digits = 1
) {

  ifelse(
    is.na(x),
    "—",
    sprintf(
      paste0("%.", digits, "f%%"),
      x
    )
  )

}

Then every TLF can use the same formatting rule.

Centralize Study Formatting

A production system should ideally have a study-level formatting configuration.

study_format <- list(

  pct_digits = 1,

  mean_digits = 1,

  sd_digits = 2,

  pvalue_digits = 3,

  missing = "—",

  date_format = "%d %b %Y"

)

A change in formatting can then be applied consistently across the reporting package.

Table Shells and TLF Specifications

The shell is the contract between the statistical specification and the programmer.

A shell may define:

• Title
• Population
• Treatment columns
• Rows
• Statistics
• Footnotes
• Ordering
• Display conventions

The production pipeline should treat these specifications as controlled inputs.

Example Specification Object

spec <- list(

  tlf_id = "T14.1.1",

  type = "table",

  dataset = "ADSL",

  population = "SAFFL",

  treatment = "TRT01P",

  title = "Demographic Characteristics",

  variables = c(
    "AGE",
    "SEX",
    "RACE",
    "ETHNIC"
  ),

  statistics = c(
    "n",
    "percent",
    "mean",
    "sd",
    "median",
    "min",
    "max"
  )

)

The important point is that the program can consume the specification rather than embedding all of these decisions directly into the source code.

Driving Multiple TLFs From One Specification

Suppose the specification dataset contains:

TLFID      TYPE       DATASET    PROGRAM
T14.1.1    TABLE      ADSL       demographics
T14.2.1    TABLE      ADAE       ae_summary
T14.3.1    TABLE      ADLB       lab_summary
F14.1.1    FIGURE     ADTTE      km_plot
L16.1.1    LISTING    ADAE       ae_listing

A driver can iterate over the specifications:

for (i in seq_len(nrow(tlf_spec))) {

  current_spec <- tlf_spec[i, ]

  run_tlf(
    specification = current_spec
  )

}

This is the beginning of true TLF automation.

Function Dispatch

One useful approach is to maintain a registry of approved TLF functions.

tlf_registry <- list(

  demographics = make_demographics,

  ae_summary = make_ae_summary,

  lab_summary = make_lab_summary,

  km_plot = make_km_plot,

  ae_listing = make_ae_listing

)

The pipeline can then select the appropriate function from metadata.

run_tlf <- function(specification) {

  fn <- tlf_registry[
    [specification$PROGRAM]
  ]

  fn(specification)

}

In real production code, additional validation should be performed before executing the function.

Validate the Specification Before Running Anything

An automated pipeline should fail early when metadata are invalid.

For example:

validate_specification <- function(spec) {

  required <- c(
    "TLFID",
    "TYPE",
    "DATASET",
    "PROGRAM"
  )

  missing <- setdiff(
    required,
    names(spec)
  )

  if (length(missing) > 0) {

    stop(
      "Missing specification fields: ",
      paste(missing, collapse = ", ")
    )

  }

  invisible(TRUE)

}

This is much better than discovering the missing information halfway through a 200-output production run.

Input Validation

The same principle applies to datasets.

Before generating a TLF, the pipeline should verify that required variables exist.

require_variables <- function(
  data,
  variables
) {

  missing <- setdiff(
    variables,
    names(data)
  )

  if (length(missing) > 0) {

    stop(
      "Missing variables: ",
      paste(missing, collapse = ", ")
    )

  }

}

For example:

require_variables(
  adsl,
  c(
    "USUBJID",
    "TRT01P",
    "SAFFL"
  )
)

Assertions Should Be Everywhere

Production code should contain explicit assumptions.

Examples include:

stopifnot(
  "USUBJID" %in% names(adsl)
)

stopifnot(
  all(adsl$SAFFL %in% c("Y", "N", NA))
)

More sophisticated validation can check uniqueness, ranges, expected values, and relationships between datasets.

Subject-Level Uniqueness

For ADSL, one of the most important structural checks is often one record per subject.

duplicates <- adsl |>
  dplyr::count(USUBJID) |>
  dplyr::filter(n > 1)

if (nrow(duplicates) > 0) {

  stop(
    "ADSL contains duplicate USUBJID records."
  )

}

The exact expectation depends on the dataset, but the principle is general: encode important structural assumptions as executable checks.

TLF-Level Validation

Validation should occur at multiple levels.

Numerical Validation

Suppose a table reports:

Treatment A
N = 100
Events = 37
Percent = 37.0%

The pipeline can validate the arithmetic:

expected_pct <-
  round(
    100 * 37 / 100,
    1
  )

stopifnot(
  expected_pct == 37.0
)

For production systems, these checks should be systematic rather than manually inserted into only a few programs.

Independent QC

Automated validation and independent QC are not necessarily the same thing.

A programming function can verify that its own calculation is internally consistent while still implementing the wrong specification.

Independent QC provides a separate opportunity to detect such errors.

Double Programming

Traditional clinical-trial programming frequently uses independent programming to verify important outputs.

For example:

Primary program
      ↓
TLF output

Independent QC program
      ↓
QC output

        ↓
Compare
        ↓
Resolve differences

Automation can support this process by automatically comparing the two results.

Automated Output Comparison

For numerical datasets, comparison can be performed using structured data rather than visual inspection alone.

compare_results <- function(
  production,
  qc
) {

  # Join using stable keys.

  # Compare analysis values.

  # Return differences.

}

A useful comparison report might contain:

The pipeline should fail or flag the output when unexplained differences remain.

Visual QC

Not every problem is numerical.

A table can contain the correct values but still have:

• Truncated labels
• Incorrect page breaks
• Overlapping figure elements
• Incorrect footnotes
• Incorrect treatment headers
• Unexpected blank pages
• Unreadable fonts
• Incorrect axis labels

Therefore, automated pipelines should include a visual review stage.

Logging

Every production run should generate a meaningful log.

At minimum, capture:

• Run date and time
• Study identifier
• Data snapshot
• Software version
• R version
• Package versions
• TLF identifier
• Execution status
• Warnings
• Errors
• Output path
• Runtime

A Simple Logging Function

log_message <- function(
  message,
  file = "production.log"
) {

  timestamp <- format(
    Sys.time(),
    "%Y-%m-%d %H:%M:%S"
  )

  cat(
    paste0(
      "[",
      timestamp,
      "] ",
      message,
      "\n"
    ),
    file = file,
    append = TRUE
  )

}

Then:

log_message(
  "Starting T14.1.1"
)

log_message(
  "Completed T14.1.1"
)

Error Handling

A production pipeline should distinguish between:

• Successful outputs
• Warnings requiring review
• Recoverable failures
• Critical failures

A basic approach uses tryCatch().

result <- tryCatch(

  {

    run_tlf(spec)

    "SUCCESS"

  },

  warning = function(w) {

    log_message(
      paste(
        "WARNING:",
        conditionMessage(w)
      )
    )

    invokeRestart("muffleWarning")

  },

  error = function(e) {

    log_message(
      paste(
        "ERROR:",
        conditionMessage(e)
      )
    )

    "FAILED"

  }

)

For regulated production, error handling should be designed carefully so that failures cannot be silently ignored.

Fail Fast vs. Continue

There are two common strategies.

The appropriate choice depends on the dependency structure of the study.

Dependencies Matter

Consider:

A pipeline should understand these dependencies rather than simply running files alphabetically.

Directory Structure

A practical project might look like:

study-project/
│
├── data/
│   ├── sdtm/
│   └── adam/
│
├── metadata/
│   ├── tlf_spec.csv
│   ├── formats.csv
│   └── controlled_terms.csv
│
├── R/
│   ├── functions/
│   ├── analysis/
│   ├── formatting/
│   └── validation/
│
├── programs/
│   ├── tables/
│   ├── listings/
│   └── figures/
│
├── tests/
│
├── output/
│   ├── tables/
│   ├── listings/
│   ├── figures/
│   └── qc/
│
├── logs/
│
├── config/
│
└── run_pipeline.R

The exact directory structure can vary, but the underlying separation of responsibilities is valuable.

Do Not Put Everything in One Script

A common beginner implementation is:

# Read data
# Create Table 1
# Create Table 2
# Create Table 3
# Create Listing 1
# Create Figure 1
# Create Figure 2
# ...
# Save everything

This becomes difficult to debug and maintain.

A better architecture is:

run_pipeline.R
      │
      ├── load_config()
      │
      ├── validate_inputs()
      │
      ├── load_metadata()
      │
      ├── load_functions()
      │
      ├── execute_tables()
      │
      ├── execute_listings()
      │
      ├── execute_figures()
      │
      ├── run_qc()
      │
      └── generate_run_report()

The Master Driver

A simplified master driver might look like:

source("R/functions/load_data.R")
source("R/functions/load_specs.R")
source("R/functions/run_tlf.R")
source("R/functions/validate_output.R")

log_message("TLF production started")

validate_inputs()

spec <- load_tlf_spec()

for (i in seq_len(nrow(spec))) {

  run_tlf(
    specification = spec[i, ]
  )

}

validate_outputs()

log_message("TLF production completed")

This driver should remain relatively small.

Its purpose is orchestration, not statistical calculation.

Configuration Files

Study-specific settings should ideally be separated from reusable code.

For example:

study:
  id: ABC123
  title: "Phase 3 Study"

data:
  adam: "data/adam"

output:
  tables: "output/tables"
  listings: "output/listings"
  figures: "output/figures"

format:
  pct_digits: 1
  pvalue_digits: 3

This makes the programming framework more portable between studies.

Environment Management

Reproducibility requires more than saving R scripts.

A TLF generated today may differ from one generated six months later if package behavior has changed.

Important components include:

• R version
• Package versions
• Operating environment
• Source code version
• Input data version
• Specification version

R projects should therefore use an environment-management strategy appropriate to the organization's validation and reproducibility requirements.

Version Control With Git

Git provides version control for:

• R programs
• Specifications
• Configuration
• Documentation
• Validation scripts
• Package code

A useful commit might say:

Fix denominator for TEAE Grade 3 summary

rather than:

updates

Good commit messages create an audit trail for programming changes.

Do Not Treat Git as the Entire Audit Trail

Source-code version control is only one component of traceability.

A production run should also establish which:

• Input data
• Specifications
• Code version
• Configuration
• Software environment

were used to generate the outputs.

Deterministic Output

Whenever possible, the pipeline should be deterministic.

Given identical:

the pipeline should produce the same result.

Random procedures require additional controls.

For example:

set.seed(12345)

may be required for simulation-based analyses.

Dates and Time Zones

Production logs can become confusing when systems use different time zones.

For critical workflows, define a consistent convention for timestamps.

format(
  Sys.time(),
  "%Y-%m-%d %H:%M:%S %Z"
)

The important point is not the exact formatting but the existence of an explicit convention.

Parallelization

If TLFs are independent, some can potentially be generated in parallel.

For example:

library(parallel)

results <- mclapply(
  seq_len(nrow(spec)),
  function(i) {

    run_tlf(
      spec[i, ]
    )

  }
)

However, parallel execution should be introduced only after the pipeline is correct and deterministic.

When Parallelization Helps

Parallelization is most useful when:

• TLFs are computationally independent
• Each TLF requires substantial processing time
• The execution environment supports controlled parallel execution
• Logging remains understandable
• Outputs cannot overwrite one another

Output Naming Conventions

Automated systems need predictable output names.

For example:

T14.1.1_demographics.rtf
T14.2.1_teae_summary.rtf
T14.3.1_laboratory_summary.rtf
F14.1.1_overall_survival.pdf
L16.1.1_adverse_events.rtf

A naming convention should be defined once and applied consistently.

Never Let Programs Invent Their Own Output Paths

This creates unnecessary operational complexity.

Instead:

output_path <- build_output_path(
  tlf_id = spec$TLFID,
  type = spec$TYPE,
  extension = "rtf"
)

The path-building function becomes the single source of truth.

File Integrity Checks

A successful R function call does not necessarily guarantee that the expected file exists and is usable.

After writing a file:

if (!file.exists(output_path)) {

  stop(
    "Expected output was not created: ",
    output_path
  )

}

You can also check that a file has nonzero size:

if (file.info(output_path)$size == 0) {

  stop(
    "Output file is empty."
  )

}

Output Manifest

A useful production pipeline generates a manifest containing every output.

This makes large production runs much easier to review.

Runtime Measurement

Runtime can be captured using:

start <- Sys.time()

run_tlf(spec)

end <- Sys.time()

runtime <- difftime(
  end,
  start,
  units = "secs"
)

Runtime monitoring can identify unexpectedly slow programs.

Quality Gates

A mature pipeline should have explicit quality gates.

What Should Cause a Pipeline to Fail?

Examples include:

• Missing required ADaM dataset
• Missing required variables
• Invalid treatment values
• Duplicate subject records where uniqueness is required
• Missing TLF specification
• Unknown TLF function
• Numerical QC discrepancy
• Missing expected output
• Unexpected empty output
• Unresolved critical warning

The exact rules should be defined by the organization's SOPs and validation strategy.

What Should Become a Warning?

Some conditions may legitimately occur but require review.

Examples include:

• A subgroup contains zero subjects.
• A table has a prespecified empty category.
• A figure has fewer patients than expected.
• An analysis has a sparse category.
• An optional dataset is unavailable.

The important point is that warnings should be visible and documented rather than silently ignored.

Automating Listings

Listings are particularly suitable for automation because they often share common operations:

• Subject ordering
• Treatment ordering
• Variable selection
• Date formatting
• Column labels
• Page breaks
• Footnotes

A generic listing function might look like:

make_listing <- function(
  data,
  columns,
  sort_by,
  title
) {

  result <- data |>
    dplyr::select(
      dplyr::all_of(columns)
    ) |>
    dplyr::arrange(
      dplyr::across(
        dplyr::all_of(sort_by)
      )
    )

  list(
    data = result,
    title = title
  )

}

Individual listings can then be driven by metadata.

Automating Figures

Figures should also be built from reusable components.

For example:

make_km_plot <- function(
  data,
  time,
  event,
  treatment
) {

  # Fit survival model

  # Generate survival estimates

  # Build standardized figure

}

The same figure framework can then be reused across:

• Overall survival
• Progression-free survival
• Duration of response
• Time to treatment discontinuation

Central Figure Themes

A study should ideally use a controlled visual theme.

clinical_theme <- function() {

  ggplot2::theme_classic() +

    ggplot2::theme(
      plot.title = ggplot2::element_text(
        face = "bold"
      ),

      axis.title = ggplot2::element_text(
        size = 11
      ),

      legend.position = "bottom"
    )

}

Centralizing the theme means that a formatting change can propagate across many figures.

Reusable Statistical Components

Some statistical operations appear repeatedly across clinical reports.

Examples include:

• Descriptive statistics
• Confidence intervals
• Exact binomial confidence intervals
• Odds ratios
• Risk differences
• Hazard ratios
• Kaplan-Meier estimates
• Quantiles
• Change from baseline
• Repeated-measures summaries

These are excellent candidates for centralized, validated functions.

Example: Descriptive Statistics Function

summarize_continuous <- function(
  x
) {

  list(

    n = sum(!is.na(x)),

    mean = mean(
      x,
      na.rm = TRUE
    ),

    sd = sd(
      x,
      na.rm = TRUE
    ),

    median = median(
      x,
      na.rm = TRUE
    ),

    min = min(
      x,
      na.rm = TRUE
    ),

    max = max(
      x,
      na.rm = TRUE
    )

  )

}

A production version should also define behavior for empty vectors and other edge cases.

Edge Cases Are Where Automation Often Fails

A function may work perfectly for the primary dataset and fail when:

• A treatment arm contains no subjects.
• A category contains no observations.
• All values are missing.
• There is only one nonmissing value.
• A confidence interval cannot be calculated.
• A denominator is zero.
• A time-to-event analysis has no events.
• A subgroup has very few subjects.

Production functions must explicitly define these cases.

Zero Denominators

For example:

is undefined.

A clinical table should not accidentally display:

NaN%

unless that representation is explicitly intended.

Instead, the display convention should be controlled:

if (denominator == 0) {

  return("—")

}

Empty Treatment Groups

An automated pipeline should not silently drop a treatment group simply because there are no subjects in a particular analysis subset.

The desired behavior may be:

Treatment A     Treatment B     Treatment C
N = 100         N = 0           N = 98

rather than:

Treatment A     Treatment C
N = 100         N = 98

Treatment-column structure is often part of the controlled output specification.

Preserving Factor Levels

This is a common R programming issue.

If treatment is converted to a factor using only observed values, an empty treatment arm may disappear.

Instead, define the expected treatment order explicitly.

treatment_levels <- c(
  "Placebo",
  "Dose 1",
  "Dose 2"
)

data$TRT01P <- factor(
  data$TRT01P,
  levels = treatment_levels
)

This helps preserve consistent table structure.

Controlling Category Order

The same principle applies to:

• Sex
• Race
• Severity
• Response category
• Visit
• Age groups

Never rely on accidental alphabetical ordering when the clinical specification defines a particular order.

TLF Numbering

Output numbering should also be metadata-driven.

For example:

TLFID
14.1.1
14.1.2
14.2.1
14.3.1
15.1.1

The pipeline should use the TLF identifier consistently in:

• File names
• Titles
• Logs
• QC records
• Output manifests

Automated Titles and Footnotes

Titles should not be manually duplicated across programs when they can be generated from controlled metadata.

build_title <- function(
  spec
) {

  paste(
    spec$TLFID,
    spec$TITLE,
    sep = ". "
  )

}

Footnotes can similarly be stored in structured metadata.

Dynamic Titles Require Caution

Automation should not make titles so dynamic that the final wording becomes difficult to review.

A good principle is: automate predictable substitutions, but keep clinically meaningful prose under controlled specification.

Table Rendering

R provides several approaches to rendering tables.

Depending on the organization's requirements, packages and output formats may include:

• gt
• gtsummary
• flextable
• officer
• LaTeX-based approaches
• HTML-based approaches

The choice should be driven by output requirements, formatting needs, validation considerations, and organizational standards.

Do Not Confuse a Formatting Package With a Pipeline

A package can make a table beautiful.

A pipeline determines:

• What data are used
• What analysis is performed
• What population is included
• What statistics are calculated
• How the output is validated
• Where the result is stored
• How the run is documented

Formatting is only one layer.

Testing Reusable Functions

Reusable functions should have automated tests.

For example, a percentage function should be tested with:

• Normal values
• Zero numerator
• Zero denominator
• Missing values
• Large values
• Rounding boundaries

A test framework such as testthat can be used.

test_that(
  "percentage is calculated correctly",
  {

    expect_equal(
      calculate_percent(
        25,
        100
      ),
      25
    )

  }
)

Unit Testing vs. TLF QC

A mature pipeline uses multiple layers rather than relying on one test.

Regression Testing

Once a TLF is known to be correct, its expected behavior can become part of regression testing.

For example, if a reusable adverse-event function is modified, automated tests can verify that previously validated scenarios still produce the expected results.

Golden Files

For some workflows, a controlled reference output can be used for regression testing.

The current output is compared with a known-good reference.

reference/
    T14.1.1_reference.csv
    T14.2.1_reference.csv
    F14.1.1_reference_data.csv

current/
    T14.1.1.csv
    T14.2.1.csv
    F14.1.1_data.csv

Differences should be investigated rather than automatically accepted.

Data Changes Should Produce Explainable Changes

A new data snapshot should change outputs when the underlying information changes.

But a change in output should have an explainable cause.

For example:

T14.2.1
Previous: 42 events
Current:  44 events

Reason:
Two additional treatment-emergent events
were present in the updated data snapshot.

This kind of traceability is extremely valuable during study updates.

Incremental Production

Not every TLF necessarily needs to be regenerated after every change.

A dependency-aware system can determine which outputs depend on changed inputs.

For example:

For very large studies, dependency-aware execution can substantially reduce production time.

Pipeline Orchestration Tools

For more sophisticated workflows, R projects can use tools designed to track dependencies and reproducibility.

Examples include:

• targets
• drake in legacy projects
• Make-based workflows
• CI/CD systems
• Organization-specific workflow engines

The important concept is dependency tracking rather than the particular tool.

The targets Model

A dependency-aware workflow can conceptually look like:

ADSL
  ↓
Demographics Analysis
  ↓
Demographics Table

ADAE
  ↓
AE Analysis
  ↓
AE Tables
  ↓
AE Listings

ADTTE
  ↓
Survival Analysis
  ↓
KM Figure

If ADAE changes, the system can identify which downstream objects need to be rebuilt.

Why Dependency Tracking Is Powerful

Without dependency tracking, a programmer may simply rerun every program.

With dependency tracking, the system can ask:

This is a much more scalable approach.

Continuous Integration

A mature programming repository can automatically run tests whenever code changes.

For example:

This approach can catch programming defects before a formal TLF production run.

Production Run vs. Development Run

The pipeline should distinguish between development and controlled production.

Do Not Over-Automate Clinical Judgment

Automation is excellent for deterministic operations.

It is less appropriate for decisions requiring clinical interpretation.

For example, automation can identify:

• A patient has a laboratory value outside a threshold.
• A treatment group has zero subjects.
• A table differs from the QC result.
• A response trajectory crosses a numerical threshold.

But the system should not automatically decide that an unexpected clinical pattern is clinically meaningful without appropriate review.

Common Anti-Patterns

Copy-and-Paste Programming

Creating 50 programs by copying one another creates subtle differences over time.

A correction made to one program may never reach the other 49.

Hard-Coded Treatment Groups

if (TRT01P == "Treatment A") {
  ...
}

if (TRT01P == "Treatment B") {
  ...
}

This becomes fragile when the study design changes.

Hard-Coded Denominators

pct <- n / 120 * 100

This is dangerous because the denominator can silently become obsolete.

Hidden Population Filters

data[data$SAFFL == "Y", ]

When population rules are hidden throughout dozens of programs, consistency becomes difficult to establish.

Formatting During Analysis

paste0(
  round(mean(x), 1),
  " (",
  round(sd(x), 2),
  ")"
)

This turns numerical results into strings too early.

It is generally better to retain numeric analysis results until the presentation layer.

Silent Error Suppression

try({
  run_tlf()
})

This is particularly dangerous in production.

A failed output should not disappear into a log without being clearly identified.

Numerical Values Should Remain Numeric as Long as Possible

Consider:

result <- data.frame(
  mean = 54.2,
  sd = 8.7
)

Keep these values numeric until the formatting stage.

Only later create:

"54.2 (8.7)"

This makes validation and downstream calculations much easier.

Long-Format Intermediate Results

A useful design is to retain statistical results in long format.

TLFID   TRT01P   PARAMETER   STAT   VALUE
T14.1   Placebo  Age         MEAN   57.2
T14.1   Placebo  Age         SD      8.1
T14.1   Dose A   Age         MEAN   58.4
T14.1   Dose A   Age         SD      7.9

The presentation layer can then reshape these results into the final shell.

This separation makes statistical validation easier.

Why Intermediate Data Matter

A final RTF or PDF is difficult to validate programmatically.

A structured intermediate dataset is much easier to inspect.

Therefore, a useful architecture is:

Traceability

For every final value, a strong system should make it possible to determine:

• Which dataset supplied the observation
• Which variable supplied the value
• Which population rule was applied
• Which analysis function calculated it
• Which specification defined the output
• Which formatting rule displayed it

This is the essence of analysis traceability.

Example Traceability Chain

T14.2.1
  ↓
Specification T14.2.1
  ↓
ADAE
  ↓
SAFFL = "Y"
  ↓
TRTEMFL = "Y"
  ↓
AESOC / AEDECOD
  ↓
Count subjects
  ↓
Calculate percentages
  ↓
Format table
  ↓
QC comparison
  ↓
Final output

Reproducibility Test

One of the strongest tests of a pipeline is: Can another programmer reproduce the output from the documented inputs?

A reproducibility test can involve:

1. Start from a clean environment.
2. Obtain the controlled source data.
3. Obtain the specifications.
4. Obtain the approved source code.
5. Install the controlled package environment.
6. Run the pipeline.
7. Compare the generated outputs.

If this process cannot be performed reliably, the workflow is not yet fully reproducible.

Reproducible Does Not Mean Unchangeable

A good pipeline should change when controlled inputs change.

For example:

The goal is not to make outputs immutable.

The goal is to make changes explainable and reproducible.

Handling Data Cutoffs

Clinical reporting often depends on a specific data cutoff date.

The cutoff should be a controlled input rather than a hidden date embedded inside individual programs.

cutoff_date <- as.Date(
  "2026-06-30"
)

analysis_data <- data |>
  dplyr::filter(
    ADT <= cutoff_date
  )

Ideally, the cutoff should come from the study configuration or controlled analysis metadata.

Study-Level Parameters

Other centralized parameters may include:

• Study ID
• Protocol number
• Data cutoff
• Treatment labels
• Treatment ordering
• Population definitions
• Decimal rules
• Date conventions
• Output directory
• Analysis version

This avoids duplicating study-specific information throughout the codebase.

Parameterizing Treatment Labels

Instead of:

labs(
  x = "Treatment"
)

the underlying treatment labels should be controlled by study metadata.

treatment_map <- data.frame(

  code = c(
    "P",
    "A",
    "B"
  ),

  label = c(
    "Placebo",
    "Drug A",
    "Drug B"
  ),

  order = c(
    1,
    2,
    3
  )

)

This becomes particularly useful when the same analysis framework is reused across studies.

Automating Subgroup Analyses

Subgroup analyses are another area where automation provides substantial benefits.

Suppose the same efficacy table must be generated for:

• Overall population
• Sex
• Age group
• Race
• Geographic region
• Baseline disease severity

Rather than creating separate programs, the subgroup variable can become a parameter.

make_subgroup_table <- function(
  data,
  subgroup
) {

  data |>
    dplyr::group_by(
      .data[[subgroup]],
      TRT01P
    ) |>
    dplyr::summarise(
      N = dplyr::n(),
      .groups = "drop"
    )

}

The same validated function can then generate multiple subgroup outputs.

Subgroup Explosion

Automation makes it easy to create hundreds of outputs.

That does not mean all of them should automatically be created.

The specification should determine:

• Which subgroups are required
• Which analyses apply to each subgroup
• Minimum sample-size rules
• Display conventions
• Whether an analysis is exploratory

Parameterizing Visit-Based Analyses

Longitudinal analyses often repeat the same summary structure at multiple visits.

Instead of writing:

week4_table()
week8_table()
week12_table()
week16_table()

parameterize the visit:

make_visit_summary(
  data,
  visit = "Week 4"
)

make_visit_summary(
  data,
  visit = "Week 8"
)

Or drive the process entirely from metadata.

Reusable Visit Ordering

Visits should be represented using controlled ordering.

visit_levels <- c(
  "Baseline",
  "Week 4",
  "Week 8",
  "Week 12",
  "Week 16",
  "Week 24"
)

data$AVISIT <- factor(
  data$AVISIT,
  levels = visit_levels
)

This prevents accidental alphabetical ordering.

Production Validation Checklist

• Input datasets are present and version-controlled.
• Required variables are present.
• Population flags contain expected values.
• Treatment assignments are valid.
• Subject identifiers satisfy dataset-specific uniqueness rules.
• Specifications contain all required metadata.
• Every specification maps to an approved program.
• Every expected TLF is generated.
• No unexpected outputs are generated.
• Numerical QC checks pass.
• Output files are nonempty.
• Output titles and footnotes match specifications.
• Warnings are reviewed.
• Errors are resolved.
• Visual QC is completed.
• Production logs are archived.
• Output manifest is complete.

Example End-to-End Pipeline

Consider a study with:

• ADSL
• ADAE
• ADLB
• ADTTE

and five required outputs:

T14.1.1 Demographics
T14.2.1 Treatment-Emergent Adverse Events
T14.3.1 Laboratory Summary
L16.1.1 Adverse Event Listing
F14.1.1 Overall Survival

The production process could be:

A More Complete Driver

main <- function() {

  log_message(
    "Production run started"
  )

  config <- load_config()

  validate_environment(
    config
  )

  data <- load_adam(
    config$data_path
  )

  validate_datasets(
    data
  )

  specs <- load_tlf_specs(
    config$spec_path
  )

  validate_all_specs(
    specs
  )

  results <- lapply(
    seq_len(nrow(specs)),
    function(i) {

      run_one_tlf(
        spec = specs[i, ],
        data = data,
        config = config
      )

    }
  )

  validate_all_outputs(
    results
  )

  write_manifest(
    results
  )

  log_message(
    "Production run completed"
  )

}

main()

Design Principles for Production Code

What Not to Automate First

A common mistake is attempting to automate the entire study immediately.

A better progression is:

1. Standardize one TLF.
2. Convert repeated calculations into reusable functions.
3. Introduce structured specifications.
4. Add validation.
5. Add logging.
6. Add automated QC.
7. Scale to multiple TLFs.
8. Introduce dependency management.
9. Consider parallelization.

This allows the architecture to mature incrementally.

Start With High-Volume TLF Families

The largest return on investment often comes from output families containing many similar tables.

Examples include:

• Adverse-event tables
• Laboratory tables
• Vital-sign tables
• Concomitant-medication tables
• Subgroup tables
• Repeated efficacy summaries

If one reusable framework can replace 20 nearly identical programs, the benefit is substantial.

Clinical Programming and R Packages

As the framework matures, reusable functionality can be organized into an R package.

For example:

clinicalTLF/
├── DESCRIPTION
├── NAMESPACE
├── R/
│   ├── demographics.R
│   ├── adverse_events.R
│   ├── formatting.R
│   ├── validation.R
│   └── output.R
├── tests/
│   └── testthat/
└── man/

An internal package can provide a controlled library of validated clinical programming functions.

Why an Internal Package Can Be Powerful

Instead of every study independently implementing:

fmt_pct()
fmt_pvalue()
make_ci()
make_ae_table()
make_listing()
clinical_theme()

the organization can maintain approved implementations.

This can improve consistency across studies while reducing duplicated effort.

Package Versioning

When reusable code is placed in a package, package versions become important.

For example:

clinicalTLF 1.4.0
clinicalTLF 1.5.0

A study's production run should identify which version was used.

Change Control

A change to a central function can affect many TLFs.

Therefore, changes to reusable components should trigger appropriate testing and review.

Documentation

A production pipeline should document:

• Architecture
• Directory structure
• Configuration
• Specification format
• Function registry
• Validation rules
• Output conventions
• QC strategy
• Execution instructions
• Troubleshooting procedures

Documentation is part of the production system.

A Programmer's Runbook

A practical runbook might state:

1. Confirm approved data snapshot.
2. Confirm specification version.
3. Confirm R environment.
4. Run input validation.
5. Execute pipeline.
6. Review production log.
7. Review output manifest.
8. Review QC discrepancies.
9. Resolve warnings.
10. Perform visual review.
11. Archive production package.

Production Package Contents

A controlled production archive may contain:

• Final TLFs
• Analysis datasets or controlled references
• Programs
• Specifications
• Configuration
• Logs
• QC outputs
• Validation results
• Software-environment information
• Output manifest

The exact archival requirements depend on the organization's procedures and regulatory context.

Automation and Regulatory Work

Automation can be highly appropriate for regulated clinical-trial reporting, but automation does not remove the need for validation.

The central question becomes:

That requires documented requirements, controlled code, testing, traceability, and appropriate review.

Computerized System Validation Perspective

When R is used in a regulated environment, organizations should consider applicable procedures for:

• Requirements
• Risk assessment
• Testing
• Change control
• Access control
• Auditability
• Data integrity
• Release management

The exact validation strategy should be determined by the organization's quality system and applicable regulations.

Risk-Based Automation

Not every component requires the same level of scrutiny.

A reusable function that calculates a critical efficacy endpoint may warrant more extensive testing than a function that simply adds whitespace to a footnote.

A risk-based framework can consider:

• Clinical importance
• Complexity
• Likelihood of failure
• Downstream impact
• Degree of reuse
• Detectability of errors

Reusable Code Has Multiplicative Risk

Suppose a function is used by 100 tables.

A defect in that function can affect 100 outputs.

But the same reuse also provides a major benefit:

A single validated correction can improve all 100 outputs.

This is why reusable code requires both discipline and testing.

Common Mistakes in Automated TLF Pipelines

1. Automating before standardizing. If every TLF follows a different convention, automation simply reproduces inconsistency faster.
2. Hard-coding study-specific assumptions. This makes the framework difficult to reuse and maintain.
3. Combining analysis and formatting. This makes numerical validation harder.
4. Using the final rendered file as the only QC target. Structured intermediate results are much easier to validate.
5. Suppressing errors. Failed TLFs must be visible.
6. Ignoring empty groups. Empty categories and treatment arms require deliberate handling.
7. Allowing alphabetical ordering. Clinical specifications should control ordering.
8. Overusing dynamic titles. Controlled clinical language should remain reviewable.
9. Skipping independent QC. Automation does not eliminate the need for independent verification.
10. Building an enormous framework too early. Start with a small, well-tested set of reusable components.

A Practical Development Roadmap

Example Production Architecture

                    ┌────────────────────┐
                    │   Study Config     │
                    └─────────┬──────────┘
                              │
                              ▼
                    ┌────────────────────┐
                    │ TLF Specifications │
                    └─────────┬──────────┘
                              │
                              ▼
┌──────────────┐    ┌────────────────────┐
│ SDTM / ADaM  │───▶│ Input Validation   │
└──────────────┘    └─────────┬──────────┘
                              │
                              ▼
                    ┌────────────────────┐
                    │ Analysis Functions │
                    └─────────┬──────────┘
                              │
                              ▼
                    ┌────────────────────┐
                    │ Structured Results│
                    └─────────┬──────────┘
                              │
                              ▼
                    ┌────────────────────┐
                    │ Formatting Layer  │
                    └─────────┬──────────┘
                              │
                              ▼
                    ┌────────────────────┐
                    │ TLF Outputs       │
                    └─────────┬──────────┘
                              │
                    ┌─────────┴─────────┐
                    ▼                   ▼
             ┌──────────────┐    ┌──────────────┐
             │ Automated QC │    │ Visual QC    │
             └──────┬───────┘    └──────┬───────┘
                    └──────────┬────────┘
                               ▼
                       ┌──────────────┐
                       │ Release /    │
                       │ Archive      │
                       └──────────────┘

The Ideal TLF Pipeline

An ideal pipeline should allow a programmer to answer the following questions quickly:

• What generated this table?
• Which dataset was used?
• Which data cutoff was used?
• Which specification defined it?
• Which R function generated it?
• Which code version was used?
• What validation was performed?
• Was independent QC completed?
• What changed from the previous run?

If these questions require manually searching through dozens of scripts and folders, the pipeline has room for improvement.

The Three Levels of Automation

It is useful to distinguish three levels.

Most organizations obtain the largest long-term benefits from Level 3.

The Most Important Design Principle

The most important concept is not a particular R package.

It is the separation of:

The specification defines what should be produced.

The programming framework defines how that output is generated.

This separation allows a single validated implementation to generate many outputs while keeping study-specific requirements explicit.

Final Production Checklist

• Study configuration is controlled.
• Input datasets are identified and validated.
• Data cutoff is explicitly defined.
• TLF specifications are complete.
• Population definitions are controlled.
• Treatment ordering is controlled.
• Reusable analysis functions are unit tested.
• Formatting functions are standardized.
• Intermediate analysis results are retained where appropriate.
• Every requested TLF maps to an approved function.
• Errors stop or appropriately flag the pipeline.
• Warnings are captured and reviewed.
• Output files are checked for existence and integrity.
• Numerical QC is performed.
• Independent QC is performed where required.
• Visual review is completed.
• Run logs are generated.
• An output manifest is generated.
• Code and specifications are version-controlled.
• Software environment is documented.
• Final outputs are archived according to applicable procedures.

Summary

Automating TLF production in R is much more than writing a loop around a collection of scripts.

A robust clinical-programming pipeline integrates:

• Analysis-ready ADaM datasets
• Controlled TLF specifications
• Reusable statistical functions
• Standardized formatting
• Input validation
• Numerical validation
• Independent QC
• Automated testing
• Error handling
• Logging
• Output manifests
• Version control
• Reproducible environments
• Dependency management

The result is a system in which TLF production becomes a controlled, repeatable transformation rather than a collection of manually executed programs.

References and Further Reading

International Council for Harmonisation (ICH). E3: Structure and Content of Clinical Study Reports.

International Council for Harmonisation (ICH). E6(R3): Guideline for Good Clinical Practice.

CDISC. Analysis Data Model (ADaM) Implementation Guide.

R Core Team. R: A Language and Environment for Statistical Computing.

Wickham, H., Averick, M., Bryan, J., et al. (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686.

Wickham, H., Bryan, J., et al. R Packages: Organize, Test, Document, and Share Your Code.

RStudio / Posit. targets: Dynamic Function-Oriented Make-Like Declarative Pipelines.