Reporting a Multisite Nutrition Study End to End

From Study Records to a Defensible Report

A multisite analysis is not complete when software prints a coefficient. Readers need to know what was measured, how groups were defined, what the model adjusted for, and whether the estimate is stable enough to interpret. They also need uncertainty and limitations stated in a way that does not claim more than the design supports.

Consider a hypothetical nutrition cohort in which adults at 24 clinics chose whether to participate in a nutrition-support program. The outcome is daily fruit-and-vegetable servings at 12 weeks. Because participation was not randomly assigned, an adjusted difference describes an association between program participation and the outcome; it does not, by itself, establish that the program caused the difference.

Define the Study Variables Before Fitting a Model

The unit of analysis is one adult. State the outcome and its units, the groups being compared, the timing of measurements, and the candidate adjustment variables. This makes the analysis understandable and helps reveal coding errors or variables that do not match the research question.

Definition: An adjusted group difference is the difference in a model’s expected outcome between two groups when the specified adjustment variables are held fixed. Its interpretation depends on the model, the variables included, and the study design.

Worked Example: Documenting the Analysis Variables

Suppose the hypothetical cohort includes 480 adults from 24 clinics, with 20 adults enrolled at each clinic. The analysis team plans to compare adults who participated in the program with adults who did not. Define the main variables before analyzing the records.

Outcome: \(Y\), fruit-and-vegetable servings per day at week 12, measured in servings/day. A larger value indicates more reported servings.

Group indicator: \(G=1\) for program participants and \(G=0\) for nonparticipants. This coding makes the model’s group coefficient the participant-minus-nonparticipant difference.

Baseline measure: \(B\), servings per day at the start of the study. Adjusting for baseline outcome can account for initial differences in diet and usually improves the clarity of a follow-up comparison.

Age: \(A\), measured in years. To give the age coefficient an interpretable scale, use age in decades relative to 50 years: \((A-50)/10\). A one-unit increase then corresponds to ten years.

Site: clinic identifier, with 24 possible values. Participants at the same clinic may share local practices or other conditions. The report should say how site is handled in the model and in the uncertainty calculation.

These definitions do not settle every analytic choice. For example, the team should decide in advance how it will handle missing week-12 measurements and whether other variables are justified as confounders. Including many variables simply because they are available can make an analysis harder to interpret and can introduce new problems.

Specify the Adjusted Comparison

A model for this example includes a common program-group difference, a linear adjustment for baseline servings, a linear adjustment for age, and fixed differences among clinics. Taking clinic 1 as the reference site, the model is:

Y_{ij}=\beta_0+\beta_1G_{ij}+\beta_2(B_{ij}-4)+\beta_3\frac{A_{ij}-50}{10} +\sum_{k=2}^{24}\delta_k I(S_{ij}=k)+\varepsilon_{ij}

Here \(i\) indexes adults, \(j\) indexes clinics, and \(I(S_{ij}=k)\) equals 1 when the adult is at clinic \(k\), and 0 otherwise. The centering values 4 servings/day and 50 years make the intercept more interpretable; centering does not change the fitted group contrast. The model assumes that the adjusted program difference is the same across clinics and that the relationships with baseline servings and age are linear over the observed ranges.

Model interpretation: In this model, \(\beta_1\) is the adjusted participant-minus-nonparticipant difference in expected week-12 servings per day for adults with the same baseline servings, age, and clinic. The coefficient is an adjusted association, not automatically a causal effect.

The site indicators account for average differences among clinics. For a site-adjusted group comparison to be estimable, there must be program participants and nonparticipants within sites; if group and site are completely confounded, their differences cannot be separated by this model. The model also assumes no program-by-site interaction. If the group difference varies substantially across clinics, a single common coefficient may conceal important variation.

For uncertainty, adults within a clinic may have correlated outcomes. A cluster-robust standard error allows residuals to be related within a clinic while relying on clinics being independent of one another. With only a modest number of clinics, this approximation deserves caution; the number of independent clinics, not just the number of adults, matters. Here, a small-sample \(t\) reference using \(df=24-1=23\) is used for the illustrative calculation.

Estimate and Report the Group Difference

For a two-sided test of no adjusted group difference, the hypotheses are \(H_0:\beta_1=0\) and \(H_a:\beta_1\ne0\). The test statistic is the estimated coefficient divided by its standard error. A confidence interval combines the coefficient and its standard error with a critical value from the stated reference distribution.

Worked Example: Calculating the Main Estimate and Interval

Suppose fitting the stated model to the hypothetical 480-person cohort gives \(\hat{\beta}_1=0.62\) servings/day and a clinic-clustered \(SE=0.21\) servings/day. Use \(df=23\), a two-sided test, and a 95% confidence level. The corresponding \(t\) critical value is approximately 2.069.

Test statistic. Substitute the estimate and standard error:

t=\frac{\hat{\beta}_1-0}{SE} =\frac{0.62}{0.21}=2.952

With \(df=23\), this gives a two-sided \(p\)-value of approximately 0.007. At \(\alpha=0.05\), the data provide evidence against a zero adjusted association under the model and its assumptions.

95% confidence interval. Calculate the margin of error and endpoints:

0.62\pm 2.069(0.21) =0.62\pm 0.4345 =(0.1855,\ 1.0545)

Rounded to two decimal places, the interval is 0.19 to 1.05 servings/day. In context, the estimated adjusted difference is 0.62 more servings/day among participants than nonparticipants. The interval describes values compatible with the data and model at the 95% confidence level; it does not mean there is a 95% probability that this fixed parameter lies in this particular interval.

Check Whether the Model Is Adequate

A coefficient and interval are informative only if the model is a reasonable summary of the observed data. Diagnostics do not prove that every assumption is true; they look for patterns that make the chosen model questionable. Review them before writing a polished conclusion, and report meaningful problems rather than presenting only the preferred estimate.

Residuals versus fitted values: Look for curvature, changing spread, or a small number of conspicuous points. A systematic curve can signal that the assumed linear mean relationship is inadequate.
Residual distribution: A normal quantile plot can reveal strong skewness or unusually heavy tails. Normality is not required for the coefficient to be calculated, but severe departures can affect small-sample inference.
Influence: Check whether a few observations have unusual predictor values or strongly affect the fitted result. Refit after investigating influential observations; do not remove them solely because their removal improves the result.
Site and group coverage: Confirm that both groups occur within clinics and that the estimate is not determined by a single clinic. Check whether group differences appear broadly similar across sites.

Worked Example: Interpreting Diagnostics and a Sensitivity Model

Suppose residual-versus-fitted and normal quantile plots for the hypothetical study show no strong curvature or funnel-shaped spread, with only mild departures in the tails. The largest Cook’s distance is 0.08. A review of that record finds no data-entry error, and a refit excluding it gives an estimated difference of 0.60 servings/day rather than 0.62. These checks do not establish that the model is correct, but they suggest that this one record is not driving the main estimate.

The team also fits a prespecified sensitivity model using change in servings/day, \(Y-B\), as the outcome, while retaining age and clinic adjustment. Suppose this specification estimates a group difference of 0.55 servings/day, with a clinic-clustered \(SE=0.24\) and \(df=23\). Its test statistic is:

t=\frac{0.55}{0.24}=2.292

The two-sided \(p\)-value is approximately 0.031. The corresponding 95% interval is:

0.55\pm2.069(0.24) =0.55\pm0.4966 =(0.0534,\ 1.0466)

Rounded, this is 0.05 to 1.05 servings/day. The alternative specification gives a positive estimate similar to the main estimate, though the estimates are not identical. This supports describing the finding as reasonably stable to this particular outcome choice, not as proven under every possible analysis.

The change-score model and the baseline-adjusted follow-up model are related but make different modeling choices. Reporting the sensitivity analysis means naming that choice and showing its estimate and uncertainty. It does not mean selecting whichever model produces a smaller \(p\)-value.

What the Final Report Should Say

A concise report should give readers enough information to assess the analysis without making them infer important choices. State the outcome and its units, the groups, the adjustment variables, how clinics entered the model, and how uncertainty accounted for clustering. Then report the estimate, interval, and test result in context.

For example: “Among adults in this hypothetical multisite cohort, program participation was associated with an adjusted 0.62 more fruit-and-vegetable servings per day at week 12 than nonparticipation (95% CI 0.19 to 1.05; clinic-clustered \(SE=0.21\); \(t=2.95\), \(df=23\), \(p\approx0.007\)). The model adjusted for baseline servings, age, and clinic. A change-score sensitivity model estimated 0.55 more servings per day (95% CI 0.05 to 1.05). Because participation was self-selected, these adjusted associations should not be interpreted as causal effects.”

The report should also describe missing data and any limitations that could change interpretation. If participants with missing week-12 measurements differ from those observed, an analysis restricted to complete records may be biased. Adjustment cannot guarantee that all relevant confounding has been removed. Clinic-clustered inference also depends on having enough independent clinics for its approximation to be credible; with 24 clinics, uncertainty about that approximation should not be hidden.

Common Mistake: Treating Adjustment as a Causal Guarantee

A common mistake is to reason, “The regression adjusted for baseline servings, age, and clinic, so the program caused the observed difference.” Adjustment only compares groups conditional on the included variables under the specified model. In this self-selected cohort, unmeasured factors may still differ between groups, and the model may be misspecified. The appropriate correction is to describe the result as an adjusted association, explain the adjustment set, and state the design limitations.

Key takeaway: A useful multisite regression report connects the measurement choices, model, diagnostics, sensitivity analysis, and uncertainty. State what the adjusted contrast means, account for the site structure, and keep the conclusion within what the study design supports.

Check Your Understanding

Answer these questions using the hypothetical nutrition cohort.

Why is week-12 fruit-and-vegetable intake a more precise outcome definition than simply “diet quality”?
In the stated model, what does \(\beta_1\) compare, and what units does it have?
Why can clinic-clustered uncertainty differ from uncertainty calculated as if all adults were independent?
What does the 95% confidence interval from 0.19 to 1.05 servings/day say, and what does it not say?
Why does adjusting for baseline servings, age, and clinic not prove that program participation caused the estimated difference?
What information does the change-score sensitivity analysis add to the report?