Why Configuration Structure Matters
A group sequential design is often represented in software by many inputs: the hypotheses and error-rate conventions, assumed effect and nuisance parameters, a boundary method, an analysis schedule, and software options. When several candidate designs are being considered, keeping these inputs in separate notes or editing them informally makes comparisons fragile. A value can be omitted, a unit can be misunderstood, or an output can become detached from the assumptions that produced it.
A machine-readable configuration is a structured record of inputs that software can parse consistently. Metadata describes the record and its provenance: what it represents, which schema and software versions were used, when it was created, and how it relates to other candidate records. The purpose is not to choose the design objectives or justify the statistical assumptions. It is to make each candidate’s specification explicit, inspectable, and traceable.
The introductory material on reproducible design workflows establishes why rerunning an analysis from recorded inputs matters. The next step is to organize those inputs so both people and software can determine exactly what each field means. A configuration is useful only if its structure is stable, its values are interpretable, and its limitations are visible.
Design a Schema Before Filling Candidate Records
A schema defines the permitted fields, their types, required or optional status, units, and allowable values. For a candidate group sequential design, a practical schema usually separates at least four categories:
- Identity and status: a stable candidate identifier, schema version, record status, and a short description.
- Statistical specification: hypotheses, endpoint scale, significance level, sidedness, assumed effect and nuisance parameters, boundary method, and any other software inputs needed to define the candidate.
- Analysis schedule: the planned look identifiers and schedule quantities, with units and definitions. This records the schedule; it does not by itself establish that the schedule is operationally feasible.
- Execution and provenance: software and package versions, configuration author or responsible role, creation or revision time, input references, and links to generated outputs or review records.
Keep distinct concepts in distinct fields. For example, write an endpoint scale and its units explicitly rather than recording an effect value with no indication whether it is a difference in means, a log hazard ratio, or another quantity. Likewise, distinguish a planned schedule from a schedule used in a later analysis. Use a field for the schedule convention, such as information fraction, and a separate field for the listed values. Avoid relying on a reader to infer meaning from a column name such as “time” or “look.”
Use controlled labels where possible. If the software accepts only particular boundary-method names, the schema should define the permitted labels and reject misspellings rather than silently interpreting them. Record numeric fields as numbers, Boolean options as true or false, and identifiers as strings. Units should be explicit even when a software input is dimensionless: for example, state that an information fraction is dimensionless and lies between zero and one.
A schema version is essential because the meaning or required fields may change. A candidate created under schema version 1 should not be silently treated as if it had been entered under version 2. Either maintain backward-compatible interpretation or apply a documented migration that preserves the original record and records the transformation. Schema versioning is not the same as software versioning: one describes the record’s structure; the other identifies the software that interpreted it.
A Candidate Record in Practice
The table shows a hypothetical record for one candidate. The values illustrate how fields can be made explicit; they do not recommend a particular design. In a machine-readable file, the same fields could be encoded as a mapping in a format such as YAML or JSON, with nested sections for statistical specification, schedule, and provenance.
| Field | Hypothetical value | Interpretation |
|---|---|---|
| candidate_id | GS-A | Stable identifier for this candidate |
| schema_version | 1.2 | Version of the field definitions used |
| design_family | Two-arm, two-sided group sequential | Concise design classification |
| endpoint_scale | Mean difference, mmHg | Scale and units for the modeled effect |
| alpha and sidedness | 0.050, two-sided | Recorded inferential convention |
| assumed_effect | 4.0 mmHg | Candidate-specific assumed effect |
| boundary_method | O’Brien–Fleming type | Method label as defined by the software |
| planned_information_fractions | 0.50, 0.75, 1.00 | Three planned looks, expressed as fractions of maximum information |
| software | GroupSeqTool 2.4.1 (hypothetical) | Software identity and version used for the record |
| record_status | Under review | Administrative state; not a statistical conclusion |
The record should also identify other inputs required by the chosen software, such as the assumed variance when relevant, allocation ratio, and the exact parameterization of the boundary method. If a field is not applicable, represent that explicitly according to the schema—for example, with a permitted null value and a reason—rather than leaving an ambiguous blank. Never let “not supplied,” “not applicable,” and “software default” collapse into the same representation.
Worked Example: Registering Two Candidate Designs
Suppose a planning team has two hypothetical candidates for a two-arm study with a continuous endpoint measured in mmHg. Candidate GS-A uses an assumed mean difference of 4.0 mmHg, and GS-B uses 3.0 mmHg. Both records are to use a two-sided significance level of 0.050 and three planned looks at information fractions 0.50, 0.75, and 1.00. The task is to represent the two candidates without blending their assumptions.
Step 1: Assign distinct identifiers. Use GS-A and GS-B as stable candidate identifiers. The records share common fields, but each has its own values for candidate-specific assumptions. Their effect fields are respectively 4.0 mmHg and 3.0 mmHg; the units are part of the field definition and must not be inferred from the number alone.
Step 2: Check the schedule representation. The three listed information fractions should be increasing and within the interval from zero to one. The adjacent differences are \(0.75-0.50=0.25\) and \(1.00-0.75=0.25\), both positive, and the final fraction is 1.00. Thus the entries are ordered and end at the stated maximum-information fraction. This check confirms only the consistency of these metadata values; it does not establish that the schedule is suitable for the trial.
Step 3: Record shared values once per candidate record. Each record should state the two-sided convention, alpha of 0.050, endpoint scale, schedule definition, software version, schema version, and status. Do not store only a statement that GS-B is “the same as GS-A except for effect.” That shorthand can become incorrect if one record is later revised, and a parser cannot reliably recover omitted values without an explicit inheritance rule.
The resulting registry has two independently interpretable candidate records. A reviewer can compare the assumed effects while also verifying that the remaining listed assumptions and software context are present in both.
Validation and a Short Reproducibility Justification
Validation should occur at more than one level. A structural check asks whether required fields exist and have the correct types. A semantic check asks whether values make sense under the schema: for instance, whether a significance level is between zero and one, a schedule is ordered, and its final information fraction is consistent with the schema’s stated convention. A cross-field check asks whether related entries agree—for example, whether a schedule lists the number of looks claimed by its metadata.
These checks do not prove that an assumption is scientifically appropriate or that the resulting design has desirable operating characteristics. They identify malformed or internally inconsistent records before those records are passed to statistical software. Statistical review remains necessary.
The practical implication is to record enough execution detail to interpret outputs: software name and version, relevant package versions, configuration schema version, and any material options not contained in the main configuration. A record should also reference the output produced from it, rather than treating the output filename as a complete audit trail. Where a software update is made, preserve the original version information and identify which run used which version.
Worked Example: Finding a Schedule Metadata Error
A hypothetical record claims to contain three planned looks, but lists information fractions 0.40, 0.70, and 0.65. Apply a basic increasing-schedule validation rule: for successive fractions \(t_1,t_2,t_3\), require \(0<t_1<t_2<t_3\leq1\).
Substituting the recorded values gives \(0<0.40<0.70\), which passes the first comparison, but the last comparison requires \(0.70<0.65\), which is false. Equivalently, the second successive difference is \(0.65-0.70=-0.05\), not positive. Therefore the record fails this validation rule.
Do not “fix” the data by sorting the values into 0.40, 0.65, 0.70. Sorting changes the association between each planned look and any other look-specific metadata, and it does not resolve whether the original values were entered incorrectly. Flag the record, identify the intended schedule from an authoritative source, correct it through a documented revision, and retain the prior version if auditability is required.
Provenance, Digests, and Evidence
Provenance records answer where a configuration came from and what happened to it: who or what created it, when it was revised, which source inputs it reflects, who reviewed it, and which analysis outputs refer to it. Separate authorship, review, and approval fields where those are different responsibilities. A timestamp or username alone is not proof that a record was authorized; the process that authenticates users and controls changes is outside the configuration file itself.
A cryptographic hash can be used as a compact fingerprint of a file’s bytes. For a meaningful comparison, the procedure must specify which bytes are hashed—for example, the exact saved file, or a documented canonical serialization. Formatting changes can alter a file’s bytes even if the parsed values are equivalent. Conversely, independently serializing semantically equivalent records can produce different byte sequences unless canonicalization rules are defined.
Worked Example: Interpreting a Digest Check
A hypothetical registry stores a digest for the canonical bytes of candidate GS-A. Later, an analyst computes a digest from a copy using the same documented canonicalization and hash procedure. The stored digest is “a81f…” and the computed digest is “b42c…”.
Comparison: The digest strings differ. Under the stated deterministic procedure, this shows that the canonical byte sequences differ. The analyst should investigate whether the copy has a changed value, a changed schema version, or an unintended serialization change. The comparison alone does not identify which field changed.
If instead both digest strings were “a81f…”, that would provide evidence that the canonical byte sequences agree, but it would not be a mathematical proof of byte-for-byte identity because a collision is possible. Nor would the match prove that GS-A’s assumed effect or error-rate convention is scientifically defensible, or that an authorized reviewer approved the record. Those questions require separate checks and governance evidence.
A sound provenance practice therefore pairs a digest with a clear definition of the hashed content, controlled access or approval records as appropriate, and a human-readable revision history. Never treat a hash as a substitute for authorization or statistical validation.
Format Choices, Diagnostics, and Limitations
YAML and JSON are common choices for structured configuration. JSON has a strict syntax and broad parser support; YAML can be more readable for hand-edited nested records but can introduce ambiguity through implicit types or parser differences. Either can work if the team defines a schema, validates files with a consistent parser, and avoids undocumented conventions. A spreadsheet can be convenient for review, but it is less robust as the sole source of truth when cell formats, formulas, hidden sheets, and manual edits can change interpretation. A database can support many records and controlled updates, but still needs a documented schema and export strategy.
Diagnostics for a configuration workflow include missing required fields, unknown field names, values outside allowed ranges, inconsistent units, duplicate candidate identifiers, malformed version labels, and output files that cannot be traced to an input record. A successful parse is not sufficient: a parser may accept a syntactically valid record whose assumptions are incomplete or whose fields are inconsistent. Review a rendered summary as well as the machine-readable file, and confirm that important inputs appear as intended.
A registry also has limits. It cannot determine whether the trial assumptions are clinically plausible, whether the selected boundary method is appropriate, or whether a planned schedule is feasible. It cannot guarantee reproducibility if important dependencies or defaults are omitted. Nor should it become an uncontrolled store of sensitive information; provenance should identify appropriate records and locations without copying confidential data unnecessarily.
- Common mistake: Treating a configuration as self-explanatory because it parses. Correction: validate its meaning, required fields, units, and cross-field relationships.
- Common mistake: Reusing a candidate identifier after changing its assumptions. Correction: preserve identity and revision history so each version can be traced unambiguously.
- Common mistake: Treating a matching hash as proof of identity or authorization. Correction: treat it as evidence of agreement under a specified procedure, with collisions possible, and use separate controls for approval.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Why should schema version and software version be separate fields?
- A schedule is recorded as 0.30, 0.60, and 1.00. Calculate the successive differences and state what this check does—and does not—establish.
- A candidate record omits the unit for its assumed effect. Why is this a metadata problem even if the numeric value parses correctly?
- Two files produce matching digests under the same documented procedure. What conclusion is justified, and what conclusions are not?
- Why is “same as candidate GS-A except for the effect” a risky way to record GS-B?
Solutions. (1) Schema version identifies the structure and meaning of fields; software version identifies the implementation that interpreted them. One can change without the other. (2) The differences are \(0.60-0.30=0.30\) and \(1.00-0.60=0.40\), both positive; the values are increasing and end at 1.00. This supports internal schedule consistency under that rule, but does not establish operational feasibility or design suitability. (3) Without a unit and defined effect scale, the number may represent different quantities, so a parser can accept a value that is not interpretable or safely comparable. (4) Matching digests provide evidence that the specified canonical byte sequences agree, but collisions are possible; the match does not prove scientific validity, authorization, or correct execution. (5) The shorthand depends on implicit inheritance and can become stale or ambiguous if GS-A changes; independently recording required fields makes GS-B interpretable and auditable on its own.