Topic: Train-Test Splits and Cross-Validation

Why Training Error Understates Prediction Error

What Generalization Performance a Split Estimates

Uncertainty in Cross-Validation Estimates

Variable Selection Within Cross-Validation

Training, Validation, and Test Roles

Tuning Models Without Reusing Validation Data

The K-Fold Cross-Validation Algorithm

Test-Set Reuse and Adaptive Overfitting

Repeated K-Fold Cross-Validation

Repeated Random Holdout Validation