Topic: Markov Chains: Definitions and Transition Matrices

One-Step Memory Versus Dependence Across Times

One-Step Prediction with a Transition Matrix

Inverse-Transform Sampling for Categorical Transitions

Initial Distributions and Fixed Starting States

Independent Sequences as Markov Chains

Homogeneous and Inhomogeneous Transition Laws

Finite-State Markov Chains: States, Times, and Sample Paths

Deterministic State Updates as Markov Chains

Designing a State That Makes the Process Markov

Constructing Finite Chains from Independent Uniforms