Topic: Markov Chains: Definitions and Transition Matrices

Why Transition Matrices Are Row-Stochastic

Weather Models with Exact Transition Probabilities

Urn Counts Under Sampling with Replacement

The Markov Property as Conditional Independence

State Ordering and Row-Vector Conventions

State Augmentation for a Non-Markov Observation

Simple Random Walks as Markov Chains

Random Walks with Holding Probabilities

Random-Map Constructions of Markov Chains

Path Probabilities from Initial Laws and One-Step Transitions