Continuous time Markov chains; uniformization, transient and limiting behavior; Brownian motion and martingales; optional sampling and convergence; Discrete time Markov chains, Poisson and renewal processes; transient and limiting behavior; average cost and utility measures of systems.