6 Continuous Time Markov Chains 6.1 Pure Birth Processes In this chapter, we present several important examples of continuous time, discrete state, and Markov processes. Specifically, we deal here with a family of random vari- ables {X(t); 0 t < } where the possible values of X(t) are the nonnegative integers. We shall restrict attention to the case where {X(t)} is a Markov process with stationary transition probabilities. Thus, the transition probability function for t > 0, P ij (t) = Pr{X(t + u) = j|X(u) = i}, i, j = 0, 1, 2, . . . , is independent of u 0. It is usually more natural in investigating particular stochastic models based on physical phenomena to prescribe the so-called infinitesimal probabilities relating to the process and then derive from them an explicit expression for the transition proba- bility function. For the case at hand, we will postulate the form of P ij (h) for h small, and, using the Markov property, we will derive a system of differential equations sat- isfied by P ij (t) for all t > 0. The solution of these equations under suitable boundary