Command Line Interface aka CLI

Markov chain is a very important math model. Many practical problems can be modeled as Markov chain, and then be solved. I would like to talk about the transition matrix, specifically how to deduce the transition matrix after step n.  Let us use a simple but non-trivial example. Let say we have only two states: State 1 and State 2. State 1 has 0.6 probability to stay at itself, and 0.4 probability to go to State 2. State 2 has 0.8 probability to stay at itself and 0.2 probability to go to State 1. We can easily to write down the transition matrix below: $$q = \begin{pmatrix} 0.6 & 0.4 \\ 0.2 & 0.8 \\ \end{pmatrix} $$ Please notice, in different literature, people tend to use different way of representing transition matrix. In our example, each row represents the corresponding probability for each state. Let us assume we start at $T_0$, which has PMF of $(1,0)$. It simply means at $T_0$, it is 100% at State 1. What happen after 1 step at $T_1$, what will the PMF cha...

By definition, memorylessness means the distribution of waiting time (in continuous case) or number of trials (in discrete case) does not depend on how much time has passed or how many trials you have tried. There are only two distributions are memoryless: exponential distribution in continuous case, and geometric distribution in discrete case. Actually geometric is analogous to exponential. Mathematically, memorylessness can be expressed as below where m and n are non-negative integers in discrete case, or non-negative real numbers in continuous case: $$P(X \ge m+n | X \geq m) = P(X \ge n)$$ Using conditional probability, we can write left hand side to $$ P(X \ge m+n | X \geq m) =  \frac {P( X \ge m+n, X \geq m)}{P(X \geq m)}$$ Since m and n are both non-negative, we can simplify the numerator as $P( X \ge m+n, X \geq m) = P(X \ge m+n)$, so above equation becomes $$P(X \ge m+n | X \geq m) =  \frac {P( X \ge m+n)}{P(X \geq m)} $$ It is not hard to prove why exponential ...