Define period for a state of a discrete time Markov chain.
Define the no-memory property for a random variable
Define the transition probability function \(P_{ij}(t)\) for continuous time Markov chains.
Give a sufficient condition for the existence of the limiting probabilities of a discrete time Markov chain.
Set up the detailed balance equations for a continuous time Markov chain.
The period of a state \(i\) is \(d_i = \gcd\{n\mid p_{ii}^{(n)} > 0\}\).
A random variable \(X\) is memoryless if \(P(X > t + s\mid X > s) = P(X > t)\).
\(P_{ij}(t) = P(X(t) = j\mid X(0) = i)\)
There exists a solution to \(\pi P = \pi\), \(\sum_{k} \pi_k = 1\).
\(q_{ij}P_i = q_{ji}P_j\) for all distinct states \(i, j\).
Capa plays either one or two chess games every day, with the number of games that she plays on successive days being a Markov chain with transition probabilities
\[P_{1,1} = 0.2,\quad P_{1,2} = 0.8,\quad P_{2,1} = 0.4,\quad P_{2,2} = 0.6.\]
Capa wins each game with probability \(p\). Suppose she plays two games on Monday.
What is the probability that she wins exactly one game on Tuesday?
What is the expected number of games that she plays on Wednesday?
In the long run, on what proportion of days does Capa win all her games?
\[p_{2,1}P(\text{1 win in 1 game}) + p_{2,2}P(\text{1 win in 2 games}) = 0.4p + 1.2p(1-p)\]
\[\begin{align} E[X_2 \mid X_0 = 2] &= p_{2,1}E[X_2 \mid X_1 = 1] + p_{2,2}E[X_2 \mid X_1 = 2] \\ &= p_{2,1}(p_{1,1} + p_{1,2}\cdot 2) + p_{2,2}(p_{2,1} + p_{2,2}\cdot 2) \\ &= 0.4(0.2 + 1.6) + 0.6(0.4 + 1.2) \\ &= 1.68 \end{align}\]
\[\begin{align} 0.8\pi_1 &= 0.4\pi_2 = 0.4(1 - \pi_1) \\ 1.2\pi_1 &= 0.4 \\ \pi_1 &= \frac{1}{3} \end{align}\]
In the long run, the proportion of days Capa wins all her games is
\[\frac{p}{3} + \frac{2p^2}{3}.\]Consider the random walk on the integers \(\{0,1,2,3\}\) which takes steps +1 (to the right) with probability \(\frac{1}{4}\) and \(−1\) (to the left) with probability \(\frac{3}{4}\), except at the endpoints where there is reflection; this means that a step from 1 to 0 is always followed by a step from 0 to 1, and a step from 2 to 3 is always followed by a step from 3 to 2.
Determine the probability transition matrix for this Markov chain.
What fraction of time does it spend in state 0?
\[P = \begin{bmatrix} 0 & 1 & 0 & 0 \\ \frac{3}{4} & 0 & \frac{1}{4} & 0 \\ 0 & \frac{3}{4} & 0 & \frac{1}{4}\\ 0 & 0 & 1 & 0 \end{bmatrix}\]
\[\pi_0 = \frac{3}{4}\pi_1,\quad \frac{1}{4}\pi_1 = \frac{3}{4}\pi_2,\quad \frac{1}{4}\pi_2 = \pi_3\]
\[\begin{align} \sum_{i} \pi_i &= \pi_0 + \frac{4}{3}\pi_0 + \frac{4}{9}\pi_0 + \frac{1}{9}\pi_0 \\ &= \frac{26}{9}\pi_0 \end{align}\]
Hence, \(\pi_0 = \frac{9}{26}\) is the fraction of time spent in state 0.Customers arrive at a shop according to a Poisson process of with rate 2 per a minute. Assume that a customer is woman with probability \(\frac{3}{4}\) and man with probability \(\frac{1}{4}\) independently of the other customers.
What is the probability that at least two men arrive in a 6 minute period?
What is the probability that exactly two men and four women arrive in a 4 minute period?
From 9:20-10:20am there were 50 customers in total. What is the probability that none of them arrived between 9:30-9:34am?
\[P(\mbox{Poisson}(\tfrac{1}{2}) \geq 2) = 1 - P(\mbox{Poisson}(\tfrac{1}{2}) \leq 1) = 1 - \frac{3}{2}e^{-1/2}\]
\[P(W(4) = 4)P(M(4) = 2)\]
TODO
TODO