Markov chain chance one state is reached before another

Hey could someone explain why this is true? I am trying to understand how to solve such a problem but I dont understand the solution.



Given a Markov chain [itex]\left\{X_{n}: n\in\ \mathbb{N}\right\}[/itex] with four states 1,2,3,4 and transition matrix

[itex]P =

\begin{pmatrix}

0 & \frac{1}{2}& \frac{1}{2} & 0 \\

0 & 0 & \frac{1}{2} & \frac{1}{2}\\

\frac{1}{2} & 0 & 0 & \frac{1}{2} \\

\frac{1}{2} & \frac{1}{2} & 0 & 0

\end{pmatrix}[/itex]

We leave from state 3. What is the chance that state 1 will be reached before state 4?



Solution:

Lets call this chance x. Lets call y the same chance but instead leaving from 2. Lets call:

[itex]f =

\begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}[/itex]

Then f satisfies P'f = f with P' the transition matrix derived from P by making state 1 and 4 absorbing states i.e.,



[itex]

\begin{pmatrix}

1 & 0& 0 & 0 \\

0 & 0 & \frac{1}{2} & \frac{1}{2}\\

\frac{1}{2} & 0 & 0 & \frac{1}{2} \\

0 & 0 & 1 & 1\end{pmatrix}*\begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}= \begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}[/itex]

Solving gives y = 1/2, x = 1/4.

via Physics Forums RSS Feed http://www.physicsforums.com/showthread.php?t=703415&goto=newpost