Markove chain Model

Answer