Markove chain Model

Solution