Marko chains are models where the system can be in any one of distinct states and we have known, fixed probabilities of transiting between various pairs of states.