Markov-chains are models where the system can be in any one of distinct states and we have known, fixed probabilities (based on state) of transiting between various pairs of states.