Markov Model

In probability theory, Markov model refers to the assumption that a future states of a process depend solely on the present state, never on the events that occurred before.

Resources

Markov model

2022-01-29

The Gittins Index
- Here, the rules are simple. Using the Multi-armed bandit slot machine problem, with a record of 0 win and 0 losses we start at column 0 and row 0 and each win we go to the right of the table and each losses we go down. Gittins index follows a Markov model, that is the future state depends only on the current state, and never on the prior events. The table assumes that the next arm pull is worth 90% of the payoff now (much like a discount), and notice that the Gittins index at the start point is already \(.7029\) or a 70% probability of payout. In other words, Gittins index gives the benefit of the doubt to a fresh start with no payout knowledge of the slot machine.
Zettel Posts

🧠 Index 🧠#