Explore-Exploit Trade-off

Everyday, we are faced with multitude of decision making whether to stay with the tradition or venture novelty. Do we buy our favorite flavor of snack that we know for sure we’ll enjoy, or do we try something new, knowing it might be our best yet flavor or the worst one. We were always reminded to “always try something new” or to “try new things” but we were never told by how much. Computer scientist have been trying to find the perfect balance between the two for more than fifty years, they even have a name for it: the explore/exploit tradeoff.

pg.31¹

Algorithms to Live By by Brian Christian and Tom Griffiths - Explore/Exploit
↩︎

2021-12-12

The Gittins Index
- In drug producing business, both the for-profit companies and the medical profession they serve are faced with the dilemma of the explore/exploit trade-off. Companies want to make as much profit as possible but also want to reinvest profits into R&D for new drug breakthrough. Similarly, doctors are bound by their oath to “do no harm” and provide the best treatment to their patients but also favors experimental studies that may save countless number of lives in the future. However, the present holds more weight into the equation and “a cured patient today is taken to be more valuable than one cured a week or a year from now, and certainly the same holds true of profits.”¹ In parlance of the economists, valuing the present more highly than the future is called “discounting.”
  1. Algorithms to Live By by Brian Christian and Tom Griffiths - Explore/Exploit
    ↩︎
Multi-armed Bandit Problem
- In Computer Science, one of the most popular problem in the realm of Explore-exploit trade-off is the Multi-armed bandit problem. The term derived from the casino slot machine “one-armed bandit”, and as the name suggests, the problem entails multiple slot machines and how to win as much as possible.¹
  1. Algorithms to Live By by Brian Christian and Tom Griffiths - Explore/Exploit
    ↩︎
🧠 Index 🧠
Book Notes

🚧 WIP 🚧#