If you’re looking for a house to buy, or you want to hire an employee, when should you stop searching?
The rational answer would be to consider all options available, and then, by going through all the data, to choose the option that is superior to all the other options. But if the decision must be made immediately, you cannot afford to do that.
Herbert Simon formulated the idea of bounded rationality; we can’t deal with the world as if we were divine beings — we need shortcuts and distortions under evolutionary pressures (even computers follow heuristics).
In The Principle of Least Effort, I discussed the idea of how intelligent systems will choose the path that requires the minimum amount of effort to achieve the minimally acceptable outcome.
A similar idea, where conservation of effort is an important component in making a decision, is the 37 percent rule (from Algorithms to Live By).
The rule seems to have originated from the secretary problem, which was introduced in 1949 by Merril M. Flood.
To understand the problem, imagine you are the owner of a business, and you need to hire a secretary.
Say you only have 100 days to do this, but you must decide whether to reject or accept each applicant (one-by-one) right after each interview. At what point do you abandon your search?
There are two mistakes you can make. One is to pick too early (you likely missed out on superior options that you did not give yourself the chance to discover), the other is to pick too late (you missed out on the best options already, but now you have to settle for an inferior choice).
To arrive at the optimal solution, you should figure out how much time you have, or how many applicants you have, and stop your search when 37% of that time (or options) has elapsed and commit to the next best choice.
Let’s assume that Charlie was the best applicant until day 36. The best strategy (the maximal probability that you will select the best candidate) would be to hire the next person better than Charlie.
Likewise, if your job offer got 100 applicants, and up to that point, Dana was the best applicant, then you should, starting at applicant 37, choose the next person who is better than Dana.
But there are some limitations. One, you need to know in advance how many days you have, and how many applicants there will be, or the maths will get complicated. Another condition is that the applicants should know that you are not using this rule, otherwise, you would get no early applicants since they know they aren’t going to get hired.
The shortest rigorous proof known so far is provided by the odds algorithm. It implies that the optimal win probability is always at least 1/e (where e is the base of the natural logarithm), and that the latter holds even in a much greater generality. — Source
Originally published at http://unearnedwisdom.com.