Automatic Evaluation and Selection of Problem-Solving Methods: Theory and Experiments

The choice of the right problem-solving method, from available methods, is a crucial skill for experts in many areas. We present a technique for automatic selection among methods based on analysis of their past performances. We formalize the statistical problem involved in choosing an efficient method, derive a solution to this problem, and describe a selection algorithm. The algorithm not only chooses among available methods, but also decides when to abandon the chosen method if it takes too much time. We then extend the basic statistical technique to account for problem sizes and similarity among problems.