Outstanding Issues:

Of course, there is no such thing as a free lunch. In essence, we have turned our original single objective problem (find a minimizer to the approximation a to predict a minimizer for the objective f) into a biobjective problem (find a point that both improves on the value of the approximation a and improves the quality of the approximation of the regions in which we have not yet sampled to predict a minimizer for the objective f). Once we do so, we raise the usual issues one finds in multiobjective optimization: to wit, how to balance the two objectives in a sensible fashion. For now, we introduce the parameter p_c and leave it to the user's control--which would be considered a feature by some users and a burden by others.

There is also the issue of how to do the optimization on either the approximation a or the merit function m to predict a candidate minimizer for the objective f. Obviously, the approximation is constructed so that it is inexpensive to compute; otherwise, we would work directly with the objective. If the objective is particularly expensive to compute, then we can afford to spend some time solving the "easier" optimization problem posed by the approximation. Nonetheless, if we are interested in a global minimizer of the approximation, finding such a minimizer becomes more problematic for all but the simplest problems in higher dimensions. When we move to the family of merit functions we have proposed, the problem is necessarily compounded. As the example should make clear, we necessarily introduce multiple local minimizers. Fortunately, the special structure of the family of merit functions suggests specialized optimization techniques for identifying potential minimizers--but this is a subject for ongoing research.