When I woke up this morning I lay in bed, trying in vain to remember the details of a particular theorem I’d learnt in grad school. I tried Bing and Google, putting forward key phrases I thought might be relevant, but it was no help. So, I decided to ask the bot instead, using freeform text in a more conversational way.

Hi Bing, I’m trying to remember something I really ought to. I think it was from computational learning theory, but I’m not 100% on that. Anyway, the gist of the result is that it was established that it would be impossible for a single optimization procedure to exist that could solve all problems. Can you remember?

I think you are referring to No Free Lunch Theorems for Optimization, which state that there is no single optimization algorithm that can outperform any other algorithm on all possible problems. This means that different optimization algorithms may have different strengths and weaknesses depending on the problem at hand.

Computational learning theory is a sub-field of machine learning that studies the formal properties of learning algorithms and tasks. Optimization is a key component of many learning algorithms, as they aim to find the best parameters or hypotheses that fit the data.

Does this answer your question?

Ahhh, of course, no free lunch! Like I said, something I really ought to have remembered, but I woke up this morning and totally blanked on it. Thank you so much.

You’re very welcome. I’m glad I could help you remember. 😊