160 CHAPTER5 Credibility: Evaluating What's Been Learned the right measure to use. This is sometimes called a 0 ­ 1 loss function: The "loss" is either 0 if the prediction is correct or 1 if it is not. The use of loss is conventional, although a more optimistic terminology might couch the outcome in terms of profit instead. Other situations are softer-edged. Most learning schemes can associate a prob- ability with each prediction (as the Naïve Bayes scheme does). It might be more natural to take this probability into account when judging correctness. For example, a correct outcome predicted with a probability of 99% should perhaps weigh more heavily than one predicted with a probability of 51%, and, in a two-class situation, perhaps the latter is not all that much better than an incorrect outcome predicted with probability 51%. Whether it is appropriate to take prediction probabilities into account depends on the application. If the ultimate application really is just a predic- tion of the outcome, and no prizes are awarded for a realistic assessment of the likelihood of the prediction, it does not seem appropriate to use probabilities. If the prediction is subject to further processing, however--perhaps involving assessment by a person, or a cost analysis, or maybe even serving as input to a second-level learning process--then it may well be appropriate to take prediction probabilities into account.