Antimalware Evaluation and Testing · Chapter 10 443 thing when they happen to you, but they're only a (shrinking) percentage of the total malware problem. So your choice of product is affected by a whole range of subsidiary detection issues; that is, how effective is it with non-viruses. How do we test AV software for effective virus detection? Top players in AV testing like Virus Bulletin, ICSA Labs, West Coast Labs, and the AV Test Centre (see the final section of this chapter for details of these testing organizations) use meticulous testing procedures and large, carefully maintained collections of malicious software (malware). Spurious samples are carefully weeded out, and supplementary techniques such as large, carefully vetted false posi- tive (FP) test sets are used.Testing procedures are scrupulously planned, documented, and followed. However, such tests take time and are technically demanding.They are also difficult to implement without the cooperation of the AV industry, where samples are normally only shared between trusted individuals.The expense entailed means that detailed results may not be readily available to non-subscribers. Sometimes the service is funded by vendors, to the disadvantage of small vendors and community projects. Many recommendations reported in magazines and even in security circles are informal, based on the apparently trouble-free performance of a live installation. Variables such as configuration and the quality of any test set used have to be taken on trust. As a result, products that display low detection rates of commonly seen viruses can be per- ceived as more effective than products that stand up much better to formal testing. And if viruses are only part of the problem, how do we test for all those Trojans, backdoors,