Once again the AMTSO, a large group of security professionals, testers and journalists came together to work on further documents trying to help improving the quality of anti-malware reviews. This time the meeting took place in Budapest, Hungary and the host was done by Virusbuster, a Hungarian anti-malware company.

In the two-day meeting we finalized new documents, among them

• Suggested methods for the validation of samples.

A very well-known problem of recent anti-malware tests is the use of damaged or non working samples in test sets. This means that products are tested against files that are not able to run and therefore are no real threat to users. By the fact that the amount of malware samples increases from day to day, it becomes more difficult for testers to ensure that the samples they use for their tests are really working and have a malicious behaviour. The document explains different methods how samples can be validated and so hopefully helps to reduce the amount of less meaningful tests in the future.

• Best Practices for testing In-the-Cloud security products

Testing products that use “in-the-cloud” technologies present new difficulties to testers, since those technologies make use of online databases. Since those databases can change within minutes or even seconds, the repeatability and reproduction – an important criterion for any tests – can be hard or even impossible. The document tries to show the difficulties and comes up with advices how to avoid errors in those product tests.

Furthermore, the members agreed on a process of how AMTSO can review an existing test of anti-malware products and started working on new documents.

Philipp Wolf
Viruslab

The latest AMTSO meeting took place last week in Cupertino, CA, USA. It was hosted by Symantec and therefore the members met in the buildings of Symantec headquarter.

Again, many representatives from major security companies attended the meeting, as well as testing organisations like AV-Test, ICSA, NSS, AV-Comparatives and the PC Magazine.

After successfully publishing important documents following the last meeting in Oxford, the group was now working on new documents, which include among others

• AMTSO Whole Product Testing
• AMTSO Review of Reviews
• Educational documents on obtaining and verifying samples

The new documents focus on testing new methodologies of AV-Products and also on how testers may obtain working samples for their tests, which hopefully helps to increase the quality of upcoming anti-malware tests.

For more information about AMTSO, please also have a look at the official website at www.amtso.org.

Philipp Wolf
Viruslab

In January 2008 representatives of Avira and more than 40 other security software technologists, testers, academics and reviewers came together to found the Anti-Malware Testing Standards Organization (AMTSO) in Bilbao, Spain.

The motivation for the foundation of AMTSO was helping improve the quality and relevance of anti-malware testing. Since then, the members of AMTSO have had several meetings and a lot of things to discuss. The latest meeting took place in Oxford, UK on October 30/31 2008.

Picture from the AMTSO meeting in Oxford end of October.

Members agreed on guidelines and set recognized standards for testing security software for the first time. Therefore there are now two documents available for the general public, “Fundamental Principles of Testing”, which reflects the basic principles for testing anti-malware products and “Best Practice for Dynamic Testing”, which defines a standard for the dynamic testing of anti-malware products, a testing methodology that becomes more important every day.

The complete documents can be found at the homepage of AMTSO, http://www.amtso.org/.

Philipp Wolf
Viruslab