Part I: Protection against 0-day malware attacks, inclusive of web and e-mail threats (Real-World Testing)
The most important category where the protective effect of products is concerned is the test against current online threats. This involves accessing known malicious websites or e-mails in order to test whether the protection product is able to ward off attacks.
Part II: Detection of widespread and prevalent malware discovered in the last 4 weeks (the AV-TEST reference set)
In order to increase the statistical relevance of the tests, further analyses are carried out with regard to a large number of current threats. This involves decreasing the complexity of the test and in turn increasing the number of test cases many times over. This test refers to the static detection of files, which includes detection with signatures, heuristics and in-the-cloud queries. AV-TEST uses two different test sets to carry out these analyses:
- All malicious files that were discovered by AV-TEST in the last 4 weeks prior to the beginning of the test, usually around 10,000 to 15,000 files.
- Extremely widespread malicious files that were discovered by AV-TEST in the last 4 weeks prior to the beginning of the test (detection of widespread malware): around 2,000 to 2,500 files.
Both test sets only use files that have been discovered and analysed by AV-TEST. In order to prevent the test sets from being influenced by the manufacturers in their favour, data from manufacturers are not incorporated into the tests. As a result, the independent analysis carried out by AV-TEST achieves a very high level of quality.
Typical actions are used to measure the influence on the system speed, for example:
- Launching popular websites
- Download of frequently-used applications
- Launch of standard software applications
- Installation of frequently-used applications
- Copying of files (locally and in a network)
Functions of the operating system, the protection program and other programs that may be disruptive are closed down prior to the beginning of the test. This includes both automatic updates and planned actions such as scans or backups. The tests are carried out on a limited number of computers that are identical in construction and have been verified as having the same system speeds in order to give all products the same chances.
Every individual action is carried out at least seven times so that a reliable average can be generated. If the standard deviation of the individual values exceeds a specific threshold, this indicates an error and the test is repeated in full. If the test achieves reliable values for all products and all test cases, these are compared with the reference system values and the difference is calculated. This difference then specifies the slowing down of the system in the case of the actions tested.
Top 5 Endpoint Security Vendors
- Intel (previously McAfee)
From 01/01/2016: Dennis Technology Labs is no longer operating.
Kaspersky Lab is the Most tested and Most awarded.
In 2015, Kaspersky Lab participated in 94 independent tests and reviews. They were awarded 60 first places and achieved 77 top three finishes. The TOP 3 metric represents the aggregate scores achieved by almost 100 vendors in some of the security industry’s most vigorous independent tests and reviews. It’s much more meaningful than a ‘one hit wonder’ performance with a single product on a single test.
Independent Endpoint Security Analysis
(Android Malware Detection Test)