HEURISTIC TESTS – RIGHT OR WRONG?
So, is it wrong to test a scanner’s ability to detect
heuristically? Of course not, if it’s done competently. Was
this a competent test? Well, we don’t really know. Only the
barest bones of their methodology has been published.
Since these people are working outside the AV research
community – which is far more collaborative than anyone
outside it will ever believe – we really don’t know whether
they know any more about this specialist area than the
average end user.
Back in the days when I was less easily depressed, I tracked
some of the ‘tests’ that were circulating at that time. Testers
were using collections of alleged viruses found on ‘vx’
websites. These were known to contain large numbers of
garbage files such as random text files, snippets of source
code, intendeds (viruses that couldn’t actually replicate, and
therefore weren’t viruses), corrupted viruses that couldn’t
work, programs generated by virus generators which may or
may not have been viable viruses, the infamous Rosenthal
utilities, and (my particular favourite) ‘virus-like’ programs
(I’ve often wondered what that meant). Even then, testers
were trying to test a scanner’s heuristic ability by generating
‘variants’. Inserting snippets of virus code at random places
in a test file. Patching presumed infected files in random
places. Changing text strings found in virus bodies on the
assumption that that was what scanners were looking for. |