AV Tester Workshop总结：WildList已跟不上时代

solcroft

"The hot topic was the impending demise of the WildList. As Andrew Lee pointed out, anti-virus testing exists primarily for marketing. Myles Jordan of Microsoft stated that the reason the industry has hung on to the WildList for so long, and will fight to continue doing so, is because WildList testing is easy to pass."
当天的热门话题是关于WildList的意义。Andrew Lee指出，反病毒软件评测的主要目的是为了让厂商推销产品。微软的Myles Jordan表示，反病毒厂商对WildList的维护，主要是因为要通过采用WildList样本的评测是十分容易的事。

"Agreement was virtually unanimous that the WildList is no longer useful as a metric of the ability of a product to protect users." - Randy Abrams, ESET
（专家都）异口同声觉得如果要评估反病毒软对用户的保护能力，WildList如今已经不管用。

全文见：http://www.wilderssecurity.com/showthread.php?t=176667

绅博周幸

EQ2 [:01:] [:01:]

The EQs

Viruses which are ITW are not included in the WildList. Viruses, which are included in the WildList are not ITW. The so-called "WildList reporters" don't really bother to monitor what is actually ITW and just keep "confirming" that the same things they have reported before are still ITW. They often keep sending one and the same sample over and over.

There are so many things wrong with the WildList that I can't hope listing them all here. Just refer to my paper on this subject. I wrote it 8 years ago but most of the problems discussed there have never been fixed - mostly due to the incompetence of the people behind the WildList (and sometimes because of their ego that does not allow them to admit that I am right and they are wrong).

Basically, the viruses that are actually ITW and the WildList have very little in common. Nevertheless, "everybody" loves it. The WildList people love it because it gives them a sense of self-importance. The testers love it because it's easier to test AV products against a small test set somebody else provides you for free than against a huge virus collection that you build and maintain yourself. The AV producers love it because it's easier to score high detection rates against 200+ viruses than against 300,000+. Of course, the only losers are the users, who are lulled into a false sense of security.

There was a valid question (I think in the VB article Mike posted) - if the WildList virus set is so easy to detect, why so many products are failing the "VB 100%" detection tests. The answer is simple - because passing these tests does not mean only detecting the viruses on the WildList. It also means no false positives, reasonably high (>90%) detection of the "zoo virus set", equal detection rate of the on-access and the on-demand scanners (sometimes there is a difference due to a bug, or an OS quirk, or a configuration issue), sometimes there is new stuff surprisingly added to the WildList and used by the testers before the AV producers can adapt and so on.

solcroft

大概翻译：

真正普遍病毒未必都列在WildList里，同样的，WildList的样本也未必是真正普遍病毒。这些所谓的WildList报告者都没有真正关注哪些病毒是在真正传播的，往往都一直只把以前上报过了的病毒继续再上报。

WildList的缺陷太多了，我在这里列不完。我八年前写过一篇讨论WildList的问题的论文，但如今很多的问题都还没有纠正，这大部分是因为WildList管理层的无能，有时候也是因为他们的傲慢导致他们不肯承认问题。

虽然WildList样本和真正普遍病毒往往都扯不上关系，但它还是受到大家的拥护。WildList人员拥护它，因为它给他们一种超越感。反病毒评测机构拥护它，因为它们可以直接从WildList得到免费的病毒库，而不必花时间和精力来收集和测试自己的样本。反病毒厂商拥护它，因为要通过一个只有200多个样本的评测，获取好成绩实在太方便了。当然，吃亏的是用户，他们都被蒙在鼓里。

Mike（即是在WSF的楼主）发布的文章里有人提出了个很好的问题，就是如果VB100%评测这么容易过关，那为什么还有这么多厂商不合格了？答案很简单，是因为要通过VB100%测试不只需要查杀所有WildList的样本，也还需要零误报，查杀一些包含非WildList样本的病毒库，还有监控和静态扫描的检出率都必须一样（有时候是软件或操作系统的bug导致问题，或杀软的设置问题），或有时候WildList突然把新样本加入名单里，厂商都来不及做出适当的反应，等等。


注：此贴作者是Vesselin Bontchev，反病毒界里的著名老前辈。帖子里所提到的讨论WildList弱点的论文，有兴趣者可以参考：
http://www.people.frisk-software ... apers/wildlist.html

solcroft

看来VB100%等其他采用WildList的杀软评测都没看头了，只能说是一种让厂商推销产品的手段。有些厂商专门为了应付VB100%测试下足了功夫，然后往往自吹自擂说自己通过了几次VB100%测试，但实际效果实在是一塌糊涂。

ianlai

IBK 很早就在WS上引用过 Vesselin 对WLO前途的看法，也曾多次在2005年的 av-comp 报告出多次引用（到了2006年下半年后似乎Vesselin这篇文章就在他的报告中消失了）

http://www.wilderssecurity.com/showthread.php?p=606364

补充一下，Vesselin 是目前世界上处理宏病毒最权威的专家，正因为他，Frisk一直是 Anti Macro Viruses 最好的安全软件之一，现在很多安全公司的工程师都是他的“徒弟”，包括Symantec

这次大集体捅破这层纸，一个很重要是 onecare 过了，并且大势宣传，原因很明显，不多说了。

hahacomcn

,看来国外杀软也是内幕重重呀，如果不掐起来（onecare的入注），还真是火在纸中烧着呢。

bidianyang

其实想开点也就没事了

评测不可能100%准确，无非是一种参考而已
不必较真

buycard

VB100也有积极的一方面，IBK说的你怎么不拷贝过来？报忧不报喜的家伙。

jpzy

呵呵，任何的评测都是一种参考了~！
这个世界上，有人的地方就有江湖啊，国外公司的产品多少也要靠一些公关能力的！