本帖最后由 【乱】 于 2013-7-6 01:26 编辑
之前错误连接：http://eugene.kaspersky.com/2012/06/20/fighting-false-positives/
本文原文：http://eugene.kaspersky.com/2013 ... ification-devalued/
= =这是卡巴尤金的博客？
PS：此贴并不特别打击或维护某些事情，随意参考参考
chrome直接翻译有点坑爹 所以....别喷太厉害哦o(∩_∩)o 英文原文在连接里（底部增加全文）
我从来没想过杀毒行业谈论时，我曾经用这句话，但是这就是它的来。你知道，在这个世界上的一切进展顺利。经济的现实和新客户的需求往往设法引诱到黑暗的一面，即使是最好的。这一次，在AV界最知名的测试实验室- AV-TEST之一-已经屈服了。
对比测试：一点背景的门外汉
你怎么去挑选最好的任何特定产品？你怎么知道这是最好的吗？好吧，你可能会开始寻找一个专业杂志，或相当于在线的比较试验结果。我敢肯定，这是不是消息给你。这同样适用于AV解决方案 - 有一些测试实验室评估和比较庞大的各种防病毒产品，然后公布结果。
现在，对于一些未知的原因（下面我会尝试和猜测到底为什么）德国知名测试实验室AV-TEST已悄然（有没有警告）修改了其认证过程。这些变化意味着，新的规则所产生的证书，把它温和，相当无用的优劣不同的AV产品。
是的，这是正确的。我正式宣布，不再允许AV-TEST认证的AV解决方案，为家庭用户的产品质量进行比较充分。换句话说，我强烈建议不要使用他们的证书列表为导向时，选择一个解决方案，以保护您的家用电脑。相信，两种产品都具有相同的认证必须等于（或接近等于）在性能上是很自然的。AV-TEST的新的认证标准，对用户的责任是仔细地调查个别测试的实际结果...他们可能会发现，挡住攻击的99.9％的产品，具有了相同的“认证”作为一种产品，只受阻55 ％。
现在，让我们来仔细看看发生了什么事，为什么 - 或速成班解释AV-TEST的结果。
理想的AV（杀毒软件）解决方案的公式被认为在很久以前。它是这样的：
100％的保护和0％的误报。
零对系统资源的影响。
没有问题给用户。
（4，而且，如果我们想进入幻想的境界，则必须提供完全免费的。）
显然，这种理想是不可能实现的，但我们至少可以向往来尽可能接近它，特别是：
抓住尽可能多的恶意程式可能，如果事情没有得到通过-能治疗 感染（并且能够被感染的计算机上安装保护）。
最大限度地减少误报的风险 - 如果他们确实发生了，尽快摆脱他们。
声称：“我们的积分是没有界限的，”我的好朋友，而且也没有任何限制的工作，进入优化使用的系统内存，处理器的运行时间，数量和更新的大小通过互联网。当然，没有影响的安全级别。
这一切听起来非常简单。但平均用户做什么时，他/她期待在几十防病毒产品吗？哪一个更好，为什么？谁可以享有他们如何密切，他们的来“理想AV解决方案”的条款？（请记住，所有这些产品作出有说服力的索赔，最好有。）
所以，我们还能相信谁告诉我们真相吗？独立测试的，当然。包括AV-TEST。
几年前，AV-TEST队创造了很好的方法测试产品，并获得证书，产品几乎完美的表现在每个测试类别所需.. 产品测试三个标准：
保护（预防感染），
修复（清理现存感染）
可用性（易于使用，性能和数量的误报）。
颁发的证书，或（视情况而定），结果（累积分数）的基础上。我们支持这个系统，它作为一个例子比较测试中的“超级联赛”的其他测试。
那么，什么样的变态已经发生在AV-TEST？为什么我们不再信任其认证系统？
首先，获取证书的必要的标准已经改变了。重要维修认证参数已丢弃。AV解决方案，它能够检测到感染，但不能治疗有什么意义？（试想一下，在牙医的：“哦，你已经有了蛀牙，但我们不会把它-我们不能把它！”）不久前我们发现，约5％的所有计算机世界AV安装被感染！每二十个！显然，AV解决方案的能力，以消除活动性感染数以百万计的世界各地的人们的高度重视。
他们的信用AV-TEST已经承诺创建一个单独的，更进步的维修测试...但将不再影响产品的认证结果，并且，最重要的是，这将是可选的！如果安全厂商，其产品的治疗的质量有怀疑，他们可以简单地退出，以避免测试结果不佳。至少，我们可以看，谁不参加测试，他们的表现如何，这本身将是一个很好的指标，以及AV解决方案可以保护您的计算机。
其次，认证的门槛已经降低-你现在只需要10分的成绩，一个可能的18收到一个“奖”。
第三，可用性现在只指误报。有这种想法的可用性和易用性，需要考虑性能之间的差异是一个世界。鉴于目前妆AV-TEST的测试标准，误报可能只是被列入保护类别作为抗衡（其他大部分测试实验室做的东西，反正）。
那么，什么是AV-Test的认证过程这突如其来的变化所带来的后果吗？
首先，他们的认证，将严重贬值。测试参与者的数量将增加 - 避开一些反病毒厂商参加测试，因为他们知道他们的防御水平的机会都没有获得证书。现在，几乎每个人都将有从AV-TEST的奖项，只有最闲置或不称职的被冷落。
事实上，根据这些标准降低，这是一个基本的反病毒程序可以实现从AV测试的“合格证”合理的结论。为什么不呢？有没有必要去寻找新的恶意软件 - 所有你需要做的是从耻骨在线VirusTotal等多引擎扫描仪监测流量。有没有需要分析任何东西 - 只需设置一个多扫描仪和“检测”的文件，其他人都已经检测（检测使用MD5，以确保不存在任何虚假阳性）。然后设计一个接口，添加一个小更新，扔在一对夫妇的Windows功能来模拟连续的保护，坚持在系统托盘中的图标，全部包裹在安装程序和宾果！把它送上AV-TEST，等待您的证书！
基本上，评估安全性之间的平衡的技术和“使用方法”已丢失。个别测试的结果仍然有效，将继续紧随其后的是安防行业书呆子。但AV-TEST认证的门槛降低，不幸的是，意味着这个认证主要是没用的平均消费者试图购买AV解决方案时作出知情决定。
有你有它......现在的下一部分。
现在的问题是：到底为什么AV-TEST办呢？为什么自己和他人使事情变得更糟吗？
知之甚少的真正原因（AV-TEST的变化几乎没有注释），但我们可以尝试为自己看着办吧。首先我们需要看经济学的测试业务。
是的，测试是一个业务与它自己的经济。我明白了，非常好。执行是一个很好的测试不只是一个大脑-它需要投资的基础设施，办公空间和薪金。和许多企业一样，有质量和利润之间的相关性。有时候，企业自觉降低质量，增加利润。在短期内，这种做法真的可以还清。但在长远来看，它会导致退化和遗忘。
难道这是这里的情况吗？从“强制性计划”，最难的考验 - 治疗受感染的系统（修复） - 已被删除。现在，所有你需要做的是测试产品对恶意软件和干净的文件和...瞧的集合！
是的，新的程序，使测试过程中可能会带来新的客户，AV-TEST，他们会很乐意抛出自己的“金牌”，使每一个杀毒网站有一个。但在测试实验室将失去它的独特性，以及整个行业依赖于技术上更志同道合的AV厂商的信赖。
我当然不是谴责的愿望，以赚取更多的$ $ $ $ $！有了正确的优先级，这是一个很好的衡量一个企业的成功，其产品和服务的质量。当然，它有助于提高质量，甚至更多，大家的利益，赚取更多的:)测试业也不例外。许多公司在这一领域的多元化发展的道路，进入新的测试龛（AV-TEST也已参与），不仅挖掘更深，但也更广泛，提高他们的专业的因果报应。也许追逐额外的仙一对夫妇将导致到一个后裔的主流，贬低他们的判决“证书”在一个公平的价格水平。
下一个合乎逻辑的问题是-为什么我们的产品仍然在AV-TEST系统？
首先，它是我们的原则立场，更多的测试，更客观的评估。我们什么都不怕。我们有信心，我们的技术和我们的保护水平，如果我们有一个测试或其他任何投诉，我们这样说，直接和公开。
此外，AV-TEST有很多其他有用的测试和认证，包括企业和移动产品部门。处理功能是为企业客户的重要性比它是为家庭用户。即使在小公司通常有一个系统管理员和备份复制，在一起能击败感染如果一个AV产品不能。
要总结
有了这个新的认证过程中，我们不建议用户采取AV-TEST认证时考虑选择一个解决方案来保护他们的家用电脑。
然而，我们相信这是确定考虑到单独的保护和性能类别的结果，当然，所承诺的维修测试。同样，我们不同意他们的个人测试的方法......我们正在采取什么恩怨，在这些测试中，要求实现“认证的问题。”
它是在AV-TEST的利益，听AV界专家的意见，以便创建一个真正的充分的认证体系，帮助用户选择产品时作出知情决定。这涉及到，例如，返回到反恶意软件测试标准组织（AMTSO）和几乎所有的领先的供应商和行业专家与代表讨论的争论点。
One step forward, two steps back.
“Everything ought to happen slowly, and out of joint, so we don’t get above ourselves, so we remain miserable and confused”
Venedikt Yerofeev. Moscow Stations
I never thought I’d ever use this phrase when talking about the antivirus industry, but that’s what it’s come to. You know, not everything in this world progresses smoothly. Economic realities and the need for new customers often manage to lure even the best over to the dark side. This time, one of the best-known test labs in the AV industry – AV-TEST – has succumbed.
Comparative testing: A bit of background for the uninitiated
How do you go about picking the best of any particular product? And how do you know it’s the best? Well, you would probably start by looking at the results of comparative testing in a specialist magazine, or the online equivalent. I’m sure this is not news to you. The same goes for AV solutions – there are a number of test labs that evaluate and compare a huge variety of antivirus products and then publish the results.
Now, for some unknown reason (below I’ll try and guess why exactly) the renowned German test lab AV-TEST has quietly (there was no warning) modified its certification process. The changes mean that the certificates produced by the new rules are, to put it mildly, pretty useless for evaluating the merits of different AV products.
Yes, that’s right. I officially declare that AV-TEST certification of AV solutions for home users no longer allows product quality to be compared adequately. In other words, I strongly recommended not using their certificate listings as a guide when choosing a solution to protect your home PC. It would be natural to believe that two products that both have the same certification must be equal (or close to equal) in performance. With AV-TEST’s new certification standards, the onus is on the user to carefully investigate the actual results of each individual test…they may find that a product that blocked 99.9% of attacks has the same “certification” as a product that only blocked 55%.
avtest_cert_balance_blue
Now let’s take a closer look at what happened and why – or a crash course in interpreting AV-TEST results.
The formula for the ideal AV solution was thought up a long time ago. It goes something like this:
100% protection and 0% false positives.
Zero impact on system resources.
And no questions to the user.
(4. And, if we want to get into the realms of fantasy, then all that has to be provided absolutely free.)
Obviously, that ideal is unattainable, but we can at least aspire to come as close as possible to it and in particular to:
Catch as many malicious programs as possible, and if something does get through – be able to treat the infection (and be able to install protection on an infected computer).
Minimize the risk of false positives – and if they do occur, get rid of them ASAP.
“Our integral knows no limits,” claims a good friend of mine, and there’s no limits to the work that goes into optimizing use of system memory, processor operation time, the number and size of updates via the Internet. And of course, none of that should impact on the level of security.
That all sounds very straightforward. But what does an average user do when he/she looks at dozens of antivirus products? Which is better, and why? Who can rank them in terms of how closely they come to the “ideal AV solution”? (And remember, all those products make convincing claims to being the best there is.)
So, who can we trust to tell us the truth? Independent testers, of course. And that includes AV-TEST.
A few years back the AV-TEST team created a very good method for testing products, and to earn a certificate, products needed to perform nearly flawlessly in each test category.. Products were tested on three criteria:
PROTECTION (prevention of infections),
REPAIR (cleaning up existing infections)
USABILITY (ease of use, performance, and number of false positives).
A certificate is issued, or not as the case may be, based on the results (number of points accumulated). We supported this system and held it up as an example to the other testers in the “premier league” of comparative testing.
So what metamorphosis has taken place at AV-TEST? Why can we no longer trust its certification system?
First of all, the necessary criteria for obtaining a certificate have changed. The important REPAIR parameter for certification has been discarded. What’s the point of an AV solution that’s capable of detecting an infection but is incapable of treating it? (Just imagine at the dentist’s: “Oh, you’ve got tooth decay, but we won’t treat it – we can’t treat it!”) Not so long ago we found out that about 5% of all computers in the world with AV installed are infected! Every twentieth! Clearly, an AV solution’s ability to remove an active infection is of high importance to millions of people around the world.
To their credit AV-TEST has promised to create a separate, more progressive REPAIR test… but it will no longer effect product certification results, and, most importantly, it will be optional! If security vendors have doubts about the quality of their product’s treatment, they can simply opt out to avoid a poor test result. At least we can watch who does participate in the test and how they perform, which in itself will be a good indicator of how well an AV solution can protect your computer.
Secondly, the threshold for certification has been lowered – you now only have to score 10 points out of a possible 18 to receive an “award”.
Thirdly, USABILITY now only refers to false positives. There’s a world of difference between this idea of usability and usability that takes performance into consideration. Given the current makeup of AV-TEST’s test criteria, false positives might just as well be included in the PROTECTION category to act as a counterbalance (something that most other test labs do anyway).
So, what are the consequences of this sudden change to AV-TEST’s certification process?
First and foremost, their certification will be seriously devalued. The number of test participants will increase – some AV vendors shied away from taking part in testing because they knew their level of protection had no chance of getting a certificate. Now just about everyone will have awards from AV-TEST, with only the most idle or inept being left out.
In fact, based on these lowered criteria, it’s reasonable to conclude that a basic AV program could achieve “certification” from AV-Test. And why not? There’s no need to look for new malware – all you need to do is monitor the flow from pubic online multi-engine scanners like VirusTotal. There’s no need to analyze anything either – simply set a multi-scanner and “detect” files that others have already detected (and detect it using MD5 to make sure there are no false positives). Then design an interface, add a mini-updater, throw in a couple of Windows functions to simulate continuous protection, stick an icon in the system tray, wrap it all up in an installer and bingo! Send it off to AV-TEST and await your certificate!
Basically, the balance between evaluating security technology and “usability” has been lost. The individual test results are still valid and will continued to be followed closely by security industry wonks. But the lowered threshold for AV-TEST certification will, unfortunately, mean this certification is mostly useless to the average consumer trying to make an informed decision when purchasing an AV solution.
And there you have it… Now for the next part.
The question is: why the hell did AV-TEST do it? Why make things worse for themselves and others?
Little is known about the real reasons (AV-TEST has made almost no comment about the changes), but we can try to figure it out for ourselves. And first we need to look at the economics of the testing business.
Yes, testing is a business with its own economy. I understand that very well. Performing a good test is not just a matter of brains – it requires investment in infrastructure, office space and salaries. And just like many businesses, there is a correlation between quality and profit. Sometimes companies consciously lower the quality to increase profit. In the short term that approach can really pay off. But in the long term it leads to degradation and oblivion.
Could this be the case here? The hardest test – treating an infected system (REPAIR) – has been removed from the “mandatory program”. Now all you need to do is test products against a collection of malware and clean files and… voila!
Yes, the new procedure makes the testing process will likely bring new clients to AV-TEST, who will gladly dish out their “medals” so that every antivirus website has one. But the test lab will lose it uniqueness as well as the trust of the more technically minded AV vendors which the entire industry relies on.
I’m certainly not condemning the desire to earn more $$$$$! With the right priorities in place it is a good measure of a business’s success and the quality of its products and services. Of course, it helps to enhance quality even more and, to everyone’s benefit, earn even more :) The testing industry is no exception. Lots of companies in this sphere are going down the road of diversification, entering new test niches (AV-TEST has also been involved), digging not only deeper but also wider, improving their professional karma. Maybe chasing a couple of extra cents will lead to a descent to the mainstream, devaluing their verdicts to the level of “certificates for all” at a fair price.
The next logical question is – why are our products still in the AV-TEST system?
First of all, it is our principled position that the more tests there are, the more objective the assessments. We are not afraid of anything. We have confidence in our technology and our level of protection, and if we have any complaints about one test or another, we say so directly and publicly.
Moreover, AV-TEST has lots of other useful tests and certificates, including in the corporate and mobile product sectors. The treatment function is of less importance for corporate clients than it is for home users. Even in small companies there is usually a sysadmin and backup copying, which together can beat an infection if an AV product can’t.
To sum up
With this new certification process, we do NOT RECOMMEND users take the AV-TEST certification into consideration when selecting a solution to protect their home computers.
However, we believe it is OK to take into consideration the results of the separate PROTECTION and PERFORMANCE categories and, of course, the promised REPAIR test. Again, we don’t disagree with the methodology of their individual tests…we are taking issue with what scores are required in these tests to achieve “certification.”
It is in the interests of AV-TEST to listen to the views of AV industry experts in order to create a truly adequate certification system that helps users make an informed decision when choosing a product. This involves, for example, returning to AMTSO (Anti-Malware Testing Standards Organization) and discussing points of contention with the representatives of virtually all the leading vendors and industry experts.
|
[复制链接]
发表于 2013-7-5 05:31:04
同意！
见人见智吧，对杀软感兴趣研究一阵子就能有自己的判断了，不用管别人怎么说