TK421
Premium
join:2004-12-19
Canada
| reply to Margolis
Re: Intel Cheats in 3D Performance Benchmarks
said by Margolis  :intel didn't cheat. This is a VERY BIASED article. the original article snipped only a small part of the 3dmark rules that they thought would back up their biased claims. The very first rule from 3dmark concerning cheating: It is prohibited to for the driver to change the rendering quality level, rendering technique or replace or alter any shaders or other parts of the workload from that what is requested by 3DMark.
and they ignore the most important part of the quote they do use. to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection
they are NOT replacing any quality parameters or any of the workload. They are just offloading parts to the cpu to help out. Always interesting when different people see something and come away with totally different interpretations.
I read the article and thought: "Naughty Intel... caught tweaking those benchmark numbers they love to stick on boxes and in ads to confuse unwary tech neophytes who do not know what system specs are truly best for them."
Nvidia and others have done the same in the past... ah well, such news is far from surprising these days.
I hope that most if not all folks here realize by now that a synthetic benchmark (3DMark being one of the most popular) is useful for getting the general picture of your system performance... it is not particularly useful for estimating how your machine will perform in any given game.
Anyway, back to the article.
I already commented in my post above that the article showed a possible motive and effects on manipulating the 3DMark score to obtain an apparent advantage over rival IGP maker's chips (AMD - who correctly informed Futuremark that Intel's drivers were not playing by the same rules to get a higher bench score). In light of some comments here that it's okay to optimize drivers, I wondered if some folks even read the actual article in the OP?
In the early days of GPUs, application-specification performance optimizations in graphics drivers were viewed by many as cheating. Accusations were hurled with regularity, and in some cases, there was real cheating going on. Some optimizations surreptitiously degraded image quality in order to boost performance, which obviously isn't kosher. Optimizations that don't affect an application's image quality are harder to condemn, though, especially if you're talking about games. If a driver can offer users smoother gameplay without any ill effects, why shouldn't it be allowed?
The situation gets more complicated when one considers optimizations that specifically target benchmarks. Synthetic tests don't have user experiences to improve, just arbitrary scores to inflate. Yet the higher scores achieved through benchmark-specific optimizations could influence a PC maker's choice of graphics solution or help determine the pricing of a graphics card.
Futuremark's popular 3DMark benchmark has been the target of several questionable optimizations over the years. Given that history, it's not surprising that the company has strict guidelines for the graphics drivers it approves for use with 3DMark Vantage. These guidelines, which can be viewed here (PDF), explicitly forbid optimizations that specifically target the 3DMark Vantage executable. Here's an excerpt:
With the exception of configuring the correct rendering mode on multi-GPU systems, it is prohibited for the driver to detect the launch of 3DMark Vantage executable and to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection. Optimizations in the driver that utilize empirical data of 3DMark Vantage workloads are prohibited.
No ambiguity there, then: Vantage-specific optimizations aren't allowed.
Intel may not be playing fair, though. We recently learned AMD has notified Futuremark that Intel's 15.15.4.1872 Graphics Media Accelerator drivers for Windows 7 incorporate performance optimizations that specifically target the benchmark, so we decided to investigate. Techreport then goes on to run their tests and discover that yup the Intel driver detects the launch of 3DMark Vantage executable and with a little voodoo succeeded in optimizing for 3DMark to get a slightly higher score than AMD's non-optimized 785G. Yet surprise surprise even when allowing for Intel's G41 to utilize the same vertex offloading optimizations for a real game (Crysis Warhead) it falls way short of matching the performance of AMD's 785G.
Not once (after clearly reading the entire article) did I ever think Techreport was making a issue that Intel should not be allowed to use vertex offloading to the CPU if it helps gain a few FPS... rather the opposite, it's encouraging that it might help give a little performance boost that is good for gamers. The only issue, and was the entire focus of the article, was that the tweak only helped Intel achieve a quotable benchmark victory over AMD's rival IGP chip... and a misleading one because it's not backed up with real game performance with or without Intel's optimizing.
The bottom line is do not rely on a synthetic benchmark to gauge the subjective value of (as TR said), "these modest IGPs." Of course we all know that but I pity the uninformed.
And Margolis:
Offloading vertex processing to the CPU is altering the rendering technique.
BTW the complaint originated from AMD. If AMD and Nvidia are required to submit drivers that do not change behavior specifically for 3DMark why shouldn't they bring it to the attention of Furturemark? I would if they were comparing skewed 3DMark scores to my product.
Intel could have scored a point *if* the optimizing helped the G41 get a performance level even close to AMD's 785G but got its ass kicked even with vertex tweaking outside of 3DMark.
Intel graphics always have been and still are the poorest choice for gamers. |
TK421
Premium
join:2004-12-19
Canada
1 edit | Nah, I realize the optimization can be beneficial but I try to stay on topic regarding the methods Intel's drivers used in 3DMark Vantage. The evidence is there that Intel was breaking the rules for driver certification in Vantage.
Tuning graphic drivers to offload vertex handling from GPU to CPU should work wonders with a graphics-only benchmark. Obviously the GPU does not have to work as hard.
The interesting thing though, was the optimizing helped Intel achieve a higher score than their rival in 3DMark only... using the same vertex processing optimization in a real game benchmark did not make Intel's G41 even close to AMD's 785G true performance level.
So what is the real benefit to gamers? Higher Vantage score - who cares? But let's be honest, IGP is not for games and no right minded gamer is going to build/buy a system built upon any Intel graphics... so who is likely to fall for suspect benchmark scores when trying to decide what to get? Intel knows 3DMark scores can look good on them for marketing purposes. |
·
·
·