Page 11 - Conclusion

ConclusionWell you have been able to see it for yourself. AquaMark 3 is without a doubt a very welcome addition towards the gaming community. It delivers us a powerful set of tools to measure performance and features of a graphics card in several ways. I'll say it immediately though, it's not a 100% DirectX 9 benchmark, it's backwards compatible to DirectX 8 and 7. So with a GeForce 4 or Radeon 8500 you'll get a very nice score .. but it's not a  DX9 score, it's an overall performance score.

My concern is this, if a shader operation can fallback from 2.0 towards 1.4, 1.3 or even lower. Then this is not a 100% DX9 benchmark and such a fallback would be very hard to see in performance. It's as much DX9 compatible as it can be as it will utilize DX9 as much as it can. Also a small side-note, AM3 does not rely so extremly on 2.0 Shaders as for example Half-Live will, AM-3 approximately generates 30% of it's screen pixels (without overdraw) through Pixel Shader 2.0 It's measures performance in DirectX 9 very well though. I've raised my concerns towards Massive here though and this was the response from Ingo, Technical Director.

The AM3 shaders have been designed with the fallback possibility in mind. For every shader which has been written in 2.0, 1.4, 1.3, 1.1 we have fallbacks, which generate a VERY similar result on the screen. Very similar means, that we do not ignore features like shadow, double caustics, detail maps etc. and therefore the benchmark provides an identical benefit for the user (identical regarding screen content) under different techniques. This concept is frequently used in other industries to compare different techniques. To make them comparable you first have to define the result and afterwards you have to guarantee that every technique generates exactly this result. This concept is often referred as basket of goods/benefit concept.

But nevertheless we try to create the same screen content with every technique, we face minor differences which arise from the internal accuracy which is smaller when we select a multi-pass technique instead of a multi-texture technique. The fallback mechanisms are optimized as heavily as possible, so we can ensure that the way to achieve the defined result is a near optimal way for all ps/vs versions. For that reason, the AM3 score is comparable, because the only fact that counts is the users benefit (ignoring image quality losses, which are negligible in AM3).

But you still have to pay attenantion regarding other resulting numbers. For instance, if you want to compare the number of triangles rendered per second, you compare apples and oranges because multipass techniques lead to significantly different numbers compared to multitexture techniques.

Best regards,

Ingo

That being said Aquamark seems to be build in a very objective way without graphics card manufacturers interference. The results you were able to witness today can never have been cheated on as the chipset designers did have not even had a chance to make modifications in their drivers. Come to think of it, it's quite sad that I once again have to mention illegal performance enhancing techniques in drivers in the conclusion of a really, really nice piece of software that measures graphics performance in a variety of ways.

I must state that with the extremely short time we where able to test Aquamark 3, we where very pleased with it. It'll be a welcome addition to our benchmark suite and sets a new standard in measuring graphics card performance.

If you like to try out the free version of AquaMark 3, which you should do at the very least you should know that we are official mirror for this software, so you can grab it from our file-servers.

Download @ Guru3D: Aquamark 3 - 63 MB

More info: www.aquamark3.com

If you like to discuss Aquamark 3 you can do so here.