A Guru's guide to 3D Benchmarking

INTRODUCTION

Hey people, yes I'm back again and making a comeback after my previous article (Detonators Explained). This time I'm here to explain some aspects of benchmarking and to show you a few good benchmarks and outline how they work. In the article we will be looking at benchmarks that are suited to games and graphics performance, I doubt anyone would care how fast your computer can run MS Office (I think Hilbert would though). Anyways, I hope you enjoy reading it and will learn something from it.

First off, for the newbies among us, Benchmarking is simply a method where you run a set of tests (or a single test) to get a result proving how good (or bad) your computer or peripheral is.

There are two different types of Benchmark ;

1. Synthetic
2. Real World

Synthetic Benchmarks

Benchmarks that are only benchmarks , nothing else, they serve no other purpose but to test and rate a PC or hardware part.

A program such as 3DMark 2000 is synthetic. All the demos in it are made for the purpose of benchmarking, we will probably never see the helicopter demo become a game in real life nor any of the other tests it performs. We do see however the trend of integrating 3D engines into benchmark software. As example 3D Mark 2001 uses the MaxFX engine to run the Max Payne party of the benchmark. The game will be released in 2001. Most Synthetic benchmarks also calculate and give a score instead of an average FPS (Frames per Second)

Pros : They are made specially for testing therefore giving a more accurate result.

Cons : It is unlikely that the tests performed in such programs would ever need to be performed in a real life situation.

Real World Benchmarks

Real World Benchmarks grade the performance of an actual program or game.

For example, Quake 3 is real world, it is a game you play and by benchmarking with it you get an accurate idea of what performance you can achieve in the game at a certain resolution and with certain settings. Real World benchmarks will represent a score on frames per second. It's a very good method to measure performance however it's likely that not all options/features/optimizations of your videocard can handle are enabled.

Pros : You know how fast your computer can handle this Application or Game should you ever need to use it or play it. The result in a real world benchmark would most probably also be useful to show how you are able to handle other games (ie. if you can run Quake 3 at 50FPS then you should be able to run other FPS games quite well).

Cons : Real World Benchmarks may not test all the things that are tested in synthetic benchmarks (special  optimizations such as SSE/3DNOW! and Hardware T&L)

Obviously both have their strong and weak points , so it is good to use a mixture of both when benchmarking your system. This is also the reason why we use both real-world and synthetic benchmarks in all our reviews.