Why are the results different than manually run results?
Q3Bench results should be VERY close to manually run benchmarks. The problem is that manually run benchmakrs have human speed interaction with Quake3. In Q3Bench's scripting benchmarks run, settings and so on are executed as fast as they can be read by Quake3. We've taken steps to minimize the differences as much as possible.

Bad Results, early exit by Quake3,"Massive" Demo, and Very Large Demos
Q3Bench can not automate extremely large demos like the "Massive" demo wich is over 1 MByte in size. Q3Bench has been tested with demos up to 429K in size. For larger demos than this, it becomes necessary to use the advanced option of dumping the Quake3 console after every fps result. Beyond this, the Quake3 console simple overscrolls with information and fps results can become lost.

For comparison, the built in (1.11 - 1.17) demo "Demo001.dm" is only 128K in size and the next largest "public" benchmarking demo is (next to Massive.dm3) is "010101.dm_48" (from the Reverend) which is 429K in size.

It is only known at this time that somewhere in between the 429K and 1 MByte file sizes is a limit for the Q3Bench scripting. Keep in mind that almost all benchmarking demos are 150K or less in size.

Quake3 keeps crashing while running benchmarks started by Q3Bench.
The changes in resolutions and stress of the benchmarks will show up driver and overclocking problems often. This is the first thing you want to consider. A full run will generate over 50 resolution changes in only a few minutes....

If Quake3 Crashes while running the benchmarks, are the results still good?
Yes! Whatever gets compiles is good. You'll just have to redo what wasn't run before the crash. The only exception to this is that Q3Bench may not be able to average the results of the last setting level it was running. Right now if you only got 1 result out of 3, Q3Bench will fill in the blanks of the missing two runs for the average...

Why run the benchmarks 2 or 3 times instead of once?
Becuase of Gremlins or the exact voltage fluctuations during benchmarking, Quake3 benchmarks can vary up to 5 FPS at the same settings. The most accepted way to overcome what can by misleading is to run at every setting 3 times and then average the results. You can choose 1, 2, or 3 runs. If you do choose more than one run per setting, Q3Bench will average the results for you.

Number of Results per condump:
You may be wondering what this advanced setting does. You have the option here of allowing up to three results between console dumps, or dumping the console after each result is generated. Some of us at G256 believe that dumping every three results provides more accurate results (which is why this is the default setting). However, by setting Q3Bench to 1 result per dump you will be able to run much longer demos and still get the results (long demos can overscroll the console befpre it gets dumped on more than 1 run).