An introduction to FCAT benchmarking - FCAT Multi-GPU Results

FCAT versus FRAPS Results

So was there a reason why NVIDIA was so keen to get the new benchmark methodology out in the open? Yes - two actually. The first is simple, they think FRAPS measures at the wrong spot as explained and we tend to agree: 

Above Radeon HD 7790

Above Hitman Absolution again. FRAPS is showing the single GPU performance of the Radeon HD 7790 with heaps of issues. Once we started seeing this behaviour (shows on all cards) an alarm bell rang with us wondering if the spikes actually are graphics card related. Loads and loads of latency spikes are charted that we do not visually see during our test run. If we'd zoom in on that chart we'd see weird and off stuff. Now admittedly, Hitman is the one very specific game title that is showing this stuff so badly. The more reason though we use it as a showcase today, you'll get the generic idea.

Let me run the FRAPS frametime test again with the R7950 Boost we have been using for this article:

So that is the R7950 measuring Hitman with FRAPS. Crazy right ? That's measured in combo with the very latest 13.3 Beta 3 drivers. If you see that you'd probably get scared to purchase the graphics card. However the spikes do not correspond at all with what we see on screen. Now let's run FCAT.

Above Hitman Absolution with the very same AMD GPU and drivers. Notice how all the weird spikes are gone? That's the advantage of FCAT, you record what you see on screen and this is in line with what we see on the monitor. So what does that say about FRAPS ? Yeah, it indeed can show a lot of stuff that really isn't there and perhaps vice versa. That doesn't mean though it is recording non-exsistant stuff, contrary. But is is inserting and leaving out stuff we can not observe visually.

FCAT Multi-GPU Results

So was there a 2nd reason why NVIDIA was so keen to get the new benchmark methodology out in the open? Yup - see one can argue about FRAPS frametime recordings, but like it or not, FRAPS did discover / expose some stuff that was out of the ordinary. Now FRAPS had shown weird anomalies with AMD Crossfire configurations that are not in line with what you expect to see. So let's fire up FCAT again with a Radeon HD 6990 (dual-GPU).

Uh-oh! Above you see the Radeon HD 6990 (dual-GPU), 2560x1440 and basically what happens is that with each frame that is rendered the next frame the latency drops and then the next goes up again. So basically high-low-high-low-high-low-high-low.

Let me show you a few lines of the data:

Data point 0 - 13.063
Data point 1 - 20.372
Data point 2 - 9.369
Data point 3 - 20.372
Data point 4 - 9.809
Data point 5 - 21.273
Data point 6 - 8.750
Data point 7 - 24.234
Data point 8 - 8.896
Data point 9 - 24.640

And so on, the data-point is frame-number and the result is displayed in milliseconds. The odd / even results could be described as microstutter and if we deduct monitor latency we are nearing framedrops. I'm being very careful here and say "could"... okay? But with the data-set at hand, we can now zoom in as well. This is a 100 frames. The scale to the left, 40.000 Micro seconds = 40ms:

Fact remains that the turnaround time for a frame is low in ms and that makes this very hard to see and observe on the monitor, I mean look at the chart above and there's not one frame passing 45ms. But you need to wonder, what the heck is happening there?

The theory is simple, basically multi GPU solutions apply AFR (alternate frame rendering), two GPUs alternate in processing frames. AMD claims to have the GPU as render as fast as possible, NVIDIA keeps things smooth by syncing maybe even delaying frames. The image above shows how that would be rendered (and pardon my l33t drawing skillzz). That would explain the phenominon shown above. But it leaves te question as to why each alternating frame is rendered faster then the previous one.

Let's look at what NVIDIA does. Have a look below.

By syncing Nvidia is likely delaying one frame to compensate the offset you are seeing with AMD. As a result, above a chart done with a GeForce GTX 690, same title, same benchmark and same resolution. Really fine scores with very tiny latency changes showing very smooth stuff. But allow me to zoom in at the exact same point as we did with the R6990, again 100 frames this time with our axis scaled and fixed:

Above the GeForce GTX 690 now showing the very same frame sequence at the same spot we just showed the AMD result, same resolution well... same everything. That really is just a massive difference. But with data-sets we can take it a step further:

And there it is, 100 frames at the same measuring point. Now remember, FCAT displays in a chart what you see on screen, keep that in mind.

Now AMD doesn't like to share information about this behaviour, except when your website name starts with an A. We undertand they are working on this, have confirmed the findings and expect in the June/July time-frame to come up with some sort of sync solution for what you are seeing there. But yeah, this basically is why NVIDIA is pretty keen about FCAT. They did their homework and tackled that challenge. Now this is all stuff for discussion as let me throw one back at ya, what if we enable VSYNC ? Correct. The problem would be near gone. Anyway .. enough brain teasing for today.