An introduction to FCAT benchmarking

trajan2448

2013-04-04 17:50

In this article we will introduce you towards FCAT benchmarking. The past couple of months we have seen a some new dynamics in measuring the framerate of your games. Basically the framerate of your ... FCAT benchmarking an article introduction

I think its GREAT you guys are doing this, as it really goes much deeper than FRAPS. I hope you will delve into the whole RUNT frame issue as well. Great site!.

#4569926

Holyneo

2013-04-04 19:00

Is there a difference between what Fraps shows us in real time, compare to benchmarks that show fps in real time. In other words do benchmarks like Heaven..etc measure FPS like Fraps?

#4569937

JJayzX

2013-04-04 19:24

However, the question arises - how reliable and jitter-free is the capture device? 😀

It doesn't matter if the capture card is not perfect or compression sucks, as long as the frames are captured. Each frame is tagged so even if you miss a frame it would be known one is missing cause there is a sequence to the tagging.

#4569998

Noisiv

2013-04-04 21:03

Hilbert, I've read your description a couple of times now and I'm not really sure why you need to store all the frame data? If all you care about is the differing colours couldn't you just store the 1 pixel from bottom left? The only reason I can see for not doing that is that it the video capture device can only store the full frame and instead you'd require CPU to pull out just the 1 pixel? But I cannot imagine that would be at all taxing as extracting a value from a known location in a frame is very simple.

They need to store all data so they can process output frames later on. So to obtain that "1 pixel from bottom left" you _need_ full frames output. Alternatively script could be running real-time and extracting just frametimes and their length, ie "1 pixel from bottom left"

#4570030

Zunaro

2013-04-04 21:51

With that said I like to conclude this article. I do hope it wasn't too confusing and/or difficult to understand. The testing methodology used is a bit steep, we know. So if it was too confusing we do apologize.

I personally think that measuring the frame output is simple in theory, perhaps not in the implementation. However, at the same time you shouldn't underestimate what your audience is capable of understanding 😉 Thanks for the great article! I look forward to seeing FCAT results included alongside FPS benchmarks!

#4570077

Anarion

2013-04-04 23:11

This is the way to go. Average doesn't tell the whole story. If this means smoother mgpu gaming in the future for both, this is really good thing.

#4570149

Koniakki

2013-04-05 00:39

Hilbert is out(end of the article 😀) and FCAT is in... Well done Hilb. Top notch work. 😉

#4570217

Lowki

2013-04-05 02:57

Yes vsync eliminates it. I wish that was a little more emphasized. All this frametime issue seems to be NVidia propaganda because nobody ever seems to say oh if you play with vertical synchronization on There is no latency issues. I am glad it was put in for clarification even though its only one sentence in the article lol(I am referring to multi gpu microstuttering). It would be interesting to know what the latencies are like with vsync on since that's how I will playing. Are they different between NVidia and Amd? Im not trying to be negative here I just forsee a lot of users linking this article here like hey don't buy amd it sucks even though nobody I know plays with vsync off. Thanks for the nice read though man

#4570306

buddybd

2013-04-05 05:14

Yes vsync eliminates it. I wish that was a little more emphasized. All this frametime issue seems to be NVidia propaganda because nobody ever seems to say oh if you play with vertical synchronization on There is no latency issues. I am glad it was put in for clarification even though its only one sentence in the article lol(I am referring to multi gpu microstuttering). It would be interesting to know what the latencies are like with vsync on since that's how I will playing. Are they different between NVidia and Amd? Im not trying to be negative here I just forsee a lot of users linking this article here like hey don't buy amd it sucks even though nobody I know plays with vsync off. Thanks for the nice read though man

I play with VSYNC off in games that I play MP. I enjoy the increased fluidity, but when I sit back and play with the controller, it's usually VSYNC'd. Great article and glad to see Nvidia didn't try to pull a fast one. Looking forward to your in-depth cards reviews.

#4570327

applejack

2013-04-05 05:56

Yes vsync eliminates it. I wish that was a little more emphasized. All this frametime issue seems to be NVidia propaganda because nobody ever seems to say oh if you play with vertical synchronization on There is no latency issues. I am glad it was put in for clarification even though its only one sentence in the article lol(I am referring to multi gpu microstuttering). It would be interesting to know what the latencies are like with vsync on since that's how I will playing. Are they different between NVidia and Amd? Im not trying to be negative here I just forsee a lot of users linking this article here like hey don't buy amd it sucks even though nobody I know plays with vsync off. Thanks for the nice read though man

vsync isn't everyone's cup of tea you know. higher input latencies, occasionally caps fps to half refresh, some games has it poorly implemented, some don't support at all. correct me if I'm wrong but last time I checked, AMD driver can only force vsync in opengl ? also cannot really set flip queue size anymore ? also heard about eyefinity vsync issues, where it hurts the most. sure there are 3rd party solutions like d3doverrider and radeon pro to counter some issues. but if so called 80% "gamers" don't even change ingame settings, how many really are into drivers, tools and tweaks ? anyway as long as benchmarks are done with vsync disabled, FCAT results are not to be overlooked. as for me, I mostly use vsync w/ max pre-rendered frames 1 and sometimes d3doverrider for triple buffer. single gpu. I'd like to see vsync enabled, triple buffered CF/SLI results for when fps is below refresh. my guess is NV-AMD avg fps differences would be much closer to vsync disabled FCAT CF/SLI results than to FRAPS which counts CF runts & drops.

#4570501

TheSarge

2013-04-05 11:40

"Each four seconds or so the IO reached a bottleneck. It will kill and ruin results. We thereoff created a storage solution that as capable of writing faster then 650 Mb/sec." So... I'm assuming that's why there's what appears to be an OCZ RevoDrive 3 X2 PCIe SSD in the pics. Wow. That's a long way to go just to get some benchmarks. Thanks for staying hardcore Guru3D! :rock: Also... The Grammar Nazi that hides inside me is compelled to inform you that the word you were looking for in that above quote is "therefore" not "thereoff". For three reasons: 1) "Thereoff" is not a word. Go buy a dictionary and read it sometime. 2) "Thereof" is the word you aparantly meant to use... but it's the wrong word. 3) The right word was "therefore"... and you didn't use it. The phrases "solved this by..." or even "felt this to be unecceptable and so we sought a remedy, and so we..." would have also been accepted. This concludes your free yearly English spelling and grammar lesson. Live long and game on. 😎

#4570626

Noisiv

2013-04-05 15:01

The phrases "solved this by..." or even "felt this to be unecceptable and so we sought a remedy, and so we..." would have also been accepted. This concludes your free yearly English spelling and grammar lesson.

that internets law about grammar nazis always failing grammar themselves - it never fails ahahhaha http://www.abload.de/img/27124291-indexg5qc9.gif

#4570657

drac

2013-04-05 16:03

Yes vsync eliminates it. I wish that was a little more emphasized. All this frametime issue seems to be NVidia propaganda because nobody ever seems to say oh if you play with vertical synchronization on There is no latency issues. I am glad it was put in for clarification even though its only one sentence in the article lol(I am referring to multi gpu microstuttering).

I'd love for it to be true, that vsync eliminated microstutter. Though as I'm seeing things, its still fairly apparent with vsync on (depending on the game engine). Some games seem to not suffer from it at all, vsync or not, or at least its not noticeable.

#4570758

MadGizmo

2013-04-05 18:49

Great article. Hilbert. Looking forward to future use of FCAT in your reviews. 🙂

#4570813

fwixgamer

2013-04-05 20:10

this is really one of the best article in the high-tech web site in the past Q1/2013 really the best evolution of how benchmark need to be made in the future PS: just wanna say in your future benchs for CFX/SLI use Nvidia Inspector/Radeon Pro (vsync..-triple buffering- MGPU mode AFR etc ) to get the best result in your FCAT Article plus always use the latest high end gpu 2x/3x with both side AMD/NVIDIA that will add a heavy nice charm to your reviews 🙂 so keep it up that way your doing it nice (guess wizzard (TPU) ll start to cover him self with new idea lol )

#4570814

Stukov

2013-04-05 20:12

I'd love for it to be true, that vsync eliminated microstutter. Though as I'm seeing things, its still fairly apparent with vsync on (depending on the game engine). Some games seem to not suffer from it at all, vsync or not, or at least its not noticeable.

To be honest, the term micro stutter only comes from issues when vsync is enabled. It's somewhat complicated and I can't explain it off of the top of my head, but what most people refer to as micro stutter is simply just plain stutter.

#4570823

IPlayNaked

2013-04-05 20:23

vsync isn't everyone's cup of tea you know. higher input latencies, occasionally caps fps to half refresh, some games has it poorly implemented, some don't support at all. correct me if I'm wrong but last time I checked, AMD driver can only force vsync in opengl ? also cannot really set flip queue size anymore ? also heard about eyefinity vsync issues, where it hurts the most. sure there are 3rd party solutions like d3doverrider and radeon pro to counter some issues. but if so called 80% "gamers" don't even change ingame settings, how many really are into drivers, tools and tweaks ? anyway as long as benchmarks are done with vsync disabled, FCAT results are not to be overlooked. as for me, I mostly use vsync w/ max pre-rendered frames 1 and sometimes d3doverrider for triple buffer. single gpu. I'd like to see vsync enabled, triple buffered CF/SLI results for when fps is below refresh. my guess is NV-AMD avg fps differences would be much closer to vsync disabled FCAT CF/SLI results than to FRAPS which counts CF runts & drops.

It doesnt matter, it should be mentioned, but painfully few sites are doing that portion of the review. While it may not be some's cup of tea to play with vsync, the same is true that no one everyone wants to play without it. The fact that the picture is drastically different when you enable it should be mentioned and investigated. Instead its glossed over. The Tech Report, which I frequent, started this mess and has yet to thoroughly investigate how vsync affects it, and has also failed to investigate frame latencies. It's unfortunate.

#4570827

drac

2013-04-05 20:29

To be honest, the term micro stutter only comes from issues when vsync is enabled. It's somewhat complicated and I can't explain it off of the top of my head, but what most people refer to as micro stutter is simply just plain stutter.

Hmm well when I switch to 1 GPU I no longer get it, so I assume its microstutter, shrug! :infinity: Oh btw, awesome article Hilbert. 🙂

#4570843

Elder III

2013-04-05 20:45

This is the best article I have read in 2013. Thank you Hilbert and keep up the good work. :rock: *As soon as I saw "FCAT" I interpreted it as "FatCat".... http://www.cats-on-tshirts.com/images/cat_designs_280x280/design_fat_cat_black.jpg

#4570849

Dictator93

2013-04-05 20:50

Incredible article hilbert. I always new about the one frame delay in SLI (being an SLI user) and have always been a tiny bit turned off by it. BUt still... great to see this research being done. I laughed at your "website begins with an "A" joke too.