Digital Foundry owes a small measure of gratitude to the PC tech media. Two years in the past, journalists from The Tech Report and PC Perspective carried out an in-depth evaluation of micro-stutter points in multi-GPU gaming PCs. The finish consequence was a brand new kind of research primarily based on FCAT – a system that sees each body produced by the graphics {hardware} marked up with a colored border. Frames are captured, the video analysed and from there, correct frame-rates might be calculated – however greater than that, the high quality of the experience comes into focus. This manifests in the type of frame-time – the period of time (sometimes measured in milliseconds) that every body persists on-screen.
And that is the place Digital Foundry’s method to PC efficiency diverges. Nvidia’s FCAT suite features a bunch of scripts for scraping the related knowledge from the video captures, utilised by a lot of the PC specialist press. We perform our personal evaluation utilizing our bespoke evaluation device – FPSGui. The captured movies are imported, the FCAT border mark-up is scanned and from there, our instruments place the knowledge on-screen alongside the knowledge.
We can extract the identical metrics from the captured video as Nvidia’s scripts however the distinction is that by overlaying the efficiency knowledge on prime of the precise video, we can choose efficiency in context. The essential a part of what FCAT provides – ‘in the second’ frame-times for each picture output by the GPU – makes much more sense when you possibly can really see what causes any points. FCAT additionally provides an additional layer of reliability to metrics – all benchmarks are measured primarily based on knowledge derived from the video output of the PC (ie what you really see on-screen), whereas instruments akin to FRAPS extract efficiency knowledge internally, which led to discrepancies in ends in the pre-FCAT period.
In the video under, we spotlight a current instance of how FCAT permits us to speak the high quality of the PC gameplay experience, utilizing Avalanche Studios’ Just Cause 3. We benchmarked a spread of playing cards in our unique characteristic, however the GTX 960 vs R9 380 comparability particularly was fascinating. AMD’s {hardware} is unequivocally extra highly effective than the Nvidia equal, and that is mirrored in frame-rate metrics that put the crimson staff in pole place. However, intrusive stutter on the R9 380 arguably made it the much less satisfying experience general. Using our instruments together with FCAT, we might isolate the areas that have been inflicting points and mixed with a number of GPU assessments throughout the stack of obtainable {hardware}, we have been fairly positive that driver points have been accountable. One month after Just Cause 3’s launch, the Radeon Crimson 16.1 hotfix driver replace resolved virtually all of the points (on the R9 380 no less than) – one thing we might affirm with A to B testing.
A dwell demo of Digital Foundry’s PC efficiency evaluation software program – and the way efficiency in context is essential to understanding the high quality of the gameplay experience.Watch on YouTube
The video additionally highlights how we take a look at GPU efficiency for our graphics card evaluations and our GPU improve information. The very same methods are used, besides that equivalent sequences – matched to the body – are captured with FCAT enabled. When the sequence video is imported, FPSGui creates a cache file with all the border mark-up knowledge, and as the content material of the video itself is basically equivalent between every take a look at run, we can bin the multi-gig captures. All of the FCAT knowledge is overlaid over pristine 1080p60 v-sync captures of every sequence, making the last benchmark movies considerably extra ‘watchable’ (no efficiency dips, no tearing and many others).
Benchmarking usually throws up fewer points than per-game evaluation, for numerous causes – firstly, drivers are usually mature, which means that points like Just Cause 3’s launch efficiency on AMD {hardware} can be a factor of the previous in a typical GPU take a look at. Secondly, essentially we can solely benchmark a comparatively small sequence of motion. Benchmarks ought to by no means be thought of as an evaluation of how nicely a specific recreation will run – it is extra a comparative evaluation of how nicely recreation engines run on every GPU. Once the total benchmark suite is full, we acquire a extra full image of how the {hardware} is performing.
However, as the Far Cry 3 benchmark run highlighted in the video reveals, we can get a grip on sure points. For instance, Ubisoft’s open-worlder calls for no less than 3GB of VRAM for clean(ish) 1080p gameplay at excessive frame-rates. Comparing the GTX 960 and R9 380 once more – two playing cards accessible in each 2GB and 4GB iterations – a few conclusions might be drawn. Firstly, that some video games actually do want that extra quantity of reminiscence – intrusive stutter is noticed on the 2GB playing cards, absent on the 4GB variants from each distributors. And secondly {that a} 2GB Nvidia card has extra environment friendly reminiscence utilisation than its Radeon equal: the AMD card stutters extra typically, and with longer split-screen freezes.
This would not occur on each recreation, however video games like this, Assassin’s Creed Unity, Rise of the Tomb Raider, Shadow of Mordor and naturally Batman Arkham Knight present that the demand for VRAM is barely transferring in a single course. For anyone contemplating a GPU buy on a higher-end £150+ card, 4 gigs actually is the method ahead and by way of our instruments, we can visualise the impression on the gameplay experience when VRAM is over-committed.
Performance in context permits us to seize and analyse the precise experience of PC gaming and whereas bar-charts and tables are invaluable in offering an at-a-glance take a look at efficiency, the work we’ve carried out right here is extra informative: quicker is not at all times higher, and usually talking, consistency is king. There’s nonetheless way more work to do right here – for instance, in higher highlighting points in the ‘in-the-second’ experience points in our at-a-glance metrics, to not point out optimising the complete course of (seize and evaluation is time-consuming) however our intention right here is to complement current metrics offered elsewhere and attempt to focus extra on the precise gameplay experience, and the way new {hardware} can genuinely enhance playability on PC titles.
Inside Digital Foundry is an occasional sequence the place the staff talk about the methods and the instruments used to provide its articles and movies. Other matters coated to this point:
https://www.eurogamer.net/digitalfoundry-2016-inside-digital-foundry-beyond-the-barchart-pc-performance
