QA Consultants Determines AMD's Most Stable Graphics Drivers in the Industry

[Raevenlord]

As independent third-party experts in the software quality assurance and testing industry for over 20 years, QA Consultants has conducted over 5,000+ mission-critical projects and has extensive testing experience and depth in various industries. Based in Toronto, Ontario, QA Consultants is the largest on-shore software quality assurance company, with a 30,000 sq. ft., industry-grade facility called The Test Factory.





Commissioned by AMD, QA Consultants used Microsoft's Windows Hardware Lab Kit (HLK) to conduct a 12-day head-to-head test comparing AMD and Nvidia graphics drivers. The test compared 12 GPU's, six from AMD and six from NVIDIA, running on 12 identical machines. All machines were configured with Windows 10 April 2018 Update. Both gaming and professional GPUs were equally represented in the testing. After running for 12 days of 24-hour stress tests, the aggregate of AMD products passed 93% of the scheduled tests. The aggregate of NVIDIA products passed 82% of the scheduled tests. Based on the results of testing the 12 GPUs, QA Consultants concludes that AMD has the most stable graphics driver in the industry.

About QA Consultants

QA Consultants is the North American leader in software quality assurance and testing services. Having successfully delivered over 5,000 testing and consulting projects to a variety of sectors including automotive and transportation, advertising and marketing, banking and finance, construction, media and entertainment, US & Canadian Federal State and Local government, healthcare, insurance, retail, hospitality, and telecommunications.

The Test Factory is the next generation of software testing, providing a superior quality, cost and service alternative to offshore providers and external contractors. The centre can handle any testing project of any size, with any application and for any industry. With full-time employees in Toronto, Ottawa and Dallas, QA Consultants supports customers by providing testing services such as accessibility testing, agile testing, test automation, data testing, functional testing, integration testing, mobility testing, performance testing, and security testing. Along with engagement models like Managed Consulting Services and On Demand Testing , QA Consultants is equipped to handle any client's request.

View at TechPowerUp Main Site

 

[HD64G]

Myth busted...

 

[Arjai]

I know I shouldn't be so skeptical. I am sure this is probably on the, up and up'. But, Amd paid for this research. What would the outcome be if Nvidea paid for it?

Hopefully the same.

It is good to see AMD being good. I like this AMD improving, thing. It has been too long. So long that I don't have anything AMD anymore! Sad. Choice is good, AMD is becoming a choice again.

 

[CosmicWanderer]

I know I shouldn't be so skeptical. I am sure this is probably on the, up and up'. But, Amd paid for this research. What would the outcome be if Nvidea paid for it?

Hopefully the same.

Just because they paid for the research doesn't mean it is biased towards them. If the results weren't in their favor, they simply wouldn't talk about it.

It is very common for organizations to pay scientific institutions/universities do to research for them and their products, and the last thing these institutions want is to have their reputation ruined due to sponsored bias.

 

[AnarchoPrimitiv]

It'll be interesting to see the mental gymnastics the ncidia fans go through to deny this...maybe I'll head over to WCCFTech to have those very fans show me the "bottom" with respect to humanity

 

[natr0n]

Tell AMD to use opengl on pcsx2. It can get unstable quickly.

 

[Arjai]

Just because they paid for the research doesn't mean it is biased towards them. If the results weren't in their favor, they simply wouldn't talk about it.

It is very common for organizations to pay scientific institutions/universities do to research for them and their products, and the last thing these institutions want is to have their reputation ruined due to sponsored bias.

I know. I probably should have just shut up. It is all this crap I read that makes me think that way. I think I may just give the NEWS article reading a rest, for a bit. I think it has made me feel different about things....well, that and the lack of sleep.

 

[GC_PaNzerFIN]

So, professional drivers from both companies are much more likely to crash than performance optimized consumer drivers? Sounds like massive variance test tools/setup playing up. Meh, you can twist and bend these test results to a point you have proved that cats are dogs!

 

[Vayra86]

Not really very valuable research except as a headline piece to alter the view on AMD drivers. And they need that positive noise to counter the negative trends of past years, right now its unfairly biased against them when in reality its a pretty stable affair across the board.

But yeah. Academic value? LOW. The most interesting bit here is seeing the 1060 fail so many tests. If I recall correctly it also suffered some bad driver releases not too long ago in the consumer space.

It'll be interesting to see the mental gymnastics the ncidia fans go through to deny this...maybe I'll head over to WCCFTech to have those very fans show me the "bottom" with respect to humanity

Was this really necessary? All I can say is: you won't be missed...

 

[the54thvoid]

It'll be interesting to see the mental gymnastics the ncidia fans go through to deny this...maybe I'll head over to WCCFTech to have those very fans show me the "bottom" with respect to humanity

Well, as a pure analysis of fact,
- 61 out of the 76 Nvidia fails happened on their Quadro cards. So, 80% of Nvidia's fail was on their professional line.
- 27 out of the 31 AMD fails happened on their Pro cards. So, 87% of AMD's fail was on their professional line.

That leaves:
- 15 desktop card fails for Nvidia
--- of which
- 10 fails for GTX1060
- 3 fails for GTX1050
- 2 fails for GTX1080ti

- 4 desktop fails for AMD
---of which
- 2 fails for VEGA 64
- 1 fail for RX 560
- 1 fail for RX580

In summary - the GTX 1060 is a donkey

And... their testing is highly questionable if BOTH vendors have 80%+ fail rates on the professional lines.

How did you like them mental gymnastics?

 

[Arjai]

Um. 76 fails vs. 31 fails. Pro cards broken down and separated like you did, is not logical. /Spock emulator

 

[the54thvoid]

Um. 76 fails vs. 31 fails. Pro cards broken down and separated like you did, is not logical. /Spock emulator

My post purely arithmetical. Logic is lord. And the donkey.

 

[Arjai]

No, 1060, is the Donkey. Logic, is never a Donkey. Unless, the answer is, "Donkey".

 

[the54thvoid]

No, 1060, is the Donkey. Logic, is never a Donkey. Unless, the answer is, "Donkey".

Live long and donkey.

 

[Xzibit]

And... their testing is highly questionable if BOTH vendors have 80%+ fail rates on the professional lines.

Did anyone bother to read the PDF?

We define graphics driver stability as a resistance to Blue Screens (BSODs), application crashes, hangs, and otherwise unexpected behavior in the presence of stressful test vectors over a period of time. We used Microsoft’s Windows HLK as a source of graphics test vectors. Windows HLK is a test framework designed to help hardware manufacturers deliver a quality product to users running Windows operating systems.
Within Windows HLK, we used the 64-bit variant of CRASH to stress the Graphics Processing Unit (GPU). CRASH is a GPU stress test tool that spans 4 hours in length and captures test cases covering S3, display resolution changes, display orientation changes, content protection, and rendering

The Windows Hardware Lab Kit (Windows HLK) is a test framework used to test hardware devices for Windows 10. To qualify for the Windows Hardware Compatibility Program, your product must pass certain tests using the Windows HLK.

 

[mtcn77]

AMD cards are stable. So much so in fact, I always mistake the card's limits for the specified limits. If I were able to restrain my curiosity, I would have still kept a 4890, 6950 and 6870 - all of which died during overclock sessions. 6870 was the hardiest, but in that case it was the vrm controller busting. Tragically, I'm not a good maintainer of them.

 

[GC_PaNzerFIN]

Did anyone bother to read the PDF?

Based on these results, both companies should fire their professional driver teams. Makes it even more funny the report is being shared by AMD.

 

[Recus]

It reminds me of FCAT and CTS-Labs and arguments why they can't be trusted because one is developed by Nvidia and other is paid by Intel. But what could go wrong if research is commissioned by AMD?

Did anyone bother to read the PDF?

QA Consulting said:
CRASH is a GPU stress test tool that spans 4 hours in length

I believe it takes more than 4 hours to render a movie so why VFX artist choosing Quadro if it can't hold long stress tests?

 

[mtcn77]

It reminds me of FCAT and CTS-Labs and arguments why they can't be trusted because one is developed by Nvidia and other is paid by Intel. But what could go wrong if research is commissioned by AMD?




I believe it takes more than 4 hours to render a movie so why VFX artist choosing Quadro if it can't hold long stress tests?

They use an array of cards for brief intervals?

 

[the54thvoid]

Did anyone bother to read the PDF?

Yes, I did. What's your point (genuinely)?

 

[Vya Domus]

It reminds me of FCAT and CTS-Labs and arguments why they can't be trusted because one is developed by Nvidia and other is paid by Intel. But what could go wrong if research is commissioned by AMD?

It was never proven that CTS was paid by Intel , stop making shit up just to spite people. Also , FCAT is just a tool, I literally never heard any complaint about it.

 

[rtwjunkie]

Based on these results, both companies should fire their professional driver teams. Makes it even more funny the report is being shared by AMD.

I’ve been saying this about Nvidia’s driver team for awhile. Thank goodness we have @qubit to beta test them for us before I try them!

Last year I made another foray into AMD land and had a 480. My experience was AMD has problems with drivers too. Their driver would frequently crash, and basically carry on if I was lucky with the basic Windows version.

So, they both need work, in my experience.

Edit: yes, I know my experience is anecdotal, not scientific.

 

[Mr.Mopar392]

Yes, I did. What's your point (genuinely)?

to remove your green shade!! that's the point!

 

[John Naylor]

AMD came out on top on an AMD funded test ... as Joe Pesci said in "My Cousin Vinny" ... "Oh there's a $#*&^%$ Surprise"

Frankly, outside CF installations from years ago, I never had a driver issue on any card.

to remove your green shade!! that's the point!

In an argument as to whether the sunset makes the sky look orange or red, isn't made by saying the sky has clouds in it. What I think is being asked is what is the relevance of those specific tests to the intended use of gaming cards. For example, if I was evaluating the suitability of a van for a Florida soccer mom,noting that it failed the tests for trail climbing and performance in deep snow would not be relevant. I'm not the tests address this or they don't; but you will need to address that to make the point.

Again, we have never had real driver stability issues with AMD in years ... not since they went from 4 to 3 digits in their product line. But in in the performance segments we build boxes for ... right now, AMD just doesn't have any products in play. Outside of that, only issues I have had with AMD driver wise were when a problem with a particular new game arises, the time to release of a driver fix was a lot longer than nVidia. Tho that's been getting better and better as time goes on.

 

[GreiverBlade]

Myth busted...

i never took that myth to be true ... after all i never had to rollback on drivers for any of my past ATI/AMD cards....

Nvidia on the other hand ... i am still on 397.93

I’ve been saying this about Nvidia’s driver team for awhile. Thank goodness we have @qubit to beta test them for us before I try them!

beta tester are the best!

Last year I made another foray into AMD land and had a 480. My experience was AMD has problems with drivers too. Their driver would frequently crash, and basically carry on if I was lucky with the basic Windows version.

So, they both need work, in my experience.

Edit: yes, I know my experience is anecdotal, not scientific.

well i might just be lucky with ATI/AMD then ...

Tell AMD to use opengl on pcsx2. It can get unstable quickly.

uh? i did on my R9-270 and 290, didn't had any issues...