Brain-storming: How to show CPU performance in CPU Database

[evernessince]

3 runs of cinebench (both single and multi core) to heat load
averge the 6 numbers put it in a table one for single one for multithreaded
done
gaming performance is irrelavent at anything over 1440p

Using a single application to measure single and multi-threaded CPU performance would not be representative of a CPU's performance. It would only be representative of that CPU's performance in Cinebench. TPU already has the numbers available for a variety of applications, I don't see why they wouldn't use them to calculate a more comprehensive CPU performance number.

How about a "potential" index for all 3197 CPU's cores x threads x frequency with adjustments for instructions higher than SSE 4.1 and "3D" cache?
It can be added - the CPU can achieve said potential when all its cores and features are properly used.
A quick idea for avoiding re-testing 3k CPU's.

I don't think subjective metrics are a good idea. Userbenchmark uses them and not only are their rankings inaccurate, they have to frequently change them which kills any sort of usefulness. Using objective data from benchmarks doesn't have this same issue as the user is free to interpret the data according to their own use case. All a potential index would amount to is an assumption based on who is making the metric, not the true potential. That's impossible to know unless you can see into the future.

 

[OneMoar]

Using a single application to measure single and multi-threaded CPU performance would not be representative of a CPU's performance. It would only be representative of that CPU's performance in Cinebench. TPU already has the numbers available for a variety of applications, I don't see why they wouldn't use them to calculate a more comprehensive CPU performance number.



I don't think subjective metrics are a good idea. Userbenchmark uses them and not only are their rankings inaccurate, they have to frequently change them which kills any sort of usefulness. Using objective data from benchmarks doesn't have this same issue as the user is free to interpret the data according to their own use case. All a potential index would amount to is an assumption based on who is making the metric, not the true potential. That's impossible to know unless you can see into the future.

we don't need ScienceDiet levels of accurancy just a ballpark
cinebench is fairly reliable and platform agnostic

 

[DemonicRyzen666]

we don't need ScienceDiet levels of accurancy just a ballpark
cinebench is fairly reliable and platform agnostic

The trouble with using Cinebench is that there are different version throughout the years & does not scale linear with age.

 

[P4-630]

3 runs of cinebench (both single and multi core)

But

For all the 3197 CPUs that we currently have in the database?

 

[HTC]

A radar graph would be a perfect fit I think:

View attachment 310790
It's perfect for visualizing the differences. In the graphic above for example, the red line could represent CPU A and the blue CPU B. You can adjust the number of points based on the number of metrics you want to measure, although this does work best with 5 or more metrics. Just off the top of my head: Single thread, multi-thread, Application performance (mixed workloads), game performance, energy efficiency. Definitely others you can add as well, price or performance per dollar for example.

Seems the best approach, IMO.

 

[OneMoar]

I'd like to add some sort of performance indicator to our CPU database, but not sure how/how to present.

As we all know, single-threaded performance is vastly different to multi-threaded, and gaming performance is even different to those 2. Also GPU-limited matters

Any ideas how to present something that's easy to understand for the average Joe coming from a Google search? and who just wants to know "what's faster?" or "what should I buy"?

you are asking for the impossible

Real performance is subjective the best you can do is provide synthetic testing if you must go down to rabbit hole of caring about 1080p CPU performance, then I would recommend using a custom unreal or unity engine scene rather than trying to benchmark something off the shelf

The trouble with using Cinebench is that there are different version throughout the years & does not scale linear with age.

Why would you run anything other than r23? If you're thinking of trying to make comparisons with CPUs, you've already benchmarked. Forget it. Waste of time. New CPU is faster than old CPU by how much doesn't matter

 

[Sunlight91]

You will never be able to show gaming performance for all 3k CPUs. The oldest ones are Pentium 4 which won't start modern titles. Even 10 year old CPUs lack some modern instruction which severely impact performance. If you want to maintain a gaming index you should limit it to the last 5 years.

To get a consistent metric for all CPUs it should be one single thread and one multi thread score. Kind of like Passmark or Geekbench do.

 

[evernessince]

we don't need ScienceDiet levels of accurancy just a ballpark
cinebench is fairly reliable and platform agnostic

We are talking about the TPU CPU database. I'd argue accuracy is key.

 

[DemonicRyzen666]

You will never be able to show gaming performance for all 3k CPUs. The oldest ones are Pentium 4 which won't start modern titles. Even 10 year old CPUs lack some modern instruction which severely impact performance. If you want to maintain a gaming index you should limit it to the last 5 years.

To get a consistent metric for all CPUs it should be one single thread and one multi thread score. Kind of like Passmark does.

If that's way you feel then he should set it to link to the most recent cpu review.

 

[Shrek]

CPU-Z gives a number for Single Thread and another for Multi Thread, but even this is not enough as it misses the benefit of L1/2/3 cache.

 

[freeagent]

GFlops!

But that is not indicative to anything related to gaming, or even single threaded performance.

Its just raw performance..

 

[W1zzard]

right now i'm leaning towards cinebench, segment cpus into age groups, to run historic versions of cinebench. and interpolate between them so the performance scaling is contiguous

 

[OneMoar]

what would be the cut off for each age group
and a certain point it stops mattering

example comparing a FX9590 to a 7950x
its no comparison you would not need data to under stand that

 

[AusWolf]

Maybe with 3 separate graphs... single-threaded, multi-threaded (Cinebench) and gaming (average game fps at 1080p). Then, everybody could look at whichever is more interesting for them.

 

[Outback Bronze]

Tough one.

Maybe a rating system that's separated into gaming and workloads.

E.g: 7800X3D Gaming 10 - Workloads 8
13900K Gaming 9 - Workloads 9

Should be pretty simple for the average joe to understand.

But how do you rate older CPU's that were once good? Do they all drop down the product stack once they become legacy? So, say 5 years from now for example.

E.g: 7800X3D Gaming 8 - Workloads 6
13900K Gaming 7 - Workloads 6

Unfortunately, this means you would have to change the complete data base every few years.

 

[HD64G]

Y-Cruncher scores are heavily influenced by DRAM. The same chips with DRAM at different clocks produce vastly different results.

It's a tough problem to solve. Aggregate benchmark results would be one solution as it would weed out the strengths/weaknesses of architectures, but how do you get those scores without months of manual input? Approximations could be made for many classic architectures like K10, where Core\HT\NB clocks*cores=perf, but once we start getting into the modern chips that have varying core configurations with entirely different architectures inside, multiple different busses, mixed instructions... It's a herculean task at best.

Agreed on y-cruncher being sensitive to memory latency and speed. So, if the CPUs are tested on normal and stable XMP profiles that don't need expensive RAM to buy (for instance, Zen and Zen 2 at 3200c14, Zen3 at 3600c16 and the same for Intel K CPUs) where is the problem with this? Moreover, this will show the 3D-cache effect that helps in many apps. Y-cruncher scales well enough with clocks and cores also, so it might be the perfect all a rounder app to test many gens and archs of CPUs with.