Why on earth is 4090 106 Fps, and 4080 - 55 Fps. That's 92.5% and it only has 70% more CUDA, what is this free performance,. 16GB not enough already?
It's likely that it is not just one factor but a combination of many factors including -- but not limited to -- memory bus width, memory clock frequency, memory bandwidth, GPU clock frequency, and other things. Game performance isn't based on one type of transistor on a GPU.
Remember that the 4090 has a 384-bit memory bus and the 4080 has 256-bit.
As mentioned repeatedly in many, many threads, NVIDIA is binning silicon and earmarking better transistors for their higher priced products. Their top GPUs end up in data centers.
Very excellent GPUs end up in their top tier graphics cards. They are also binning VRMs, VRAM, and other silicon. Not all GDDR6X chips are the same in the same way that not all DDR4 chips perform equally. Not sure if you've noticed that.
All of these slight improvements add up.
There's also a very real possibility that the software driver used for these comparisons was optimized for the 4090. After all, that was the first Ada Lovelace card to be released so undoubtedly NVIDIA engineers prioritized that GPU.
This is yet another example why one can't look at a single game benchmark for at a single display resolution and make conclusive statements. Some games will benefit from more raster cores, some games can take advantage of RT and ML cores. Other games might favor fast and wide memory transfers, others just a lot of VRAM. Some games rely more on the CPU. And not each card works equally well with all graphics APIs. Some cards are better for DX11 games, others are better for DX12. And game developers sometimes end up writing code that favors a particular architecture, occasionally because game development was sponsored by a GPU manufacturer (AMD, NVIDIA, and now Intel).
So in the end, it's more than counting CUDA cores.