TSMC stated that the N5/N4 cost is about 2x higher than N7. And Samsung's 8N was cheaper than that (that was effectively a 10 nm class process).
As for your math, unfortunately it is not that simple. Process nodes have gotten so complicated, that the traditional nanometer metric does not apply anymore (with stuff like finfets, EUV, wider metal pitch and who knows what else, I do not even understand any of it). That is why they came up with those weird names.
TSMC's N5 has triple the density over Samsung's 8N. The N4 even slightly better than that. There is also a separate 4N process that NVIDIA is using here, and I do not know what the difference is compared to N4.
But AD102 has 2.7x more transistors with a slightly smaller die compared to GA102. The die size being so similar means that Ada chips are over 2x more expensive to make currently.
The cost increase is there, but their margins are another thing. As I said in another thread, maybe it is time to go back to making smaller and less power hungry dies.
Give up performance increases for one generation, instead utilize the efficiency of the new process and make the flagship model consume 250 W, sell it for $600-700 and call it a new generation of environment friendly technology. Would that not be a win in a world struggling for resources?
I think this is what AMD realized. I think they will let NVIDIA keep the pointless performance crown while they sell extremely efficient and affordable cards. I cannot wait to find out.