Can you explain what is effective bandwidth?
Already basically did, it's the combination between the caches + vram bandwidth. AMD used this some times in their slides, giving a huge number in the mid 1 TB/s or higher.
No, 4090 does not have a ton of cache.
The full chip has 96 MB, regardless, 72 MB IS a lot of L2 cache. You seem to compare it to the wrong metrics (probably L3 cache). Even AMD reduced it to 96 MB max. - it is a lot.
Not going to google right now but IIRC AMD caches have more bandwidth almost throughout the whole thing.
No, L2 cache is generally faster than L3 cache, does not matter which kind of processor. AMD can not dream about having faster cache especially because it now goes into chiplets, not even on the same chip anymore, which decreases efficiency further.