Latency, bandwidth, and capacity restraints can really all cause major performance slowdowns and what you describe sounds kind of like a classic case of it, but given you say with Optane there very well could and probably is something else cause performance issues. It's interesting that doesn't seem to make a pronounced difference and yet SAM has a enormous difference on the minimum frame rate. There has to be a logical explanation to that matter I would imagine. To be pair on the optane matter isn't NVME actually better in certain regards in terms of performance worse latency yet higher bandwidth!!? On another note with Optane did you try NTFS compression? You can net a bit more bandwidth like that. In a lot of cases what you describe is generally one of the issues I mentioned. Some games are inherently poorly coded though like EQ2 with the shadows being CPU rendered it's a design pisser for certain.
In the case of RDR2 might even be clip plane view distance matter I ponder too much loading quickly in instance at once at high details and not enough culling and detail reduction of the distance resources of textures and animations relative to the system hardware. There is a lot open world games that have problems trying to load way to much stuff at once in the distance that cause hitching and slowdowns if you don't tame those settings a bit. It's a good looking game and that kind of thing can kill performance quickly especially if VRAM or other system resources is a bit borderline.
What happens when you reduce memory frequency with the same latency to the FPS I ponder. If memory frequency impacts heavily the game defiantly is memory bandwidth limited to a degree. Still that doesn't explain how SAM is yielding such enormous FSP minimum gains to me that implies a bottle neck from storage or the PCIE interface itself. It could very well be the PCIE BAR size is the actual bottleneck in this instance. Really w/o increasing the BAR size there are still ways around the problem, but not necessarily as easy or simple at the same time.