I've been thinking on this a little bit. Not sure if it really applies here but when it comes to audio you have ABX testing to compare components. There is this huge debate of the dorks on how valid it is because 9 times out of 10, people can't hear differences in most things, where plenty will insist they're so major only the deaf or the deluded couldn't hear them. It's a gold standard in many areas of science, when done right there isn't much room for doubt. It's done with things like amplifiers because when lining-up steady-state measurements with very well-established perceptual baselines, they *should* sound totally identical, but subjectively there isn't a solid agreement. A big criticism is that not enough has been done in audio. The difficulties in actually getting these tests right is tough, it's all about eliminating bias, simply to see if a person actually can detect a difference. Google around, you'll find a good couple of decades of "do amps sound different?" debates.
The reason I mention it is that one thing that makes it really difficult... say you're switching between two amps. It takes but a split second before your memory of the first one is pretty much gone... like it could be such a short interruption that you don't notice, but it was enough that your brain forgot it. From that point on, it's not possible to get reliable data on what was heard, and what was constructed in your mind.
What ends up happening a lot, when people try to do these tests themselves, is that they do them badly and convince themselves they've heard something they really haven't. So it's possible that with the way they ran their comparo, it could be playing tricks on him. He knows when he's looking at the RT or not, which can be a problem. In audio they call that a sighted test... not considered valid because simply knowing things like how it looks, buzz, price, can influence how you experience it. Hide those things and suddenly they can't guess which is which, or even when it was changed. So if you think the RT is supposed to be better and maybe you're into it, when you do a comparison like that, you're likely to 'see' confirmation. Though, just as often people will say they didn't hear a difference between the $100 amp and the $1000 one. No fully understood reason why, or even if people even can legitimately hear them. So someone who's not thinking much of RT or doesn't buy it at all might have a negative bias that prevents them from picking up on things that their senses are actually trying to tell them.
These biases are really tricky to get around. It takes people, money, and resources. Linus and crew would be among the few who could do it on a scale that actually means something. Would be interesting to see an in-depth look at how reliably people could really tell the difference in a truly 'blind' setting. Might make hardcore believers uncomfortable though... or maybe it would be the doubter's who'd be eating their leftover pride for the next week.
I'm really contriving something very different... but seeing these RT compar-o's, something seems familiar from watching subjectivist and objectivist audiophiles banter and bicker over etiological phenomena. I don't know as much about visual phenomena as I do psychoacoustics. I'm sure all of the tolerances are different. But in principle... maybe not far off?
That said, I don't think RT *should* make an immediate, earth-shattering difference, unless the lighting was done terribly wrong from the beginning. What I notice most when I don't go comparing, just turn it on and leave it there, I feel like what I'm seeing is more plausible. It's not that it *looks* amazeballs - it just *feels* more real. There's better depth and environments feels engrossing, especially in more dynamic scenes. What that really is, I don't know. I just know that if I turn RT, play for a while, and try to go back, it's much harder for me to get 'into' what I'm seeing on the screen. A trained eye might see it, but it goes back to the ABX thing. I don't have a way to prove the difference to someone else. I don't have this thing I can produce or tell you "look at this" and have you know it. So what good is it? Iunno...
Screenshots or even video don't really capture what I think is happening. It's hard to convey if you've never experienced it. Going back to audio, its sort of like watching speaker demos on youtube where a guy put a mic in a room with speakers playing... and then you're playing that back thinking you'll understand the experience of hearing them. I think it's something you can only notice when you're actually interacting with what you're seeing, as though it kicks off a different mode in your visual processing. When you already know it's an illusion, no need to analyze - you know. Brains are lazy, they hate extra work. But when you're actually playing a game that's getting all sorts of hormones going, the neurons start firing and the lazy brain gets to work. It wants to start picking at things. No mistaking, it's still a lazy brain, like always. It's hoping to peel at something that will reveal the trick, so it can go back to sleep and stop picking over so many details in the background.
I think that many times, the faked effects do such a good job, you really can't tell. But there are going to be those times where they don't suffice. It also comes down to how it is used. Control had nearly every surface triggering reflections. Nobody could say that didn't make a difference. The specularity for every material changed.
