Gamespot PC/PS4/X1 visuals experiment. Can you tell the difference?

Kassen said:
But the people who did the visual experiment do not even know what water is made of. Isn't that terrible? How can they judge visuals if they can't even draw a picture of a water molecule?
Click to expand...
Ignorance is not stupidity, just lack of knowledge. I've known people that don't have a lot of academic knowledge that could think people into circles. You have to be shown what a water molecule looks like before you can draw it. Maybe they have just forgotten what it looks like because most people have zero occasion to make use of this amazing proof of intelligence. Honestly, what actual good does that knowledge do unless you are a molecular biologist?

I have a problem with this rationale. It is not a good indicator of intelligence, just training.
 
The final test is perfectly valid. It shows how much they struggled when the platforms were different.

At that point, they expected the test to be difficult and didn't question it. If the initial tests were easy, they would have spotted the trick.
 
Frozpot said:
To a degree, but the last two (the guys who got the other right) were saying that they SAW a clear difference in clarity and frame-rate. That means their brains were filling in to what they expected, not what they were actually seeing.

If there was intellectual honesty( so-to-speak) they would say the look the same because the DO, making the decision very difficult. They were, however, confident they had it right. because they could see a difference. THAT is what he is getting at.
Click to expand...

I understand. I agree, some of them were confident they saw real differences. As you say, their brains were filling things in. Or, their brains were taking little random differences in the portions they saw and drawing conclusions from them, or doing some other subconscious manuever. And then, they felt confident about it (some did, anyway). Perception always seems like reality, unless we have some reason to question it, because we're not aware of all the subtle unconscious filtering we're doing.

My point is that the experimenter and situation set up an expectation/demand for precisely this sort of thing to happen. This is not a naturalistic experiment, Mutual of Omaha's Wild Kingdom studying fanboys in the wild or something, it's a very artificial and deceptive context, where anyone would be really stretching it to draw inferences about anything in the real world (or in the console wars). But I suppose people will try to do that.
 
Anderson said:
No, I'm saying what I said above, that the situation was set up in a way that the subjects expected to find differences, were tasked with finding differences, and were implicitly rewarded for finding differences. And so, they "perceived" differences, which were imagined or subconsciously manufactured out of random variations. It's a common phenomenon. It goes under the labels of expectancy effect or demand effect.
Click to expand...
The important point is that expectations can affect perception. If someone believes that one is supposed to be inferior/superior, they will, to an extent, perceive them regardless of the reality

EDIT: I liked that the producer made note of the low level of scientific relevance due to sample size.
 
Frozpot said:
Ignorance is not stupidity, just lack of knowledge. I've known people that don't have a lot of academic knowledge that could think people into circles. You have to be shown what a water molecule looks like before you can draw it. Maybe they have just forgotten what it looks like because most people have zero occasion to make use of this amazing proof of intelligence. Honestly, what actual good does that knowledge do unless you are a molecular biologist?

I have a problem with this rationale. It is not a good indicator of intelligence, just training.
Click to expand...
It's not exactly ignorance if all of these people were taught in primary school what H2O is. It would be like saying you forgot how to write or forgot who George Washington is. So forgetting the formula for water is stupidity. But that's not the point. The point is we don't know if any of the participants are fit to partake in the experiment. We don't know if any of the participants are smart or dumb, but there are people out there that will ignore that fact because they don't like it and it doesn't agree with their viewpoint.
 
The funny thing is the obvious responses from certain posters.

Any unbiased multi-format gamer will see and tell the differences.
PC versions will look better in almost every comparison.
 
Kassen said:
It's not exactly ignorance if all of these people were taught in primary school what H2O is. It would be like saying you forgot how to write or forgot who George Washington is. So forgetting the formula for water is stupidity. But that's not the point. The point is we don't know if any of the participants are fit to partake in the experiment. We don't know if any of the participants are smart or dumb, but there are people out there that will ignore that fact because they don't like it and it doesn't agree with their viewpoint.
Click to expand...

They know what H2O is. They didn't all remember it was Hydrogen and Oxygen. Some of them couldn't draw it- so what. It is not in any way essential information, and they may even have dim recollections, but that isn't an indicator of how dumb they are- just that that formulaic information has been utterly unused for the majority of their lives.

I won't argue that I feel there are a lot what I see as "dumb" people out there, but this is not a good test of it. You write every day. George Washington is a recurring figure in culture, but I doubt many people know much more than that he was the first president.

Regardless, the intelligence of the participants is irrelevant. They are gamers. The biggest issue is sample size, and that is noted in the video. It's not meant as some kind of thesis- more food for thought.
 
VaLLiancE said:
The funny thing is the obvious responses from certain posters.

Any unbiased multi-format gamer will see and tell the differences.
PC versions will look better in almost every comparison.
Click to expand...
Ironic, and missing the point.

You didn't even bother watching the whole thing before chiming in.
 
Frozpot said:
They know what H2O is. They didn't all remember it was Hydrogen and Oxygen. Some of them couldn't draw it- so what. It is not in any way essential information, and they may even have dim recollections, but that isn't an indicator of how dumb they are- just that that formulaic information has been utterly unused for the majority of their lives. I won't argue that I feel there are a lot what I see as "dumb" people out there, but this is not a good test of it.

Regardless, the intelligence of the participants is irrelevant. They are gamers. The biggest issue is sample size, and that is noted in the video. It's not meant as some kind of thesis- more food for thought.
Click to expand...
Why do you say the sample size is an issue? Didn't you just that intelligence of the participants is irrelevant because "they are gamers?"
 
Frozpot said:
The important point is that expectations can affect perception. If someone believes that one is supposed to be inferior/superior, they will, to an extent, perceive them regardless of the reality

EDIT: I liked that the producer made note of the low level of scientific relevance due to sample size.
Click to expand...

Yes, I agree that expectations affect perception. That's what happened here. They expected to see differences, and they saw differences. Of course, the console warriors will say, the X1 and PS4 are actually different consoles, not the same console, but in this experiment, you only had PS4s. And they have a point. You can't really translate one situation into the other.

I didn't watch closely enough to see if, on that third trial, they said it was X1 because of some quality they perceived as inferior. I'm not sure they gave their reasons. It would be nice to have a big sample size, with everyone saying why they thought it was X or Y. Then we could get at the superior/inferior thing you're mentioning. I think that does happen a lot (confirmation bias), but I don't think we can get there from this little experiment.

The expectation thing cuts both ways, too. People expecting to not see differences will tend to not see differences.
 
VaLLiancE said:
Certain groups of gamers(worst version owners) claim they see no difference in resolutions and fps.

These groups are comical
Click to expand...

Fps is easy to see for me. Resolution is more difficult unless there are cleary more jaggies or something.
What about AF? Can you tell when a game has that implemented or not?
 
Kassen said:
Why do you say the sample size is an issue? Didn't you just that intelligence of the participants is irrelevant because "they are gamers?"
Click to expand...
Because you have to have a large enough sample group to have any kind of inkling of the accuracy of the findings. You have to be able to diminish out-liers in order for the findings to have a chance at representing that demographic as a whole. They could just be unlucky by having so many who couldn't see a difference.

Maybe only two people in the building could actually tell the difference, and they just happened to randomly be in the test, but if you just used this data, it would say that a 30 percent of people could feasibly do so, instead of an actual percentage of . 002 or something. Larger sample sizes give you a more accurate statistic.

Of course, I only took a semester of statistics, and I hated it, lol. But that's a ballpark of my understanding.
 
Anderson said:
Yes, I agree that expectations affect perception. That's what happened here. They expected to see differences, and they saw differences. Of course, the console warriors will say, the X1 and PS4 are actually different consoles, not the same console, but in this experiment, you only had PS4s. And they have a point. You can't really translate one situation into the other.

I didn't watch closely enough to see if, on that third trial, they said it was X1 because of some quality they perceived as inferior. I'm not sure they gave their reasons. It would be nice to have a big sample size, with everyone saying why they thought it was X or Y. Then we could get at the superior/inferior thing you're mentioning. I think that does happen a lot (confirmation bias), but I don't think we can get there from this little experiment.

The expectation thing cuts both ways, too. People expecting to not see differences will tend to not see differences.
Click to expand...

That's really all I was saying. I wasn't technically referring to the X1/PS4 difference. It has more to do with understanding fanboys, lol.
 
Frozpot said:
That's really all I was saying. I wasn't technically referring to the X1/PS4 difference. It has more to do with understanding fanboys, lol.
Click to expand...

I think we should fund a naturalistic study of fanboys in the wild. National Geographic's "Fanboys of the mid-Atlantic" or something.
 
  • Like
Reactions: Frozpot
Anderson said:
Yes, I agree that expectations affect perception. That's what happened here. They expected to see differences, and they saw differences. Of course, the console warriors will say, the X1 and PS4 are actually different consoles, not the same console, but in this experiment, you only had PS4s. And they have a point. You can't really translate one situation into the other.

I didn't watch closely enough to see if, on that third trial, they said it was X1 because of some quality they perceived as inferior. I'm not sure they gave their reasons. It would be nice to have a big sample size, with everyone saying why they thought it was X or Y. Then we could get at the superior/inferior thing you're mentioning. I think that does happen a lot (confirmation bias), but I don't think we can get there from this little experiment.

The expectation thing cuts both ways, too. People expecting to not see differences will tend to not see differences.
Click to expand...

I've seen comments that suggest that because AC U performs so erraticly that it can seem like three different versions at the same time. Lol that probably has some merit.
 
Frozpot said:
Because you have to have a large enough sample group to have any kind of inkling of the accuracy of the findings. You have to be able to diminish out-liers in order for the findings to have a chance at representing that demographic as a whole. They could just be unlucky by having so many who couldn't see a difference.

Maybe only two people in the building could actually tell the difference, and they just happened to randomly be in the test, but if you just used this data, it would say that a 30 percent of people could feasibly do so, instead of an actual percentage of . 002 or something. Larger sample sizes give you a more accurate statistic.

Of course, I only took a semester of statistics, and I hated it, lol. But that's a ballpark of my understanding.
Click to expand...
But none of that sample size stuff should matter if "because they are gamers." Do you see the contradiction you made? But anyways, yes, sample size matters as I have been trying to explain in this thread. Having a small sample size means statistically large variation in test subjects; meaning some could be smart, some could be dumb, some could be blind, some could be tetrachromatic, and some could just be space cadets. Having a larger sample size helps to reduce that problem and allow you to better gauge what results you would hypothetically get from sampling the majority or the entirety of your target audience. Having a small sample size means you can't assume you know anything about the participants, whether you wish to assume them to be smart or to be dumb, etc.
 
Kassen said:
But none of that sample size stuff should matter if "because they are gamers." Do you see the contradiction you made? But anyways, yes, sample size matters as I have been trying to explain in this thread. Having a small sample size means statistically large variation in test subjects; meaning some could be smart, some could be dumb, some could be blind, some could be tetrachromatic, and some could just be space cadets. Having a larger sample size helps to reduce that problem and allow you to better gauge what results you would hypothetically get from sampling the majority or the entirety of your target audience. Having a small sample size means you can't assume you know anything about the participants, whether you wish to assume them to be smart or to be dumb, etc.
Click to expand...

Sample size always matters. The demographic is the important thing, and intelligence level isn't intrinsic to that. "Gamers" encompass all kinds of people, so the IQ of the participants is irrelevant. The only requirement is that they play games and qualify as "layman", I guess. I don't see the contradiction.

I'm not sure we are in disagreement, lol.
 
Frozpot said:
Sample size always matters. The demographic is the important thing, and intelligence level isn't intrinsic to that. "Gamers" encompass all kinds of people, so the IQ of the participants is irrelevant. The only requirement is that they play games and qualify as "layman", I guess. I don't see the contradiction.

I'm not sure we are in disagreement, lol.
Click to expand...
Their "smartness" is very relevant. It's the reason why someone in this experiment would not see the difference and the reason why we shouldn't use this experiment to dismiss differences in fidelity between platforms. Because people as a whole as "dumb" and not a good measuring stick for what is noticeably different or not. Which I remember now is what I was actually trying to argue. But JimmyD was a childish distraction. Reminds me why I don't like arguing in thread like these. But yeah, it's all the same. We aren't disagreeing.
 
Last edited:
VaLLiancE said:
Very noticeable even more so when witnessed 1st hand.
Click to expand...
Arent you the same person that said that a 1080p Blu-ray disc has better screen resolution than a 1080p digital copy of the same content? Im amused.:laugh:
 
I've taken this experiment to real life and it works fab. I tell my subconcious that the X1 game I am playing is 1080p and 60fps and then it controls my mind to see exactly that. The results are glorious for my eyes, but I am still trying to figure out how to trick myself into making the gameplay fun.

:really::grin:
 
Kassen said:
Why would you ask the participant that? Since the participant is dumb, he or she will always give the wrong answer.
Click to expand...
But they didn't get it wrong so they're 1/2 as intelligent or guessing?
 
This reminds me of my college days. A girl I knew had one of those magic eye posters on the wall (those things where you try not to focus and see something in a seemingly random image). The only problem, there really was nothing in the picture. It was just random stuff. She'd have people stare at it and stare at it and most would eventually "see" something.

It isn't a flawed test, it shows that people are flawed. I was one of the few who was honest and said I could never see it. :laugh:
 
Kassen said:
Their "smartness" is very relevant. It's the reason why someone in this experiment would not see the difference and the reason why we shouldn't use this experiment to dismiss differences in fidelity between platforms. Because people as a whole as "dumb" and not a good measuring stick for what is noticeably different or not. Which I remember now is what I was actually trying to argue. But JimmyD was a childish distraction. Reminds me why I don't like arguing in thread like these. But yeah, it's all the same. We aren't disagreeing.
Click to expand...
I'm a childish distraction and you're broad-brushing these/all people as stupid providing a youtube H2O water molecule drawing video as proof?
 
VaLLiancE said:
The funny thing is the obvious responses from certain posters.
Click to expand...

Indeed.

VaLLiancE said:
Soda Jack thinks 1080p digital/streaming and 1080p Blu-Ray look the same.
Click to expand...

Where did he say "streaming"?

A Blu-ray holds a digital copy of a 1080p movie. Now, if you are trying to STEAM digital content over the internet or through On Demand boxes like DirecTV or Charter, then no question the 1080p version on those sources are going to get nailed with artifacts and internet slow down.

But if you have a digital copy of a 1080p movie that you purchased and have on your hard drive or on a USB stick or what have you, the quality will be equal to, if not better than the Blu-ray copy (better than the Blu-ray if you have a crappy Blu-ray).
 
Can we keep this on topic, please.
 
You must log in or register to reply here.
Top Bottom
Back