Gamespot PC/PS4/X1 visuals experiment. Can you tell the difference?

[Frozpot]

Their "smartness" is very relevant. It's the reason why someone in this experiment would not see the difference and the reason why we shouldn't use this experiment to dismiss differences in fidelity between platforms. Because people as a whole as "dumb" and not a good measuring stick for what is noticeably different or not. Which I remember now is what I was actually trying to argue. But JimmyD was a childish distraction. Reminds me why I don't like arguing in thread like these. But yeah, it's all the same. We aren't disagreeing.

I don't think the point has anything to do with whether or not the games themselves have "noticable" differences. It's about how expectation affects perception. When I watch the vid , I see that the first bit was more about identifying the people who can identify the differences (and kinda what percentage of that group could- which is the part where sample size is relevant in this portion's vacuum,), because the trick test would be useless on the people who couldn't tell.

The findings there have some relevancy as people who know enough to tell differences continued to see differences when there where none. The second half is the more important, imo. Without establishing those who could differentiate the versions, the last bit would fall flat.

 

[Kassen]

I don't think the point has anything to do with whether or not the games themselves have "noticable" differences. It's about how expectation affects perception. When I watch the vid , I see that the first bit was more about identifying the people who can identify the differences (and kinda what percentage of that group could- which is the part where sample size is relevant in this portion's vacuum,), because the trick test would be useless on the people who couldn't tell.

The findings there have some relevancy as people who know enough to tell differences continued to see differences when there where none. The second half is the more important, imo. Without establishing those who could differentiate the versions, the last bit would fall flat.

The point is everything to do with noticeable differences. Hence why they ask "do you see the difference? " That trick they pulled with all versions being ps4 is a part of it, but you are dismissing 90% of the experiment.

 

[Plainview]

The WiiU gets left out again...

 

[Frozpot]

The point is everything to do with noticeable differences. Hence why they ask "do you see the difference? " That trick they pulled with all versions being ps4 is a part of it, but you are dismissing 90% of the experiment.

I know?

It's not about if the games have noticeable differences- but how expectation and perception can affect how they are perceived. I'm not dismissing anything. To be effective, it has to establish who can see them in the first place.

Hell, this test even demonstrates that the term "noticeable" is a bit subjective. The difference is noticeable to some, but not to others. The interesting bit is how differences manifest in perception even when they aren't there (as evinced in the latter test).

 

[Andy]

The interesting bit is how differences manifest in perception even when they aren't there (as evinced in the latter test).

Not to beat a dead horse, but my point was that the experimenter set it up in such a way as to make that outcome nearly inevitable.

 

[Frozpot]

Not to beat a dead horse, but my point was that the experimenter set it up in such a way as to make that outcome nearly inevitable.

I'm not convinced that is the case. You can apply that logic to them needing to assign the version to one or the other, and in that aspect you are right. They "know" that there are 3 versions.

However, the fact that they said they actually saw clear differences is the point, not that they would get them wrong or right in that particular portion. The experiment is looking at two different things. The first half establishes ability, the second is about seeing what isn't there based on expectation.

In their minds, they assigned the values, or quality of what they are seeing based on PC>PS4>X1.

 

[Andy]

I'm not convinced that is the case. You can apply that logic to them needing to assign the version to one or the other, and in that aspect you are right. They "know" that there are 3 versions.

However, the fact that they said they actually saw clear differences is the point, not that they would get them wrong or right in that particular portion. The experiment is looking at two different things. The first half establishes ability, the second is about seeing what isn't there based on expectation.

In their minds, they assigned the values, or quality of what they are seeing based on PC>PS4>X1.

I don't think you're following what I'm saying. I sort of exhausted myself in my replies to Jimmy, so please see those for further explanation. In brief, the experimenter created a situation that was primed to create differences in their minds, when there were none. He did that in several different ways. Because of that, he got an outcome that anyone could have predicted -- people "perceived" differences where there were none. It's a well known problem in experimental design, usually considered under the heading of things that threaten the external validity of the experiment (that is, the ability to draw conclusions from the experiment and apply them to the real world). They are known by labels such as experimenter effects, expectancy effects, and demand effects. This was a highly artificial and manipulated scenario, and it can't be validly extrapolated to the real world.

That last sentence is your inference, but I don't think the data are there in the video to back it up. Do the people in that last trial actually say, "This version had poorer resolution, so I figured it was Xbox." I'd have to watch again, but I don't remember any of them giving reasons for their choices. (Maybe one or two them did? I can't remember.)

 

[Frozpot]

I don't think you're following what I'm saying. I sort of exhausted myself in my replies to Jimmy, so please see those for further explanation. In brief, the experimenter created a situation that was primed to create differences in their minds, when there were none. He did that in several different ways. Because of that, he got an outcome that anyone could have predicted -- people "perceived" differences where there were none. It's a well known problem in experimental design, usually considered under the heading of things that threaten the external validity of the experiment (that is, the ability to draw conclusions from the experiment and apply them to the real world). They are known by labels such as experimenter effects, expectancy effects, and demand effects. This was a highly artificial and manipulated scenario, and it can't be validly extrapolated to the real world.

That last sentence is your inference, but I don't think the data are there in the video to back it up. Do the people in that last trial actually say, "This version had poorer resolution, so I figured it was Xbox." I'd have to watch again, but I don't remember any of them giving reasons for their choices. (Maybe one or two them did? I can't remember.)

I cut to the spot for ya Those are the two guys who got it right on the first segment. The others said they couldn't tell.

 

[pravus]

If someone feels like digging it up, we could just use that 'IGN Call of Duty XB1 v PS4 720p v 1080p' thread as a case study.

The one where it was pre-final release and both versions were actually running at 720p at the time; yet IGN stated one version was 1080p.

 

[Kerosene31]

people "perceived" differences where there were none

Also known as being wrong

In all seriousness, to do this right, you'd just have to add "I can't tell" as an option. TV1 is: a) X1, b)... d) Don't know

I'd love to see that done on a larger scale with bigger screens. I bet other than people being able to find PC more consistently, you'd see the same results - people really can't tell that well. Actually I'd eliminate PC altogether and just do the two consoles.

 

[jimmyD]

If someone feels like digging it up, we could just use that 'IGN Call of Duty XB1 v PS4 720p v 1080p' thread as a case study.

The one where it was pre-final release and both versions were actually running at 720p at the time; yet IGN stated one version was 1080p.

I'm guessing plenty here went off as clearly noticing the difference?

 

[pravus]

I'm guessing plenty here went off as clearly noticing the difference?

I wish I could find the thread(s) but I just remember being amused, I hope I don't have any dumb comments in there myself.

 

[Andy]

I cut to the spot for ya Those are the two guys who got it right on the first segment. The others said they couldn't tell.

Unless I missed something, only one guy made the PC>PS4>X1 attribution. The second said she didn't see any noticeable graphical differences, just that one had a cat in it, which is funny but also helps me recognize that the clips they saw actually did have differences in them; they weren't exactly the same. The third guy said he couldn't articulate the difference and wasn't saying one was better/worse. So, really just one person explained the differences as being better/worse, along the lines we're led to expect.

Again, I'm not saying this didn't happen (people assigning imagined differences based on preconceptions about PC>PS4>X1), just that the data aren't there to support it, because the subjects don't give their reasons. Well, one does, but that's a pretty small subject pool.

 

[Oblong]

Heck, I see these side-by-side DF pics of X1 and PS4 versions clearly labeled and half the time I can't figure out which is "better", the other half I need a magnifying glass to figure out the difference. If that is what a 30-40% difference looks like then it's going to take about 100% difference for me to start caring.

 

[Kassen]

Heck, I see these side-by-side DF pics of X1 and PS4 versions clearly labeled and half the time I can't figure out which is "better", the other half I need a magnifying glass to figure out the difference. If that is what a 30-40% difference looks like then it's going to take about 100% difference for me to start caring.

Not only that, but those DF comparisons also like to slow-mo the living daylights out of it or freeze-frame on a single blown-up still image and then slowly throw the slider left and right to show X1, PS4, and PC differences. That's kind of admitting how similar these new visuals are across platforms. At that point, the game is lost. You can go no deeper in. Pixel counting has reached its ultimate height and can grow no more.

 

[TDbank24]

The Wii U gets left out again...

I've been getting requests to portray a Wii U fanboy. Not even TD can sell that sh!t. You kidding me? Half my material wouldn't even work. The Wii U is the pinnacle of consoles at 720p? Though i could sell Nintendo's first party games as being the best and make these graphics comparisons look even sillier than they already are. That could be fun. I'll think about it. People think i'm like the Will Ferrell of the console wars though. I ain't that good.

 

[Kvally]

I thought all the games on the Wii U were 1080p?

 

[Kassen]

All this talk about whether this is representative of "the average gamer." It's all foolish. Who would ever want to take pride in being "average?" I'm not like you all. I take pride in being better than you!

 

[Troublegum]

I thought all the games on the Wii U were 1080p?

Most of them are 720p.

I think these games run at 1080p...

Super Smash Bros.
Monster Hunter
Wind Waker HD
Rayman Legends


There might be a few others.

 

[TeKPhaN]

I thought all the games on the Wii U were 1080p?

No but many are 60fps.