Survey on mental imagery or more information go to: tstbl.co/793-242#neuroskyence#visionscience#psychscisky

Be careful with ordinal data! I simulated within-subject experiments with Likert responses and compared the p-value from a naive t-test, Wilcox test, and ordinal regression. T-tests and Wilcox had way more "significant" results than an ordinal regression. Probably have to make a blog post. #stats

That last sentence sounds almost like a threat 😉

Also, in many of our studies I suspect we often test the least noisy subjects first. The 3 black dots in the scatterplots are non-naive observers. We didn't see any obvious difference in noise here but it can certainly be the case. We tested with & without them to ensure this doesn't affect result.

Yeah, correlations are often very high with small N but then converge down as sample grows. After all, it's not hard for 3 points to lie in a fairly straight line. But of course the BFs would be near 1 for that. But I was a bit worried we'd hit our BF10>10 criterion at n=6 already 😉

But we should probably add the scatter plots to our appendix in a revision.

I was surprised by that too at first, but you got to keep in mind this is psychophysics. Not actually that surprising to get such strong correlations. What surprised me more is that correlation between Ebbinghaus & Delboeuf is almost as strong as test-retest of Ebbinghaus.

Perhaps once you actually watch it, that can be true. I just find I don't have the patience to watch it in the first place. Also, frequently this happens when I'm somewhere I cannot simply watch an instruction video, like on the train or somewhere else in public 😉

More importantly, the stimulus difference is also not the biggest problem. Our new data at least suggest that the task makes a huge difference. Fast, brief stimuli in forced-choice task only share relatively small amount of variance with unconstrained tasks - and especially with adjustment.

Definitely. Although I actually offered to send them the code (in fact, may even have sent it). They could have spotted the problem if they had run/looked at it... But I'm not bitter. The past two months have taught me that all this matters very little in the big scheme of life.