Experimental Philosophy

Mturk Now More Expensive

As many readers have probably heard, Amazon's Mechanical Turk will soon (July 22, 2015) be charging substantially more for its services. Before, Mturk took 10% of what researchers ("requesters") paid workers. Now it will be effectively 40% (in the rare case that you need fewer than 10 distinct participants, then it's only 20%).

Naturally, some researchers are upset, as documented at The Chronicle of Higher Education ("Researchers Complain About Changes in Amazon Tool Used for Surveys").

Two quick downsides that stand out to me:

(1) Running even a small x-phi study on Mturk will more likely require grant money, which is scarce in the humanities.

(2) An increase in operating costs will probably motivate researchers to still get the same mileage out of their funds. This makes it even less likely that workers will get paid a living wage, which is an ongoing ethical issue with use of the site.

Any reactions from the x-phi community?

(Note: If interested, this blog recently featured an excellent post on the status of Mturk as a research tool.)

FREE (!!!) Statistical Consulting

Have you ever wondered how you can calculate 95% confidence interval for effect sizes in SPSS? Have you ever wondered how many participants you need to achieve 80% power for a 2 IV SEM in Tetrad? Well, I have. And when I had those questions, I really wished there was someone I can easily turn to who will give me an answer for free. One can dream, right?

On some US campuses, the statistics department offers pro bono consulting. However, such services are by no means universal. Luckily, the Center for Open Science has just started a free (yes, really) statistical consulting service for everyone!

Check out their webpage here: http://centerforopenscience.org/stats_consulting/

I'm gonna write in a question now. You should too!

How to Better X-Phi

Many readers of this blog have probably already seen the latest issue of Emotion Review, which features Nina Strohminger's review of Colin McGinn's book on disgust. There is also a terrific set of commentaries that expand on the themes prompted by Strohminger's review.

In particular, David Pizzaro's commentary is focused on the challenges of interdisciplinary research. Pizzaro rightly joins Strohminger in excoriating McGinn's data-free approach to theorizing disgust, and noting the essentiality of data to making scientific claims. Pizarro also rightly emphasizes the importance of intellectual humility in doing interdisciplinary research. Toward the end of the commentary, Pizarro writes,

Philosophers interested in empirical questions about the mind, but who have not themselves been trained in experimental methods or statistical analyses, are more likely [...] to selectively report results that support their argument [...]

I find this claim curious. Is it the case that experimental philosophers who lack formal background in experimental methods or statistical analyses are more likely to selectively report results that support their argument?

Given the recent discussions regarding replication and research practices in psychology, what can experimental philosophers do to avoid being seen by other philosophers and psychologists as p-hackers? The replication project in X-Phi is a good start. Perhaps we should be p-curving ourselves more? Perhaps we should be registering our projects on Open Science Framework?

What other measures do you think would be helpful toward a better x-phi?

Big Data Aesthetics?

Experimental philosophers aim to use empirical methods to inform philosophical enquiries. Since many of the early, prominent, influential works make heavy use of the contrastive vignette technique, it is easy -- especially for those already hostile to experimental philosophy -- to think that experimental philosophers don't do anything but CVT studies.

That's a mistake. Methodologically, experimental philosophers are a heterogeneous bunch. Meskin, Phelan, Moore, and Kieran conducted an experiment that involved visual priming. Chandra Sripada used structural equation modeling to shed new light on the side-effect effect. And, to come back to one part of the title of this blog post, Nicole Hassoun and collaborators harnessed the "big data" from the micro-lending institution Kiva to uncover how people think about distributing aid.

Onto the other part of the title: aesthetics. Can "big data" inform philosophical aesthetics?

Before that question can be answered, we need to consider another, even more pressing question. Where can such data come from?

Any proposed "big data" study faces a difficult challenge right from the start: getting the data. The kind of information that is most accessible, such as the ones that can be gleaned from art history textbooks, is also inevitably heavily biased toward the relatively few iconic artworks. Datasets constructed this way would be unlikely to give a fully informative picture of the artworld. So when it comes to highly abstract philosophical questions, say about the nature of art, such datasets seem to be only of limited use.

Where else can we look? Thankfully, as I learned from the wonderful blog Museum 2.0, many museums -- notably the Tate -- are now making datasets of their collections openly accessible. Although the Tate dataset has only been available since the beginning of this month, there are already fascinating, if preliminary, findings from it. (Of course, researchers also encountered some serious problems in the mean time: notably, not all objects in an art museum's collection are artworks.)

As a first-pass reaction, there seems to be much that can be learned from "big data aesthetics". What do you think? Which questions in philosophical aesthetics, on your view, would be most amenable to empirical investigations using the museum collection datasets?

[x-posted at Experimental Philosophical Aesthetics]