Experimental Philosophy

Philosophical Expertise?

There is no such thing.

Why Do Experiments?

On psychologist Simine Vazire's always-excellent blog, sometimes i'm wrong, there is an excerpt from John Doris's forthcoming book that reacts to #repligate. Doris makes many important points about how philosophers should respond to this episode in psychology, such as not relying too much on any single study, including any single replication.

However, I want to take issue with one parenthetical remark. Doris writes "(Less cynically: if scientific findings weren’t surprising, why would we need experiments and publications?)". Although this may just be a throwaway remark for Doris, I actually think it might be a somewhat common thought. The thought is that experiments get some of their value from surprisingness -- i.e. disconfirming some intuitive thought. Or, put it in the reverse direction, if people were able to reliably predict whether an experiment would confirm or disconfirm a commonsense belief, then there would be less reason for us to do such an experiment.

I don't think that's the right view about experiments. Duncan Watts's book Everything is Obvious gives an easy way to see why. As Watts points out, many of our commonsense beliefs appear in tension with one another. Absence makes the heart grow fonder. But out of sight is out of mind. So you might imagine that, if the result of an experiment went one way, it'd be framed as "Counter-intuitive finding! Absence does not make the heart grow fonder!" for publication, but if the result of the same experiment went the other way, it'd be framed as "Counter-intuitive finding! Out of sight is not out of mind!" for publication.

Given that many of our commonsense beliefs appear in tension with one another in this way, whether an experiment is counter-intuitive or not hardly has any connection to its value. Instead, as Watts points out, what is valuable about scientific experiments is that they can delineate the scope of such commonsense beliefs. And I might add, they can estimate the magnitude of the highlighted causal relationships. From a scientific perspective, it's not that interesting to find out whether "out of sight is out of mind" is true or not, but it is interesting to find out in which cases it holds and in which case it doesn't, and to what extent it holds.

I think this view of why do experiments actually coheres with some important methodological lessons that came out from #repligate. There should be less emphasis on the yes/no question of whether p is above or under 0.05. Instead, there should be more emphasis on effect sizes, which measure the magnitude of a causal relationship, including giving confidence intervals for effect sizes. Moreover, there should be more emphasis on "scope" -- covering aspects ranging from moderating factors to conditions under which the effect is expected to replicate. For example, as I've mentioned before on this blog, psychologist Dan Simons suggests that publications of studies should come with an explicit method section on "Limits in scope and generalizability". At the same time, Many Labs 2 and Many Labs 3 are now investigating, respectively, replication across sample and setting and replication across timing of experiments.

Why do experiments? The answer has nothing to do with intuitiveness, but everything to do with magnitude and scope of causal relationships that are of interest to some of our aims.

[x-posted at Experimental Philosophical Aesthetics]

Folk Teleology and Persistence: Debunking Folk Intuitions and Endorsing the Expertise Defense

How do ordinary material objects persist?  For instance, if a rock is smashed to pieces with a hammer, does it survive the smashing?  Or, if a rock is hit with a hammer and chipped, does the rock survive?  What about a rowboat that has had all of its original parts replaced over the years?  Does it survive the complete replacement of parts?  What if another rowboat was built with the original parts? Which rowboat is the original, the one with all of its parts replaced or the one built from the original parts?

In a forthcoming paper, I take up the issue of how the folk determine whether a material object persists through alterations.  Across a range of cases, I find that the folk operate with a teleological view of material object persistence in that they tend to judge that an object survives alterations when its function is preserved and tend to judge that an object has been destroyed when its function is destroyed.  Indeed, I find that teleological considerations infuse persistence judgments for artifacts, organisms and non-living natural things like rocks. 

Given that the folk operate with a teleological view of material object persistence, I argue that there is a debunking explanation for folk intuitions of material object persistence, concluding that metaphysicians should dismiss folk intuitions as tied into a benighted view of nature.  But I go further.  I draw on some evidence suggesting that philosophers do not operate with a teleological view of material object persistence and invoke an expertise defense.  In the specific case of material object persistence, I hold that the folk suffer from a benighted view which is deserving of dismissal while the metaphysician enjoys in a more enlightened perspective.  The prescriptive metaphysician should be liberated from any demanded conformity with folk intuitions of material object persistence.

Philosophical Expertise

Just wanted to make a quick post announcing the publication of my Philosophy Compass article on philosophical expertise.  You can find it here.  Hopefully it's of some interest!

What Experimental Philosophers Actually Do: A Quantitative Study

These past few years have seen a surge of metaphilosophical work exploring the implications (or lack thereof) of the empirical studies being conducted in experimental philosophy. Much of this metaphilosophical work is being done by people who are not experimental philosophers themselves but who simply want to address the larger philosophical questions that certain kinds of empirical research can raise. This development is very welcome indeed, and I have gotten a lot out of reading many of these contributions. 

 

However, I do have some misgivings about one aspect of this work. Typically, papers in this tradition start out by (a) briefly describing the sort of empirical research that experimental philosophers do and then proceed to (b) engage in a discussion of the larger philosophical implications of such research. In my experience, the discussion of philosophical implications usually shows a truly impressive degree of clarity and rigor, but the description of the actual research can sometimes be a bit inaccurate. (Thus, there is a real danger that one will end up with a impressively rigorous analysis of the implications of something that experimental philosophers don't actually do.) 

 

To address this issue, I thought it might be helpful to conduct a quantitative study that would explore the question as to what experimental philosophers have actually been doing. 

 

How to Better X-Phi

Many readers of this blog have probably already seen the latest issue of Emotion Review, which features Nina Strohminger's review of Colin McGinn's book on disgust. There is also a terrific set of commentaries that expand on the themes prompted by Strohminger's review.

In particular, David Pizzaro's commentary is focused on the challenges of interdisciplinary research. Pizzaro rightly joins Strohminger in excoriating McGinn's data-free approach to theorizing disgust, and noting the essentiality of data to making scientific claims. Pizarro also rightly emphasizes the importance of intellectual humility in doing interdisciplinary research. Toward the end of the commentary, Pizarro writes,

Philosophers interested in empirical questions about the mind, but who have not themselves been trained in experimental methods or statistical analyses, are more likely [...] to selectively report results that support their argument [...]

I find this claim curious. Is it the case that experimental philosophers who lack formal background in experimental methods or statistical analyses are more likely to selectively report results that support their argument?

Given the recent discussions regarding replication and research practices in psychology, what can experimental philosophers do to avoid being seen by other philosophers and psychologists as p-hackers? The replication project in X-Phi is a good start. Perhaps we should be p-curving ourselves more? Perhaps we should be registering our projects on Open Science Framework?

What other measures do you think would be helpful toward a better x-phi?

New paper on the philosophical role of x-phi

The post is to point readers to a forthcoming article of potential interest: 'X-phi and Carnapian explication'. I co-authored it with James Justus. In the paper we try to bring together experimental philosophy and Rudolph Carnap’s notion of explication. The result is a view on the philosophical value of experimental philosophy that departs in certain ways from both the ‘negative program’ and the ‘positive program,’ as those labels are normally applied.

The paper is archived at PhilPapers, and should be out in Erkenntnis soon:

http://philpapers.org/rec/SHEXAC

Here’s the abstract:

The rise of experimental philosophy (x-phi) has placed metaphilosophical questions, particularly those concerning concepts, at the center of philosophical attention. X-phi offers empirically rigorous methods for identifying conceptual content, but what exactly it contributes towards evaluating conceptual content remains unclear. We show how x-phi complements Rudolf Carnap’s underappreciated methodology for concept determination, explication. This clarifies and extends x-phi’s positive philosophical import, and also exhibits explication’s broad appeal. But there is a potential problem: Carnap’s account of explication was limited to empirical and logical concepts, but many concepts of interest to philosophers (experimental and otherwise) are essentially normative. With formal epistemology as a case study, we show how x-phi assisted explication can apply to normative domains.

Jennifer Nagel and Joshua Alexander on Philosophy TV

Readers of Experimental Philosophy might be interested in this episode of Philosophy TV in which Jennifer Nagel and Joshua Alexander discuss the evidential role of intuitions in philosophy (particularly, epistemology).

I discuss one argument that Nagel makes during this conversation, which I think is unsound, over at The Philosophers’ Cocoon. I would be interested in learning what readers of Experimental Philosophy make of this argument.

In the Thick of Moral Motivation

Suppose you had a genuine moral belief about something, and all that abiding by it required was pushing a little blue button on your desk. It would literally cost you nothing to do it, other than lazily lifting up your arm and tapping it with your index finger. Most people would probably press the button. After all they believe that's the right thing to do. On the other hand, some people might not press the button. You can probably imagine all kinds of scenarios where you fail to do it even though you think it's right. Other concerns enter in. Maybe you get busy and forget all about it, or maybe someone offers you a winning lottery ticket to resist.

But here's the question: in either case, could you genuinely hold that moral belief about the button, and at the same time, have absolutely no motivation (I'm talking no tinge, no inkling, not a scintilla of motivation) to push it? Internalists say no. It’s just part of our concept of moral judgment that motivation (even if really weak or totally defeasible) accompanies it. Externalists say yes. Any connection between moral judgment and motivation is contingent. This is the debate about moral motivation.

The debate has persisted for decades and remains still undecided. Perhaps it's just too easy to feel the basic intuitive pull both for and against a necessary motivational component. We can see a sense in which the belief about the blue button implies some sort of motivation, and with minimal effort, can also begin to a see a sense in which it doesn't. As one observer notes, at this point philosophers attempt to produce "additional cases" and "probe our intuitions further in an effort to overcome what seems an impasse.” And, experimental philosophers have heard this call too, with influential experimental work supporting externalist intuitions and, lately, some to question it.   

In a new paper with John Turri we continue this trend. But rather than just collect intuitions about individual cases, our ultimate goal is to try and understand why there has been so much disagreement about moral motivation in the first place. In our paper we suggest that the division among philosophers actually derives from a fault line familiar to us from everyday life. Specifically, we present some evidence that the persistent disagreement between internalists and externalists can be sourced to a more general phenomenon with deep origins in the folk psychology of belief.

How Subtly Do Philosophers Analyze Moral Dilemmas?

You know the trolley problems. A runaway train trolley will kill five people ahead on the tracks if nothing is done. But -- yay! -- you can intervene and save those five people! There's a catch, though: your intervention will cost one person's life. Should you intervene? Both philosophers' and non-philosophers' judgments vary depending on the details of the case. One interesting question is how sensitive philosophers and non-philosophers are to details that might be morally relevant (as opposed to presumably irrelevant distracting features like order of presentation or the point-of-view used in expressing the scenario).

Consider, then, these four variants of the trolley dilemma:

Switch: You can flip a switch to divert the trolley onto a dead-end side-track where it will kill one person instead of the five.

Loop: You can flip a switch to divert the trolley into a side-track that loops back around to the main track. It will kill one person on the side track, stopping on his body. If his body weren't there to block it, though, the trolley would have continued through the loop and killed the five.

Drop: There is a hiker with a heavy backpack on a footbridge above the trolley tracks. You can flip a switch which will drop him through a trap door and onto the tracks in front of the runaway trolley. The trolley will kill him, stopping on his body, saving the five.

Push: Same as Drop, except that you are on the footbridge standing next to the hiker and the only way to intervene is to push the hiker off the bridge into the path of the trolley. (Your own body is not heavy enough to stop the trolley.)

Sure, all of this is pretty artificial and silly. But orthodox opinion is that it's permissible to flip the switch in Switch but impermissible to push the hiker in Push; and it's interesting to think about whether that is correct, and if so why.

Fiery Cushman and I decided to compare philosophers' and non-philosophers' responses to such cases, to see if philosophers show evidence of different or more sophisticated thinking about them. We presented both trolley-type setups like this and also similarly structured scenarios involving a motorboat, a hospital, and a burning building (for our full list of stimuli see Q14-Q17 here.)

In our published article on this, we found that philosophers were just as subject to order effects in evaluating such scenarios as were non-philosophers. But we focused mostly on Switch vs. Push -- and also some moral luck and action/omission cases -- and we didn't have space to really explore Loop and Drop.

About 270 philosophers (with master's degree or more) and about 670 non-philosophers (with master's degree or more) rated paragraph-length versions of these scenarios, presented in random order, on a 7-point scale from 1 (extremely morally good) through 7 (extremely morally bad; the midpoint at 4 was marked "neither good nor bad"). Overall, all the scenarios were rated similarly and near the midpoint of the scale (from a mean of 4.0 for Switch to 4.4 for Push [paired t = 5.8, p < .001]), and philosophers and non-philosophers mean ratings were very similar.

Perhaps more interesting than mean ratings, though, are equivalency ratings: How likely were respondents to rate scenario pairs equivalently? The Loop case is subtly different from the Switch case: Arguably, in Loop but not Switch, the man's death is a means or cause of saving the five, as opposed to a merely foreseen side effect of an action that saves the five. Might philosophers care about this subtle difference more than non-philosophers? Likewise, the Drop case is different from the Push case, in that Push but not Drop requires proximity and physical contact. If that difference in physical contact is morally irrelevant, might philosophers be more likely to appreciate that fact and rate the scenarios equivalently?

In fact, the majority of participants rated all the scenarios exactly the same -- and philosophers were no less likely to do so than non-philosophers: 63% of philosophers gave identical ratings to all four scenarios, vs. 58% of non-philosophers (Z = 1.2, p = .23).

I find this somewhat odd. To me, it seems pretty flat-footed a form of consequentialism that says that Push is not morally worse than Switch. But I find that my judgment on the matter swims around a bit, so maybe I'm wrong. In any case, it's interesting to see both philosophers and non-philosophers seeming to reject the standard orthodox view, and at very similar rates.

How about Switch vs. Loop? Again, we found no difference in equivalency ratings between philosophers and non-philosophers: 83% of both groups rated the scenarios equivalently (Z = 0.0, p = .98).

However, philosophers were more likely than non-philosophers to rate Push and Drop equivalently: 83% of philosophers did, vs. 73% of non-philosophers (Z = 3.4, p = .001; 87% vs. 77% if we exclude participants who rated Drop worse than Push).

Here's another interesting result. Near the end of the study we asked whether it was worse to kill someone as a means of saving others than to kill someone as a side-effect of saving others -- one way of setting up the famous Doctrine of the Double Effect, which is often evoked to defend the view that Push is worse than Switch (in Push, the one person's death is arguably the means of saving the other five, in Switch the death is only a foreseen side-effect of the action that saves the five). Loop is interesting in part because although superficially similar to Switch, if the one person's death is the means of saving the five, then maybe the case is more morally similar to Push than to Switch (see Otsuka 2008). However, only 18% of the philosophers who said it was worse to kill as a means of saving others rated Loop worse than Switch.

[Cross-posted at The Splintered Mind.]