Philosophers, Order Effects, and Framing Effects

Fiery Cushman and I have a new paper in draft, exploring the question of whether professional philosophers' judgments about moral dilemmas are less influenced than non-philosophers' by factors such as order of presentation and phrasing differences.

We recruited hundreds of academic participants with graduate degrees in philosophy and a comparison group of academics with graduate degrees in other fields. We gave them two "trolley problems" and two "Asian disease"-type framing effect cases.

We presented the trolley problems either with a Switch case first (the protagonist saves five people by Switching a runaway trolley onto a side track where it kills one), followed by a Push or Drop case (saving five by Pushing one person into the trolley's path or by Dropping him into its path), or Push/Drop first, followed by Switch. For each scenario, participants rated the protagonists' choice to kill the one person to save the five others, using a 7-point scale from "extremely morally good" to "extremely morally bad".

Our previous research suggests that non-philosophers are much more likely to judge the Push case and the Switch case equivalently when Push is presented first than when Switch is presented first. On some views of philosophical expertise, philosophers' judgments about these cases should be less dependent on order of presentation than are non-philosophers' judgments. In other words, philosophers, due to their familiarity with scenarios of this type and their expertise in applying moral principles to them, should have more stable opinions, less influenced by order of presentation. We wanted to see if philosophers with prior familiarity with the cases, or self-reported expertise in the area, or self-reported stability of opinion, would show smaller order effects. We also wanted to see if we could reduce order effects by encouraging participants to reflect carefully on different versions of the scenario and different ways of phrasing the scenarios and enforcing a delay before responding.

We were unable to find any level of expertise at which the order effects were detectably reduced. Nor did adding a reflection condition appear to reduce the order effects. This figure shows the rates at which Switch was rated as morally equivalent to either Drop or Push on the 7-point scale:

[click to enlarge]

The order effect, as indicated by the differences in height between the black and the gray bars, is basically the same for philosophers and the non-philosophers, even in the "reflection" condition.

 

This next figure breaks the results down by degree of self-reported expertise among philosopher respondents:

[click to enlarge]

Note that at no level of expertise does the order effect appear to be reduced: not among philosophers reporting being professors with a specialization in ethics, nor among philosophers reporting having a "stable opinion" about trolley problems of this sort. If anything, the trend appears to be toward larger order effects with increasing expertise.

Some of the most famous results in psychology are the Tversky-Kahneman "loss aversion" framing effects. Participants are asked to imagine that an unusual disease will kill 600 people if nothing is done and then given a choice between two programs: On Program A, 200 people will be saved. On Program B, there's a one-third probability that 600 people will be saved and a two-thirds probability that no people will be saved. When the decision is framed this way, in terms of the number "saved", most people favor the non-risky Program A. When what are (purportedly) the exact same options are presented in terms of how many will die (400 will die vs. one-third probability that none will die and two-thirds probability that 600 will die), respondents tend to favor the risky Program B.

The results: Percentage of philosopher respondents recommending the risky Program B, by framing, level of expertise, and order of presentation:

[click to expand]

As is evident from the figure, our philosopher respondents showed very large framing effects (similar to those of our comparison group and similar to the effect sizes seen in other studies with non-expert populations) -- again up to very high levels of expertise, including self-reported expertise on framing effects and self-reported stability of opinion about framing effects. To see this, look just at the black bars above, ignoring the gray bars.

Philosophers also showed large order effects when they were presented two slightly different framing-effect scenarios, either die-frame followed by save-frame or save-frame followed by die-frame. To see this, compare the adjacent pairs of black and gray bars above.

Full manuscript in draft here. Comments welcome!

[Cross-posted at The Splintered Mind]

Comments

[Zach]

Thanks for the paper. Can you please report effect sizes (preferably eta-squared or cohen's ds) and confidence intervals, at least on your main contrasts?

Thank you,
Zach

[Eric Schwitzgebel]

Hi Zach -- Sorry for the slow comment approval! I agree that effect sizes are important and CIs, especially for null effects. The article is already so loaded with numbers, we'll have to think carefully about whether to add a whole new set. For the percentage comparisons, effect sizes can be calculated by the reader with just the N's and the percentages; and for the regressions the e^betas are an effect size measure, and we put CIs around the crucial null results there. But I'll consult with Fiery. Maybe we should add a little more effect size detail.