“The Heart Attack” Season 2, Episode 10

Lovable anti-hero George Costanza is having a salad for lunch when he suddenly clutches at his chest and declares, “I think I’m having a heart attack!” His companions, Jerry and Elaine, seem remarkably unconcerned with George’s seemingly life-threatening predicament. As George breathlessly lists off the symptoms he’s experiencing (Tightness… Shortness of breath… Radiating waves of pain!), Jerry arrives at a less catastrophic diagnosis: “I know what this is. You saw that show on PBS last night, Coronary Country.” Turning to Elaine he explains, “I saw it in the TV Guide. I called him and told him to make sure and not watch it.”

George isn’t faking a heart attack – he’s merely imagining the symptoms after hearing about them on TV. Apparently this isn’t the first time he’s done this. As Jerry tells it, “he saw that show on anorexia last year – ate like an animal for two weeks. […] I could have predicted this to the minute.” He may have been on to something. New research, published this year in the Journal of Psychosomatic Research reveals that this is not a Costanza-specific neurosis.

Photo: Tau Zero

Michael Witthöft and G. James Rubin conducted an experiment in which they exposed 147 participants to 15 minutes of a fake Wi-Fi signal and then measured how many symptoms (such as headaches or tingling sensations) they reported experiencing. Before the fake Wi-Fi, though, half of the participants were randomly selected to watch a short documentary about the potential health effects of exposure to Wi-Fi signals that had been previously aired on the BBC; the other half watched a similar documentary about internet security. Despite the fact that there is no solid evidence supporting any of the symptoms, 54% of people who had watched the film about health risks reported experiencing symptoms (a significantly higher proportion than among those who watched the other film). In short, watching the film made people imagine the symptoms. What’s more, George may have made a particularly easy target – according to the research, people who were generally high in anxiety were much more vulnerable to experiencing symptoms following the movie.

Another factor working against George was that heart attack symptoms – tightness, shortness of breath, and pain – are all easy to imagine. Research conducted in the 1980s by Steven Sherman found that people who were asked to imagine the symptoms of a disease thought they were more likely to contract it, but only when those symptoms were easy to imagine, like fatigue or aching muscles. When the symptoms were more abstract, like an inflamed liver or a malfunctioning nervous system, people thought they were less likely to contract it. Broadly speaking, we have a tendency to think that things are more likely to be true the easier it is for us to imagine them (Kahneman and Tversky call this the availability heuristic [PDF]). When we see symptoms vividly illustrated on TV, it just makes it that much easier for us to imagine having them.

Photo: Darren Cullen

Imagined symptoms – also known as psychosomatic symptoms – are not only experienced after watching television shows. They are also common after hearing about the side effects of medicines, a phenomenon known as a nocebo effect (the opposite of the better-known placebo effect). Nocebo effects pose a particular conundrum for doctors who, while they have an obligation to be honest with their patients about the possible effects of a drug, also want to avoid unnecessarily increasing the risk of symptoms. The question for them is: do you tell a patient about a rare side effect of their medication when you know that telling them makes it more likely that they’ll experience it – or do you keep it a secret? The answer is not an easy one. The New Yorker’s Gareth Cook suggests that doctors could get patients’ permission not to tell them about minor side effects.

Still, this solution doesn’t help the George Costanzas of the world who are intent on watching medical documentaries. The media should certainly be responsible in their reporting and avoid sensationalizing or exaggerating possible concerns – local news reports explaining the countless way you could be killed by ordinary household objects probably don’t help – but even if they stick strictly to the truth, our imaginations are likely to run wild.

[Watch the video]

Paul Rozin

I’m not sure how it’s possible that until yesterday I had never seen Paul Rozin speak. However it happened, I corrected a huge mistake by going to see him give an invited address at the Midwestern Psychological Association meeting in Chicago titled, The Aesthetics and Pleasures of Temporal Sequences. The talk spanned far more than that topic, but as Rozin’s research predicts, it was made extremely memorable by ending with a bang.

If you don’t know Rozin’s work you should absolutely check it out. Among many other things, his work helps us understand how disgust and morality are contagious (people don’t want to drink out of a sterilized glass if they know there’s been a cockroach in it (PDF), and won’t wear a sweater that had been owned by Hitler), how the process of moralization (PDF) works (how preferences become values), and why people come to consider some land to be sacred (PDF).

The research is not only interesting psychologically, it also has important real world implications. For example, Rozin is helping to figure out how to encourage people to overcome their disgust with drinking purified sewer water in regions facing water shortages, and to become more comfortable supplementing their diet with nutrition-rich insects. I’m still trying to decide how I would feel about a delicious-looking chocolate chip cookie made using 2% ant flour (given that, for cookies, I’m very liberal with the 3-second rule, I’m guessing I’d be ok with it).

Rozin is an extremely endearing iconoclast with as many interesting, meaningful, and important ideas about human psychology and behavior as anyone I can think of. He has very little patience for the rules that most psychologists fall in line with, which helps him to be incredibly generative, but often underappreciated. This works out fine for me as a rapt spectator, but may sometimes be more problematic for a graduate student who desperately needs publications.

In yesterday’s talk Rozin illustrated how fads come to dominate psychological research, which has the consequence of leaving huge swathes of psychology underexplored. He also commented on the state of publishing in the field, concluding that pre-publication peer review may not, on balance, be worth the costs it imposes, preferring open access publishing instead (putting him on the same page as a growing number of reform-minded researchers). He also noted, provocatively, that while there’s nothing wrong with hypothesis-based research, some of the great discoveries in science did not start with a clear hypothesis. Sometimes, he argued, journals should be far more liberal about publishing the mere existence of an interesting effect, even when it doesn’t yet have a good explanation, in order to let others take a shot at providing one.

Noma’s live ants. Photo: Carl Reinholdtzon Belfrage

For me, though, the best part of the talk was when Rozin turned his attention back to the original theme, discussing how the sequences in which we experience things like food and music can have a profound effect on how we enjoy them. Unlike most academic talks, Rozin treated his audience to a photographic tour of incredible meals he had enjoyed at cutting edge restaurants around the world, including Denmark’s Noma and Spain’s El Bulli. On the menu: live ants, cantaloupe caviar, and sushi accompanied by an aerosolized ginger mouth spray.

Rozin’s research, building on Danny Kahneman and colleagues’ work on the peak-end rule (PDF) and duration neglect (PDF), suggests that Americans’ food preferences are not well suited to creating optimal memories of dining experiences. Unlike many other nations, Americans’ favorite foods tend to come in large portions in the middle of meals as a main course. Rozin finds (PDF), however, that people generally best remember the first and last things they eat and that, in retrospect, they are surprisingly insensitive to the amount of their favorite food that they actually consume.

Rozin also shared some fascinating ideas and findings about the importance interruptions. In one line of work, done in collaboration with Brian Wansink, Rozin finds that mindless eating can be cut dramatically by a simple interruption. People who are eating potato chips out of a tube eat half as many chips when every tenth chip is dyed red (PDF).

Rozin also described work on a different aspect of interruptions that he did in collaboration with his son, Alexander Rozin, a music theorist (with a background in astrophysics!). Rozin noted that symphonies have an intermission, thereby dividing the audience’s experience into two parts and providing them with twice as many opportunities to experience a memorable beginning and end. Bringing us back to food, he explained how at El Bulli, half way through their 34-course meal, his party, that had initially been seated outdoors, was moved inside. It made me wonder whether something as simple as changing the tablecloth, or asking people to change their seats, might be enough to create a similarly powerful interruption.

If the talk had ended there I would have left with the memory of a great experience. But in keeping with his own advice, the session ended on a high note in the question and answer period. A man in the front row raised his hand and wondered whether, like meals and symphonies, our experience of sex should be thought of in a similar way because it’s also an experience that ends with a bang. Before Rozin could answer, though, a woman seated behind me interjected dejectedly: “Sometimes.”

In case you’ve never had a chance to see Rozin speak either, here he is addressing the MAD symposium, the conference organized by Noma chef Rene Redzepi.

Diederik Stapel (Photo Credit: Jack Tummers)

Along with many other psychologists, I’ve been closely following (and participating in) the ongoing discussion about finding ways to effectively improve the shortcomings in our field’s research methods. Given that the Stapel fraud case was an important spark to these discussions, I read Yudhijit Bhattacharjee’s article, The Mind of a Con Man, in this week’s New York Times Magazine with great interest.

Bhattacharjee paints a very humanizing portrait of Stapel and the struggles that those around him have had coming to terms with his fraud. He certainly doesn’t let Stapel off the hook, but I was disappointed that the story essentially conflates Stapel’s fraud with the sins of many other psychologists, with the difference being only a matter of degree. In doing so, it unfairly portrays many honest psychologists and presents a simplistic – and, in my opinion, deeply misguided – understanding of the real problems in psychology.

Although he calls it a “cynical point of view,” he suggests that actions like Stapel’s are on the same “continuum of dishonest behaviors that extend from the cherry-picking of data to fit a chosen hypothesis — which many researchers admit is commonplace — to outright fabrication.” I could not disagree more strongly. I will be among the first to acknowledge that current practices in psychology are in need of improvement. Some psychologists make mistakes when they should know better; others do know better but find ways to cut corners through excuse making or self-deception. But fabricating data crosses into an entirely different territory.

In Bhattacharjee’s story, Stapel explains that it is, “The extent to which I did it, the longevity of it, [that] makes it extreme. […] Because it is not one paper or 10 but many more.” Sorry, Diederik. That’s not what makes it extreme. Adding a post-hoc covariate or dropping an inconvenient outlier is a questionable research practice – one that should be eliminated – but making up fake data? That’s fraud.

Ultimately, I agree with Bhattacharjee that fraud “might represent a lesser threat to the integrity of science than the massaging of data and selective reporting of experiments.” I’m grateful that there are a growing number of scientists and scientific organizations who are making methodological reform a priority. As I’ve written previously, we need to make sure that at the very least psychologists recognize the effects that some of these practices can have. But that still doesn’t place most psychologists on the same continuum as Stapel. It’s a very big step from reporting unplanned analyses of real data to fabricating fake data from studies that were never conducted. Pretending like these are more or less the same thing is not only misleading, but it impedes efforts at reform by putting on the defensive those people whose support is critical if the reforms are to succeed.

In an article full of self-incriminating quotations, when it comes time to accuse psychologists of deliberately falsifying their data, suddenly  Bhattacharjee’s sources go silent, and no evidence of this damning indictment is presented besides what “several psychologists” told him. I find it ironic that immediately after accusing an entire field of acting in bad faith in the “pursuit of a compelling story no matter how scientifically unsupported it may be,” he is guilty of doing precisely that himself.

Important!

Image courtesy of Alex de Carvalho

There is an academic achievement gap in the United States. Compared to their White peers, African American and Latino American students earn lower grades and are more likely to drop out of school. Compared to “the haves”— that is, students with greater economic means – “the have nots” also have lower grades and higher drop-out rates [1]. These achievement gaps are there when students come to school in the fall and widen as the year goes on. Recently, a small intervention, aimed at easing the psychological burdens that impair minority performance, has been found to interrupt this downward trajectory, improving the performance of minority students, narrowing the achievement gap, and with long lasting effects. This month, a new paper [2] in the Journal of Personality and Social Psychology, reports findings from two field experiments led by David Sherman of the University of California, Santa Barbara, along with eight collaborators*, that replicate this pattern of results among Latino American middle school students, helping to better understand how the intervention works and why it has its long lasting effects.

The Psychological Roots of the Achievement Gap

For most students, middle school is hard enough as it is, but for minority students it can present an added set of psychological obstacles. They have to take the same tests as everyone else but, if they perform poorly, they risk confirming negative stereotypes about their group. This phenomenon, known as stereotype threat adds a cognitive burden that can lead to underperformance [3, 4]. In school settings stereotype threat tends to affect minorities as well as women in math and science but, more broadly, it can affect anyone (including white men doing math while being compared to Asian students [5]). Worse still, the process can build on itself recursively.
First, stereotype threat impairs performance; in turn, poor performance increases the threat, which then further depresses performance. This recursive pattern helps explain the downward trajectory in students’ grades over the course of the year. The goal, then, is to intervene early and interrupt this pattern. Geoffrey Cohen, also a co-author of the new research, has been able to do just that with African American middle school students [6, 7]. This new research, led by David Sherman, replicates Cohen’s findings with a population of Latino American students, a group that had received relatively little attention in stereotype threat research, despite its growing numbers. Using student diaries, the new research was also well positioned to provide insight into how the intervention affects students in their daily lives.

Sherman and his team ran separate field experiments in two middle schools, each of which had a substantial Latino American population integrated. One school had a substantial economic disparity between White and Latino American students. In both schools, the intervention improved core course grades (Science, Social Studies, English, and Math) among Latino American students by the end of the school year by an average of roughly 0.3 points (on a 4.33 scale), reducing the achievement gap by 20-30% (the intervention had no effect on the White students’ grades). The intervention also sharply reduced the downward performance trajectory. Moreover, students in one school were followed for two years after the intervention had been completed and its effect on their grades persisted, even as some students made the difficult transition from middle school to high school.

The Intervention

At each of the schools, the intervention was introduced early in the school year, when it had the best chance of interrupting the recursive cycle. The intervention itself was administered by teachers, as the researchers were careful to avoid presenting it as an intervention but rather as a regular part of school. Teachers remained unaware of the students’ randomly assigned condition by handing out sealed envelopes containing the materials.
Inside each envelope were instructions for a writing exercise**. All students were presented with a list of eleven values such as, “being creative,” “being with friends and family,” and, “being good at sports.” Half the students were in a control condition and were asked to write a few sentences about why some of the values that were not important to them might be important to someone else. The other half of students instead wrote about the values they considered most important and why. This sort of writing exercise is known as a self-affirmation. Although the name may evoke self-help, the effectiveness of the exercise has been replicated dozens of times since first introduced by Claude Steele in the 1980s [8, 9, 10]. Affirmations reduce physiological stress and make people more open to uncomfortable information about things like their health and their political beliefs [9].
As David Sherman put it, “When you look at what the students write, you see that they are generally not boosting their egos or self-aggrandizing. What they do is remind themselves about who they are, and what is important to them. They are reaffirming a narrative about themselves that they are okay people who have core values that will be with them through the ups and downs of school. And this helps the students see threatening events from a broader perspective, and these threats become less of a stressor and less disruptive of their academic motivation and efficacy.” [11]

In the second of the studies, in addition to the affirmation exercise, students in both conditions (affirmed and unaffirmed) also completed a series of surveys and took part in a writing exercise at six different time points. They wrote about their experiences with daily adversity (e.g., “Today I feel stressed out at school.”), identity threat (e.g., “Today in school, I am worried that other people might judge me based on my race.”), and their sense of academic fit (e.g., “Today, I really feel like I belong at [my school].”). Variants of these structured writing assignments were given throughout the year. With the cooperation of teachers and administrators, they were developed to be intelligible and impactful for the students at the specific schools.

How it’s working

Because Latino American students must contend with the threat of being negatively stereotyped, they are also likely to be particularly vigilant to this possibility. As Claude Steele recently put it, “When you feel under threat, you know that based on an identity you have, something bad could happen. You don’t know whether in fact it will happen. You don’t know precisely what could happen or when or where it could happen […] It’s like having a snake loose in the house. It’s a terrible feeling. When you are in this situation, most of your cognitive resources are devoted to vigilance.” While increased vigilance is a rational response to a potential threat, it also carries a cost. In addition to being cognitively taxing, which can impair performance, vigilance also narrows people’s focus. When in a state of vigilance, people tend to neglect long term goals in favor of attending to the present, and are more likely to see threats in ambiguous situations than they might otherwise be. The affirmation, then, may be effective because it helps threatened students take a step back and see things from a broader, and perhaps less vigilant, perspective.

One indication that this may be happening is that, when affirmed, Latino American students report seeing things from a more abstract perspective. In a survey, students are asked to answer questions that identify whether they are seeing things more abstractly or concretely. For example, ringing a doorbell can be seen concretely as “pushing a button,” or abstractly as “seeing if someone is home.” Latino American students in the affirmation condition saw significantly more of the items in an abstract way than those in the control condition (66% vs. 51%).

The students’ writing exercises provide additional evidence supporting this view. When unaffirmed Latino American students reported adversity, they were also more likely to report identity threat and poor academic fit. They were also more likely to see “adversity [as] a sign that the stereotype is in play and evidence that they do not belong. [2]” For affirmed Latino American students the relationship between the adversity and identity threat was absent. So even though all students may have been experiencing similar levels of daily adversity, for unaffirmed students experiences of adversity spilled over into their academic narrative, but it did not for those who were affirmed. It appears as though the affirmation changes the way Latino American students experience adversity. They are much less likely to interpret something going wrong as a sign that they don’t fit in or that the academic environment is hostile towards people with their identity.

Without the added psychological burden, students become better able to perform to their potential. As Cohen explains, “The intervention taps into what’s already there, alleviating underperformance. It obviously won’t teach kids to spell or offset the effects of harsh disadvantage. But under the right circumstances, when threat inhibits performance and infrastructure supports student growth, it will produce significant gains and these kinds of interventions may even synergize the effects of other reform efforts that don’t take into account the psychology of the student.”

By intervening early, it appears that it is possible to interrupt a self-reinforcing cycle of threat and underperformance, improving Latino American students’ grades and narrowing the achievement gap. The new research also illuminates the process by which the affirmation is having its effect: “Together,” the article’s authors explain, “these results paint a rich portrait of the psychology of identity threat and how affirmation works. Affirmation opens people up to a broader cognitive perspective, and it helps them to construct a less threatening narrative around adversity.” With the burden of identity threat diminished, Latino American students become less likely to interpret the adversity that all students inevitably encounter as potentially related to their ethnicity. Instead of seeing adversity as confirmation of their fears, they are able to take a step back and see their situation from a broader perspective. This not only improves their performance on the next test, but it interrupts a downward trajectory in performance and helps narrow the achievement gap. These findings and interventions hold potential for not only minorities, but for any group that is operating under psychological threat, such as test anxious students [12] or women in advanced math [13]. Indeed, who may be experiencing underperformance due to threat or stress may vary substantially by context. It is not minorities who should benefit per se, but any student, regardless of race, who is underperforming due to threat or stress. Many questions remain to be addressed, both in terms of the psychology and the possibility of implementing such interventions on a larger scale, but this research offers a promising step towards effective, psychological interventions that can play an important role in reducing the achievement gap.

————————————————

*Full disclosure: I am the eighth of these nine authors. I took part in the grant writing process and helped to write and edit the article, but I was not involved in administering the intervention, or in data collection and analysis.
**There were three different versions of the affirmation exercise that took place over the course of the year; each is described fully in the article.

References:

[1] Reardon, S. F. (2011). The widening academic achievement gap between the rich and the poor: New evidence and possible explanations. Whither opportunity?: Rising inequality, schools, and children’s life chances, 91-116. Russell Sage Foundation Publications. Link to PDF

[2] Sherman, D. K., Hartson, K. A., Binning, K. R., Purdie-Vaughns, V., Garcia, J., Taborsky-Barba, S., Tomassetti, S., Nussbaum, A. D., & Cohen, G. L. (2013). Deflecting the trajectory and changing the narrative: How self-affirmation affects academic performance and motivation under identity threat. Journal of Personality and Social Psychology, 104, 591-618. doi:10.1037/a0031495

[3] Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613–629. doi:10.1037/0003-066X.52.6.613

[4] Schmader, T., Johns, M., & Forbes, C. (2008). An integrated process model of stereotype threat effects on performance. Psychological Review, 115, 336–356. doi:10.1037/0033-295X.115.2.336

[5] Aronson, J., Lustina, M. J., Good, C., Keough, K., Steele, C. M., & Brown, J. (1999). When White men can’t do math: Necessary and sufficient factors in stereotype threat. Journal of Experimental Social Psychology, 35, 29 – 46. doi: 10.1006/jesp.1998.1371 (http://dx.doi.org/10.1006/jesp.1998.1371)
[6] Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial achievement gap: A social-psychological intervention. Science, 313, 1307–1310. doi:10.1126/science.1128317

[7] Cohen, G. L., Garcia, J., Purdie-Vaughns, V., Apfel, N., & Brzustoski, P. (2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324, 400–403. doi:10.1126/science.1170769

[8] Steele, C. M. (1988). The psychology of self-affirmation: Sustaining the integrity of the self. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 21, pp. 261–302). New York, NY: Academic Press. doi:10.1016/S0065-2601(08)60229-4

[9] Sherman, D. K., & Cohen, G. L. (2006). The psychology of self-defense: Self-affirmation theory. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 38, pp. 183–242). San Diego, CA: Academic Press. doi:10.1016/s0065-2601(06)38004-5

[10] Sherman, D. K., & Hartson, K. A. (2011). Reconciling Self-Protection with Self-Improvement. Handbook of self-enhancement and self-protection, 128-151. Guilford Press. Link to PDF

[11] Shnabel, N., Purdie-Vaughns, V., Cook, J. E., Garcia, J., & Cohen, G. L. (2013). Demystifying Values-Affirmation Interventions Writing About Social Belonging Is a Key to Buffering Against Identity Threat. Personality and Social Psychology Bulletin. doi:10.1177/0146167213480816

[12] Ramirez, G., & Beilock, S. L. (2011). Writing about testing worries boosts exam performance in the classroom. science, 331(6014), 211-213. doi: 10.1126/science.1199427 

[13] Miyake, A., Kost-Smith, L. E., Finkelstein, N. D., Pollock, S. J., Cohen, G. L., & Ito, T. A. (2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science, 330(6008), 1234-1237. doi:10.1126/science.1195996

Source: Jason Ford for The Boston Globe

Is it possible to get gifts down to a science? With gift giving season upon us, there has been a flood of advice for consumers on how to navigate their purchases scientifically. The Wall Street Journal explains that re-gifting is an acceptable practice, while the Boston Globe counsels that fancy wrapping can backfire; Cass Sunstein (President Obama’s regulations guru and the co-author of Nudge) tells us what lessons behavioral economics has to share on gift giving. I even got in on the action myself, writing a piece for the Chicago Booth website on some new research by Yan Zhang and Nick Epley about when people appreciate a gift’s thoughtfulness. (There’s also a nice review of the research by Kate Reilly at the always interesting Psych Your Mind blog).

Reviewing the research – and the journalism – got me thinking about the strengths and the limitations of bringing a scientific approach to exchanging presents. There is a lot to be said for studying gift giving, as long as we remain mindful of its limitations.

Take the work by Zhang and Epley that I wrote about. The research demonstrates that when we give gifts we tend to believe that the thought we put into the gift will be appreciated. In reality, people often don’t pay much attention to the thought that went into a gift (unless there’s something that triggers them to consider it). The finding is built on an important insight that may seem obvious, but is frequently overlooked: people can’t read your mind.

When we give a gift, we know about the hours of thought we put into it, but usually they can’t see that just by looking at the gift. We know that we picked just the right tennis racket or sweater out of the dozens there were to choose from, but all they see is the one we chose.

In some cases it’s easy to see – like when your eight-year-old rips open the packaging of the video game that he desperately wanted. He doesn’t take the time, amidst all the jumping up and down and screaming for joy, to consider how you took the time to talk to his friends or read online reviews to make sure you got the right game. You’re lucky if you get a breathless thank you.

On the other hand, there are other cases in which the finding just doesn’t hold up – like when your grandmother knits you a sweater. It’s really obvious that a lot of thought and effort went into the gift and, in those instances, we usually do appreciate the thought (the research actually shows that we’re particularly likely to appreciate the thought that went into an unexpectedly bad gift, but I’m pretty confident that we appreciate the thoughtfulness of grandma’s sweater even if we really like it).

It’s the in between cases that are tricky and where the research proves the most valuable. For example, your intuition might tell you that picking a wedding gift off a registry is impersonal, and your college roommate would appreciate something more thoughtful. The research (by Francesca Gino and Frank Flynn link to PDF) says you would be wrong – people tend to like things off their registry better than vigilante gifts and they consider them more thoughtful.

In short, the research helps us to confront our intuitions; those cases in which it feels like we’re right, but in reality we’re not. We think people will like a gift more the more expensive it is, but they don’t (link to PDF). We think that people will get mad at us if we re-gift something they’ve given us, but they generally seem not to mind (link). And we tend to think that people will like the same things we like, but often they don’t.

So has science solved the gift conundrum? Are we about to leave behind our misguided intuitions and become perfect little Santas? Maybe not. Sometimes it may seem like science has all the answers, but the problem for science is not a shortage of answers but that it’s not necessarily asking all the right questions. If we can tell science what outcomes we value then science does a decent job telling us how to get those outcomes. But science is not necessarily good at deciding on its own what outcomes we should value.

Take the example of how (rational) economics tackles the question of gifts. As George Loewenstein and Cass Sunstein point out, gifts tend to destroy value because “the value of gifts to their recipients is typically far lower than the money that was spent on them. [Joel Waldfogel] found that of the $65 billion spent on winter holiday gifts in 2009, about 20 percent was wasted, in the sense that the gifts were worth that much less to the recipient than they cost.” From a purely rational economic perspective this makes no sense; what we should do, if we insist on exchanging gifts on Christmas morning, is each pass an envelope of money to the person on our right. Of course that misses the point of gift giving. An economic perspective can arm you with tools that help you destroy less value, but it doesn’t ask the right questions to make you a better gift giver.

We can try to console ourselves with the notion that psychology (and behavioral economics) does a better job when it comes to gift giving, but let’s not kid ourselves, psychology has blind spots, too. Just like economists, we focus on one outcome while we neglect others that were too hard to measure, that we dismissed as unimportant, or that we simply didn’t think of.

For example, reading the first three experiments in Zhang and Epley’s paper might lead you to the conclusion that there’s no real upside to spending a lot of time trying to think of the perfect gift. After all, the time you put in is unlikely to be appreciated. If you were to stop reading there, you’d have an answer to the question of how much is the thought that goes into a gift appreciated.

Luckily, Zhang and Epley did not stop there. In their fourth experiment they found something surprising. The benefits of thoughtfulness often did not go to the person receiving the gift – they didn’t appreciate the gift any more when the giver had put a lot of time into it. Instead, the benefits went to the gift giver who, having spent time and effort thinking about what the recipient of their gift might like, felt closer to that person and more socially connected to them. That certainly would have been easy to miss had the researcher’s only question been about gift appreciation. Of course we fail to ask the right questions all the time and it’s a very difficult problem to detect. How do you know when you’re missing something? Certainly science has done its job answering the question it was presented, but science can’t always tell you what other questions still need to be asked.

Anuj Shah

There’s a really interesting new paper out in Science by my colleague Anuj Shah at Booth (the business school at the University of Chicago, where I teach). It explores how poverty can change people’s psychology, leading people with few resources to begin with to make decisions that only undermine their financial situation. This approach, which he has been working on with two of the leaders in the area, Eldar Shafir and Sendhil Mullainathan, doesn’t pin poverty on a poor person’s inability to reason properly, or their shortsightedness, or their lack of motivation. It shows how – for anybody, rich or poor – facing a scarcity of resources (i.e., not having enough of something you need) produces a sense of urgency that changes the way we make decisions. This perspective not only helps us understand these problems better, but steers us in new directions when we’re thinking about how to fix the problem.

On a personal note, I’ve just started to do some writing for Capital Ideas, the soon-to-be revamped Chicago Booth publication that highlights new ideas and research from Booth faculty. They’ll be fully up and running in the new year, but for now I thought I’d share a short piece I wrote for them on Anuj’s research, which I’ve reproduced below.

Does Being Poor Lead to Poor Decisions?

One of the obstacles that keeps the poor from rising out of poverty is the tendency to make costly financial decisions – like buying lottery tickets, taking out high interest loans (PDF), and failing to enroll in assistance programs – that only make their situation worse. In the past, these poor decisions have been attributed either to low income individuals’ personalities or issues in their environment, such as poor education or substandard living conditions. New research published this month in Science by Booth Assistant Professor of Behavioral Science Anuj Shah points to a new answer: living with scarcity changes people’s psychology.

Source: http://flic.kr/p/6sqgFQ

The basic idea is that when resources are scarce – when people are short on time, or money, or food – each decision about how best to use those resources takes on greater urgency than when resources are abundant. This focus can have positive effects in the short term, but it comes at the expense of neglecting other, less urgent demands. For example, when they are under the press of urgent expenses like rent and groceries, people may neglect to do routine maintenance on their car and end up with costly (and avoidable) repairs down the road.

Shah, along with colleagues Sendhil Mullainathan of Harvard and Eldar Shafir of Princeton, published five studies in which he studied the effects of scarcity on decision making in various games in which people were paid according to their performance. In each of the studies some people received ample resources with which to play, while others received very few. Moreover, in some studies the players had the opportunity to borrow additional resources with interest. The researchers then observed how scarcity affected the players’ borrowing behavior, their performance, and the psychological processes at play.

Across the studies Shah found that for people who had very few resources, the games took on more urgency. They became more focused on the task at hand in order to make the best use of their scarce resources, but that this added focus came at a price, including mental fatigue, costly borrowing decisions, and poor overall performance.

For example, in an Angry Birds-type of game, in which the object was to knock down as many targets as possible, players who could take only three shots per round spent more time aiming each shot than players who had fifteen shots. This added focus improved performance, but it had downsides. When players were given the opportunity to “borrow” a shot, by giving up two shots in a later round of the game, players who had fewer in shots made counterproductive borrowing decisions that hurt their overall performance.

These simple experimental games are obviously a step removed from the actual problems facing people living in poverty, but they capture the essential features of the situation. The findings tell us that decisions that are considered routine by the relatively affluent can take on great urgency for the poor, and that this added urgency can lead to poor decision making in other matters by monopolizing attention and draining cognitive resources. These findings also suggest the sorts of interventions that are likely to be effective, such as clearer explanations of the costs of payday loans that are less cognitively demanding, as has been shown to be the case in research by Booth Professors Marianne Bertrand and Adair Morse.

One set of reforms that has been at the forefront of recent debate has taken issue with the problem of False-Positive Psychology (FPP), which I’ve written about previously here. Published just last year in Psychological Science, the three authors (Joe Simmons, Leif Nelson, and Uri Simonsohn; pictured) make the case that researchers often make methodological choices that, although they may seem benign, can dramatically inflate the chance that they will generate a false-positive result – or, in other words, they are p-hacking (i.e., engaging in various methodological maneuvers that assure that the target significance level of p < .05 is reached). (Read the paper here; for more discussion see 1, 2, 3, 4.)

Despite the attention they have received, the FPP recommendations (or any similar ones) have not been widely adopted. Although, anecdotally, they have inspired some change at the individual level (one example), not much has happened at the institutional level. Journals do not yet require the transparency that would help fix the p-hacking problem. Why not?
In a basic sense, change is hard. In addition to simple inertia, there are obvious benefits to the researcher who engages in p-hacking, including more significant results, less data to collect and less associated cost, and more publications. The cost, however, is that these publications are much more likely to be filled with false-positive results. So while the researcher may benefit on some level in the short run, in the long run the science suffers. And, ultimately, I strongly believe that, personal motivations notwithstanding, the vast majority of researchers are deeply committed to the pursuit of the truth.

On the surface, the easiest solution would be for journals to adopt a policy that requires researchers submitting articles to be transparent in the reporting of their methods. For example, researchers should be encouraged to determine their sample sizes in advance; but, if they peeked at the data then added subjects, they should be required to report it. The trouble is that it’s not necessarily that easy.

For one thing, there is a great deal of research that is currently ongoing. So even if a researcher were to adopt the FPP recommendations today, they would still be sitting on a pile of unpublished research that does not necessarily adhere to the new standard. They would either have to discard the research or submit it with the knowledge that their voluntary transparency would likely preclude publication based on current standards that value clean, consistent results. Moreover, there is no unequivocal consensus in the field as to which policies should be adopted. Some practices are unquestionably problematic, but others are more controversial. While I find the FPP recommendations reasonable, not everyone agrees. One could, for instance, require transparency to the extent that hypotheses and data analysis strategies would have to be formally registered in advance. If different journals were to institute different policies things could get very confusing very quickly, so the journals are – perhaps understandably – reluctant to lead the reforms. (Note: Eric Eich, the editor-in-chief of Psychological Science recently put out what seems to be a very encouraging proposal for important new initiatives that hopefully the journal will adopt in some form. More on that next week, but you can check it out here.)

If the changes are not going to come from the top down, then they will have to rise from the bottom up. Already, as a result of the renewed attention paid to these issues, the discussion of best practices and necessary reforms is underway. Professional societies such as the Society for Personality and Social Psychology and the Association for Psychological Science are hosting events on these topics at their annual conferences. The discussions are taking place among colleagues, in departmental meetings and colloquia, and over the internet through email, blogs, and listservs.

The authors of the False-Positive Psychology paper themselves have just written a short piece for the SPSP newsletter in which they make a case for transparency, addressing many of the objections to their proposals. It’s short, funny, and compelling; you should read it if you haven’t already. But more importantly, they encourage researchers to take a step beyond simply discussing the problem.

To those researchers who are on board with their recommendations they say:

 There is no need to wait for everyone to catch-up with your desire for a more transparent science. If you did not p-hack a finding, say it, and your results will be evaluated with the greater confidence they deserve.

If you determined sample size in advance, say it.

If you did not drop any variables, say it.

If you did not drop any conditions, say it.

They even offer a template for doing so, using just 21 words: “We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study.” That’s pretty concise. Since they posted the piece, people have suggested improvements to make the statement clear and even shorter (see here).

The goal is to change the field’s norms from the bottom up. Although not everyone may agree right now, “a small but energetic choir” may bring the rest along. Will it work? It’s certainly possible that it will be ignored and that too few people will clearly signal the transparency of their research methods for the change to take hold broadly. But it can’t hurt to try. Reading papers that carry the “21 words” may send an important message to those who are wary or reluctant to reform. For researchers it will make it clear that these changes are already underway, which may prompt them to get on board in order to improve the science they are doing, or at least to make sure that they are not left behind. And for reviewers seeing the “21 words” not only allows them to evaluate those submissions with greater confidence, but also to ask for greater transparency from those submissions in which they are absent. With a little time, norms may change. We are a field that is built on conventions and social norms — there’s nothing inherently meaningful about a significance level of p = .05 — so there’s no reason not to add transparency to the list of things that we expect of responsible scientists, and hopefully it will eventually take for granted.

For New Yorkers, Daddy Bloomberg decided that the limit was at sixteen ounces of soda. If you want more than that you’re going to have to get a whole other cup. Bloomberg’s ban provoked angry protests from New Yorkers who insist that they aren’t children and that they can make their own soda choices. In this week’s New Yorker, James Surowiecki argues that while it’s true that the soda ban is paternalistic, it could still be very effective at reducing soda consumption. And I believe he may be right – the research he reviews is compelling – but there is other research that suggests that the ban could backfire, or at least that heavy handed paternalism may not be the best way for the Mayor to achieve his goal.

Where Surowiecki is certainly right, is in his claim that people’s desires can be incredibly malleable. Our preferences are strongly influenced by all sorts of things we are usually not even aware of. As Surowiecki explains it:

An executive at the American Beverage Association has dismissed [Bloomberg’s] plan, saying that “150 years of research finds that people consume what they want.” Actually, the research shows that what people “want” has a lot to do with how choices are framed.

The evidence that Surowiecki marshals to support this claim are convincing. Making sixteen ounces the default and forcing someone to actively choose to get two separate cups if they really want all that soda is likely to be effective. Taking away the biggest option is also likely to transform the sixteen ounce size from “medium” to “large”. And, just having less soda in front of you is likely to make you feel satisfied with less soda. Mayor Bloomberg’s hope is, presumably, that the furor over the ban will subside over time and New Yorkers will be left with a choice set that is likely to make them healthier.

The potential downside, though, is that there is also evidence that imposing choices on people, as Bloomberg is doing, can have just the opposite effect. Every parent knows that the best way to get a kid interested in something is to tell them they aren’t allowed to have it. The psychological term for this phenomenon, introduced by Jack Brehm in the 1960s, is reactance. As Columbia University’s Sheena Iyengar put it in an article in the Dallas Morning News:

Most of us would probably agree that we’d be better off if we drank less soda, but any government action designed to control consumption puts us on alert.

[…]

If we want to implement a sweetened beverage tax and maximize its effectiveness, the best approach would be to dissociate it from the larger issue of individual choice and focus on its immediate practical benefits, such as the revenue it produces.

The key point is that, whatever approach you favor, it’s probably best to avoid drawing attention to the fact that you’re forbidding people from doing something. We all know how that worked out for Adam and Eve, and Romeo and Juliet may have quickly lost interest in one another if their families had just left them alone. That’s why in their book, Nudge, Dick Thaler and Cass Sunstein favor a subtler approach. As Thaler commented in response to Bloomberg’s soda ban:

To state the obvious: a BAN is not a NUDGE.The opposite in fact. So don’t blame Bloomberg’s ban on large soda cups on us.

— Richard H Thaler (@R_Thaler) May 31, 2012

This weekend Robert Frank wrote about the respective roles of luck and skill in achieving success in this Economic View column in The New York Times. Frank’s piece hits on many of the same themes as my post from a couple of weeks ago on how the way we think about luck and merit affects how we see the world. That post, incidentally, has been cross-posted at the Creativity Post website, which is well worth checking out (thanks to Sam McNerney for helping to make that happen).

Here are the opening paragraphs:

There may be no topic that more reliably divides liberals and conservatives than the relationship between success and luck. Many conservatives celebrate market success as an almost inevitable consequence of talent and effort. Liberals, by contrast, like to remind us that even talented people who work hard sometimes fall on hard times through no fault of their own.

It’s easy to see why each side is wary of the other’s position. Conservatives, for example, understandably fret that encouraging people to view life as a lottery might encourage them just to sit back and hope for the best. Liberals, for their part, worry that encouraging people to claim an unrealistically large share of the credit for their own success might make them more reluctant to aid the less fortunate.

Frank goes on to describe a study that tests the role of luck and merit on the success of a music recording. In the study (PDF), sociologists Matthew Salganik, Peter Dodds, and Duncan Watts set up an online “market” in which people can download and rate a set of 48 relatively obscure indie songs (the downloads are free in exchange for providing a rating). In a control condition people simply listen to the music and rate it. This serves as an objective rating of the songs’ quality.

In other versions of the experiment, the researchers randomly made some songs appear to be more popular and have better ratings than others. The effect on subsequent ratings, as the study’s authors put it, was that “the best songs rarely did poorly, and the worst rarely did well, but any other result was possible.” Frank describes how one song, that had ranked 26^th out of 48 in the control condition, was ranked as low as 40^th in one version, but all the way up at #1 in another. Here’s his takeaway from the study:

Early success — even if unearned — breeds further success, and early failure breeds further failure. The upshot is that the fate of products in general — but especially of those in the intermediate-quality range — often entails an enormous element of luck.

In the end, both conservatives and liberals views were confirmed, at least in part: you have to be good to be successful but you also have to be lucky. With the hard work of these researchers, Bob Frank, and others (as well as a healthy dose of luck) maybe partisans on both sides will be willing to lay down their caricatured versions about what the other side supposedly believes causes success.