SPEAKER_01: This message comes from NPR sponsor Bank of America, with powerful digital solutions for local and global businesses so you can make every move matter. Visit bankofamerica.com slash banking for business. Bank of America N.A. Copyright 2023.
SPEAKER_00: This is Planet Money from NPR.
SPEAKER_07: About a decade and a half ago, this one idea started showing up everywhere. The nudge.
SPEAKER_08: The idea of the nudge came from behavioral economics. Basically said, if you really understood how people think, their psychology, you could make a huge difference in the world just by doing these little tweaks. Just by nudging people in the right direction. Like if you want people to use less electricity, use social pressure.
SPEAKER_07: You can send them a letter saying, hey, you're using way more electricity than your neighbors.
SPEAKER_08: Or organ donation. How do we get people to sign up to be organ donors? Turns out the secret has to do with the form at the DMV.
SPEAKER_06: This is one of the most famous behavioral economists around, Dan Ariely.
SPEAKER_08: He's giving a TED talk that's gotten more than 10 million views. And yeah, he's right. Some European countries require people to opt out of being organ donors instead of opting in when they're getting their driver's licenses. And making people opt out, that gets a lot more people on the organ donor list. Ariely here in a different TED talk. He's given about a dozen.
SPEAKER_07: So what have we learned from this about cheating? He's talking about some of his own research, focused on how you can get people to be more honest. With simple reminders. A lot of people can cheat. They cheat just by a little bit.
SPEAKER_06: When we remind people about their morality, they cheat less.
SPEAKER_08: Ariely has had some blockbuster research into the subject. Why people cheat and how to make them more honest. In one study, he asked people to recall the 10 commandments. You know, like, thou shall not this, thou shall not that. And then he had them play this game where they would be tempted to cheat. The moment people thought about trying to recall the 10 commandments, they stopped cheating.
SPEAKER_06: Another one of Ariely's big deal papers was about this one simple trick that would make people way more likely to tell the truth when they're filling out forms or paperwork.
SPEAKER_07: For this one, he collaborated with a bunch of researchers, including another huge star of behavioral science, Francesca Gino. Here's Gino talking about that trick in 2015 on Planet Money. We should sign at the top of the form.
SPEAKER_05: That's it. Instead of signing at the bottom, we should sign at the top.
SPEAKER_07: Exactly. It's that simple. Makes sense. When you fill out a form and then at the very end, you're asked to swear that everything you wrote was true. Uh, was it? Close enough. But if you swear to tell the truth ahead of time, your brain is primed for honesty.
SPEAKER_08: Gino and Ariely, they became these academic superstars for all this nudge research, especially for their papers on how to get people to be more honest. And nudges? Well, nudges became all the rage. In the UK, the government created an entire organization. It's called the Behavioral Insights Team. Or as they were nicknamed, the nudge unit. That's Michael Sanders. Back in 2014, he was the nudge unit's chief scientist. It was his job to test out nudges all around the world.
SPEAKER_07: Yeah, and Michael still remembers the moment this famous sign on the top idea was brought before his team. He was at his office in London, and one of Gino and Ariely's co-authors came in to give a presentation on the research. What was the mood like in the room?
SPEAKER_08: I think, you know, we're British, so we don't have electrified rooms. People don't sit in.
SPEAKER_02: People don't buzz like they do in America. But, you know, I exchanged glances with a couple of senior colleagues. It's like, we could do this in this area. We could do this in this area. There's so many opportunities for this.
SPEAKER_08: This sign on the top idea? It was the quintessential nudge. So clever and so easy to implement and so surprisingly powerful. It's just so simple.
SPEAKER_02: It is. Many lies are simple. Allegedly.
SPEAKER_07: That quintessential nudge idea was clever and was powerful. And also, it was a lie. Hello and welcome to Planet Money. I'm Nick Fountain.
SPEAKER_08: And I'm Jeff Guo. Dan, Ariely and Francesca Gino have played this central role in taking nudge theory mainstream. And they've done really well for themselves. Gino's this big shot professor at Harvard Business School with all these speaking gigs and book deals. Ariely's got this fancy job at Duke with multiple best selling books. One of them is even being turned into a TV show by NBC. But now their famous sign on the top paper we were talking about before.
SPEAKER_07: That paper is at the center of a massive scandal. And these two superstars, famous for showing how to nudge people to be more honest, they are being accused of dishonesty themselves. Today on the show, the truth about dishonesty, dishonesty about truth, and the spreadsheet detective who cracked the case wide open.
SPEAKER_01: It was a lot of work to condense more than a century of contract law
SPEAKER_08: into a single 20 minute episode.
SPEAKER_00: There was an article that you sent me that had like an old English font at the top of it. And I was immediately like, I'm not reading this one, Jeff. I'm sorry. You got to be more selective.
SPEAKER_08: It's called research. How we did ours for our recent story on the rise of Internet contracts and how they got so sneaky. That is in our latest bonus episode for Planet Money Plus listeners, whose support helps make this show possible.
SPEAKER_03: This message comes from NPR sponsor Crow. Don't avoid or resist the unknown. Face it head on. Crow offers top flight services in audit, tax, advisory and consulting to help your business take on today's biggest challenges. Visit embrace volatility dot com.
SPEAKER_08: Back in 2014, before two titans of behavioral science were engulfed in scandal, and back when nudges were still becoming the hotness, Michael Sanders was the chief scientist at the British government's Nudge Unit. It was his job to take some of these nudges and test them out in the field. You guys were like sort of like special economics agents bringing behavioral econ
SPEAKER_07: to the far flung corners of the world. That was sort of your mission.
SPEAKER_02: I mean, that sounds very grand, doesn't it? Yes, that's I'd say that was our mission.
SPEAKER_07: Which is why when the government of Guatemala had a problem getting its citizens to pay their taxes, Michael was like, this is perfect for us. Guatemala has among the, if not the lowest level of tax compliance of any country across Latin America.
SPEAKER_02: And so the opportunity for up the opportunity to improve things was just enormous in that context.
SPEAKER_07: You felt like you could make a real difference there?
SPEAKER_08: Yeah. The Guatemalan government was losing hundreds of millions of dollars every year to tax evasion, money that could go to building roads and schools and whatever else. Maybe Michael, armed with all his little nudges, could help fix that. So Michael and a small team flew to Guatemala City to meet with tax authorities there
SPEAKER_07: and asked them how they collect taxes. And they said, well, one of the ways is called the value added tax, which is kind of like a sales tax. Every month, businesses are supposed to fill out this form where they declare how much they owe in VAT. And most people fill in the form, most people who do fill in the form say, oh, I don't know any.
SPEAKER_02: I don't know any sales tax. And that's just not really credible, right? I feel like in places like the UK and in Europe, the VAT is just part of the sales price, right?
SPEAKER_02: I mean, I'm not going to take lectures from Americans about how you can run a tax system.
SPEAKER_08: Michael, as you might expect, immediately thought back to that famous paper by Dan Ariely and Francesca Gino, the one where if you make people promise to be honest before they fill something out, they're much more likely to tell the truth than if they promise after they fill it out. Michael and his colleagues realized they could use this idea on the Guatemalan tax website. Before people entered their taxes, they would first have to pledge that they were going to tell the truth.
SPEAKER_07: And Michael tells the tax authorities there's real research behind this done by professors from Harvard and Duke. And also to prove to you that this works, we're going to run an experiment to test it. Some people will get the new honesty pledge at the start and others won't. And what did they say?
SPEAKER_02: They said, let's do it. Let's go. So we were in Guatemala for three days and we went from, hi, my name's Michael, to we're running this study and we're going to launch it and we're going to run it for three months with three million tax returns. And we'll call you back in four months with the results.
SPEAKER_08: Four months go by, Michael gets the data set and he stays up late crunching the numbers. Did the intervention work? It worked not at all.
SPEAKER_02: Not at all? Not at all. There are few things in my life that I am in a statistical sense more confident of than that this didn't work. But it should have worked.
SPEAKER_07: The original research behind the whole sign on the top of the form thing had been done by a team that included two of the biggest stars in behavioral science. And Michael's thinking, I can't even repeat their research.
SPEAKER_02: I can't even copy them, right? Right? It's not like I'm trying to come up with something new and innovative and I'm struggling at that part. I just can't even copy them in a way that creates the right results. But here's the thing.
SPEAKER_07: Michael wasn't the only one having trouble getting the right results. Around the time he published his paper describing his failure in Guatemala, other researchers started noticing their experiments were failing too. Eventually, one of Gino and Ariely's co-authors on that original sign on the top paper
SPEAKER_08: convinced them that they needed to revisit their findings too. So they tried to redo one of their experiments from the paper. And this time, it straight up didn't work. This was a big deal.
SPEAKER_07: One of the most famous nudge studies couldn't be replicated. But that didn't necessarily mean anyone had done anything wrong. This is how science works. Sometimes you just get fluke results.
SPEAKER_08: In 2020, a bunch of researchers, including Ariely, Gino, and the other original co-authors published something that says, yeah, maybe this sign on the top intervention doesn't actually work. But more importantly, they also published for the very first time all the original data, all those Excel spreadsheets that they had used. Which, if you had some suspicions about what was going on here, might be very interesting.
SPEAKER_07: I'm Yuri.
SPEAKER_05: I'm a professor of behavioral science at a side of business school in Spain. In addition to teaching behavioral science, Yuri Simonson has a little side project called
SPEAKER_07: Data Colada, where he and some colleagues evangelize to other behavioral scientists about how they should conduct and analyze their experiments. But what the Data Colada folks are really famous for are their investigations into research
SPEAKER_08: that they think smells a little fishy. Maybe a researcher juiced his regressions a little bit to get the results he wanted. Or maybe he played fast and loose with his Bayes factors. Those tricky Bayesians.
SPEAKER_07: Anyways, Yuri in particular loves digging through people's spreadsheets for clues on how exactly they were put together and why. It's like a puzzle that nobody knows a solution to and then you solve it.
SPEAKER_05: I mean, it's like true crime. That is why true crime is so interesting to people and trying to figure it out. And it makes for good podcasts, too.
SPEAKER_07: Hopefully. Because of Data Colada's reputation, some tipsters reached out to Yuri and said, hey, you know how the original sign on the top paper had three different experiments in it? You might want to take a closer look at experiment number three, the one about car insurance. Yeah, the car insurance experiment.
SPEAKER_08: You know how your car insurance company occasionally asks you what the mileage is on your odometer? That is a sneaky way of asking how much you've been driving because car insurers want to charge people who drive more, more. So drivers have an incentive to lie, which is where this sign on the top intervention comes in. One of the superstar researchers, Dan Ariely, worked with a car insurer.
SPEAKER_07: Ariely said he asked the company to send out different versions of the same form to around 13000 customers, requesting them to update the odometer readings the company had on file for them. Half the people got forms with honesty pledges right at the top, and half of them got them
SPEAKER_08: on the bottom. And according to the paper, the ones who signed at the top, they were way more honest. But the tipsters who reached out to Yuri said there's lots of weird stuff going on in the data.
SPEAKER_07: Like even though when people originally reported their odometer readings, they often rounded to like the nearest thousand or 500. They did not do that when they updated their results. And the tipster said, look at the fonts.
SPEAKER_08: Why are half the cells in this column in Calibri and the other half are in Cambria? We were puzzled. Like, why would this happen?
SPEAKER_05: Yuri is like, fine, fine, fine.
SPEAKER_07: But immediately he hones in on this one thing. This was like piece of evidence seven.
SPEAKER_05: And I was like, I don't care about piece of evidence one through six. Like this is it. Piece of evidence number seven, the distribution of miles driven.
SPEAKER_08: OK, so usually if you took a bunch of people, what you'd expect is that most of them are driving like a normal number of miles in a year, right? Like, say, 12000 miles. And a few people might be driving way less, only like 1000 miles a year. And maybe a few people might be driving way more, maybe like 50000 miles. But overall, the data would look like a bell curve with a big hump of people in the middle. But when Yuri looked at the distribution of miles driven in Dan Ariely's experiment,
SPEAKER_07: he did not see a bell curve at all. He saw basically a flat line. It just had an impossible shape.
SPEAKER_05: Like there were just as many people who drove a thousand miles as there were who drove 2000 miles as there were who drove 10000 miles as there were 40000 miles all the way to 50000 miles. And that's just crazy. Yours says anyone who works with data would immediately be like, there's no way.
SPEAKER_07: Like I share that with people.
SPEAKER_05: People started laughing hysterically looking at the screen like it was just like a like just a little reaction. So it was just self evidently faked. This data set was faked.
SPEAKER_08: And as if it could not get any more obvious, even though there were lots of people who drove nearly 50000 miles, it all cuts off right there. Not a single customer apparently drove more than 50000 miles. When he saw that, Yuri immediately knew the exact spreadsheet command whoever faked the
SPEAKER_07: data had used. Round between.
SPEAKER_05: So give me a random number in between zero and 50000. And if you do that, you get exactly this spreadsheet. So it's kind of remarkable.
SPEAKER_07: So you know how they did it.
SPEAKER_05: Yeah. Yeah, we figured out every step of the way like how the spreadsheet was generated. Yuri had done it. He cracked the case. It just so I've never seen anything so blatant in my life. It just incredibly as far as smoking gun evidence goes this this for you is like, it's this
SPEAKER_07: is it. Yeah, nothing will ever match this.
SPEAKER_08: But there was another unsolved mystery here. Who was responsible for this fake data? Yeah, what I wanted to look for was things that would point to who did it.
SPEAKER_07: And there was not a great way for him to figure that out. Before Yuri and his colleagues published what they had found, they reached out to the authors of that famous paper to give them a chance to respond. Usually when they do this, he says it can get kind of contentious. But this time, everyone was pleasant.
SPEAKER_05: That was very unusual because we were saying, hey, you have some fake data there. And and they were like, oh, we should do sorry. And so they usually we get I mean, we were not expecting this. So almost immediately, Dan Ariely replied and said, just to be clear, I was the only person responsible for this in this team. This meaning Ariely was responsible for getting the data from the insurance company.
SPEAKER_08: But Dan Ariely was not taking responsibility for forging the data. He basically blamed the insurance company. He said, quote, the data were collected, entered, merged and anonymized by the company and then sent to me. This was the data file that was used for the analysis and then shared publicly. So Yuri knew the data had been forged, but without access to the original data from the
SPEAKER_07: insurance company, there was no way to prove who did the forging. The case was at a dead end.
SPEAKER_08: But not for long. In another experiment in that same paper, Yuri would find another smoking gun. And this time there were fingerprints. Also, after the break, we uncover a little evidence ourselves.
SPEAKER_07:
SPEAKER_03: Hi, this is Daniel Alarcon, host of NPR's Spanish-language podcast Radio Ambulante.
SPEAKER_04: Our new season features surprising stories from Latin America. In Mexico, a sculptor confounds archaeologists with brand new antiquities. In Costa Rica, gentrification sparks a war in defense of endangered turtles. In Colombia, a journalist's military I.D. is issued inexplicably with the photo of Cristiano Ronaldo. New stories every Tuesday, wherever you get your podcasts.
SPEAKER_07: So here's where we're at. After Yuri and Data Collada pointed out that the car insurance experiment in that original Sun on the Top paper had fake data in it, the authors of the paper retracted it. If you look up the paper online, right there on the top, there's now a warning in red that basically says this paper is no good. But it's not like anyone lost their jobs over it.
SPEAKER_08: Now, Yuri and his team had actually gotten another tip that there might be something wrong with the work of one of the paper's co-authors, Francesca Gino. And they start looking at a lot of her research, including one of the experiments she worked on for the Sun on the Top paper. So they start digging in. Maybe this time they can do more than just show that the data was obviously wrong. Maybe they can find definitive proof of how the data had been manipulated.
SPEAKER_07: So the experiment in question asked people to solve some math problems. They were told they would get paid a certain amount of money for each one they got right. They did their math, and then they got to say on this official-looking form how many problems they'd gotten right. The ones who signed on the top of the form, the paper said, were way more honest. But unlike with the odometer data, there was nothing obviously wrong with the math problem
SPEAKER_08: data. Yuri looked at it. He looked at it again. And then he thought back to this one secret little quirk about the spreadsheet program
SPEAKER_07: Excel. Excel remembers. Excel remembers.
SPEAKER_08: And when I first heard about this, I could not believe this was true. But if you use formulas in Excel, and everybody uses formulas in Excel, Excel essentially keeps track of where you move stuff. And it records all your moves in a secret file called calcchain.
SPEAKER_05: Calcchain is something that I think 99.9% of people have never heard of before.
SPEAKER_08: But Yuri, of course, is part of the 0.1% who has heard of it. He loves it.
SPEAKER_05: Yeah, it wants to figure out it's very addictive. Like you get a data set and you start just checking the calcchain file, you know, like immediately.
SPEAKER_08: So Yuri pulls up the calcchain file for experiment number one and realizes that a bunch of those spreadsheet cells, they have been moved by somebody. And all of those cells that have been moved, every single one happens to strengthen the finding that signing on the top increases honesty. Like, how convenient, right?
SPEAKER_07: Yes. And this time, Yuri and his colleagues at Data Colada could do more than prove that the data was fishy. They could prove that somebody involved in the research had gone in and actually butts with the data. And that really sort of wraps it up in a very compelling way because you have like
SPEAKER_05: confirmatory evidence from a complete different source of information.
SPEAKER_07: And by all accounts, this was Francesca Gino's experiment. Run under her supervision, the results analyzed by her. So just to reiterate, two experiments by two different superstars of the field, two
SPEAKER_08: researchers who are famous for showing how to make people more honest, their own studies about honesty are based on lies on fraudulent data. Like this is bananas.
SPEAKER_07: And it's even bigger than that. Yuri's team has identified three more suspicious experiments done by Gino pointing out what they call, quote, evidence of fraud. She's on administrative leave from Harvard, and two more of her papers have already been retracted. Dan Ariely's story is also getting more complicated.
SPEAKER_08: Remember back in 2021, Ariely more or less blamed the insurance company for the fabricated data that had appeared in his odometer study. He said, quote, the data were collected, entered, merged and anonymized by the company and then sent to me. This was the data file that was used for the analysis and then shared publicly. Now, for years, the insurance company, which is called the Hartford, hasn't said much
SPEAKER_07: of anything about all this. But a couple of days ago, we convinced the company to finally go on the record. The company told us in a statement that they'd pulled the original data set they sent to
SPEAKER_08: Dan Ariely, and it looked dramatically different from the published data from Ariely's experiment. The company said they'd only given Ariely data for about 3700 insurance policies. But in Ariely's paper, he claimed he had data for more than 13000 policies. That's a difference of almost 10000.
SPEAKER_07: In their statement, the company told us, quote, though some of the data in the published study data is originally sourced from our data, it is clear that the data was manipulated inappropriately and supplemented by synthesized or fabricated data. The company basically confirmed everything that Yuri and the data kalata people had said
SPEAKER_08: about Ariely's numbers. And the company made it clear that whatever had gone wrong with the data set had gone wrong after they had already given it to Ariely. We shared parts of the insurance company statement with Ariely, and he responded in an emailed
SPEAKER_07: statement, quote, as I said two years ago, I was responsible for the relationship with the insurance company that provided the data for the paper. I got the data file from the insurance company in about 2007, and I can't tell now who had access to it. Getting the data file was the extent of my involvement with the data. We also reached out to Francesca Gino, but she didn't make herself available to be
SPEAKER_08: interviewed for this episode, nor did representatives from Harvard or Duke. The co-authors of the original paper declined to be interviewed. But the rest of behavioral science, they are commenting aplenty. So this is a disgrace. This is Michael Sanders again. He's the researcher who tried to bring the sign on the top thing to Guatemala, who left thinking that he had been the one who somehow messed up. Now he feels like the whole field was duped, not just him.
SPEAKER_02: Sometimes it does make me feel like we are stupid when you see Dan Ariely, who famously says everybody lies a little bit, and Francesca Gino says, here's my book on how you can succeed at work if you don't follow the rules. And it's like you walk in with an eye patch and a tricorn hat and a cutlass, then a hundred pages in the book later, you're like, oh my God, they were a pirate. I never saw it coming. But Michael says the problem with all these fake studies is bigger than the feelings of
SPEAKER_07: a bunch of academics like him. In Guatemala, there was this big investment. Hundreds of thousands of people were put through this experiment. That was never going to work. All of that time and money could have been spent on something different, a different
SPEAKER_02: intervention with a better chance of working if we'd known. This is not a petty academic squabble about he said, she said. This has real impact in the real world. So I'm pissed off about that. I'm also pissed off by how stupid it is. So when you see like eminent Harvard professor has committed research fraud, what you want is really sophisticated, clever ways of cheating. Not, oh, I thought I'd copy and paste these observations because they were higher and I'll make my result come together. This is, this is dumb. And that annoys me. But it's also, it's also a source of terror because that means there'll be people out there who have cheated in a really sophisticated way who we haven't caught and who we may never catch. Where Michael is, that's where his whole field is right now.
SPEAKER_08: Because of the questions around Dan Ariely and Francesca Gino's work, this cloud of uncertainty now kind of hangs over all of behavioral science. Like what else might not be true? Ariely and Gino's research offered this really tempting story that if you're just
SPEAKER_07: clever enough, you could come up with all these powerful nudges, one simple trick after another that could solve all the world's problems. But now that story is starting to look like just that, a story. If you know of anything sketchy going on in your neck of the woods, let us know. We're at planetmoneyatnpr.org. I'm at nfountainatnpr.org. We're also on all the socials media. This episode was produced by Emma Peasley with help from Willa Rubin.
SPEAKER_08: It was edited by Keith Romer and fact checked by Sierra Juarez. It was engineered by Robert Rodriguez. Alex Goldmark is our executive producer. I'm Jeff Guo. And I'm Nick Fountain. This is NPR. Thank you for listening.
SPEAKER_01: This message comes from Jackson. Let's face it, retirement planning can be confusing. At Jackson, we're working to make retirement clear for everyone, starting with you. Our easy to understand resources and user friendly digital tools help simplify your entire experience. You can have confidence in your retirement with clarity from Jackson. Seek the clarity you deserve at Jackson.com. Jackson is short for Jackson Financial Incorporated, Jackson National Life Insurance Company, Lansing, Michigan, and Jackson National Life Insurance Company of New York. Purchase New York.
