SPEAKER_06: This is Planet Money from NPR.
Economist Florian Eterer tells the story of how he got pulled into a fight over an anonymous economics message board as a sort of string of happenstances. The first was one day last spring.
SPEAKER_04: The real catalyzing event in all of this is just a conversation with my friend Kyle Jensen,
SPEAKER_05: who is not an economist.
SPEAKER_08: Kyle Jensen is an engineer with experience writing computer code. He lectures at Yale's School of Management. And that day, the two of them were taking a stroll around New Haven. Kyle had recently sat in on some econ seminars and he was telling Florian how aggressive the conversation had gotten.
SPEAKER_05: And I told him, look, we have a relatively aggressive tone in seminars, but there's much, much more aggressive behavior in economics. You should read the type of stuff that gets said on the Internet about various people. And he said, what do you mean what gets said on the Internet? People discuss economists on the Internet? And I said, well, there's this platform called Econ Job Market Rumors.
SPEAKER_06: Economics Job Market Rumors. One half of the website is a sort of job information wiki where people anonymously post about what's going on inside the black box of economics departments. The anonymity lets academic insiders pass on information without fear of professional consequences, stuff that's directly relevant to people applying for jobs.
SPEAKER_05: But which universities have sent out invites for interviews or which have given offers out or even who has been receiving those offers?
SPEAKER_08: The other half of the website is a discussion forum where anyone with an Internet connection can ask the economics hive mind whatever they want. Also anonymously.
SPEAKER_06: Which made it pretty popular. Economics is this notoriously hierarchical field. It can be hard to navigate for people outside the elite echelons of academia. And this place was democratizing potentially useful information.
SPEAKER_08: People also post rumors about things like harassment or plagiarism. You can find out what people really think about some powerful famous economist and their work. Or about your peers. Or yourself.
SPEAKER_06: Or you could just blow off some steam.
SPEAKER_05: There is more willingness to voice that discontent on an anonymous platform. It's kind of an emotional labor market.
SPEAKER_06: Yeah, I think so.
SPEAKER_05: Yeah, it's where people will go and vent.
SPEAKER_06: And all that venting can get scaldingly hot. Debates over academic papers can spill over into outright attacks, sometimes even threats. Florian himself has gotten roasted on there.
SPEAKER_08: And the Econ jobs board has become most notorious for its sexist and racist posts, often attacking specific women and people from marginalized backgrounds. A famous paper back in 2017 actually demonstrated that statistically on the site, women are disproportionately discussed with unprofessional and lewd terms.
SPEAKER_06: Florian explains the site's whole deal to Kyle, and when Kyle gets home, he pulls up the website. And at first, scrolling through all these anonymous posts, he's just kind of shocked by how gnarly some of them are. But here is the next happenstance. When he looks at the website, he sees something Florian hadn't.
SPEAKER_08: To give posters anonymity, the site was assigning them a kind of codename for each thread they posted in. They were called the four character ID numbers. But those super secret codenames are in a format instantly recognizable to anyone with programming experience, like Kyle.
SPEAKER_02: To me, it was immediately obvious how those usernames are put together.
SPEAKER_08: How do you feel when you solve a little puzzle like this?
SPEAKER_02: Oh, I'm super into puzzles for sure. This wasn't a very long puzzle.
SPEAKER_08:
Hello and welcome to Planet Money. I'm Mary Childs.
SPEAKER_06: And I'm Alexi Horowitz-Gazi. The story of the Econ Job Market Rumors website gets to this kind of fundamental tension about the internet. When you allow people to be anonymous online, you give them the freedom to share all the things they maybe couldn't otherwise, but also to say things they maybe really shouldn't.
SPEAKER_08: And because the posters on the jobs board are anonymous, no one could really say for sure. Was all this toxicity just the work of a bunch of fringy internet trolls? Or was it a symptom of a much deeper problem within economics itself?
SPEAKER_06: Today on the show, Florian and his colleagues discover that all the clues they need to answer that question have been sitting out in the open all along. And what they find sends a shockwave through the profession.
SPEAKER_07: This message comes from NPR sponsor American Express Business. The enhanced American Express Business Gold Card is designed to take your business further. It's packed with features and benefits like flexible spending capacity that adapts to your business, 24-7 support from a business card specialist trained to help with your business needs, and so much more. The AmEx Business Gold Card, now smarter and more flexible. That's the powerful backing of American Express. Terms apply. Learn more at AmericanExpress.com slash business gold card.
This message comes from NPR sponsor Grammarly. For over 10 years, Grammarly has been powered by AI technology that you can trust. With one click, you and your team can brainstorm, rewrite, and reply quickly with suggestions based on your goals. More than 70,000 teams trust Grammarly to help them with their emails, presentations, and more. Start being more effective at work by making your point and getting things done efficiently. Go to Grammarly.com slash podcast to download for free. That's Grammarly.com slash podcast.
SPEAKER_08: So engineer, not economist Kyle Jensen has just started poking around on the Econ Job Market Rumors site. And he's noticed that the format of the anonymous four character code names the website is assigning to its users look extremely familiar.
SPEAKER_06: The website seems to be using an algorithm called a hash to transform some piece of information about every poster. He thought probably their IP address to generate these code names.
SPEAKER_08: There are a couple popular algorithms Kyle suspects the site may have used to do this. So he calls up his economist friend Florian, asks for his IP address and proposes a test.
SPEAKER_02: Let me see if I can guess what your username would be knowing what your IP address is.
SPEAKER_06: He asks Florian to make a new post in a particular thread on the website. His plan is to enter the threads ID number along with Florian's IP address into a series of the most commonly used hashing algorithms.
SPEAKER_08: And sure enough, on the second algorithm he tries with just Florian's IP address and the topic he was posting about, Kyle is able to guess the theoretically anonymous four character code name the site had assigned to him.
SPEAKER_02: So it really didn't take very long.
SPEAKER_08: This is just fun for you. This is just like a little Sunday morning puzzle that you're doing.
SPEAKER_02: Yeah, I thought it was interesting. Yeah, for sure. I thought it was interesting, but it also it was a super obvious thing.
SPEAKER_06: And the super obviousness of this puzzle, Kyle says, revealed something huge about the site that whoever had set it up back in the late 2000s had not taken one of the most basic precautions when it comes to using one of these hashing algorithms. They forgot to add salt.
SPEAKER_02: Which just means you sprinkle a little bit of other data in there. That kind of obscures where those posts are made from. And the lack of assault meant that it was feasible for us to do some statistics on these usernames.
SPEAKER_08: Those statistics could potentially point to which IP addresses were responsible for each individual comment on the site.
SPEAKER_06: And while Kyle found all of this mildly amusing in a Sunday morning puzzle kind of way, when he explained what was going on to Florian, Florian immediately sees they might have something much bigger on their hands, a kind of treasure trove of data hiding in plain sight.
SPEAKER_08: Because the anonymity that had cloaked the toxic speech on the Econ jobs board seemed like it might unravel with the application of a little statistical analysis, an analysis they could then convert into the only language economists seem to understand, an academic research paper. What were you setting out to do with this paper?
SPEAKER_06: What was the plan?
SPEAKER_05: I think we wanted to just document who posts on Econ job market rumors. I had heard this criticism before of saying, oh, you know, it's just people that are not relevant to the center of the profession. It's just disenfranchised people who are unhappy at low ranking universities. They're posting all that information on there. And that's why we should just ignore it. And I think what I wanted to know is, well, is that really true?
SPEAKER_06: So it's like, are all of these toxic posts attributable to a few rotten apples kind of in the periphery of the discipline? Or does this rot go all the way to the top?
SPEAKER_05: Exactly.
SPEAKER_08: So now it is math time. Florian ropes in another friend, a mathy one, Paul Goldsmith Pinkham of Yale. Kyle had figured out the easy way to crack the code on a single post, but now they will have to do it at scale. They will need to do a version of that puzzle to every comment on the site.
SPEAKER_06: Which is an enormous task. For every comment, there's a starting pool of 4.3 billion potential suspects, 4.3 billion possible IP addresses in the world that could have been responsible. So they're going to have to do that short, obvious puzzle, a preposterous three quadrillion times. That's quadrillion with a Q.
SPEAKER_05: On our very first attempt, it took us 230 hours. Wait, the first time you crunched these numbers, it took 10 days?
SPEAKER_05: Yeah, exactly. It's the first time it took 10 days. Yeah. It's just once you program it, the code runs and you just wait.
SPEAKER_06: And what happens when it's done? Is there like a toaster sound? No, it just sends you a message that the code that you ran is completed.
SPEAKER_08: But that completed code still doesn't answer the question of which IP address was responsible for any given post. Because once those calculations are done, Florian and the team are still left with around 65,000 IP addresses that could have conceivably left any single comment. 65,000 down from 4.3 billion, but still.
SPEAKER_06: So the next part of the process is to take these 65,000 possible IP addresses for one single comment and to cross-reference them with these 65,000 possible IP addresses for every other comment across the site over the course of a week. If an IP address is on the list of possibilities for one comment and then again on the list of possibilities for another and then another, it becomes more and more statistically likely that you found a real IP address that was the source of multiple comments. Those super posters are the signal in the mathematical noise.
SPEAKER_05: Then you have for about 67% of all posts on Econ Job Market Rumors, you have an IP address associated with it. And then you can start the analysis. IP addresses for two thirds of all the posts on the site, which Florian and his colleagues
SPEAKER_06: could now plug into a sort of IP address directory.
SPEAKER_08: These directories help narrow down which zip code a given IP address has come from. They're not super accurate for locating an exact physical address within a city, but...
SPEAKER_05: What is very, very accurate is the allocation to particular institutions because they tend to have IP addresses specifically allocated to them and that very, very rarely change. And so that allows actually the identification of universities very, very easily.
SPEAKER_08: And when they narrowed it down to look at just the bad posts, the racist and sexist stuff, what they found was that the toxic posts on Econ Job Market Rumors were very much coming from within the house of economics, from institutions across the discipline.
SPEAKER_05: It's not that all the rotten apples are in one barrel, but there's at least one rotten apple in every single barrel. It's just everywhere.
SPEAKER_06: Including many of the top institutions in the field. Places like Stanford, Harvard, UChicago, even the Federal Reserve. All of them seem to have contributed toxic comments to the Econ Jobs Board.
SPEAKER_08: It might be someone using the library computer, or it might be someone writing policy suggestions or who will go on to work in the next presidential cabinet.
To Florian, this seemed like an important enough finding that he was willing to risk stirring the hornet's nest.
SPEAKER_06: Did you imagine that your work might put you in kind of direct confrontation with this online forum? No, I never imagined this.
SPEAKER_05: My work is on antitrust economics, but I do enjoy confrontation a little bit. So when I started doing this project, I was very much aware that I was going to take on anonymous hordes. But sticks and stones can break your bones, but words will never hurt me.
SPEAKER_06: After the break, Florian reveals what he's found out about the ugly id of economics to economics itself. And everyone freaks out.
SPEAKER_07: This message comes from NPR sponsor Intercom. Intercom is a complete AI and human customer service platform and the only platform to combine a fully featured inbox, tickets, Help Center, AI chatbot, phone and more. Making it not only complete, but also one of the most powerful support platforms out there. Intercom, every single tool you need, all in one place, enhanced with AI so you can give every single customer the fastest, most personalized experience imaginable. Learn more at Intercom.com.
SPEAKER_03: Hey, it's Kenny Malone with a quick but very sincere thank you to our Planet Money Plus supporters and anybody else listening who donates to public media. After all, public media means you, the public, support it. Everything you hear from the NPR network really cannot exist without your contributions. And for anybody listening who isn't a supporter yet, right now is a great time to change that. For you to get invested in creating a more informed public that is after all our whole mission at NPR. That's why we're here. If you like perks, well, Planet Money Plus offers sponsor free listening and also bonus episodes of the show featuring extended interviews, behind the scenes content and more. And if you want to make a tax deductible donation to your favorite station or stations in the NPR network, that is fantastic as well. What really matters is that you are part of the community that makes this work possible. Journalists across the NPR network need resources to do their best work and those resources have a cost. Microphones, laptops, safety gear, software, and whatever amount you can pitch in really does make a difference. So please give today at donate.npr.org slash planet money. You can also explore NPR Plus at plus.npr.org. And thank you.
SPEAKER_06: So Florian and his team found that toxic posts on econ job market rumors are coming from everywhere in economics. They're going to present their paper at a big conference put on by the National Bureau of Economic Research.
SPEAKER_08: And ahead of the presentation, Florian tweets out a little teaser. He posts an abstract of the paper which says that they've managed to identify the majority of posts on the site, including ones that they have categorized as toxic, and that a lot of the posts are coming from universities.
SPEAKER_05: This was by far the most viral post I ever put out on Twitter. I think it got something close to 2 million views and reached well, well, well beyond economics.
SPEAKER_06: And the economics world itself freaks out. In particular, a lot of those individual posters on econ job market rumors, including this one tenured professor who we are going to call Terry. Terry asked that we not reveal his identity, his real name or voice, so we're going to use a voice actor for him. He says he's worried there would be professional repercussions for having posted on the site, even though he assures us he is quote, a nobody.
SPEAKER_01: I'm not like a super famous person. I mean, most people in the field don't even know who I am. So Terry found out about the econ jobs board over a decade ago.
SPEAKER_08: And he says under the site's alleged cloak of anonymity, he had passed along some insider e information about his academic department's hiring decisions. And he says he also engaged from time to time in some light political trolling.
SPEAKER_01: It wasn't racist or misogynist, but like I've made some posts where I sound like a right wing extremist. It's kind of like a Stephen Colbert, which meant that when he read Florian's tweet, Terry
SPEAKER_06: was extremely distressed. Terry feared that if his posts were unmasked, people wouldn't understand that he'd been joking, and that there might be real professional consequences.
SPEAKER_01: I was very stressed. I was feeling dread because I had no idea like what information they had, what they were going to release, what their timeline was.
SPEAKER_06: For people who had posted on the econ jobs board, the worry seemed to be if Florian and his team can identify specific universities where some of the toxic posts were coming from, could they also identify specific people who had made the posts? Were they going to be doxing people who'd posted?
SPEAKER_08: And this worry was exacerbated when in a leaked early draft of the researchers presentation slides, one slide showed the IP address of one anonymous user that the authors had managed to identify, the site's administrator.
SPEAKER_06: The econ jobs board itself was inundated with hundreds of comments, puzzling, worrying, and fuming about the paper. People were speculating about how Florian and his co-authors got ahold of this information, whether this project might fall afoul of academic ethics rules, or if it might even be illegal.
SPEAKER_08: And they're freaking out not just because some of them had posted racist, sexist stuff, but because some of them had also posted all sorts of other kinds of sensitive information, in which they, say, trash talked their boss, or posted about a mental health crisis, even about considering suicide. Stuff they posted because they believed they were anonymous.
SPEAKER_06: So this past July, when the day finally comes for Florian and Paul to present their findings, everyone is watching. Some people who'd been harassed and attacked on the jobs board, too frequent posters fearing they were about to be doxxed. When Florian and Paul arrive at the Royal Sonesta Hotel in Cambridge, Massachusetts to present the paper, they discover they've accidentally worn matching suits, and that
SPEAKER_06: they will have a packed audience. I just wanted just to mention something a little bit different.
SPEAKER_08: There is a reporter in the room. The session is also being streamed on YouTube. We often sort of feel like we're just us in the room, but actually there may be quite a few people watching. More than 2000 people have tuned in, which is pretty high for an academic conference session. So with that warning ahead, let me give, get the stage over to Florian.
SPEAKER_05: Thank you very much, Catherine. Thank you very much for everybody for coming.
SPEAKER_04:
Florian introduces the paper.
SPEAKER_05: I believe a very important topic for everybody in this room and for everybody in our profession, namely toxic speech in economics.
SPEAKER_06: And puts up on the screen behind him screenshots from the Econ jobs board. Examples of useful posts on the site like, how to run non-linearly squares with fixed
SPEAKER_05: effects, mini tutorials economists offer up to each other, to make your Stata graphs more
SPEAKER_05: beautiful.
SPEAKER_08: And then he puts up examples of the bad stuff. This with sexist language and racial slurs. And then he delivers the headline finding of their study.
SPEAKER_05: These posts here come from IP addresses at Harvard, at Stanford, at Yale, at Chicago, and even the NBR headquarters at 1050 Mass Ave.
SPEAKER_06: Florian goes on to explain to the audience the whole story, how they'd figured out the site's method for anonymizing posters.
SPEAKER_04: A allocation scheme that did not use a salt. The tragicomic lack of salt. It's a random secret that sort of gets added to the stuff that is...
SPEAKER_06: And how they use statistics to identify which institutions these toxic posts were coming from.
SPEAKER_05: And that was also in plain sight for more than a decade.
SPEAKER_08: As Terry and the thousands of other viewers watch from around the world, it gradually becomes clear that this was as much as Florian and Paul were going to reveal. Just the names of universities and government offices. No names of individual posters. They had IP addresses, but nothing more granular.
SPEAKER_06: And IP addresses are not people. To even try to link an IP address to an individual, somebody would need to sift through tons of posts to collect biographical clues and triangulate them. And that was just not part of the paper.
SPEAKER_08: And as for that slide identifying the site administrator's IP address, the only reason the researchers had been able to get that one was because of a special setting the admin had used for one year that allowed the researchers to link all of those posts together. Also, the researchers took out that slide for their final presentation.
SPEAKER_06: Instead of doxing anonymous Econ jobs board users, what they're doing is kind of doxing the profession of economics itself. This is pervasive.
SPEAKER_05: It's everywhere.
And I think we should not look away. Thank you very much for your time.
SPEAKER_04:
Great. So as we were doing before, line up at the mics and we'll take...
SPEAKER_08: There are a lot of questions and comments. People ask about the terms they'd used in the paper. Power users and power laws are like not showing up in your data. Some of the questions are more technical about methodology. Any changes on this behavior? Some people just want to express their gratitude to Florian and his colleagues for helping
SPEAKER_06: bring this problem out into the open.
SPEAKER_08: I don't have a question. I just want to say thank you. Others want to take the findings into the real world.
SPEAKER_08: That petition, started by a professor at UC San Diego, asked the American Economic Association to explore ways to identify those responsible for the worst posts, the threats and libel, stuff that is legally actionable, so that the people who posted it could be held actually responsible.
SPEAKER_06: And then after this flurry of comments and just a little over 50 minutes, the presentation comes to an end.
SPEAKER_04: Thank you very much for all your time and all your questions.
SPEAKER_01: Thank you.
SPEAKER_08: It has now been five months since that presentation. For all the hubbub this summer, not a lot has changed. None of the universities whose IP addresses showed up as the top 14 sources for toxic posts seem to have taken any official action. We emailed them to ask and the few that did respond sent over statements about being disappointed or not tolerating harassment.
SPEAKER_06: Since news of the paper dropped, the Econ jobs board administrator has changed how usernames are given out on the site. Users are now assigned the name of a famous economist instead of a four-character codename.
SPEAKER_08: A lot of people still posting on the site, along with its administrator, see this paper in the context of freedom of speech and censorship. They point out that there are real societal benefits to anonymity for exposing things like wrongdoing, corruption, plagiarism, and that there's a cost to undoing anonymity.
SPEAKER_06: For his part, Florian Eterer says it was never his intention to destroy the site. He just wants economics to find a better way to talk to itself. As for doing more to de-anonymize the toxic posters, Florian does not see that as his role.
SPEAKER_05: I think at that point you're just veering too much into doxing, I think, and then I have just bigger ethical concerns. Would it be a net positive to expose the worst actors on that platform? Perhaps, but in economics we tend to think of trade-offs, and it is a trade-off maybe that I'm not willing to make at this point.
SPEAKER_08: Florian does acknowledge that his work may have opened the door for others who view those trade-offs differently.
SPEAKER_06: Terry, the anonymous Econ job rumors poster who watched Florian's presentation with bated breath, the whole thing has not convinced him to quit the economics job market rumors site or EJMR as he calls it.
SPEAKER_01: I think EJMR is bad for the econ profession. I think it's bad for society. I think the econ profession would be better without EJMR, right? So if I could like, you know, wave a magic wand and make it go away, like I would do that. But like whether I visit the site or not, the site's going to exist. And so that's kind of why I continue to visit the site.
SPEAKER_08: So he does now use a service that disguises his IP address, just in case.
SPEAKER_06: Argentina has a new president who campaigned on the promise to dollarize his country, make the U.S. dollar their new national currency. An economist says there could be one big upside. It's like tying yourself to the mast so as to not go towards the siren's call.
SPEAKER_07: And the siren's call is print more pesos. Exactly.
SPEAKER_08: Print more pesos.
SPEAKER_07: Right. Exactly.
SPEAKER_08: Why dollarize? How to do it? And will it even work? That's on the next Planet Money.
You can email us at planetmoney at npr.org or we are on some of the social media platforms at at planet money.
SPEAKER_06: Today's episode was produced by Willa Rubin with help from James Snead and Samuela Horst-Kessler. It was edited by Keith Romer, engineered by Josh Newell and fact checked by Sierra Juarez. Alex Goldmark is our executive producer.
SPEAKER_08: Huge thank you to Catherine Tucker at MIT and to NPR One producer Julian Ring for voicing Terry. I'm Mary Childs.
SPEAKER_06: And I'm Alexi Horowitz-Gazi. This is NPR. Thanks for listening.
SPEAKER_07: This message comes from St. Jude Children's Research Hospital. This holiday season, Best Buy is joining in to help give St. Jude families a gift that could last a lifetime. Text the word GIFT to 626262 to donate now.
