Google

Episode Summary

The podcast discusses the origins and impact of Google's search algorithm. It begins by describing how poor search technology was before Google. Search engines like Lycos were easily manipulated and returned irrelevant results. Google founders Larry Page and Sergey Brin initially set out to analyze citations between academic papers at Stanford University. This led them to realize the potential for a search engine that analyzed links between webpages. Page and Brin developed an algorithm that ranked search results based on how many other credible webpages linked to them. This helped Google return far more useful results than existing search engines. After overcoming early challenges, Google's search caught on and the company grew rapidly during the dot-com boom. The podcast then examines some of the major economic impacts Google has had. It has led to time savings, price transparency for consumers, and enabled niche products to find markets. However, Google's dominance of search also raises concerns. Its opaque and ever-changing algorithm has huge influence over businesses trying to rank well in results. Overall, the podcast tells the origin story of Google's revolutionary search algorithm and analyzes its broad economic effects, both positive and negative. The development of Google search improved the usefulness of the internet enormously but also made Google a uniquely powerful gatekeeper of information.

Episode Show Notes

The words 'clever' and 'death' crop up less often than 'Google' in conversation. That’s according to researchers at the University of Lancaster in the UK. It took just two decades for Google to reach this cultural ubiquity. Larry Page and Sergey Brin – Google’s founders – were not, initially, interested in designing a better way to search. Their Stanford University project had a more academic motivation. Tim Harford tells the extraordinary story of a technology which might shape our access to knowledge for generations to come.

Producer: Ben Crighton Editors: Richard Knight and Richard Vadon

(Image: Google logo and search box on a screen. Credit: Yui Mok/PA Wire)

Episode Transcript

SPEAKER_04: Amazing, fascinating stories of inventions, ideas and innovations. Yes, this is the podcast about the things that have helped to shape our lives. Podcasts from the BBC World Service are supported by advertising. SPEAKER_02: Introducing Carvana Value Tracker, where you can track your car's value over time and learn what's driving it. It might make you excited. Whoa, didn't know my car was valued this high. It might make you nervous. Uh oh, market's flooded. My car's value just dipped 2.3%. It might make you optimistic. Our low mileage is paying off. Our value's up. And it might make you realistic. Car prices haven't gone up in a couple weeks. Maybe it's time to sell. But it will definitely make you an expert on your car's value. Carvana Value Tracker. Visit Carvana.com to start tracking your car's value today. SPEAKER_01: 50 Things That Made The Modern Economy with Tim Harford Dad, what happens when you die? SPEAKER_03: I don't know, son. Nobody knows for sure. SPEAKER_03: Well, why don't you ask Google? SPEAKER_00: Evidently it's possible for children to grow up with the impression that Google knows everything. Perhaps that's to be expected. SPEAKER_03: Dad, how far is the moon from the Earth? What's the biggest fish in the world? Do jetpacks really exist? SPEAKER_00: All efficiently answered with a tap of a touch screen. No need to visit the library to consult the Encyclopedia Britannica, the Guinness Book of Records, or who knows how a pre-Google parent would have discovered the state of the art in jetpack technology. It wouldn't have been straightforward. Google may not be clever enough to know if there's life after death, but the word Google does crop up in conversation more often than either clever or death, according to researchers at the UK's University of Lancaster. It took just two decades for Google to reach this cultural ubiquity from its humble beginnings as a student project at Stanford University in California. It's hard to remember just how bad search technology was before Google. In 1998, for example, if you typed cars into Lycos, then a leading search engine, you get a results page filled with porn websites. Why? Well, owners of porn websites inserted many mentions of popular search terms like cars, perhaps in tiny text or in white on a white background. The Lycos algorithm saw many mentions of cars and concluded the page would be interesting to someone searching for cars. It's a system that now seems almost laughably simplistic and easy to fool. Larry Page and Sergey Brin were not initially interested in designing a better way to search. Their Stanford project had a more academic motivation in academia. How often a published paper is cited is a measure of how much credibility it has. And if it's cited by papers which themselves are cited many times, that bestows even more credibility. Page and Brin realized that when you looked at a page on the nascent World Wide Web, you had no way of knowing which other pages linked to it. Web links are a bit like academic citations. So if they could find a way to analyze all the links on the Web, they could rank the credibility of each Web page in any given subject. To do this, Page and Brin first had to download the entire Internet. This caused some consternation. It gobbled up nearly half of Stanford's bandwidth. Irate Webmaster showered the university with complaints that Google's crawler was overloading their servers. But as Page and Brin refined their algorithm, it soon became clear they'd stumbled on a new and vastly better way to search the Web. Put simply, porn websites with tiny text saying, cars, cars, cars, don't get many links from other websites that discuss cars. If you searched Google for cars, its analysis of the Web's network of links would be likely to yield results about, well, cars. With such an obviously useful product, Page and Brin attracted investors and Google went from student project to private company. It's now among the world's biggest, bringing in profits by the tens of billions. But for the first few years, Page and Brin burned through money without much idea about how they'd ever make it back. They weren't alone. This was the time of the dot com boom and bust. Shares in loss making Internet companies traded at absurd prices based purely on hope that eventually they'd figure out viable business models. It was 2001 when Google found theirs, and in retrospect, it seems obvious. Pay per click advertising. Advertisers tell Google how much they'll pay if someone clicks through to their website, having searched for terms that the advertiser specified. Google displays the highest bidders ads alongside its organic search results. From an advertiser's perspective, the appeal is clear. You pay only when you reach people who've just demonstrated an interest in your offering. That's much more efficient than paying to advertise in a newspaper. Even if its readership matches your target demographic, inevitably, most people who see your advert won't be interested in what you're selling. No wonder newspaper advertising revenue has fallen off a cliff. The media's scramble for new business models is one obvious economic impact of Google search, but the invention of functional search technology has created value in many ways. A few years ago, consultants at McKinsey tried to list the most important. There are time savings. Studies suggest that Googling is about three times as quick as finding information in a library. And that's before you count the time spent getting there. Likewise, finding a business online is about three times faster than using a traditional printed directory such as the Yellow Pages. McKinsey put the productivity gains of this into the hundreds of billions. Another benefit is price transparency. That's economist jargon for being able to stand in a store, take out your phone, Google a product you're thinking of buying and seeing if it's available more cheaply elsewhere, then using that knowledge to haggle. Then there are long tail effects. In physical stores, it makes no sense to display aisle after aisle of obscure products that will be bought only rarely. Focus on a limited range of best sellers instead. But a decent search facility makes it easy to find a needle in the product haystack. And that's what enabled the rise of online stores offering more variety. Customers with specific desires are likelier to find exactly what they want, rather than having to settle for the closest thing available in the local supermarket. And entrepreneurs can launch niche products more confident that they'll find a market. This all sounds like excellent news for consumers and businesses. But there's a problem. Google dominates the search market, handling close to 90 percent of searches worldwide. Many businesses rely on ranking highly in its organic search results. And Google constantly tweaks the algorithm that decides them. But it isn't transparent about how it ranks results. Indeed, it can't be. That would give away the information necessary to game the system. We'd be back to searching for cars and getting porn. You don't have to look far online, starting with Google search naturally, to find business owners and search strategy consultants gnashing their teeth over the company's power to make or break them. One blogger complains that Google is judge, jury and executioner. You get penalized on suspicion of breaking the rules and you don't even know what the rules are. You can only guess. Trying to figure out how to please Google's algorithm is rather like trying to appease an omnipotent, capricious and ultimately unknowable God. You may say this is no problem. As long as Google's top results are useful to searchers, it's tough luck on those who rank lower. And if those results stop being useful, then some other pair of students at Stanford will spot the gap in the market and dream up a better way, right? Maybe. Maybe not. Search was a competitive business in the late 1990s. But now it may be a natural monopoly. In other words, an industry that's extremely hard for a second entrant to succeed in. The reason among the best ways to improve the usefulness of search results is to analyze which links were ultimately clicked by people who've previously performed the same search, as well as looking at what the users searched for before. Google has far more of that data than anyone else, which suggests that it may continue to shape our access to knowledge for generations to come. SPEAKER_01: An essential history of Google's algorithm is John Patel's book, The Search. For a full list of our sources, see BBCWorldService.com slash 50 things. SPEAKER_00: Just before you go, I'd like to recommend another podcast series from the BBC World Service. It's a personal favorite of mine. More or less, Behind the Stats. This is your weekly guide to the numbers all around us in the news and in life. And you know, sometimes I even present it. Check it out.