SPEAKER_02: taylor swift overrated or icon undeniably talented but the best her music's market domination dramatic lacks diversity in themes shake it off a friedberg i mean it kind of attempted a joke right this is hot off the presses it was new and fresh wow um it did it in maybe five like a
SPEAKER_03: I mean, it's pretty great when you think about it, because it did the video, the audio and the script is what you're saying.Yes.So this is like the whole complete and it didn't do a full episode.But if it can do nine seconds, it'll be able to do 90 seconds next year and nine minutes the year after and then 90 minutes a year after that.So this is unbelievably good.
SPEAKER_00: This Week in Startups is brought to you by Vanta.Compliance and security shouldn't be a deal breaker for startups to win new business.Vanta makes it easy for companies to get a SOC 2 report fast.Twist listeners can get $1,000 off for a limited time at vanta.com slash twist. Eight Sleep.Good sleep is the ultimate game changer.Now you can add the pod cover to any mattress.Go to eightsleep.com slash twist to check out the pod cover and get $200 off the pod plus free shipping. And the Equinix Startup Program offers a hybrid infrastructure solution for startups, including up to $100,000 in credits and personalized consultations and guidance from the Equinix team.Go to equinixstartups.com to apply today.
SPEAKER_03: All right, everybody, welcome back to This Week in Startups.It's AI Tuesdays.We record on Monday.We ship it on Tuesday.Thisweekinstartups.com slash AI.Or go to YouTube and just type in This Week in Startups.Do me a favor, subscribe.Put the subscription bell on so you see all of our great shows.A couple episodes a week.And the episodes that people are going crazy for right now are when I interview a founder.
We go deep on that.Or when AI demos are done by... My pal, Sandeep Sunny Madra, who is the GM of Grok Cloud.You can go to console.grok.com, console.groq.com.It's the fastest and cheapest inference in the world, according to everybody.Things are going well.How many total devs you got there now, Sonny?
SPEAKER_06: I think as of this morning, 70,000.20,000 apps.20,000 apps with Grok on it.It's crazy.It's a really... Really special.We're very excited.Yes.
SPEAKER_03: Is it when you're doing inference, does it cost people on average to do a returned query?I know it depends is the answer that everybody gives, but just generally speaking, when people send a query up and they get a response back, It's costing a penny, four pennies, fraction of a penny for an average input, average output.
SPEAKER_06: Fraction.So basically, let me kind of give you the numbers top down.We charge, so our most popular model is Mixtral, which is a eight by seven, eight mixture of experts.We charge 27 cents input, 28 cents output.And on average, we get about 10,000 tokens of input. And we get a small amount of output, one-tenth of that.And so what are people doing is they take huge things and say, hey, summarize this for me.And it's, like I said, it's 27 cents for a million input token.So it's fractions of that.So like I say, you take a million, divide it by 10,000.
That's kind of what you're looking at, right?Or sorry, 10,000 divided by a million.That's what you're looking at in terms of cost.So fractions, fractions of a penny.
SPEAKER_03: So when we talk about the disruption that Google might face and the increasing cost, are you part of the camp that we saw last year?I think Bill Gurley was saying and Brad Gerson were saying a bunch of this, like you cannot make Google search work at scale yet because it would bankrupt the company.They would be spending more per search than they were making.Is that still true in 2025?Yeah.
SPEAKER_06: Well, it's moving down quickly.So let me kind of give you a rough sense of where costs are at.The best open source model right now is Mixtral.The cost to run that's about 27 cents per million tokens. then if you look at GPT-4, it's $10 per million token input and $30 per million token output.So it's on the order of, I'd say, 50 times more expensive on the output and then 150 times more expensive on the input and 150 times more expensive on the output.So we've already seen the prices come down that much with open source.And I think that'll be the key. smaller, more efficient models will be cost effective to do search.And, you know, we're going to see that today.
We have a lot of search examples coming up.And it's a really good insight that you're kind of queuing up for us here.
SPEAKER_03: Yeah, you know, if you look at Google, there was somebody who did a really nice analysis, semi-analysis.Let me pull this up here really quick because it's important for our discussion, I think, to be honest about this and to track it.So let me share my screen real quick here and I'll show you this.And this is from February of 2023.This is well over a year old right now.So if we look at this revenue per query, currently Google makes a penny. and a half, 0.0161 revenue from a query.They took all the queries that they estimate that Google does and they just divided their total revenue by it, I guess.And they say the cost per query is 0.0106. So they're making about 0055 queries per second, 320 annual revenue, 162 billion annual cost, 107 billion operating income, 55 billion.
And then if you were to do this for ChatGPT, they say it would cost per query 50% more.So even at the same amount of money. It would basically eat up massively Google's, crush Google's efficiency.So this is a year old.
SPEAKER_06: What are your thoughts?Yeah, so from that time already, you know, Chet, so when that was launched, we didn't even have Chet GPT-4. And or GPT-4.And so we've already seen GPT-3, 5 come down 100x in price.I sort of am in the camp of one, we should get Dylan to rerun that analysis. with the latest open source pricing, I think that'd be a fun exercise.And I fundamentally believe that people are already taking advantage of that cost reduction that's happening.And if you look at something like Perplexity, who we're going to talk about today, they have 50 million active users already.That's scale now, and they can run that.And I think that's pretty impressive.
from their standpoint, and I think fundamentally aligns to what we've seen with cost reductions happening in the inference space.
SPEAKER_03: So it's basically happening already, thanks to the inference cost, thanks to open source models, and ChatGPT, of course, has themselves reducing their costs, or what they charge 90%, right?Seems like they do that every year, they cut out 90% of the cost.
SPEAKER_06: Yeah, I think someone was saying there probably needs to be like a new Moore's law because it's moving.It's like, you know, Moore's law was like doubling every 18 months.This is like coming down 10x every 12 months.You know, it's really crazy.
SPEAKER_03: Yeah.Yeah.It's just compression.It's the discount for using this hardware just is unbelievable. And that means people will be able to do crazier and crazier offerings for free.So if you remember the YouTube moment before YouTube or Dropbox, storing stuff on the web, especially video, large files, was just an impossibility.And big data... was an Achilles heel for most companies, if they started storing a bunch of data, they just had to start deleting their log files.Remember those discussions?Hey, because where we got to compress them, put them in offsite storage, we're never going to use them, what are we going to do here with all this data?
And then, of course, cloud computing followed its own type of Moore's law, although slower.And we saw the cost of storage go way down.So these things are happening in parallel.So somebody will be able to offer things for free that people were previously paying a lot of money for.
SPEAKER_06: I completely agree.And look, we've seen a lot of this throughout our eras.And sometimes you have to just remind yourself how cheap... storage got right and and you know you can buy like a 10 terabyte drive for like 100 bucks now right
SPEAKER_03: It is truly insane if you look at the cost of hard drive storage.It is nuts.It has just come down and down in cost.It is nuts how much it costs.Average cost per gigabyte since 2017.Let me share this.This is Backblaze, which is like a backup service that is popular with some folks.Here is storage. BackBuy's average cost per gigabyte since 2017.And now they buy a lot of drives there.
And what you can see is the cost per gigabyte started in 2017.This is 2017, right?I'm not going back to 1997 or anything here.Yeah.It was at 32 cents.And it's now in, it hit 15 cents just in 2022 or so.So it went down more than half in the last five years, which is truly astonishing if you think about it. Wow.
SPEAKER_06: Yeah, I would love to get a longer... Yeah, you can't even... Probably the arc is too big.I mean, I'm just on Amazon looking at a 10 terabyte hard drive.It's 200 bucks.Yeah, it's nuts.Just getting like some... really uh that's 10 000 gigabytes jk that's unbelievable 10 terabytes 195 dollars that's just wild that's 10 000 gigabytes it's nuts you have to divide this number by 10 000 to get the price per gigabyte
SPEAKER_03: But it'd be interesting to see over time also is like, what's the hosted cost with electricity and everything just fully baked, like a terabyte in the cloud.What does it cost?10 terabytes in the cloud.What does it cost?The hard drive is now, it seems, one of the cheapest parts of this.The physical hard drive now is so affordable.My understanding is in a lot of these storage facilities, when one of these drives goes bad, they remotely turn it off. but it's not worth the cost of going into the rack and taking it out.So they just power it down and they just leave the dead thing.They leave the carcass in there.
SPEAKER_06: And they just do a sweep at some point, right?
SPEAKER_03: At some point, they probably, when they're going to take the whole rack out and upgrade it from four terabyte drives to 10 terabyte drives, that's when they decommission it and they just don't even worry about it.That's what somebody who's in the rack space told me. Listen, a strong sales team can make all the difference for a B2B startup.But if you're going to hire sharks, you need to let them hunt and you can't slow them down with compliance hurdles like SOC 2.What is SOC 2?Well, any company that stores customer data in the cloud needs to be SOC 2 compliant.If you don't have your SOC 2 tight, your sales team can't close major deals.It's that simple.But thankfully, Vanta makes it really easy to get and renew your SOC 2 compliance.
On average, Vanta customers are compliant in just two to four weeks.Without Vanta, it takes three to five months.Vanta can save you hundreds of hours of work and up to 85% on compliance costs.And Vanta does more than just SOC 2.They also automate up to 90% compliance for GDPR, HIPAA, and more. So here's your call to action.Stop slowing your sales team down and use Vanta.Get $1,000 off at vanta.com slash twist.That's vanta.com slash twist for $1,000 off your SOC 2.So let's get started.
Let's do some demos.
SPEAKER_06: All right.Okay.So what we're going to start with actually is just a little level set on where things are at because we had a shifted moment last week.And so this LMSYS... chatbot arena leaderboard is generally considered the gold standard for chatbot experiences.So, you know, for your individual use cases, different models are merging.But why I wanted to highlight this was last week, Claude three Opus by Anthropic passed over chat GPT for 1106 preview and 0125 preview. This is a big moment.And so we've seen it pass and we've seen it kind of come in up top with the score of 1255.
SPEAKER_03: And so if you go to Hugging Face, which is a hosting company for different open source models, they host these leaderboards.These leaderboards will test LLMs, large language models, against certain tests. And this is the chatbot arena.And this is where they test human preferences to rank LLMs.And they use the ELO ranking system.If you've heard of that before, that's the one that's used for chess.Basically, you're taking competitors.And if you can beat somebody with a higher score than you, your score goes up.If you lose to somebody who's better than you, their score goes up dramatically. Or if you lose to somebody who you're better than, their score goes up dramatically, yours goes up minor.
It's all those kind of concepts.You can look up ELL ratings or rankings.So they test these things.And yeah, so parity is here.That's for sure.The thing that's interesting here is the first 10 are proprietary models.Proprietary models are still... winning the day and then if you scroll down you start to see some open source ones from alibaba yeah coheres model yeah if you scroll down even more then all of a sudden you start to see a ton of them uh mistral etc llama and other this is a fine tune of llama too so you're starting to see like different ones but you know it's really interesting what we're starting to see happen here yeah So what do you take from the fact that the top rankings, the 1100s or more, are still primarily proprietary models?What does that tell you?
SPEAKER_06: Well, I kind of see two or three different things in here from my perspective.One, what this is showing us is... These folks are training with data sets that are above and beyond what's available in the public domain.And that's what's making them better.Now, that could be things that they've licensed.It could be synthetically created data.We don't know. Two, what it also means is they may have more humans involved in the loop to improve the experience with humans.So more reinforcement learning with human feedback, right?
SPEAKER_03: And so people who know Claude... is Anthropic's model.So the company Anthropic, you keep reading about that Amazon just did their second investment.Their second tranche of an investment.That is the makers of Claude.And you can check out claude.ai.What this is saying, to be clear, is people like Claude as a chatbot.And it scores head-to-head with other services higher. based on this ELO ranking.Yeah.
SPEAKER_06: That's a good, that's a good way to put it.And so, you know, what it also means is that, and the last thing I'd say is no one has a defense.No one has a lead that's insurmountable.Yes. That's the most important takeaway perhaps here.Okay.
SPEAKER_03: So let's keep going.Let's do some demos.
SPEAKER_06: Let's do some demos.Okay.So what we're going to do today is we are going to actually dive into, and J. Cole, you have like a really, I don't know if like, because I don't think you even read some of these notes coming in.So I'm really blown away by how you have this like knack to pull this off. But what we're going to do today is the segues.Well, it's not even the segues because we were talking about this as we were leading into this.So what we're going to do, what we're going to talk about today is we're going to do kind of three examples of let's call it modern answer engines.And so the first is you.com.Are you familiar with this?Yes.
SPEAKER_03: You.com is, you know, making a chat GPT Google like competitor.
SPEAKER_06: Yes.And so the query that I've been using with these folks is, what is the chance the Warriors make in the playoffs?Right.And what's interesting is, you know, one, we can see here, obviously it comes up with an answer and it pulls some different data sets.And I think these folks are doing a really interesting job.Clean UI.They come up with an answer.They provide the references on where they got these things from. Right.
SPEAKER_03: So they do citations.The first citation comes from Mercury News, the local paper in the Bay Area.Yeah, the local paper.Exactly.And it gives the Warriors a 48.6% chance of reaching the playoffs.Yeah.Most likely through the play-in tournament.Yeah. And the ringer gives the Warriors a 92% chance of making the play in tournament and a 26% chance of making the playoffs.In other words, not having to do the play in tournament, being six-seeded or above.
So the question is, are these accurate or where did these come from?These could be before the season even started, let alone with 10 games left in the season when we're taping this.
SPEAKER_06: Yeah.And so, you know, they allow you to kind of click out and you can go see where this came from.And this article you can see here is from a couple months ago now, right?So maybe not the best shot, right?And so, look, I really like what these folks are doing, but I have an overall theme here. The next is we'll go to perplexity.And one thing I want you to really notice here is that they're kind of, which I'm going to call this out on purpose because I think it's interesting.What these answer engines are doing is they're having to use someone's search behind the scenes because they haven't gone and built their own search index yet.
SPEAKER_03: They're not doing the crawl of the web.So they're either doing a Google search.Maybe they're using Brave has a search API out right now.That's really good.Exactly.Open crawl is no good.That's all that's just done every month or so.So that's not a live search engine.Yahoo no longer has Search Monkey.Bing, I think, has a search API.
So I think the only two search API choices now are Brave and Bing.I don't think Google has a search API.
SPEAKER_06: No, Google does.There's a bunch of companies that are doing it off the Google one as well.So that's what's kind of interesting here.And then sort of, you know, what I wanted to talk about here is, you know, I built this.
SPEAKER_07: Make sure we, you know, we blew it so it is open.
SPEAKER_06: Are the chances of the Warriors making the playoffs?So this is something that I built.And what it does is you can kind of go here and you can look underneath the hood.I use actually SERP API.And what mine did was it did a search.It got this back from the DraftKings Sportsbook, right?And basically it gave us that as the final answer here. And so really what we want to talk about here a little bit, Jake, and I want you to chime in because of your experience with Mahalo.Yeah.What these folks are trying to do is use a search API.
You already named what they are.And then they're trying to take into account, I guess, the temporal nature of the question and other aspects, create an experience.Yeah. With what we just started with, with the cost of what's happening with these LLMs, what advice would you give these folks that are building these things in terms of really creating the next generation experience?Because I feel like they're on the path, but there's now a bigger opportunity than where they're at.
SPEAKER_03: I would say that how you get to the answer is super important because anybody can copy the presentation layer.And so this presentation layer is not trademarkable or copyrightable.So if it turns out that the best way to give an answer is to write it in a chat format with bullet points and citations, And then put some photos or video links, which we used to call comprehensive search, not just 10 blue links, but you would include image search, video search, message board search, etc.All of those little innovations are probably not enough to get people to switch from Google, it's really hard to get people to switch off Google.So I think, yeah, the the format matters, you know, the presentation layer, so how they're presenting it, I think will appeal to some people more than a Google search result. Getting people to change their behavior is extremely hard.It has to be 10 times better in order for people to change their behavior.So what you have to ask yourself here is, is this interface that we've seen here, where it gives citations and it writes it in a narrative, is that actually 10 times better than Google search? If it's not 10 times better, I don't think people switch.
And so these become, you know, I don't want to say roadkill, but essentially, it's not enough for people to switch their usage.Now, one percentage point of search used to be worth a billion dollars in market cap.I don't know what it's worth now, because you just take Google's market cap and then take out stuff like YouTube.It's, you know, one point of market cap might be 10 billion now, right?It might be a trillion dollar market.
SPEAKER_06: So I'm going to pull up two things here.This is my Google that doesn't have access to the latest.And like this, it's the same exact question.Yeah.This is terrible.Like Google has fallen really far behind.
SPEAKER_03: So where did you get that answer at the top from?That looks... Like it was taken from somewhere.
SPEAKER_06: Well, it's right here, right?About featured snippets.
SPEAKER_03: What people need to understand about snippets is Google will pull data and format it.It used to be called OneBox.Now it's called snippets from somebody's website.This is a very controversial thing, but nobody really stands up to Google about it.But as you can see, they've intercepted you. from going to that source.So the question is, will that source allow that to happen over time?I don't know the answer to that question.Yeah.
SPEAKER_06: Yeah, but what I'm really struggling with, J. Cal, just going back here, is that between the startups and between Google, Google is really falling behind here.Because whether I'm here and I ask the question about what are the Warriors' chance of making the playoffs, and I get a bunch of references and some may be out of date, or if I'm over at Perplexity, which does a better job of pulling in and showing me the dates from different...
SPEAKER_03: Yeah, here in the perplexity one, they understood that they should put the date, which is a presentation level thing.Just whatever the article says the publication date was, show it.So that's a smart thing.When they're doing references, they said always include the date.The date here says the first one is March 5th.They have a 42% chance of making it.On March 26th, basketball reference said a 26% chance of making the playoffs.Now, and then you also have the definition.If we talk about the playoffs or the play-in. Right.
So that's like a very odd, unique thing.The play in tournament gets you the last two slots in the playoffs.So, you know, this is like one of those searches.You picked a really good search because getting the proper answer is hard.So, yeah, you might argue that, you know, perplexity and you.com presenting it or chat GPT with Bing support or Bing's doing it would be a better experience for this query.Yeah. And this is a really unique query because the data changes every day.And there's not a live data feed.Now, if you said, what's the stock price of Tesla or what's the stock price of Microsoft, Google is going to nail that.I don't know if these services are going to actually do very well on that.
If you ask it, hey, what's the price of Tesla stock right now?Let's see. Yeah, so I just declawed, and I don't have the top version, and it's not giving me anything.It says, unfortunately, I don't have access to live stock data.So they haven't even put a live stock data feed into some of the stuff yet.What does it do when you ask those to access the stock price?
SPEAKER_06: I just asked perplexity, so let's pull it up.And I think they've done a good job with these live resources.So it says 173.81.
SPEAKER_03: if i start putting a citation on it that means it's not taking it from a data feed it's taking it from no that that's accurate that's it's 173 76 so that's accurate yeah chat gpt4 i asked it what is the stock price of tesla and the current stock price of tesla is 173 32 so interesting I guess some of them are now pulling in live data feeds.The issue is there's no live data feed of odds for sportsbooks, essentially, I think.
SPEAKER_06: Well, this is where I wanted to kind of keep building on this.What these different services can do is use LLMs to parse that because these odds exist at any given time on different sites, whether it's DraftKings or TakeYourPick. And the power of these kind of modern answer engines is that they can look at that entire page and get your answer out of it.And I think that's what's really, really powerful.And what blew me away and why I wanted to do these series of demos is when you ask these very specific questions, it does feel like Google has fallen way behind.
SPEAKER_03: I'm not sure what these services do is they summarize the pages that the first five pages that come up in Google, Google can add that I think it's going to become a copyright issue.I think that's probably Google hasn't done it at scale. I think summarizing the articles, the way this is being done without permission is a jump ball as to whether people feel good about it or not.
SPEAKER_06: But doesn't this go back to our conversation from last week where you said as long as they put a snippet and then link out and send you back there?
SPEAKER_03: Yeah.So I think I wouldn't have a problem as a content creator with one sentence being in there or two sentences and then clicking it.If it's the entire article, I think that's where we're going to have questions. So here, you know, if you summarize the entire New York Times story, I think the New York Times would feel bad about it.If you summarize and make it two sentences or three sentences, would they reasonably be upset about it?Would that be fair use?It'd be a negotiation.And so... The question is, will these actually drive people to those sources?And when you press that source, when you press that number, does it take you directly to the source?
Yeah, it opens another window and takes you there.So they'll make some minimal amount of advertising here.But I think what we'll see is these folks will ask for maybe some compensation if you use their data over time.
SPEAKER_06: Interesting.So where do you rank these with respect to Google?Because for me, they're doing a much better job than what Google is doing right now.
SPEAKER_03: It's going to depend on the query.You know, if I'm doing a flight query or the price of something, a stock price, or I'm just trying to get to a specific news story that I know, like there's a news story that I want to read. I probably would not experience much of a difference between ChatGPT and Google.If I wanted a summary of, hey, you know, what are the best restaurants in, you know, Austin?Or what's the best barbecue?You know, I do wonder if the Yelp search result, the Google search result, or one of these new ones would be a better experience for me. elite performers have a secret and it helps them stand out in their field.You want to know what that secret is?I can tell you.It's sleep.
Whether you're an athlete, an entrepreneur, or you're just a career-driven professional like the people who listen to this podcast, quality sleep is the fuel that powers your success.Well, how do you get deep, restful sleep? Very simple.You do what I do.Eight sleep.I got two eight sleep mattresses.I never miss a night on my eight sleep and they're selling more than just mattresses right now.You can sleep like a champion with the eight sleep pod cover.That's right.You don't have to get rid of your existing mattress.
You put the cover on, slips right on top of your bed and instantly transforms your sleep game.We're talking about dual zone heating and cooling.So if you have a partner and you're both in the same bed, you know, one might sleep hot and one might sleep cold.I like it a little chilly willy. And this smart temperature adjustment keeps you in your ideal zone for a deeper uninterrupted rest.In other words, you set how you like it, but it adjusts the temperature to make you sleep more.This is really interesting science, folks.And this advanced sleep tracking will help you monitor your metrics like your heart rate, And if you can measure it, you can manage it.That's the secret of Eight Sleep.
And get that Eight Sleep pod cover.If it's any mattress, it's going to work well with whatever you got.And when you maximize your sleep, you maximize your potential.You know that.So whether you're having sleep problems or you just want to optimize, here's your call to action.I want you to invest in the rest that you deserve.Go to eightsleep.com slash twist and get 200 bucks off the pod cover.What a great deal.Eightsleep.com slash twist for $200 off the pod plus free shipping.
SPEAKER_06: So do you think we have a chance to convince you to try these for the next week over your usual?
SPEAKER_03: Oh, yeah.No, I did that for a while.I did that last year.I made my default screen, but I will make my default search.Which one should I make my default search for a week?
SPEAKER_06: Uh, I think maybe like perplexity for, you know, like to me, I think, look, they're very similar.So I'm going to, I would give them the same grade between you.com and perplexity right now.I give them both a B. I think it's like, where's the room for upside for them?Cause I'm a bit higher where, where, where, what can they do on upside?
SPEAKER_03: I think it has to be correct data.And so I think, you know, the, I don't trust it. I think it was very misleading, the first one you showed, because it didn't have the dates of the stories.So I think their summarizing there was quite wrong.Because they presented four different sources with four different pieces of information, but they didn't say, as of this date, this is the chances as of this date.So perplexity seemed to actually get that nuance.So I give perplexity a B+.I give the other one a C+. I'll get the UFC class for this query.
SPEAKER_06: So you didn't like that they didn't put what these were and what you like.
SPEAKER_03: I just don't think it's the correct answer.I think they're kind of leading you. down the wrong road, they're actually confusing this, they're potentially confusing the situation.And again, you picked a really wonderful query, because it's a confusing query.It's kind of like the query of like, how much does it cost for a flight between these two locations?It's like, it depends, depends on what time you want to go.It depends on if you're in business class or first class, it depends on if you want to travel, you know, at Monday morning at 6am, or you want to go in the middle of the day, like all of these factors come into play. And I think that this is where these language models could ultimately have a massive advantage is that they might actually think about the second and third search and kind of pull that in.And so if you say, hey, what does it cost to do a flight?This is a good one to try.
How much is a business class flight between San Francisco and Dubai?And this is like really getting the right price on this. That's one that takes a little bit of work, right?Because you have all those sub questions, which airline, what days.
SPEAKER_06: Where I was going to go with this with you, let's see what result you get.But Jason, what is your, like, if you were trying to look at Warriors odds to get the playoffs, because we want them to get in, you know, we want to see our boy dream on just, you know, have another shot at it.We know once they're in there, everything changes. What is your process?Because again, I keep coming back to Mahalo, like you've thought about this before.So what, and obviously it's changed, but what is your J-Cal algorithm for figuring out what the best odds are right now?What would you do?
SPEAKER_03: Yeah, I would just type odds of making the NBA playoffs.I wouldn't put the Warriors in there.And then I would try to find a website that had the playoff and the odds there.So if you just type odds of making the NBA playoffs into Google, it'll take you to playoffstatus.com and sportsbetting.com.
SPEAKER_06: So those two is the one who they did the one box off, by the way.
SPEAKER_03: Yeah.So playoff status seems to be the best result.And yeah, that's what I would do.
SPEAKER_06: I would try to find a website that had those.You think it's the best result or they paid for that?We don't know.
SPEAKER_03: No, you don't pay to be in one box.One box you can't pay to be in.So you have to earn that editorially.I think because this is a single page with a nicely formatted table, that's why this one wins.This is why that one wins the Google SEO reach.Having a domain name that answers the question Like their domain name, playoffstatus.com answers the question.You know, it's like they took the time to get that domain name.Therefore, and it's a single purpose.When you go to the homepage, it just gives you the exact probability, which is wild when you think about it.
It's also wild that my Knicks are ahead of the Warriors are. In the Eastern Conference, yeah, we were like 10.Not bad.2% chance of champions, 5% chance of the finals, 16% conference champions.Round two, 52%.Round one, 99%.Yeah, we know we're going to make the finals.
SPEAKER_06: Well, your boy, Jalen Brunson, has got to stop trying to get his stats and get wins instead.
SPEAKER_03: Oh, come on now.No, he only, we have three starters out right now.So when we have two or three starters out, it's really hard for us to, we have three starters out.It's the fact that we're still in it.So that's what I would have done.I do think these things have great potential. So I think in another year or two, this will be a very interesting conversation.And can you change people's habits?Do people just like getting the 10 blue links, the one box, you know, and then this perplexities app and, you know, is it too much information?People like the process of just, you know, being transported to another website or getting a quick answer in the one box.
SPEAKER_06: I think you're boomering yourself here, J. Cal.Might be.Yeah.I think people just want the answer.
SPEAKER_03: Oh, no, I definitely think people just want the answer.I'll agree with that.The question is, is that answer correct?Is I think the key issue and is summarizing a bunch of web pages.Yeah, you know, the best way to get the correct answer.You know, I don't know that it is.I think going to data sources is going to be the way to get the correct answer.And that's where data source relationships are so important.Having a lot of sports scores, weather. getting all those data sources together is going to be... And having a licensing arrangement with them would be really great.
SPEAKER_06: One more double-click, though, J. Cal.When we look at that, what you were just talking about, the playoffstatus.com, are they pulling those in from other sources?Or is this just... Because I find that really interesting.Or... versus, you know, DraftKings or something like that, any of the sports makers like, you know, odds makers like Vegas, they really nail it over kind of the mathematical chances.It's a great question.Yeah.
SPEAKER_03: Okay, cloud computing has revolutionized startups over the past decade, you know that.But the reality is a fully cloud based solution is not right for every startup.Sometimes a hybrid solution is your answer.Like if you're working with sensitive data that can't be trusted to cloud, or if you need to connect to multiple cloud providers at once, or Maybe you just want a much more cost-effective solution.In that case, you need to check out Equinix.Equinix Metal will give you direct access to physical servers, but you still get all the benefits of the cloud, so no need to rack and stack your own servers, no.Equinix provides on-demand infrastructure in over 25 major cities. Here's the best part.They have an amazing startup program for you.
The Equinix Startup Program offers personalized consultations and guidance from the Equinix team.And of course, you'll get up to $100,000 in startup credits.So here's what I want you to do.Head to EquinixStartups.com to apply.And when you apply, James from Equinix is going to reach out to you directly.That's EquinixStartups.com to apply.E-Q-U-I-N-I-X Startups.com. You know, the sportsbettingdime.com is like the number two result.And it seems like they're posting articles.So again, is it data or is it an article is a key piece of this.
Yes.And so, you know, are you... And it's pulling together... interesting this website is pulling sb sports betting dime.com is pulling odds from other um sports books so they're intercepting traffic i think and they and their ads are all sports books so you know that game of seoing and intercepting traffic for data and then clicking through to get an ad like i think that whole game could be coming to an end yeah And those ads, by the way, are served up by Google.So when you start thinking about what's really happening here, Google has had an incentive to keep things inefficient.And I'll explain this here on an SEO basis.If you're not looking, this is an article from March 26.It's April 1st when we're recording this.So this is an old article, but it came up as like number two in the Google search rankings.So pretty good job of Google to find this article.
But this website is not the sports book itself, to the best of my knowledge. What it does is it's pulling together a bunch of data, making charts, et cetera.And it says odds of March 26 at Bet365.Bet365, odds makers, proper odds makers.Proper odds makers.So they have ranked above that group because they don't have to do all the work that Bet365 has to do of running a sports book.All they have to do is SEO and make a page that's better than sports books. intercept them in the google search and sure enough when you scroll through this what do you see here's an unlabeled ad but this ad literally has no label on it but it says get promo and you can see and by the way that's against the law to do that technically to have an ad without like an ad label but i guess it's a partnership maybe it's in their terms of service somewhere and then odds of March 26.And then you have five offers here, Caesars bet MGM sports.
So again, these are not exactly labeled, they say claim offer.So maybe you could argue that that's an offer.But you know, they're there.That's the real interesting thing that's going to go away this whole race on the web of SEO. to try to beat other people out, to then put ads on your page.That was Google's favorite thing to do.If it was inefficient, you clicked on a couple of ads, either on the destination page or on their page.If you give an answer, you don't need to click on an ad.So that is the fundamental existential issue that is correct to think about with Google.If you just got the answer, do you need to click on stuff?
Maybe. Maybe not.You know, if I did a search for these odds in a chat GPT format, actually would look good to have the offers on the bottom.So they know I might be interested in those offers.So those offers would work just as well.And so actually, in a way, what perplexity and you.com are doing is what this SEO person is doing.
SPEAKER_06: They're creating a page.Exactly.
SPEAKER_03: They're creating a page to intercept traffic.
SPEAKER_06: But they can do it more dynamically because they're using their knowledge of AI models and all that.
SPEAKER_03: And they have all of the content they've scraped from these other sites.So everything's eating its tail.
SPEAKER_06: Well, they didn't scrape anything.Let's be careful.I don't think they're scraping anything.They're doing it all real time because when you do it, they're looking at the results that come back and their engine is going through those results.
SPEAKER_03: Yeah, and summarizing them.Yeah.Yeah. Now, if you ran that software on your computer, the summarizing software, there's nothing illegal about that, by the way.Yeah.But if I build an advertising business based on that, maybe there is something illegal about that.Oh, that's interesting.
SPEAKER_06: So you're saying if I could download a plugin that would run a model locally, I'm allowed to do that?
SPEAKER_03: Yeah, what you do with content on your machine is your business.So it's when you take... If you downloaded all the Star Wars films and all... If you had all the Marvel films that you paid for and you started editing them on your computer and making your own versions and videos, there's nothing illegal about that.You can do a remix, whatever you want.It's when you republish it and you put it into commerce in the world.And try to monetize it, yeah.Yeah, an interesting thing to do would be eventually we all have an index of the entire web on our laptops. It sounds crazy to build this idea, but it could be done.And it could be a very powerful service if you think about it.Like if I took the entire Wikipedia, had it stored local, if I had all the YouTube channels I subscribed to, I downloaded all those videos, which I think technically would be against their terms of service, but whatever.If I had the transcripts of them, like you could start to build this like local LLM that might be resistant to legal issues.
Yeah. Or more resilient.It's like saving.If you want to print something from the New York Times, they can't stop you, right?Yeah.And if you want to save it to your local computer, they can't stop you, nor do they try.Remember there was a time when you would go to print something or save a PDF and they would kind of put some roadblocks on it?Yeah, yeah, yeah.Remember there were some websites you would highlight some text, you'd go to copy and be like, you can't do that.Share it using this feature.
yeah exactly yeah and that kind of eventually went away and now what people do is you hit file print it shows you a pdf you hit save on the pdf and you send the pdf to somebody you know if it's like a behind the paywall story that's what you see people doing in the back channel so i think this is a really interesting discussion um i i do think the chat interface will be the winning interface i think the talking interface will ultimately be the winning interface
SPEAKER_06: And you're still, you're at B for perplexity and C for you.Okay.
SPEAKER_03: Yeah.I think that these things have a ways to go before I change what I'm doing.But I think the voice interface is going to be the winning one.Having it in your ears, your AirPods, your Google Pixel Buds, whatever.
SPEAKER_06: Well, one more thing.I just want to close out on that one, J. Cal.I just saw this in the news.So next week, Meta is adding AI to those smart glasses.Sure.Why not?Yeah. this is kind of going in the direction of what you're saying.So it can identify things.And so you can basically sit there and do that.
So I think that's pretty interesting.
SPEAKER_03: So what's interesting about that is that's going to chip away at searches.So Google searches has, you know, continue to grow over time, but apps like Yelp, you know, or Airbnb or other ways, which they bought, but you know, other searches got picked off by apps and, Or other browsers like Brave that has its own search engine.What this says to me is one of my searches there in JetGPT was I took a picture of a vintage car to see what kind of car it was at a stoplight.Yeah.And it gave me the right answer.And so these things are going to do that.So when you take out your phone to go do a search and say, what is this? Or how many calories is this?Or what is this fruit?
Or tell me my directions.These things could conceivably give you good answers.It's almost like meta is going to be getting into the search game.Everybody's going to be in the search game.Actually, that's the conclusion I come to.Everybody's going to be in the search game.Everybody's going to be chipping away at Google search.It'll be built into 20 different products, 200 different products, and you'll get answers in other places.
SPEAKER_06: I give both the previous folks a score.I really like what they've done, and they've changed my behavior, and I'm using it more so.So I give them both. a minuses and for me what i want the ability to do is customize my search algorithm if they gave me that where i can say i want you to you know you can do some things in chat gpt like tell it to always respond with tables and stuff like that i want the ability to do that and if i could get that oh man that would be well and if you said like listen business insider i'm not interested in news from business insider because it's like toxic and it's
SPEAKER_03: They're just reblogging other people and I don't want the Huffington Post because they don't do our original work and I don't want, well, Vice is out of business and Business Insider probably be out of business soon.I'm not sure.But Vice is definitely out of business and then BuzzFeed.I was thinking of BuzzFeed is out of business too or close to it. So anyway, you could just say, I don't want those kind of rebloggers in my feed.So just don't show them again.And that can happen through like a very interesting discussion, you know?I think so as well.
SPEAKER_00: Don't show me that anymore.
SPEAKER_03: So my letter grades were for perplexity.I think they did a slightly better job.So I'll give them a B. And for the first one, U, I'll give it a C. I think they both need a lot of improvement just on making sure the results are tight.Too harsh, Jake.Too harsh.I mean, listen, it's a search. I want people to work harder.That's it.Make the answers correct.I still stand on the fact that AI is, in many cases, unusable because of the hallucination and the eras.
This year has to be the cleanup year.There's just too much bad information coming out of these models.
SPEAKER_06: Okay.All right.Okay.I give it to you.All right.Let's keep going.We got a couple more.That one, that was a good session though.I really liked it.I think it's, it's going to change the landscape of the internet.
So I think it's kind of a good one.
SPEAKER_03: It's a really good strategic discussion of how SEO, how this like micro publishers doing these landing pages and how these folks perplexity and you.com doing, you know, it's, it's very easy to replicate this by the way, to what perplexity is doing, what you.com is doing.Anybody can replicate that very quickly.And they're both, I think using chat GPT four.
SPEAKER_06: So they have different set of models.I think it's easy to get to like a baseline replication, but I think that's where, you know, they can stack on and they can differentiate right now.I actually think it's getting harder and harder.I think a simple thing is easy to do.
SPEAKER_03: Okay.All right.What's going to happen is like Google will have all of these features.Bing will have all these features.Facebook will add all these features.I think it's just way too hard to displace Google.And I think Google will just add these features as they need to. We'll keep coming back to it.I would do a long bet with you that in five years, those companies will have less than 1% search market share.You want to make that bet?
SPEAKER_06: Wait, but if they ever go over 1%, if they ever go over 1%, I win.
SPEAKER_03: I'm giving it within five years.
SPEAKER_06: But they just have to cross it once.I'll take that bet.
SPEAKER_03: Yeah, they have to cross it on, you know, whatever, in a quarter. Yeah.In a year.I don't know.Pick a duration.It can't just be like they had one day and they spiked because they were on the cover of the New York Times.
SPEAKER_06: Not one day, but over some, like a quarter.Any of the next five years.
SPEAKER_03: Not a whole year.They hit 1%.Okay.In a quarter.In a quarter or a year.
SPEAKER_06: Add it to the bets.We got it.
SPEAKER_03: So 5K.Yeah.Bet. You and perplexity will not break 1% market share in a quarter within the next five years starting April 1st.I disagree.Well, you do it on a quarterly basis.In a quarter, that will be five times four, 20 quarters.
SPEAKER_06: I'm going to track that crazy.
SPEAKER_03: Yeah.There's 5K at stake here, so we'll see who wins.Okay. I don't think they're going to break 1%.Add it to the bets.
SPEAKER_06: Okay, let's do another demo here.Next one.Okay, next one is Infinity AI.I really like this as well.This one is one of the ones where they basically don't have a UI.You have to do it on Discord.And so what I did was I said, do a short clip of the all-in podcast where they debate about Taylor Swift.Okay.It's kind of interesting.And what we're going to do is I'm going to play this.
So this was my prompt. This is the short clip.
SPEAKER_03: And it's called Infinity AI?
SPEAKER_06: Infinity.ai, yeah. And basically, I'm going to play this for you.
SPEAKER_03: And these folks use Discord because you don't have to replicate a chat infrastructure.You can just use their chat infrastructure.
SPEAKER_06: So what Discord does is it kind of interestingly brings like community chat, all kind of mixed into the same thing.So it's like, it'd be like back in the day, launching your app on ICQ.So if you launched it there, like people can start talking about it and using it and connect it.So create some virality to it. And, you know, it's really good in the early days because people are just always doing it.So, ready?I'm going to play this.
SPEAKER_02: Taylor Swift, overrated or icon?Undeniably talented, but... The best.Her music's market domination?Dramatic.Lacks diversity in themes.Shake it off, eh, Friedberg?
SPEAKER_03: I mean, it kind of attempted a joke.
SPEAKER_06: Right?And look, this is hot off the presses.It was new and fresh.Wow.They did it in maybe five, like a couple of minutes.Wow.Not bad.
SPEAKER_03: I mean, it's pretty great when you think about it because it did the video, the audio, and the script is what you're saying.Yes.So this is like the whole complete, and it didn't do a full episode, but if it could do nine seconds, it'll be able to do 90 seconds next year and nine minutes the year after, and then 90 minutes a year after that.So this is unbelievably good.
SPEAKER_06: I think your framework is right.The timeframe is going to be a lot quicker.Okay, six months, every six months.
SPEAKER_03: Yeah.So in six months, it's 90 seconds, and another six months, it's nine minutes.Yeah. And in a year and a half, it'll be 90 minutes.So I would say 18 months for them to do a full episode.That is not laughably bad.Yeah.
SPEAKER_06: And like a short little clip.Okay.Here we go.Oh my God.Okay.
SPEAKER_03: Oh my God.Full 90 minute episode of over under 18 months.Can do a full episode that is like actually watching.Wow.And you would like watch the episode. Cause it would be funny and intelligent.It wouldn't be stupid.Just nonsense.Like this was almost watchable.Like you'd almost be want to say, I'll keep listening because it was, you know, like I kind of wanted to hear more.
I was like, yeah, I didn't want to hear more.Like, where's this going?This discussion is kind of appealing.So, uh, what do you say?18 months, they can do a 90 minute podcast that is, uh, worth listening to.Like, it's not, uh, gobbledygook it's actually a coherent episode in 18 months over under what would you take i take the under i'm always taking the over there it is another bet 18 plus from now 5k oh my god this is gonna be shipping bestie coins back and forth this has gone from the small game to the big game it's become the big game big game should we be doing five thousand dollar bets i don't know where i even came up from that maybe it should be a thousand dollar bets because i thought we thought we were gonna have maybe like five and we're like oh that's interesting and So now it's like, this is going to be 20 bets.It'll be a hundred K at work here.Hopefully these things wash out a little bit.
There's a second bet.All right.Okay.So let's do one more demo.That one was infinity.I give it.Well, no, I give it an a minus because it's only nine seconds.I give it a B plus at nine seconds. If they get it to nine minutes, I probably would have given this an A-.And if they get it to an hour, I would give it an A+.
So I'm giving it a B- right now.I'm intrigued.I'm intrigued.Also, I wonder, how long did it take for this to get back to you?
SPEAKER_06: Oh, like three, four minutes.
SPEAKER_03: You think there was any human intervention where somebody looked at it and massaged it a bit?I don't think so.
SPEAKER_06: I didn't need to pick all you guys.No way.Discord is moving pretty fast, right?So there's stuff happening.
SPEAKER_03: How does it know... What is it trained on, Infinity AI, that it has all of the video and our voices?Are our video and voices already just built into AI?Is it like the podcast that popular that they've ingested all podcasts?Or they've ingested all YouTube channels?What's the training data there?
SPEAKER_05: Oh, that's a good question.
SPEAKER_03: How did they get that?Because if they did it that fast, did they do a web search for all in podcasts and then index it, do you think?Or was it already in their index? Based on your knowledge.Did they have done that with a live web search?
SPEAKER_06: No, they couldn't have done it live.You can't go and train the voices and the videos.Oh, no, I was thinking in the context window.
SPEAKER_03: Do you think they went and found a recent episode, put it in the context window, and then built it off that episode?
SPEAKER_06: I just don't think it's possible to do it that quickly.This is a really good question.I mean, yeah.
SPEAKER_03: So that means they already had in their model the all-in podcast.So then the question is, how did they get it in there?Are they scraping all of the top YouTube channels and videos, or are they scraping all podcasts? interesting i don't know you notice all podcasts now have transcripts apple spotify they're all doing transcripts now yeah well you've been doing it a long time that you would have to pay someone before right well we used to pay somebody before but it was just like it didn't even come out that great it was too much work to do it and not okay not enough juice from the squeeze is what i would say all right let's do one more here Okay.I'm going to give these guys an A. Of course you are.
SPEAKER_06: Yeah.Of course you are.You're generous.That's not generous.It's really hard to do.Like you said, look at all these things that they pulled off in a prompt.It's incredible.All right.Give me another one.Hit me again.
So, J. Kelly, you had Humon.Yeah.Give us a quick rundown.Episode 1922. Okay, awesome.And what was your impression?Because they have an empathetic voice.So I want to bring that up.That's my last demo for today.
SPEAKER_03: Yeah, I mean, the demo is unbelievable.They have trained their AI to very quickly understand your facial expressions, your emotional expressions.And then what developers choose to do with that is up to them.The things that would come to mind might be on a personal basis, therapy or companionship. And so I've been pitched on a lot of people doing therapeutic models.Oh, you sound like you're a little bit exhausted there, Sonny.Tell me how you're doing.And then you talk to it, but it really understands your tone of voice.Now, imagine it with the chatbots we just did.You're like, no, I don't want the Warriors' chances of hitting the playoffs down.
From last year or from six months ago, I want to know it right now.Now, imagine ChatGPT was like, oh, you sound a bit exacerbated.I could do a better job.Let me get right to work, right?Or you call customer support.Now you get into the enterprise, right?So there's consumer applications, enterprise applications.There's a company we invested in, Real Voice.And then there's another one, Gong.That's study how...
salespeople talk during conversations, and they say, Hey, match your 1000 person sales team.And we'll tell you the top 10% of performers and how they talk to their clients on the phone.And then it's like, oh, the ones who asked the most questions and get the client talking are the one who closed the biggest deals.Great. who's not who's talking too much on their episode on their uh calls tell that person to change their statements change it up a bit change it up a bit right so i do think there's a range of interesting things here and we got into a massive ethical debate about this too which we can get into but let's do the demo of hume.ai okay we're gonna pull it up here hi well hey there you sound a little i'm really concerned about the warriors making the playoffs i see
SPEAKER_07: Ah, the warrior, huh?What's got you thinking about them?
SPEAKER_03: Got you the playoffs.For those people who are listening, it's telling you what emotions, what top three emotions Sonny is conveying with his voice.And it's really spookily accurate.And then you can pick personas.And so imagine you were calling United and you were frustrated. And it knows you're frustrated and it knows you're like platinum status.It's like, Sonny, we really appreciate your business.We know your platinum status.We're going to work this out for you.Your bag's lost.
We know that can be incredibly frustrating.I found it.It's in Nashville.We're going to get it to you.You could really start to make these things.And what it also does is when you're talking to a chat and it doesn't know to release to let the other person talk, Yes.Super frustrating when you do talk mode and chat GPT-4 specifically.They know through facial expressions that you've finished your thought.Like I just did right now.
Which means they can insert themselves.So the ability of them to insert themselves, it makes the latency in the conversation much better. If you're just using audio, think about all the latency that you experience with Siri and Alexa.One of the reasons Siri and Alexa suck so bad is you ask it to do something and the amount of time it takes is greater than you changing the song on Spotify, right?Or you looking up the weather yourself.But with these tools specifically, if you were looking at your phone or your computer and it had the camera on, It would know if you said, hey, how much snow is there in Tahoe?It would immediately tell you.It wouldn't wait.Yep.
Oh, it waits like three seconds, I think, to know you've stopped talking.That kills the whole experience.
SPEAKER_05: Yep.
SPEAKER_06: Yep.And look, I think as we move towards what you tied back into the first conversation, if voice is going to be the interface that you think wins here, understanding when the human's getting frustrated, understanding when they want something quickly, understanding when they want it to be slower, that's incredible that we now have models that are able to predict that for us.
SPEAKER_03: Now imagine I start using it for nefarious purposes.I'm TikTok. I want to create chaos in America, or I'm Putin, and I want to create chaos in America.And remember, the Russians were doing all those boiler rooms and setting up.I know Sax will say this never happened, but this happened.Trust me, folks.You can look at the Mueller report.They had boiler rooms.The question is, was it effective?Now, setting up boiler rooms where you try...
to, you know, put up memes of Hillary Clinton versus Trump or whatever, like, did that have an effect?Probably didn't sway anybody's votes.Now you look at this technology, if it was talking to you, and it was creating content, Could it possibly steer you into a direction in terms of your political activism?Well, we're already seeing on TikTok that, and you can be on whichever side you prefer to be on.I'm on the side of like, stop the wars and treat people with civility and kindness. You know, if you start looking at the Palestine issue, it's incredibly pro-Palestine, anti-Israel on TikTok, and then on other platforms, it might be the opposite.You can really start to steer people with these AI assistants into a certain way of thinking. just very subtly with knowing their emotion.So it's very dangerous, I think, to know psychologically where somebody's at and what their vulnerabilities are.
You could just as easily make somebody feel better on a customer support call as you could radicalize them into being a terrorist.Now that sounds farcical, but people are radicalized into terrorism every day of the year.It might be a small number of people and maybe only 1% of the population is susceptible to it, or maybe even it's 0.1%.But if this technology could get their hands on that one in a thousand who could become a lone wolf or is suffering from some mental disability and then convince them to do things.This is really dark.And they actually have a really interesting code of conduct on their website.Imagine it for gambling.If it was encouraging you to gamble, if this is built into one of the gambling sites and it was like subtly encouraging you to place more bets, you know,
SPEAKER_05: Brave new world, J. Cal.Brave new world.
SPEAKER_03: I mean, I give this Hume thing like an A- right now.It was scary interactive, scary accurate in my interactions with it.And I do think it's going to change the latency and make the interaction with AIs 10 times better.So back to that 10 times better thing I said earlier with products.This does make a product 10 times better.
SPEAKER_05: Yeah.
SPEAKER_06: I'm in the same spot.I think we're on the verge of a huge amount of deflation, technology-powered deflation for society, which means hopefully we could just focus on better things.And in order for us to get technology to become better human interface, this is what's going to be needed there.
SPEAKER_03: Exactly. All right.This has been another amazing episode of This Week in Startups, our AI edition.You can follow us both on X, x.com slash Sandeep, x.com slash Jason, formerly known as Twitter, and interact with us.The show is TWI Startups.You can search This Week in Startups on YouTube.Subscribe there.Watch it there when you're not listening.And you get to see all these great demos.You can see all the demos and all the episodes at thisweekinstartups.com slash AI.
We keep that page updated.Wonderful job.We'll see you all next week.Bye-bye.
