SPEAKER_05: So this is from our friends over at Deepgram.I don't know if you remember this.I'm going to just play a little snippet.
SPEAKER_04: Steve, there's a thousand bloggers at Microsoft and bloggers that people at Apple, I understand, are not allowed to blog.
SPEAKER_05: You remember this, right?Yeah.Deepgram has released its new tool. Where if you give it an audio file, they can do an intelligence analysis of it.So you can basically have it break out the intents of the conversation.So 25% they felt was about requesting clarification about blogging.Yep.9% was about the RSS feed.11% was about inquiring about podcasts.And then the last bit was the zigger about the email.
SPEAKER_02: Oh, ask for Zinger email.I love it.That's hilarious.It got the Zinger.
SPEAKER_03: You remember the end of that one.I made the joke and the audience laughed.
SPEAKER_02: Yeah.
SPEAKER_03: Wow.
SPEAKER_00: This Week in Startups is brought to you by LinkedIn Ads.To redeem a $100 LinkedIn ad credit and launch your first campaign, go to linkedin.com slash thisweekinstartups. Coda, a new doc that brings words, tables, and teams together.All your valuable data, plans, objectives, and strategies in one place.Go to coda.io slash twist to get a $1,000 credit.And Wistia is the all-in-one video platform for business with tools that help you create, manage, and measure the impact of your videos.Try Wistia for free at wistia.com slash startups.
SPEAKER_04: All right, everybody, welcome back to This Week in Startups.I'm here with my guy, Sandeep Madra.He works at a company called Grok. Yeah, Definitive Intelligence was acquired by Grok.G-R-O-K is your Grok.G-R-O-K is the Grok that Elon spun out from Twitter.So just so we're clear, you're two weeks into the acquisition.How's it going so far?I see a number of developers using the Grok platform has gone parabolic or it's doubling every couple of weeks.
SPEAKER_05: It's going really, really fast.We are over 60,000 developers. over 15 000 apps that's awesome yeah we are rocking and rolling so it has been really fun uh we're adding capacity really fast it is a bit of cat and mouse for us because every time we add capacity people consume it as fast as we can and so you know love all the support we're getting I was actually at a hackathon yesterday that we kind of co-sponsored with the Mistral team.Oh, yeah, I saw that.
SPEAKER_04: Was that in the city?Yeah, it was.
SPEAKER_05: It's a great space there.I don't know if you've been there.Shack 15?Wow.Yeah, it's the second level of the Ferry Building. And I didn't even know this thing existed.It's a little bit like a WeWork slash SoulHouse.
SPEAKER_04: So you buy a membership.I did hear about this.Yeah.And here's the thing.There's so much space available when you go through the boom-bust cycle.And I've seen this before during the dot-com and the Great Recession. People after the dot-com bust or after the great recession were like, you know, there's tons of office space for like 1200 bucks a month.The five of us could be in a cool office.Maybe we'll spend 15 grand a year having an office.It'll be fun.
Right.Then it became, oh my God, you know, my, we were for like five desks, like $9,000 or something.It was like, yeah. okay, we're spending $100,000 a year and this just makes no sense.What are we doing?Yeah, yeah.And that boom-bust cycle is now, we're in that trial where people are like, oh, there's space available and it's free basically or close to free or- Yeah.Minimizing cost.Oh yeah, might as well get a space.And so that's really interesting when you think about space, people get inspired by space too.
They see a space, they're like, wouldn't it be cool if a bunch of people came here and did something like a restaurant?This space was a 10 out of 10.It was a 10 out of 10.
SPEAKER_05: It's the best space I've ever seen in the city for like a startup slash kind of, you know, community vibes.
SPEAKER_04: Yeah.So I think we're going to start to see a lot of that, you know, because why not?
SPEAKER_05: We should do a This Week in Startups AI event there, maybe.
SPEAKER_04: Oh yeah, we should totally do it.Yeah, absolutely.Easy to do.We just pick a day and then we just put a bunch of speakers and we're done.Yeah.Great idea.Some demos too?So we'll get that going.Okay.Yeah.
Actually, that's what we do is just... Demo or die kind of situation.A couple of news items, J. Cal.Yeah.Oh, there's news.Yeah.Okay.So news and then demos.Okay.So quick news in the AI world.
SPEAKER_05: Yeah.Let's go.And so I would say there's like one interesting thing, which is the CEO and founder of Stability Diffusion, he basically stepped down from the role.Oh. And that's one thing in and of itself.Let's not speculate too much on that.I think he said something interesting in his comment in stepping down, which is he feels as though there is, yeah, and exactly this is it because you're not going to beat centralized AI with more centralized AI.And so I find that to be a really interesting comment in terms of stepping down. And, you know, they're open source focused.So I'm not fully sure.
Well, I can speculate on what he's trying to mean here, but I want to see what you have to think when you read that comment.Because they're open source first.
SPEAKER_04: I met him, the gentleman in question, briefly when I was in the Middle East.He was in Dubai and we were at a salon together.Seemed like an interesting cat.But yeah, he's doing an open sourced image thing. creator, right?And what is the market for these things, I guess, is going to become the question.So I think we're past the honeymoon phase, and we're now getting into brass tacks.And so what investors are starting to ask, I believe, is, okay, how many subscriptions do consumers and businesses need to AI?Okay, if you are using Notion, I'm paying extra for AI.We're using Zoom right now.
We've got the AI Summary, AI Companion. I pay for everybody on our team to have open AI.I paid also for Poe.And now I'm pretty promiscuous in trying all these things.But I do think businesses and investors are going to start to say, okay, what's essential?What do we need?What are we going to use?And maybe the idea is not many people need image generation. Maybe that's like a niche thing.How often are you creating images?
Maybe for fun, you and I could sit there and go crazy all day.But I think maybe we only need two of these in the world, three of these in the world.Maybe we don't need 20 people making image creators.Maybe there's just not that much. work.And so the one built into Adobe's suite, right?They have one that's legal and built into there.Then there's DALI, you know, and stable diffusion.It just doesn't need to be that many of them.And that's my read into his comments, which is we can't compete with OpenAI.
That would be my read.OpenAI plus Microsoft plus Google Gemini, or maybe the Google Apple Alliance, which seems to be emerging, we talked about last week.So if you have these two competing factions, OpenAI plus Microsoft, and then over here you have Google and Apple, and then the open source community and the Google app, and then you have the meta open source community.Maybe there's enough going on here that the world just doesn't need number four or five, and maybe he believed they would be number five. And, you know, that's like this.I watch this happen with Uber, Lyft, DoorDash, Just Eats, Postmates, Sidecar.There was a long tail of people going after that space.And at a certain point, consolidation happens.And this has been a rapidly evolving space.
Maybe we're witnessing consolidation happening or capitulation.You know what?
SPEAKER_05: That was a very similar read to mine, which is the big folks are already here. you know it wasn't hard for them to get there from a technology standpoint and it's very difficult to be you know like you said third you know maybe not even third but fourth fifth sixth player because you've got the consumer folks covering it off you've got the original pro folks covering it off with adobe and then you have the up and coming new folks covering it off with mid journey and so really like you know to your point that it's it's difficult to win in that space.And maybe, you know, he'll come at this from a different point of view as he rethinks what he's going to do next.
SPEAKER_04: Yeah.Or there could be something else to him leaving the company.Maybe people didn't want to work for him.We don't know.Right.You sort of speculated who knows why he's leaving the company.Yeah.When CEOs leave a company, There could be burnout, personal issues that they've got.Could be bad news inside the company.
Maybe people don't want to work for them.Maybe they're not a great manager. It could be any of those things.And so sometimes when people leave the playground, they take their ball with them.So he could be leaving the playground or being pushed out of the playground.It's like, I'm going to take my ball with me.This space sucks, you know, but it might just be, he couldn't compete.Maybe he just didn't have the energy to fight this fight for the next five or 10 years, which would be a mature thing to do.If you don't think you can fight a dog fight for five or 10 years to just be honest about it and just be like, it's hard. What's the point?
You know, and sometimes you have to face that as an entrepreneur.I've had to face it in my career many times where like you're playing a certain game on the field.The game changes.I did a print magazine and then the world didn't want print magazines and I did blogs and the world didn't want blogs and I did podcasts.Maybe someday the world doesn't want podcasts and want shorts.
SPEAKER_05: Actually, the world wanted blogs.You sold too early there.
SPEAKER_04: Yeah, you know, I had a little bit of scar tissue from not selling too late the first time.So that was like my 18 months in, I'm selling this thing and I'm getting my first check.
SPEAKER_05: So I don't have to sweat.The world wanted Mahalo, just Google didn't want you to be it.And now we have a version of it with AI.
SPEAKER_04: And perplexity, yeah.And perplexity is using the same concept we're using.Just AI wasn't ready 12 years ago when I thought, you know, these 12 blue links would be better if it was like content plus services, plus Q&A.I mean, I had the right idea.I just was way too early.And I think that's like, you can forgive yourself as an entrepreneur.I think it's an important message here.Sometimes you're too early.Sometimes you sell too early.Sometimes you're too early to the party.
Sometimes you're too late to the party.And who knows what's happening here.But it does seem like, um, maybe there just wasn't an opportunity.Maybe they didn't perceive there was a huge opportunity.And also with the sale of, um, to Microsoft of, um, hi, what was the parent name?Um, last week inflection inflection.Yeah.That feels like, okay.And we said this, like how many,
large models do there need to be how many assistants do there need to be yeah you know it does seem to me like verticalized services in ai will win the day and then there'll be three or four of these foundational models and that's exactly what's happening right navigating the B2B maze can feel really tough, huh?You're trying to hit the mark with all those top-tier executives.You want them to pay attention to your enterprise product.But where can you find all those big fish, the whales, the ones who call the shots and make the buying decisions for corporations, for startups, and everybody in between.Well, here's where LinkedIn ads is going to solve that problem for you.And I've used this.It is one of my secret weapons.LinkedIn means business.
Business equals LinkedIn in people's minds.When you're on LinkedIn, you're in the business mindset.So you're going to really be thinking about business products and services.You're open to those opportunities.And LinkedIn recently passed a billion users.180 million of those billion are senior executives, 18%. But hey, we all know about the 1%.10 million C-suite executives.That's your CFO, CTO, CIO.These are the people who are always looking for a new product or service to make their organization run better.
But they are on LinkedIn.That's why LinkedIn's ad platform delivers two to five times greater return on investment compared to other social media platforms.So easy to understand why this is because this is where all the business people are and they're in that business mindset.Super easy call to action.Make your B2B marketing everything it can be. and get $100 credit on your next campaign.Go to linkedin.com slash thisweekinstartups to claim your credit.That's linkedin.com slash thisweekinstartups.No spaces, no dashes.Terms and conditions apply because they're giving you a hundy.
SPEAKER_05: Jay Kyle, you always do this magic to do it.A great segue into just the last news piece because I do want to touch on it. which is A16Z, at least a really good report.And I just want to touch on a couple of things for all the builders out there, because we do this for a lot of builders.And yesterday, a lot of folks came up at the hackathon and they were like, oh, it was so awesome.You guys do the demos.So they're really cool to see it.But You should read the whole report, but I'm going to touch on three interesting bullets from this report and its generative value in the enterprise.First, you can see here the amount of average spend for LLMs is going from $7 million last year to $18 million.
SPEAKER_04: This is for enterprise companies.So large companies are saying their average spend in 2023, the average spend across foundation model APIs, self-hosting and fine tuning models was 7 million across dozens of companies we spoke to.So they spoke to a couple of dozen companies.Yeah, it does look like the average spend for enterprise companies. Went from 7 million to 18 million is the estimate.So the actual was seven.So they're two and a half times in it.What would this be the equivalent of the cloud computing revolution?
SPEAKER_05: Even faster, right?Like, you know, this, this is a Kager, which is like, you know, growing really, really fast.Right.And I think it's faster than anything we've seen before and look like, you know, they call it, it's like anywhere from two to five X, right.And more workloads going into production.So this is good for folks that are building, right.
SPEAKER_04: I thought this was interesting in terms of where the money is for general AI coming from.So they asked 25 enterprises, I'm guessing here.
SPEAKER_05: It says dozens.They didn't say two dozens.They said dozens.
SPEAKER_04: Dozens of companies.Okay. Oh, dozens of companies.Yeah.So when you say dozens of companies, I would mean like 24 or 36 companies, probably.So in the second one, where do they allocate this from or reallocate?So they're saying the budget doesn't exist.Well, one company cited saving $6 million for each call served by their LLM power customer service for a total of 90% cost savings. That's a reason to increase your investment in Jenny.I eat full.
Okay.So you're using one anecdotal story there.
SPEAKER_05: Got it.But what I would say here, J Cal is when you're building in the earliest phases, and I've seen this in, you know, two previous lives, I've seen it when I had a mobile business and I had a cloud business. A lot of the early budget comes from the innovation teams because they're the ones within the enterprise that are tasked with looking at this.But to see that we're already at a place where 75% of the budget is not coming from innovation, this is really good.
SPEAKER_04: Got it.So this isn't the innovation budget where like experimental stuff goes.Okay.You know, vision pro comes out AR VR.Okay.We'll put a little bit of money into that.Okay.Yeah.Here, 24% innovation, 76% other business unit budgets, 21% to next highest.That means somebody in a business unit, like customer support said, Hey, I want to allocate my budget towards this and away from something else.
reallocated from the it group 15 new net new it budget for llm so that's 19 so somebody said hey i need to increase our budget yes give me some money for this reallocated from other ai 13 rnd product development eight percent so yeah it's definitely immediately having an impact on enterprise is your point i would can't argue with that here yeah
SPEAKER_05: And I have one more that I want to pull up and then we'll get into our demos here, which is interestingly showing, you know, which models are enterprises using.And so let me bring the share back here.
SPEAKER_04: And by the way, I found in the first paragraph here, over the past couple of months, we've spoken with dozens of Fortune 500 and top enterprise leaders and surveyed 70 more to understand how they're using, buying and budgeting generative AI. We were shocked by how significant the resourcing and attitude storage Gen-I had changed over the last six months.Though these leaders still have some reservations about deploying generative AI, they're also nearly tripling their budgets, expanding the number of use cases that are deployed on smaller open source models and transitioning more workloads from early experimentation into production. That says to me that they know it's not ready for prime time, but they're trying to make it ready for prime time.So there's some recognition that it's hallucinating.
SPEAKER_05: How would you interpret that?I would say that they are ready to push into production.And these next two charts, which will be the last ones, and we won't go over the whole report, but this is really fascinating because you can basically see which model providers or enterprises are using.And you can see here... that um you know overall um you know open ai still the clear leader which is you know which is 100 of the enterprises are trying to open ai are open ai is that that's that's impressive so 70 of the 70 yes in fact 70 companies or 70 leaders said they were using open eye yeah
SPEAKER_04: And 63% said they were using Microsoft.No, no.I'm sorry.Google.Google.Yes.Yeah.And 50% of it was testing.13% of it was production.Yes.
Llama, 30% testing, 11% production.Anthropic, 34% were using, but not in production.Mistral, 17%.Oh, here, 17%.So basically... Yeah, people are testing everything and slowly putting stuff into market.Yes.Makes sense.
SPEAKER_05: And then the last one, and we'll get into our demos, which is right here.Open source is booming.And you can see here where the shift is really happening across this little pie chart.
SPEAKER_04: Okay, so we have a pie chart here with three sections, and the three choices are what?
SPEAKER_05: Yes. For those people who are listening, yeah.Yeah, the three sections are no plans to increase open source.That's 18%.Then 41% say they will switch to open source when the performance matches.And another 41% saying will increase or already use majority open source.This is really hard.Wow.
SPEAKER_04: So this means 82% of enterprises want open source.They don't want a closed model.
SPEAKER_05: Yes.
SPEAKER_04: Wow.You know, this is so interesting because you remember in the early days of open source 20 years ago, Everybody was like, yeah, nobody wants to use it.Now enterprises are basically defaulting to it.Yes.It's incredible what open source has done.And this means you would be long llama anthropic.
SPEAKER_05: No, anthropic's not open.No, no.You'd be long llama. Lama, you'd be long one of the Mistral models, Mixtral.Yeah.Right?And then you would be short.And then the open source Google ones.
SPEAKER_04: And then there's an open source Google one.Yeah.It makes sense.I think this technology is so powerful that putting your entire company in the hands of Microsoft and OpenAI or Google and Google Cloud, some folks might feel uncomfortable with them having so much say in what they're doing, right?That's how I would read it.And they want to be able to not have that dependency, right?So that's what I think I would read in here is open source means not dependent, not taking a huge risk, which is so funny when you think about it, because we used to have this expression, nobody ever got fired for selecting Microsoft and picking IBM and then Microsoft.
SPEAKER_00: Yeah.
SPEAKER_04: So, you know, now it's kind of the opposite.Nobody would ever got fired for picking an open source project because with an open source project, you know, that you can't get rug pulled.You can't get price gouging.You're just fascinating, which makes it even more perplexing why open AI went closed because the statistics here are showing don't go closed, right?That's what the statistics are saying.The statistics are saying the market enterprise, at least the enterprise market wants open this solution in a certain way, and people are just ignoring them.So there you have it, folks, the great ignoring of the consumer base.Okay, listen, I got a lot on my plate, got to do a couple podcasts, I got found a university launch accelerator and all my personal life and you know, things I want to do.So there's just a lot, but I'm able to manage it all with an amazing piece of software called Coda.
Coda is the all in one platform that combines the best of documents, spreadsheets, and apps. Here's an example.We use Coda to run the Founder University.Every week, we ask all of our founders to submit a progress update.And then all of that goes into Coda as a database.And then we can sort through all of those weekly updates and look for trends.And Coda also allows us to send automated reminders to all those founders to send in their updates.And then we track week over week growth by generating charts.So if we see strong growth, we'll reach out and we'll invest in that company.And it's all done through this beautiful product called Coda, C-O-D-A.
Here's your call to action.You can use this software to solve any business problem.It is extraordinary.And if you want that platform to empower your startup to strategize, plan, and track goals effectively, you can get started for free.Coda wants to support founders, so they can give you $1,000 in credit.$1,000 in credits at coda.io slash twist.What a generous offer.It's a limited time offer, so I want you to get it right now.Coda.io slash twist.That means you can begin planning right now to make your startup just really talented.
tight C O D a.io slash twist to get started for free and get that thousand dollar credit.Can't beat the price.
SPEAKER_05: I love Coda and you will too.You know, there's a, there's this notion that is, you know, kind of cross pollinated between enterprise and consumer.I believe definitely in Silicon Valley more than other places, which is moat.And, you know, obviously in consumer that makes, you know, it's important, but I think in enterprise, the notion of moat is, Being tied to something proprietary or closed is a bad idea, but folks maybe tend to go in that direction versus something that is like performance or price.So this lock-in thing, I think Silicon Valley and SaaS and everything that's happened in the last 10 plus years are still working through this.
SPEAKER_04: yeah makes sense all right let's do some let's do it yeah of course let's go well i mean we're starting to see where the industry is going to wind up and it's it's you know which if you dovetail the two stories like stable diffusion then is an open source project with a close with a hosting project that would argue that they were more valuable because enterprises would want the open source solution so there's something with the founder leaving that's a little bit Wasn't that the same founder who allegedly screwed his co-founder out of their shares and bought a bunch of the shares back from them?
SPEAKER_05: There was some story related to getting rid of a co-founder or something.And then getting all their shares.Okay, let's get started.Okay, first one today.I really like this one.They are under a lot of pressure today because it's been going viral.What it is, it's called Gatekeep.ai. And what they do is you, you generate something.I did this already because it's, it's, it's been slammed today, but you ask it for a concept.They have some prebuilt ones here.
I did one myself, which I'll share here, which is tell me about put options.Cause I know, you know, I know, I know how much Jay trading loves, you know, trying to do options trading.
SPEAKER_04: I got to get to that Nancy Pelosi level here of taking more risk.
SPEAKER_05: Just follow her trades there.You don't even need to follow her trades. So what I did was I just said, tell me about put options.And what this did was, and I really liked this a lot.It created a little short video.I'll play it in a second.It gave me a chat about it.And then it gave me a little summary down here in a transcript.So I'm just going to play this for let me 30 seconds.
SPEAKER_01: Put options are financial contracts that give the owner the right, but not the obligation, to sell an underlying asset at a specified price, known as the strike price, within a specific time frame.Let's visualize how a put option works.Imagine we have a stock whose price is decreasing over time.
SPEAKER_05: Okay, I'm not going to play the whole video.I think this is awesome. It made this explainer video, the graphics, the voiceover and the graphics, the voiceover and gave me a little chat.So imagine, you know, you're trying to work through trying to understand something or you're doing something with your kids when they're trying to learn.You know, this is a real combination of the video, the chat experience.It's Khan Academy in a box.
SPEAKER_04: It's like a Khan Academy.
SPEAKER_05: Oh, a good way to put it.
SPEAKER_04: Yeah.It's AI Khan Academy.Let's try.I want to try one.Explain to me how a keystone in an arch works.
SPEAKER_05: To me, in an arch works.It does take them a few minutes.
SPEAKER_04: Yeah, it's all good.Well, because it's generating a video.This is generating a how-to video that you could then publish on YouTube.And now YouTube is going to be flooded with these.And hopefully they're correct. Because if they suck, then it's wrong information.That's not helping.So this is where synthetic data or the moment where data can be... Degenerative data revolution is going to pollute the hell out of the internet.So it's almost like you want to have the human internet ends at 2023.And going forward, YouTube is going to be filled with this stuff.
Now, is this stuff been checked?Is it correct or not?We're going to have to create a whole cleanup crew. Because as we know, it gets things wrong all the time.And so what if this is wrong?What if it explains the Pythagorean theorem wrong?What if it explains how the keystone works incorrectly?And now we start flooding the internet.My replies are frequently on X Twitter generated by AI.I can tell somebody's got an AI reply bot.
And it's like... Oh, you enjoy beef ribs in Texas.Beef ribs are a nice thing to enjoy.I hope you enjoyed your beef ribs.Continue enjoying your beef ribs from Terry Blacks.
SPEAKER_02: And I'm just like, what's going on here?Yeah.Okay.
SPEAKER_04: And it's obvious, like, we're going to pollute the hell out of the internet with this garbage.And so I'm just worried about YouTube.The incentives are really messed up right now.So we had content farms, which Mahalo... was definitely not part of, but there was like eHow and other people who were doing content farming, which was just throw anything on a page, get SEO, link it together, grab the ads.And if you don't answer the question or if it's incorrect, that's even better because then people click on the ads.So it's a frustrating page. What do you do when you land on a frustrating page that's supposedly going to answer your question?
SPEAKER_02: Yeah.
SPEAKER_04: You either hit the back key and go back to Google or you click one of the Google or the ads on the page.Yeah.And so that's what led to the whole Panda update and Google not wanting to link to content sites that were scaling up content.And it killed Mahalo and a bunch of other sites along with it.If it was accurate, what was their issue with that? Their issue was that there were a group of people who had figured out SEO and had to build a large number of content pages and that people would start going direct to those.And they just didn't want these content networks to take over the top ranks.And so they just said, this is the maximum amount of traffic that can go to each site.And they wouldn't tell you what was going on.So we went from having like, as but one silly example, how to cook salmon, we had the number one result on Facebook.
And for some period of time, Mahalo just had, you know, like literally 25,000 people a day, day in and day out going to that page.So we just started investing more on that page, more on that page than we did how to cook cod.
SPEAKER_02: Of course, yeah.
SPEAKER_04: We did like little how-to content and then links to the best stuff on the internet.Yeah. They started to consider this like a little bit competitive.And then they started taking from other sites content and then putting it in the one box.So their vision of the world was don't go to these pages.We'll just put the information on the Google search results.So they started getting competitive.And you see that today where Google is competing heads up with a lot of the sites that they used to link to.So you used to get stock quotes, sports scores, recipes, shopping at other websites. those got built into the google search results right if you ask for yeah you know if you ask for a stock price google just gives it to you if you ask for sports score it just gives it to you if you ask for a recipe it just gives it to you or if you're asking like what temperature should medium rare salmon be it just gives it to you yeah and so yeah you know this is the pollution of the internet is going to be crazy in the coming years and that that's where figuring out if this is in fact
ai generated or human generated and then as long as it's correct that it doesn't matter but that's the question i have for you is when when do we think we can start trusting the results from ai and in what sectors it's just the the amount of wrong information is the challenge i have right now with fully embracing a lot of this let's do it let's do a real-time test okay this it just got generated this is the one that you asked how a keystone in an arch works, yes.Here's the summary.Okay, the keystone is a wedge-shaped stone placed at the top of an arch.
SPEAKER_01: It rolls and should be weighed evenly.
SPEAKER_05: Let's play it a little bit faster here.
SPEAKER_01: Let's play it faster.Here we go.Welcome to this video explaining the importance of the keystone in an arch.An arch is a curved structure that supports weight above an opening, and it has been a crucial architectural element for centuries. Let's first look at an arch without a keystone.The stones in the arch push against each other and rely on friction to stay in place.It's not an arch.Now let's introduce the keystone.The keystone is the final wedge-shaped stone placed at the top of the arch.When the keystone is inserted, it locks the other stones in place by distributing the weight evenly.
SPEAKER_04: The keystone prevents the other stones from sliding outward, providing stability to the arch. A bunch of bricks on top of each other.Totally bungled.
SPEAKER_05: I think it didn't get the drawing right, but its concept is correct.Its concept is correct, so proving my point.
SPEAKER_01: The keystone is crucial for maintaining the integrity of the arch.
SPEAKER_04: It is the final piece that completes the arch and ensures its strength and stability.In terms of drawing it.It got the other parts correct. Yeah.I mean, for anybody that doesn't know, if you think of an arch, there's a keystone, that's the middle stone.All the stones lean into that and all the pressure goes into that stone and keeps the arch up.And for those of you who are listening, not watching, this video just showed a stack of squares with a triangle laid in the middle of them.It's exactly the opposite of how it works.So gatekeep.ai, great, but... I guess garbage in, garbage out, or good data in, garbage out.
It's almost like we have a problem here where you get good information into these LLMs, and then they screw it up.So there's still work to be done here.I think it got the information from Wikipedia relatively well, so the copy wasn't bad, but the drawing was terrible and wrong.So I don't trust any of this stuff right now. Okay, so your boss tells you you got to make a product tutorial and a customer support video.And I know what you're thinking.I need a video producer.I need an editor.And man, I need a new boss.Well, think again.
With Wistia, anyone can make professional quality videos right in their web browsing.Do you need a script?Just use Wistia's AI script generator and it will work magic. Need to be on camera?Use Wistia's built-in recorder to capture your face, your screen, or both at the same time.And when it comes to editing, Wistia's text-based editor makes things a breeze.Just highlight the mistakes and hit delete.Take out those mistakes, the ums, the ahs, the hiccups, whatever you got.You can edit it in real time and export your video.You can even add music with Wistia's smart tracks, and these are composed specifically for business videos, so they're not going to be corny.
Most importantly, they're They're royalty free and they automatically adjust to the length of your video.Seriously, if you need to make polished videos fast, Wistia is your go-to because they make amazing products for startups.So I want you to go to wistia.com slash startups.Even if you've got a big company, they're great for big companies too.W-I-S-T-I-A.com slash startups.
SPEAKER_05: Yeah.I mean, look, conceptually, the way you put it, which is like kind of an on-demand Khan Academy.Awesome. The idea is very powerful.I mean, this just, yeah, it's got to be right.So I think this is an interesting spot where there could be a convergence of human, you know, what do they call them on Twitter or X now? the people that do those.Oh, the community notes.Yeah.Community notes.
SPEAKER_04: Yeah.Like Wikipedia editors.So yeah, you need to have teachers authority.You need to have humans who can say, this is correct.Getting into this model.I give this an F for the output and an A for the concept.
SPEAKER_05: Okay.Okay.So what does it average out to?
SPEAKER_04: I give it a D as in it's dumb.
SPEAKER_03: It's a dumb answer.I give it a D.
SPEAKER_05: Okay.
SPEAKER_03: As in dumb.
SPEAKER_05: Well, the answer was right.No, the answer was right.The image was wrong.
SPEAKER_04: It didn't.Okay.And, and learning an arc, there's plenty of videos on YouTube right now where you can learn it.So this is D for dumb.I give it a D. Okay.Okay.
SPEAKER_05: It'll make you dumb.No, no.
SPEAKER_04: I think the concept's an A. Okay.I love the concept that it would, you know, make me a unique video of anything I'm trying to understand.It just, it's worthless if it's not correct.It's actually harmful.Yeah. I might go to an F now.I'm talking myself back into an F. Oh, my God.
SPEAKER_05: I think... You're so kind.No, it's hard to build these things, Jake, how you're... Yes, I know.Maybe I'm frustrated on a Monday.
SPEAKER_00: Yeah, I know.
SPEAKER_05: Exactly.You know, for me, it's a C because, you know, if there's not a way... you know, for the system to tell you this might not be right.And so they should maybe provide some, some type of score on that one to say, we're not really sure because like I said, I was playing around with the prebuilt ones and the stock option one was pretty good.But yeah, that one wasn't.And so, But that's an easy thing for the team to iterate on is provide a confident score or something like that.And look, they did do one thing.And when I refresh the site, they said they did change their model because they were getting overloaded.Maybe their older model is not running as well.
SPEAKER_04: They might have gone to a cheaper one since everybody came there to make their... Anyway, I'm giving you guys and gals, whoever's over there, say them, the whole group. That's a unit.It could be one person.It could be a hundred, but I'm going to give, I just want to share it.
SPEAKER_05: It says right up here on this.You see that, right?
SPEAKER_04: Yeah.Yeah.Our servers are overloaded.So anyway, if it was correct, I would have given this like a B plus an a minus, like it's really a great compelling concept.So I, I, I, but I give it a D on the execution. but I think the concept is wonderful.So I grind it out and get the results to equal the vision like anything else.Listen, first Tesla was a little rough too, man.Yeah.The roadster was a, was a bit of a rough experience.
And you know,
SPEAKER_05: No charging network, none of that stuff, right?
SPEAKER_04: You were just... No supercharger, you know, 100 and something range.It was a little rough, you know, when you hit a bump, pothole in the roadster, you'd feel it in your teeth.It was a hard experience.You know, I drive that car sometimes and then I drive my Model Y and it's like... Oh my God.You want to talk about going from like a D to an A plus like, yeah, there you go.You know, the roadster was the D like it was, it was, I mean, but it was also delightful in that it was like so peppy or whatever, but you do have to make it comfortable for everybody and it has to be accurate and safe.And yeah, here we go.So you're on your journey.How long before these things in your mind start to give enough consistent, good answers that you would trust it to teach your kids?
So when would you trust it to teach your kids for the SATs?
SPEAKER_05: So here's my issue.I think if you asked me the question three months ago, I would have given you my really conservative, sorry, my aggressive answer.Like, oh, within six months, this will be solved.There's a weird thing that's happening where, and I'm probably not uniquely positioned to say I'm being fair about it because of what I've been working on for the last couple of years. Yeah, where so many people just blindly rely on whether it's chat GPT or which LLM of your choice or something like this, that it's there's like this virtuous cycle of people saying, oh, you know what, let me just count on the LLM to produce whatever it does.And I think. Now, there's so many of these things out there that do that.It's been stretched out.So I would have said by the end of this year, we'll be in a spot where you can trust these things and it'll be grounded.Now, I think that's going to be harder because there's so much being created that we have to go through the first arc of many people.
um, kind of dealing with that.And it's a unique thing for computers because we never really count on computers to be fundamentally probabilistic, right?You know, you don't break out a calculator and go like, no, it might not be all the time, you know?
SPEAKER_04: So, yeah, no, I mean the whole, this is the conundrum of this technology is we're so used to that.If you put numbers in a spreadsheet, there is no way for the spreadsheet to to say cell A1 plus cell B1 equals cell C1.The formula can't be wrong.It's not possible for the computer to do a wrong formula in Excel or for spellcheck not to work.Fundamentally, when you type the word the, we expect it to be correct. and you know if it's not correct like your keyboard's broken like something is something really wrong that's easy to identify here it's like you you get these hallucinations or whatever it's it's like it's almost like a drunk human you know like if you were to had a great let's say you had a great history professor and they got hammered yeah so like you were listening to them like they gave a great talk like and then that night they got bombed and you're at you know, the kitchen table and they're drunk out of their mind telling you history and it's not accurate.Yeah.But during the day, they told you really accurate stuff.
Like, that's what we're trying to deal with is this Jekyll and Hyde.I think we got a lot of work to do here.I don't know if it's two more jumps or three more jumps, but we're not being, I think, harsh enough on accuracy.And so, this needs to be the year of accuracy.I'm going to dub this the year of accuracy.I'm looking at every single... demo going forward accuracy is my number one thing i'm not wowed by it anymore i'm not wowed by what i just saw i like the i'm wowed by the vision but i'm judging things on accuracy going forward i'm just letting people know starting today march 25th 2024
SPEAKER_05: I think when it comes to building quickly and putting things out there, when it comes to accuracy, that's something you really should... You want to go fast, but I think on accuracy, it's important.
SPEAKER_04: I'm for accuracy to play a role here.No more gifts of A pluses, but you got not accurate.I want to see a little accuracy.Okay, let's go to another one.
SPEAKER_05: Okay.So the next one, you'll hopefully get a kick out of this one, Jake.This is a good one. So this is from our friends over at Deepgram.And I don't know if you remember this.I'm going to just play a little snippet.
SPEAKER_04: Steve, there's a thousand bloggers at Microsoft and bloggers that people at Apple, I understand, are not allowed to blog.
SPEAKER_05: Okay.You remember this, right?You're basically asking about blogging at Apple, right?And what was going to happen.So Deepgram has released its new tool where if you give it an audio file, what they can do is they can do basically an intelligence analysis of it.So you can summarize it, topic detection, intent detection, and sentiment.So I took that video and basically, obviously you have the summary, which is the speaker discusses the issue of blogging at Apple, the need for employees not to spread false information, ironically, right?Yep. They also asked the other speaker if they can add a feeder into their podcast.
Like I think you were talking about something about OpenStream.
SPEAKER_04: I was talking about iTunes supporting podcasts back 10 plus years ago.
SPEAKER_05: Exactly.So now what they have this tool is so you can basically have it break out the intensive conversation.So 25% they felt was about requesting clarification about blogging. Yep.9% was about the RSS feed.11% was about inquiring about podcasts.And then the last bit was the zinger about the email.
SPEAKER_02: Oh, ask for zinger email.I love it.That's hilarious.It got the zinger.
SPEAKER_03: You remember the end of that one?I made the joke and the audience laughed.Yeah.Wow.
SPEAKER_04: This is really scary how interesting this is.And then you have word level sentiment.Wow.Yes.So negative, positive.That's interesting.
SPEAKER_00: Yes.Yeah.
SPEAKER_04: So this is, I think, really important.This is really cool.Yeah.Because, you know, you could take an entire podcast. Yes.And run it through this and start to understand.And people are doing this for sales calls, obviously, with things like Gong or whatever.Yeah.So I feel like this is another one of these really cool tools for you to understand how you're being perceived. And I could see this working really well.
We do 60 or 70 calls with founders per week, and we invest in, on average, two companies a week, right?Okay.So if you're doing 70 calls, and we're doing two investments a week, it's like one in 35 eventually wind up getting an investment.Now, they had to beat out a bunch of people to even get on that phone call, yada, yada.We have the accelerator. But we now are recording these calls and transcribing them, et cetera.So it'd be really good to have the associates do this.And we could see both the associates at our firm.We call them RAAs, researchers, analysts, associates.
SPEAKER_00: Those are the three levels.
SPEAKER_04: You start as a researcher out of school.You become an analyst after 500 calls.And then you become an associate after 1,000 calls or something like that is the program created at launch.Okay. Jobs at launch.com.There you go.I would like to actually have run some of my RAAs through this and see how they are perceived, if they're perceived as smart or not.And then the way I hire people now is I have them, I have somebody on our team pitch, do a mock pitch, and I pay people a hundred bucks who we're considering hiring to do a call and we record it where they're given the script of how you take a pitch call, right?An introductory call.Okay.
So we pay them for that, but I would write it through this now.So I'm going to use it.I give this an a, um, this is a for awesome.Yeah.
SPEAKER_05: And yeah.A for awesome.Okay, great.And it's, and it is accurate.Yeah.And you know, that one's particularly interesting because it shows how the sentiment changed over time throughout the very short conversation.It was like a minute long. Q&A at a conference.A bit aggressive.And then obviously, Steve says, hey, you'll be able to add whatever you want.
We're working with open standards and ends with the email comment.So I thought this was really, really great.Really cool.There's a lot of other things you can define custom intents that you want it to detect as well. So if there's in your case, if there's like a certain type of intent, like question or, you know, listen, you know, things that you're focusing on with your employees.Very, very cool.
SPEAKER_04: I mean, I want to I want to keep the calls positive. And I want to keep them moving quickly, right?And keep them positive and be respectful of people's time.And also make sure that we under my most important thing is making sure that the associates and researchers at our firm understand the product.So the founders feel that we understand their vision.And so I have them say at the end of the recording, okay, can I repeat back to you? your vision for definitive intelligence.I just want to make sure I understand it.Let me know if I got it right.And by adding that to the script, I got rated higher by founders on our calls because sometimes founders were like, yeah, I don't know if JCal understood our vision.
Now, if you don't invest in somebody, they're like, they didn't invest because they don't understand our vision and how genius we are.I get it.Like I'm an entrepreneur too.But if I say at the end of the call, Hey, just to be clear, Sandeep, your vision for Definitive Intelligence is to use AI to sort through enterprise companies and their big data sets and help them leverage that to achieve their business goals.And you charge them a fee.And the fee's not cheap.It's a couple hundred thousand dollars a year.You would feel like I understood what Definitive Intelligence did.Yeah.
yes and so there my custom thing here would could we tell sentiment of feeling understood feeling heard oh so you would put a custom intent in there my custom intent would i want the founders to feel that we understand them or they feel heard so i don't know you're high on the empathy these days Well, you know, your reputation in venture is really important.Yeah.And, you know, you have moments in this job where you have to be a bit of a hard ass, you know, like people want to sell their company and they put like 90% of the deal value in an employment and then screw the investors.Like we deal with all kinds of weird stuff that occurs, you know, people rent themselves an apartment for 20 K a month and you're like, well, what are you doing?Like, this is not, ethical or it's you know would be perceived as like impropriety and so when we don't have to when we we don't have to deal with those situations i'm trying to have like a very high level of empathy and connection with the founders yeah i know that sounds groovy and hippy dippy but you know there are unique individuals in the world founders right
SPEAKER_05: They are.But feeling heard, like having been on all sides of it, right?Having been a multiple time entrepreneur, having an investor, having been selling companies, acquired companies.I think that little trick that you're talking about is really important.I've felt that frustration many times.I felt it selling a product to someone, not feeling like the person I'm selling to does not understanding.Felt it during a pitch to people.So I like that.
SPEAKER_04: I call these devices, and when you have a device like this, I try to make devices like we're doing here when we give it a rating.A device can, in a process or a system, these kind of devices in a system or a process... can really manifest like a certain brand, right?So if you even have the concept of an introductory call, we frame it as an introductory call because it's not designed to be the be all end all.It's just to introduce you to our team and what your vision is.And then for you to understand who we are and how we invest.Oh, we're not series A leads, we're seed.Okay, we have programs, right?So sometimes we need to... you know, help with them understanding what we do, right?
So that's why I created the concept of the introductory call and the device of, let me repeat back to you the vision so I make sure we understand it.I do it myself now.It's like slowing down to speed up in a way.You slow down and you reflect back to the person what they're saying, right?It's also a good way to be a good partner to a spouse or to be a good friend.
SPEAKER_05: It is.The other thing is reflection is a big thing in AI. as well.I'll explain why.When an LLM starts to produce output, it's predicting X tokens, but one of the things it doesn't have is a backspace key.Once it starts down its journey, and if in its prediction tree, it makes a prediction that's incorrect, it just has to continue. But when you ask it, are you sure?Are you sure?It gets a chance to run it again and take a different path.And so the same thing works with LLMs.
SPEAKER_04: Yeah.So that's interesting.So confirming, are you sure?Like if we did that when we did the Keystone thing in the training, if we said, are you sure that's how a Keystone works?And he was like, let me go check.
SPEAKER_05: But yeah, in the inference, like, yeah, let me check because I kind of made a mistake and I just have to continue with it because I was, and there's no way to stop because it's just streaming the tokens at you.So it's an important concept.It's reflection.There's some good papers written about it.We'll drop a link to everyone.
SPEAKER_04: Yeah. Yeah, awesome.Let's do one more demo here.
SPEAKER_05: Yeah, I do.Okay, I'm going to give that one an A as well, though.So that's an A for both of us.
SPEAKER_04: Yeah, I mean, it's so awesome.I'm a sucker for... these kind of tools that help you understand humans more.And it seemed, I give it the A for accuracy because I know that was me talking.I know it was Steve Jobs replying.I know, you know, it was Walt Mossberg, you know, teeing it off to me.Like I've seen that clip a couple of times.I know it's accurate.So it felt very accurate.
SPEAKER_05: So credit to them.Okay.We are going to close out with our friends over at Anthropic.Okay.And so I have this queued up. And so I basically have a famous picture of, you know, Michael Jordan.Oh, the dunk.
SPEAKER_04: Yeah.
SPEAKER_05: It's a famous dunk of his where he takes off at the free throw line.Yeah.Sam done competition.You know, we've all seen it. And basically, the team over at Anthropic have launched a new model or a variant of a model.In this particular one, it's called Cloud 3 Haiku.So it's smaller and faster.And it's multimodal.So in this case, for folks that are listening, and I can run this again just real time here, and you can see it's going to come back pretty fast.
SPEAKER_04: You said describe this image, and it was instant.The image depicts a professional basketball player in midair dunking a basketball through the hoop during a game.The player is wearing a red jersey, appears to be in the midst of an impressive dunk, and with the crowd in the background visible and cheering, the court and basketball arena suggest this is a high-level basketball competition, likely an NBA game.The dynamic action and energy captured in the image convey the excitement and athleticism of the sport.
SPEAKER_05: And so what we can see here is What we said at the end of last year, that multimodal AI is going to become really important.These folks have a model.It's fast.It's accurate.It's incredibly useful for where things are going now. What I wanted to really talk about with respect to this, because we've seen a few of these now, this is one of the ones that's available, like hosted along with like GPTV.I want entrepreneurs who are building to think about this in the following way.If you were to, oh, let me actually find another image really quickly and pull it up.
SPEAKER_04: Well, let me also ask you a question while you're doing this.If you were to increase temperature and the max tokens to sample, there's two bars there.Explain to the audience what that would do here.
SPEAKER_05: Yeah, so temperature is going to increase the variance in its response.It's going to make it much more, I would say, I'm just going to try to describe it.When it's going down its path of probabilities, it's going to make it a little bit more random.Sometimes it won't always pick the highest probability, is what I would say.And so if we run that again, and then I will pull up another image here as well. So we'll run this.At a 0.6.It's kind of very similar.But I'm going to pull up an image.Yeah, 0.6.
SPEAKER_04: And this model, Haiku, this is their cheapest, fastest, smallest, right?Like this is their... You want to grab an image and describe it real fast.This is the way to do it.This is the way you want to do it.
SPEAKER_05: This is exactly the way you want to do it.Okay.So what we're going to do is, and I want really entrepreneurs to think about this particular example.I've got this image of a nice, healthy food bowl.That is a healthy food bowl.Okay.And what I've done is I'm saying, describe this image and return the results in JSON format, include the type of food, the size and calorie count. And we're going to do this.And why this is really important, this one knows, okay, I want more details, okay?So it simplified, oh, let me turn the temperature down.
This is probably why as well.Let's take this here.Let's run this again.Why this is very powerful, I'm going to do this one more time.Hold on. Okay.
SPEAKER_04: And for people who don't know, JSON is an open standard file format.Yeah.Human readable text to store and transmit data.
SPEAKER_05: Right.Okay.So here's why this is important.If you were to try to do this a year ago, you would have to have had something take the image. then process the image, and then go through an individual element on each thing that was there, and then cross-reference that against another database with food information.Now, all of that's been collapsed into a single model, which is basically saying, telling me there's about a cup of roasted potatoes in there, sweet potatoes calorie count calories it's telling me there's about a half a cup of sauteed mushrooms um there's some of you know friedberg's favorite it's a half a cup of quinoa in there there's half an avocado some roasted tomatoes and some turmeric roasted cauliflower about half a cup i didn't know it's turmeric because of the color yes because of the color yeah so Why this is amazing is if you're now developing something, you don't have to use a whole bunch of services inside AWS or GCP or Azure to decompose the problem.You can basically have the LLM decompose the problem for you.If you wanted to make the next generation calorie counter app, it's literally maybe...
Five lines of code to pull that off.
SPEAKER_04: Amazing.And, you know, if you also wanted to take I mentioned this, like every Charlie Rose episode ever, you know, and understand, like, who is a film director, who is an author, who is a journalist? And then, you know, start to understand what topics they're talking about.Then looking for those topics, you know, in history, in journalism, like stitching this stuff together is getting faster and smoother.You don't need to do multiple things. Choices like a Quentin Tarantino interview on Charlie Rose in 2002 would include these films, but it wouldn't include the following films.A 2010 interview would include those.And so just understanding, you know, the 17 seminal Charlie Rose interviews with Quentin Tarantino or just Quentin Tarantino generally. Putting those all together and then understanding the arc of what he's talking about, what topics over time, and being able to pin those to his filmography, etc.All of that's starting to be included in these models.
SPEAKER_05: Yeah.And what I think for developers, and I saw this at the hackathon I was at yesterday to close out, is what you can pull off in a short amount of time has been fundamentally rewritten. I think I'm excited to do, we should do, you know, like we said, a demo hackathon thing together, JCal.And I really want, you know, folks that are out there that are building, and I know many already are, but if the folks that haven't made that jump yet, it's easier than ever to do these things.
SPEAKER_04: You're going to be able to really solve a lot of problems very quickly as these platforms, you know, just keep extending the data and the reasoning, right?I mean, the reasoning part of this is super interesting because you could ask it, once you have this information, you could say, what should be removed from this and what should be added to make it healthier, right? Yeah.Right.So you could say like, hey, optimize this salad for me to reduce the amount of calories in it, but increase the amount of fiber.You didn't ask it for fiber, but we could get fiber in there and you could say, I need to, you know, get more fiber in my diet and I need to reduce, you know, the number of calories.So what's a way to do that?Oh, you know, get rid of the quinoa and go with channel, you know, like. here's a different, this would be the easiest way to solve that problem, right?This is just absolutely astounding in how fast it is and how cheap it is.
Which means, I think, on a venture basis, we were talking about this $40 billion fund that the PIF was considering doing in AI and how we might allocate it.It does seem like the fundamental language model is like, they're getting cheaper and cheaper.I'm wondering, like... How much money is to be made there?I mean, I like your business, the hardware business, the hosting business.Running them, yeah.The models themselves, if they're open source and they're going to trend towards free, right?Like... How much is it going to cost to do this?
SPEAKER_05: I think the way the allocation is being put out was accurate.I think it's like just give developers access to these models and let them build.And I think the market will stabilize itself when it comes to who's running them and who can run them the cheapest and fastest.
SPEAKER_04: It's going to be amazing.I mean, the cost plummeting and the quality increasing means this accuracy issue, I think, gets solved.I don't know when, but I think today's theme is accuracy.And yeah, so I'll give this one that seemed highly accurate to me.I'm going to give this one a B plus.Yeah, just having not tested it across many things, but I just feel like, yeah, significant.I think they've done a good job.
SPEAKER_05: They've made it fast.They've made it accessible. And now we just want to see a lot more people building with it.I think that'll be really exciting.Keep building with it.
SPEAKER_04: Keep building.And the number of developers is going to greatly increase, I think, with these tools being able to teach you to be a developer or be a maestro amongst a bunch of agents that can write code.So I'm really excited for that.I know that's going to take a little while, but I really want to see the number of people who can write code or push code.I just say push code because they're not writing it, but who can oversee code production.I want to see that increase and the velocity of new products increase.All right, man. It's been another great episode.We'll see you all next time.Follow x.com slash Sundeep, x.com slash Jason.
You get us this week in startups.com slash bets to see the bets we've placed on our AI future.And this week in startups.com slash AI to get the AI playlist.Go to this week in startups on YouTube so you can watch the video and see the keystone not get placed in the right spot if you want to.Great job, Sonny.And we'll see you all next time.Bye-bye.Awesome.Bye.
