SPEAKER_06: open source can win if there is an incentive system for people to contribute their niche data.
SPEAKER_04: Ah, interesting. So if it has a license to use Falcon or a license to use this open source model, and then if I put my data in, I own Getty archives, I say I'll put my Getty in there. But I want, you know, 3% of the pool, or whatever it happens to be. So that requires some centralization, which, you know, oh, necessarily. So and this, this is blockchain, don't say blockchain, I was gonna say blockchain.
SPEAKER_02: Are you telling me that after all this crypto drama, it's coming back?
SPEAKER_05: 100% The blockchain is coming back. Oh, God, no.
SPEAKER_00: This Week in Startups is brought to you by the Equinix startup program provides hybrid infrastructure solutions for startups, including up to $100,000 in credits, and personalized consultations and guidance from the Equinix team. Go to Equinixstartups.com to apply today. Fitbot, tired of doing the same workouts at the gym? Fitbot will build you personalized workouts that help you progress with every set. Get 25% off your subscription or try out the app for free when you sign up now at fitbod.me slash twist and LinkedIn marketing. To redeem a free $100 LinkedIn ad credit and launch your first campaign, go to LinkedIn.com slash This Week in Startups.
SPEAKER_04: Alright, everybody, welcome back to This Week in Startups. Everybody knows generative AI is having a massive impact on the creative industries, chat, GPT, amazing to, you know, brainstorm or workshop some dialogue, maybe work on your screenplay, mid journey, Dolly, generating amazing images from text that you can then use for a logo for your company or in a magazine or website. And of course, we're seeing all these AI video editing tools to make clips from your podcast or your web show, put captions on them, make the host of the show speak a different language. It is amazing how fast this has happened. And one area where AI is making a huge impact we all know is gaming making games is incredibly, incredibly expensive. And convey is bringing multiple tools together to create a platform where game developers can create AI powered NPCs. What's an NPC these are non playing characters. It's used as kind of a joke in online culture. If you hear somebody say he's an NPC, she's an NPC, basically means you're a mid, which is another cultural reference and a way to say somebody is like an average person or unimportant. So NPC non playing character. If you were playing Grand Theft Auto, that might be the shopkeeper might be the bartender or if you're on Tiktok, it might be pinky. Now, of course,
SPEAKER_07: ice cream. So good. Ice cream. So good. Pinky, who slays is not an NPC. She's
SPEAKER_04: pretending to be an NPC because NPCs are boring and do repetitive things over and over again. So the fact that she can mock them and make hundreds of thousands of dollars a year doing tik tok live and people giving her all kinds of virtual gifts shows you how popular the concept of an NPC is in popular culture. Well, my guest today is the co founder and CEO of convey and it's spelled co n v AI if you're looking for it, but it's convey as in convey something. The CEO of the company and co founder is pronounced do mukherjee. Did I get it right? Mukherjee? That's right. Yeah, you got it right. Yeah, incredible. See, this is the thing. pronounced do the harder the name is like Indian names, Sri Lanka names, Greek names, I get them. And then sometimes like weird English names. I get them wrong. So anyway, welcome to the program. You heard my little preamble there. How did you come up with the idea to make tools to make NPCs were in the video game industry before how did you come up with this opportunity?
SPEAKER_06: Yeah, absolutely. I mean, the thing is, multiple things came together, which made it so compelling that I was like, I have to do this or I'll be reading. And, and there was multiple threads, if I were to like, say the most important ones were, I've been thinking about how the human mind works since I was a kid, and was pretty fascinated of how, how our mind works. And I've been spending a lot of time with that. And then, you know, like, I did all my thesis work, you know, like on question answering systems, natural language processing. And after, after my graduation, I started at Nvidia as a deep learning engineer, and did similar work, they're primarily focusing on generative AI, large language models. And that's where I faced the problem myself, you know, where one thing is like you were working on text based systems, right. But when you want to create a character that has a consistent backstory that has some expertise, you want to add voice, you know, like have voice to voice conversation, and then you want to connect it with an avatar, or a non playing character, let's say become the mind of that character. Right? That's a that's a challenging task, especially you have to do that in low latency with different voices and different, you're basically trying to replicate a mind that can be almost human like, so now that the problem that I faced, and my passion and interest and what I've been doing, I was also a competitive gamer before, you know, as growing growing up. So so like, all of those things came together. And I was like, yeah, my life has been gearing up towards this, I have to do it. So So last year, I started conveyor, this was before the charge GPT boom thing, but started interestingly, the NPCs are going to become more like
SPEAKER_04: main characters because of your work if you're successful. So in a way, you're the savior of NPCs.
SPEAKER_06: You could say that, making them more human like more, less like that the NPC meme, more human, more human than human, perhaps.
SPEAKER_04: So show Yeah, show us, show us how you make one. And then yeah, how they act in the game, because the kind of joke of NPCs is they usually they have like a script that, you know, as pinky doll was showing us here. It's usually a script of like six or seven items. And yeah, if you go to six or seven items, he gets repetitive and funny.
SPEAKER_06: Absolutely. Yeah, I could I could walk you through that. I actually have all the whole process. Maybe I give a brief on that. And then we create one. Yeah, sure. Okay, so I actually have a new share your screen. And for people listening will describe
SPEAKER_04: what's happening on the screen here. Because this is again, a tool to make characters in a video game. And these NPCs are typically the non playing characters are typically boring. And make them a little less boring is the goal. Yeah. Yes.
SPEAKER_06: So what I'll do here is I'll present this particular slide in the all the steps that happens. So this is all the step all the way from creating the character primarily on top of LLM to adding voice to adding avatar to putting into the game engines. So first thing when you on top of the character, you give them a backstory, they will have personalities, they will have emotions, they can even have memories, you can put guardrails around them in terms of things that they can or cannot say. They can speak different languages, you can add vector databases, which augments the knowledge and you know, make them expert on certain topic, you can follow a particular narrative. And once you've added all of these over text, which we will see right after that you can add voices on top of them. So like they can listen to you understand all different kinds of language, speak different languages, and speak them with emotions, you know, like, as you see, my voice is having intonations and whatnot. Once you have this whole voice to voice character, you connect them with the avatars, right, whatever kind of avatar you can use them with metahumans with various different kind of avatar systems that are there. We are avatar agnostic, but we support all kinds. And we also support all kinds of engines, right? So you can integrate them with Unreal Engine Unity, anywhere you can create 3d words, you can embody these characters with the AI that you just created with the character, right? So lip sync will work facial expressions, gestures will work. And then finally, once these are like avatar eyes, you have provided the mind, it means that they can like perceive the environment around, you know, carry out different actions, you know, they will move around the world, you know, you can say, Hey, go pick up my kit, pick up something, and they will bring it to you, you know, like, so they can do all of these things, right? So we solve the whole pipeline end to end pause there on that last part, before you go
SPEAKER_04: any further, on that last part, you're going to have them go interact with the environment. Yeah, which means you don't know exactly what their response is going to be because I as the player could tell them, Hey, I want you to go, you know, find me all of the loot and bring it back. And they go do that for you. Okay, fine. Yeah, you're gonna say, Hey, I want you to go and insult everybody else playing the game. And you know, in the style of do that, too, you know, Jeffrey Ross, the famous insult comedian. Yeah, and I want you to use a Trump voice or something and go insult everybody. So now you've essentially created chaos inside the game. So I guess you're, you're, you're unleashing a little bit of craziness. No,
SPEAKER_06: that is true. But like, it's up to the game designers that you know, will you allow them to have different voices, usually they will probably stick to a singular voice so that it stays consistent with the character. But maybe somebody might might design something like that, you know, to it's gonna hit some language model. And as we've seen, you
SPEAKER_04: can kind of convince language models to change their mind about things. So things things are gonna get a little weird. Yeah, things will but but I think game is hopefully Yeah, in
SPEAKER_06: a good way. But like the thing is, this is a developer tool and the game designers are going to spend their time trying to perfect the character so that it stays in character, somebody may want something like that. But most cases 99.9% of the cases, they would want a consistent character. But yes, they will have the dynamism where you can tell them, hey, go tell this thing to this friend of mine on the other side of the map. And they will they will go running inside the world and tell them whatever message you send them. Yeah, awesome.
SPEAKER_04: Cloud computing has revolutionized startups over the past decade. We all know that. But reality is a fully cloud based solution is not right for every startup. Sometimes you need a hybrid solution. Like let's say you're working with sensitive data that you just can't put in some random cloud, right? Or maybe you need to connect multiple cloud providers at once. Or maybe you just want a more cost efficient solution, right? Because sometimes these cloud bills can get crazy. If that's the case, you need to check out Equinix. Equinix metal will give you direct access to physical servers, but you still get the benefits of the cloud. So there's no need to rack your own servers. Equinix provides on demand infrastructure in over 25 major cities. And here's the best part. They have an amazing startup program. The Equinix startup program offers personalized consultations, guidance from the Equinix team and wait for it $100,000 in credits. That's not a typo. So here's what I want you to do. Go to Equinix startups.com to apply. And when you apply, James from Equinix will reach out to you directly. That's how serious they take the startup community. That's EQ UI n i x startups.com EQ UI n i x startups.com equinix startups.com get a call from James. Well, I mean, if you asked it, I want you to speak French and I want you to bark like an English Bulldog. It could go find in the language model, those attributes and start talking French and barking like a bulldog. Yeah,
SPEAKER_06: you can you can. So if you if you want to try that we actually have support for multiple languages where if you speak French, you can you can speak French actually, here we are in
SPEAKER_04: the interface. And this looks like a piece of, you know, software, you got a bunch of different characters, we pick a character, or we can create a new one. Okay, I'm creating new one. Let's
SPEAKER_06: say maybe I can create you Jason. Okay, so put Jason,
SPEAKER_04: characters voice.
SPEAKER_06: Let's use a male voice for now. But we can we can even add your voice later on. How do you want to say it? Can you use AI to make character voices? Like
SPEAKER_04: could you say give me a pirate and even if you didn't have pirate already in there, we go find pirate. Yeah, we can we can we can voices in this as well. But but yeah, let's let's go
SPEAKER_06: with the default one. Maybe I'll just say Jason. Hello, anise is the host of all in podcast. And I'll see what what GPT three
SPEAKER_06: generates. Yeah, so. Okay, put it in there. Hey, this is a all in podcast where
SPEAKER_04: you've mastered the art of captivating your audience with your insightful and thoughtful provoking discussions. Your existence is anchored in the world of tech. You just pasted that in? Or no, I just generated that. Oh, you just generated it. Yeah, yeah. So based on Jason Calacanis is the host of all in, it generated the paragraph. Fantastic. Yeah, yeah. Yeah. So we'll create that. And once the character is created, it
SPEAKER_06: basically gives a placeholder avatar, but we can actually have your own meta human character inside the game engine when we take it there. That will look like you feel like you and all that. So so we have like all of these options to craft it further. So like you were saying it currently speaks English. So let's just start with it. Hey, how are you? Hey there. I'm doing great. Thanks for asking. How about you? Almost sounds
SPEAKER_07: like you almost if I had a very deep voice, it would sound a
SPEAKER_04: little like me. Yeah, yeah. Um, what do you do? I'm the host of
SPEAKER_07: the all in podcast where I discuss tech and investment topics with industry experts. I'm also a venture capitalist and entrepreneur. So I'm always looking for new opportunities to invest in. Pretty accurate. Yeah, true. True. I think you
SPEAKER_04: have this knowledge bank, let's say you have your own set of
SPEAKER_06: files, let's say all the all in podcast stuff that's tagged by your name, you can upload that and it'll know everything about what's going every episode or something. Yeah, sure. Exactly.
SPEAKER_04: And so this is the state of mind, like how you were feeling
SPEAKER_06: as I'm asking this. So let's say and this will drive the facial expressions, the tone of the voice and all of that. So So let's say I say, my dog passed away, which is not true, but I'm just testing it. That's terrible. I'm feel so bad. So
SPEAKER_06: let's see if it changes the emotion. I'm so sorry to hear that. Lo Sing a pet can be really difficult. Yeah. Is there
SPEAKER_07: anything I can do to help? So you see it kind of goes into the grief mode. Yeah, which is showing the emotion wheel. I
SPEAKER_04: forgot what they call that emotional wheel. Yeah, it's called blood chicks wheel. Blood chicks wheel. Show me
SPEAKER_06: the blood chicks wheel. Blood chicks wheel. Yeah.
SPEAKER_04: Neat. So it shows you the emotional state. Yeah. That's interesting. You can also craft the the you know, like,
SPEAKER_06: how is your personality? Are you an agreeable person? Or yeah, you can hear it's giving you that it's giving you like a
SPEAKER_04: little slider and you can say how openness, meticulous, extraversion, agreeableness, sensitivity, like kind of a Myers Briggs and you can just put me as like 100% extra word, whatever. medium emotional, whatever. And it will just kind of build my psychographic there. My personality trait. Yeah, yes, yes, yes. So exactly. This is
SPEAKER_06: the big five personality traits. We initially thought of doing Myers Briggs, but it seemed like this would be more appropriate. Yeah, to have this. Yeah. And then there is like, you can have long term memory settings that you can turn on. Oh, you can have you can have guardrail. So this is the part it protects from saying certain things. So let's say this. Yeah. So it says limit responses, stick to character description and
SPEAKER_04: knowledge bank only. And then all the other other end is talk about other areas. So you can just go crazy. Or you can go somewhere in the middle. So yeah, just script the person's going to be so you're basically creating Westworld here. When Elon or somebody figures out robots, you can put this into them and see if they'll stay on script and always stay a samurai or always stay. Yeah, cowboy. Interesting. Exactly. And then
SPEAKER_06: we have this narrative design aspect, which is basically like, you know, you can create a narrative one after the other, you know, like where this thing could be connected to the next thing and you can craft a story basically. Wow. Yeah. So like, maybe I can show you an existing one. So let's say you're making
SPEAKER_04: what they call that a story tree or something. Yeah, like a
SPEAKER_06: dialogue tree or story tree. However, the difference from the traditional ones would be that this is going to be open ended. So it will still stay within the story. It will try to follow the story, right or whatever your objective that you provided it and yet be open ended. Right. So, so so that's, that's, that's another feature that so that they can play play the role that you asked them to write. So. So once you have done all of these things, right, you take this character ID, which is the mind of the character that once you've crafted it, now you take it to the game engine. Right. So maybe I'll show you a quick one. So let's say, I'll go to this, you've got a token, a unique token that represents that
SPEAKER_03:
SPEAKER_04: character, that'd be very hard to guess. You plug it into the Unreal Engine. Exactly. It's now connected.
SPEAKER_06: Yeah. So you take it, you know, like, there's a particular character there, you put it in the mind of that character, that could be a look alike, or, you know, however you want it, and then it's inside the game world, you know, like, and you can you can talk to it. This is something that is not shared yet publicly, but this is an internal discussion. But it's mainly a scratch video internally. But this is something we plan to launch in a couple of weeks. Oh, cool. Which which is what I was seeing you that these people can actually carry out different actions.
SPEAKER_04: All right. So here we go. We got three people in the woods. Looks like they're in the jungle or something action. For example,
SPEAKER_03: three NPCs hanging out. Sure. I can follow you. And you're
SPEAKER_04: telling it Oh, follow me. There you go. Okay, let's try
SPEAKER_06: something. So now we're doing something more complex, like, Hey, can you go there? Do something more complex? Like,
SPEAKER_02: Hey, can you move to those trees and back to me again? Yes, I can move to the trees and back to you again. Oh, yeah.
SPEAKER_06: So that's one thing. The next one, we asked this guy to like, do a show us a dance. Maybe I'll share this one, basically. So on site.
SPEAKER_05: Yeah, this is gonna get crazy. So we're here. Yeah. And we'll
SPEAKER_06: ask the character to bring us the axe. No, no, tell her to
SPEAKER_04: take the axe and kill the other two NPCs.
SPEAKER_08: Sure, I'll bring you the I'm joking. Don't tell her to kill
SPEAKER_04: the other NPCs.
SPEAKER_02: Thanks for watching this demo from convey. So you can think of
SPEAKER_06: like many different things which we were discussing before and each of them were like avatars, still avatars where we put in a character ID. And now they are aware of the environment, right? And you can tell them, accompany me or go there, do that, tell this to another friend, etc, etc. Right. You can make them look like however you want to make them look. So yeah, this is
SPEAKER_04: interesting. And you could see people becoming addicted. If these things have history, every time you come back, you start talking to these people, you can start developing relationships with them, like obviously, famously in the movie her, where you know, you're kind of making relationships. So this will change video games forever. If the characters like some people play these games for a decade, the world of Warcraft people have been playing forever. You know, whatever, you could start having these characters build deep, meaningful relationships where you talk to them about your life and your kids and this is happening, hey, why haven't you played the game for so long? Will you come back to the game? And then this kind of now they start to become sentient in a way, right? They have their own lives inside the game. And if you don't come back, it's kind of reminds me of Toy Story in a way, like where the the boy grows up and the toys don't have what he doesn't have anybody to play with. And so this is there's the could go beyond video games here in terms of like Second Life had the game Second Life. It was all human characters. But if you introduce AI NPCs, well, now you got a whole different world. I could create my own world. And you never know. They might be already doing that, you know? Yeah. So now people can be
SPEAKER_04: putting those in there. Yeah. All right. You know, I've been on a health kick over the past year. And you know, I care about data driven solutions. And if you listen to this podcast, I bet you do too. So let me tell you about fit bot. This is a data driven workout app that blends machine learning with exercise science. Fit bot creates custom dynamic workouts programs based on your fitness goals, your experience. And most interestingly to me, the available equipment, let's say you got a bunch of kettlebells, or let's say you're at some, you know, sparse gym at a hotel, or you're on vacation, you got nothing. Well fit mod will maximize your fitness gains by varying the intensity and the volume between your sessions and leverage the equipment you have or don't have. As the case may be, you can customize the length of your workout, what muscles you want to target and so much more. So let's say you want to get a 30 minute workout in and I want to do chest triceps and abs, but I'm staying at an Airbnb, there's no equipment, fit bot can create a perfectly optimized workout for me based on these parameters. And it will do it for you to check it out. It's amazing. The design of this app is extraordinary. I was able to invest in it. That's how impressed I was with it. Fit bot takes the guesswork out of fitness, just open the app and start making progress. You deserve it get 25% off your football subscription or try out the app for free. When you sign up now at fit bod.me slash twist, that's fit bod.me slash
SPEAKER_03: TW is T for 25% off.
SPEAKER_04: And so pretty obvious how you make money you charge developers to use these tools. Do you charge them on a per use basis? Anything like that? Or?
SPEAKER_06: Yeah, so basically for developers, it's all the way from free through, you know, like 20 $99 a month, but primarily when the devil up and ship it, you know, then the real traffic starts, right? So we we have our cost, pretty much close to chat GPT cost per token, you know, like, but we call it by interaction. Because rather than you know, like, for interaction, we kind of assume 1000 to 2000 tokens are being used because there's backstory and whatnot. And, and it's not a fixed cost.
SPEAKER_04: If these things start getting really popular, there's a usage base here. So the game has to have some kind of monetization in it, or ability to pay for it. So that but that and but it's going to be de minimus. Yeah.
SPEAKER_06: Yeah, but we have done the math and done the numbers business models for various, you know, business models all the way from free to play games, to fixed costs, one time purchase games and subscription model games. It can with the right design in place. It can work in with all of those models. Yeah. So this is something that got unity in a lot of trouble
SPEAKER_04: recently, because they started moving from flat rate to usage based or I guess publishing based. I wanted to make more money. Let's be you're aware of that, I guess. What's happened in the developer community, because you're also adding variable pricing to groups that might have flat rate pricing, like, you know, Candy Crush, maybe doesn't want to pay for or Angry Birds doesn't want to pay for something like this, if they're flat rate.
SPEAKER_06: But where we see this going is that and this kind of like brings us to the market size discussion, actually. So, you know, if we do a bottom self understanding, so every game, every game that has characters, you know, like whichever way, you know, all the way from absolute triple A, 3d cyberpunk like games through even small mobile games that has characters, you could potentially have these LLM driven experiences for these characters, right? And, yes, with and my thought is this that if players find it more engaging, and they stay longer, the retention is good, they will spend more money and the studio will make more money and does it won't be a big deal to pay for us to take this games like Apex Legends and whatnot they spend because any MMO massively multiplayer online games has a huge 10s of millions of dollars they spend in server costs, right? So it is not super new to spend server costs for more engaging time. So they blend the sick, they blend server costs into there.
SPEAKER_03:
SPEAKER_04: So you're just like a hard drive or some CPU that makes sense or GPU? Yeah, what did what was it like to work at Nvidia? When did you work there? And tell me a little bit about the ascension of that company. It's been crazy. Sure, absolutely. I mean,
SPEAKER_06: I absolutely love the company. If I didn't have this feeling of I absolutely have to do this, I would not have left. And there's I don't know if other companies I mean, I haven't worked in a ton of companies, but the culture there is absolutely amazing. You know, like, in my experience, at least so loved working there. Love the people still in touch with them. And thanks to them. I mean, Jensen actually showcased one of our demos that we created worked on together and I don't know if you remember that, but that went pretty viral. If you if you want to see I can share that but like the the demo, the it was this
SPEAKER_06: ramen shop demo. Oh, yeah, sure. Sure. Sure. We partnered with
SPEAKER_01: framework and avatar toolmaker called conv AI convey. And together, we developed this demo you're about to see. Okay, run please. Everything is real time. Hmm. So you're going up to the ramen counter? Yeah. Hey, Jen,
SPEAKER_08: how are you? Unfortunately, not so good. How come? I'm worried
SPEAKER_08: about the crime around here. It's gotten bad lately. Sure. Ramen shop got caught in the crossfire. Can I help? Yeah, if
SPEAKER_08: you want to do something about this. I have heard rumors that the powerful crime Lord kuman Aoki is causing all sorts of chaos in the city. Oh, he may be the root of this violence. We're gonna have to do something. Where can I find him? I have
SPEAKER_08: heard he hangs out in the underground fight clubs on the city's east side. Try there. Okay, I'll go. Careful guy. So
SPEAKER_04: that interaction is happening and you got your quest but that's happening in real time. Yeah, yeah. We gave no Yeah. Oh,
SPEAKER_01: here. A character, a backstory. His story about his ramen shop and the story of this of this game. And all you have to do is go up and talk to this character. And because this character has been infused with artificial intelligence and large language models, it can interact with you understand your meaning and interact with you in a really reasonable way. All of the facial animation completely done by the AI. We have made it possible for all kinds of characters to be generated. They're all domain, they have their own domain knowledge, you can customize it. So everybody's games different. And look how wonderfully beautiful they are and natural they are. This is the future of video games. Not only will AI a lot of tribute to rendering and so I guess the question is
SPEAKER_04: now from here, can I take this and make my own character and then put it inside of this? Remember the this is Black Mirror if you miss the episode of Black Mirror? Yeah, this is a
SPEAKER_06: horrifying, horrifying guy when the little robot is chasing you
SPEAKER_04: and you built the software to power this because you could write that you are just absolutely evil murderous robot. Please go out there and this is your backstory. You are all your so congratulations. I think you're the guy in the Terminator movie, who accidentally unleashes all of this. I'm kidding. Hopefully not. I don't Well, I mean, here's the truth you could program and people have programmed and humans are using robots right now. Like quad copters and drones to go do crazy things in Ukraine. You know, and in all kinds of wars and stuff like that. This next wave, the next wave of weapons is going to be built with tools not your tool, but not a totally dissimilar tool, which might be, hey, these are the bad guys. Here's kung fu. Here's martial arts. And you know, if you had some robot, here's all the weapons you can use and use your best, you know, the AI would in real time. Yeah, teach it how to fight different opponents or, you know, solve different challenges. Absolutely. And I
SPEAKER_06: think, I mean, it's definitely happening, whether we do it or not. Yeah, the the, but it may actually even help people to not kill people. You know, when these are robots, they can take bullets and not back.
SPEAKER_04: Yeah, you just just hey, disarm the person sit on top of the person. Yeah, exactly. So you Yeah, you can write one that's just like Andre the giant where it's just like, do the worst laying on top of the person. Yeah, until help comes, hold
SPEAKER_06: them and handcuff them. And that's how you fight wars. You don't have to kill them. You know, like, just guessing. Yeah,
SPEAKER_04: what, what, how did this whole generative AI stuff start? You had some thoughts on that? What's your take on how we got to here? Because this has been a crazy year, we're getting right on the year anniversary of chat, GPT 3.5, I believe. Yeah. When we tape this in the fall of 2023. So how did this all start? And then what do you think about the pace? Yeah.
SPEAKER_06: I think multiple things are reaching to an inflection point, which is causing this. So and you'll know this list. So basically, over the last 20 plus years, we have been building this body of data over the internet collectively, everyone, you know, like everybody contributing to the internet. Yeah, the hardware has been getting better. And finally, and the architecture, the neural network models has been there for way longer. Yeah, the theory has been there for way longer. And now finally, we are going into so so initially, it was all about vision, you know, like, vision models got quite well, we started doing it on text. And now we are going multimodal, you know, GPT five, GPT four vision already has that. Yeah, GPT five probably will be about videos, you know, like, who has the most amount of videos. So in terms of like, and then what we are doing, and I think the next frontier, and which is something that actually I wrote about back in 2017, that before starting at Nvidia was that, and I saw that while doing my thesis work that you know, language models are where it said it was not called large language models back then. And, and, you know, I wrote that while these language model systems are getting better, and will continue to get better. It is not exactly how we humans learn. We learn from the 3d world around us. And, and then we basically have like, when we were kids, or babies, you know, we attach the words, yes, the images that we see, like, this is mama, this is Papa, this is food, you know, like, and that's
SPEAKER_03:
SPEAKER_04: very unique thing about cognition for humans are big, large brains, put words on it. And I wonder, at what point in evolution, do people start in their cognition? You know, we know consciousness starts like, you know, at different points in time for different species. And then the sort of advanced cognition where you're saying this is this, and you have a word. And this is what I want to do. And I'm here. And this is what I'm going to do next. Right? Like, yeah. So very interesting. I think the only difference and I hope we
SPEAKER_06: are super careful on this, especially the big companies who are working on this is our objective function is survival, all the way from a multicellular organism to like a small creature. That's the objective function that we are trying to optimize on the today's language and procreation. And so, which is kind of the same thing, right? So that's like survival of your species, survival of your genes, I should say, yes, survival of your genes. So that's why parents would sacrifice themselves to save their kid, right? survival of their genes is is the objective function that we optimize on. So the the and for the language models right now, it's like predict the next word, or praise the next token, that's like their objective function. However, where I think every with all the multimodality is coming in, researchers are going to see that encoding survival is going to be necessary for the best AI. And that is something that we have to be super careful about. We don't want maybe we don't want that. Yeah. Why
SPEAKER_04: because encoding survival be so dangerous? Because encoding
SPEAKER_06: survival because now then we are creating a different species. You know, like, effectively, who is going to start thinking for themselves? Right? What objective function and maybe this is the movie that I robot that? Yeah, the tenets of you know, what, what we what we would want to encode as their objective function becomes important. Yeah, we have to tell
SPEAKER_03:
SPEAKER_04: them their objection. Their objective is to protect human life. Yeah. And to be useful to humanity. Yeah. And but useful to humanity might be, oh, you know what, you know, fentanyl is really dangerous. So let me just go kill everybody who is using or near fentanyl. I mean, it might take it literally, right? Yeah, yeah. cancer is bad cancer exists in people's bodies. So crazy way to get rid of cancer is to kill the bodies that have cancer. And it's like, no, no, no, no, everybody's got a little bit of cancer inside them probably. Yeah. All right,
SPEAKER_04: business to business founders need to understand two very important things. B2B marketing is hard. We all know that it's really hard. Number two, it's simple on LinkedIn ads. Why is B2B marketing so hard? Because the decision makers, hey, people like me, right, the founder of the company, or maybe you, you're an executive listening to the show, they're hard to identify, right? And they're really hard to target over the internet. And how do you know you're reaching them in the most respectful environment, people who are decision makers, you cannot reach them through that 1990s way of reaching executives know they need social outreach. And the way to get that social outreach dialed in is with LinkedIn. LinkedIn solved this problem. There are 70 million decision makers hanging out on LinkedIn and executives like me. Yeah, the decision makers were on LinkedIn, we're deep into LinkedIn every single day. And LinkedIn ads helps you get your B2B message in front of the right people like me. And people are on LinkedIn, they want to do business, LinkedIn equals business, business equals LinkedIn, each social network has its vibe. And the vibes at LinkedIn are business, it's business vibes only. And it's happened. Finally, LinkedIn has hit 1 billion users to march to a billion has been achieved. Congratulations to all my friends at LinkedIn. So here's a call to action get started today. See why LinkedIn is the place to be for B2B will even give you a handy right now $100 credit on your next campaign. Go to linkedin.com slash this weekend startups to claim your hundred dollar credit. Why wouldn't you take the hundred go to linkedin.com slash this week in startups, the links are in the show notes, terms and conditions do apply. So do you think this regulation that just came out this executive order this week? Do you think that's overreaching? Do you think it's going to stifle innovation? Do you think it's worth getting the government involved this early? Or is it too late to get the government involved? Well, how do you look at it?
SPEAKER_06: I pretty much take the same take that you all have, you know, David Sachs have in your podcast that I think it's definitely early. I haven't read the whole thing through, to be honest, but from what I understand, the it may not be great for startups. It's great for incumbents. Of course, yes, you have more resources to report in, you got resources to hire people who
SPEAKER_04: used to work for the government. It's regulatory capture and Bill Gurley's Yeah, amazing talk at its peak, its peak regulatory capture. It's like, we can't wait to be regulated because we have 10 billion in our bank. And you know, you have 10 million your startup and the next startup has 1 million. And you know, the two folks in a garage or at Y Combinator have 100k. Right. So yeah, it's so family unfair for later, but open source is going to win the day you think or do you think closed AI is going to win?
SPEAKER_06: That's, uh, hmm. The thing is, whoever has the most amount of niche data are going to win. Right. So let's think that through. The Google became a monopoly because it had the most niche data of people clicking the right links and base rank doing its job. Right. So it was actually human feedback that got it the network effect, right. So chat GPT is trying that to some extent, but is that enough data where when you give the thumbs up, thumbs down? It's not as much as Microsoft, Google and
SPEAKER_07: Apple have. Exactly. Right. So have billions of users. Yeah.
SPEAKER_06: Right. So so they are collecting human feedback data, but is that differentiated enough? And I think open source has a chance, in the sense that, till we exhaust the amount of, you know, like internet niche data that we can collect and clean. But I do feel that the open source, thanks to meta is is like, and some other organizations like the Falcon ones and others who are open sourcing with licensing will enable people to contribute their niche data. Right. And, and, and they would want a way, I think, open source can win if there is an incentive system for people to contribute their niche data.
SPEAKER_03:
SPEAKER_04: Ah, interesting. So if it has a license, to use Falcon or a license to use this open source model, and then if I put my data in, I own Getty archives, I say I'll put my Getty in there. But I want, you know, 3% of the pool, or whatever it happens to be. So that requires some centralization, which, you know, Oh, okay, necessarily. So and this, this is a blockchain,
SPEAKER_04: don't say blockchain, I was gonna say blockchain.
SPEAKER_02: Are you telling me that after all this crypto drama, it's coming back? 100%? The blockchain is coming back? Oh,
SPEAKER_02: God, no, no, here's the thing. See, I was absolutely against I
SPEAKER_06: was absolutely against the whole hype train. Sure. 90% of the people were there for making a quick month quick buck. Sure. Right. But if you see the fundamental tenets of, you know, like blockchain with with zero trust protocols, and, and such, it can enable a lot of other applications that are otherwise not possible. Right? If you have a token to, and then you and I,
SPEAKER_04: we could create a token. Yeah, this weekend. And then we'll keep like 10% of it in a wallet that we forgot the, you know, and then we'll own 10% of the global AI wallet share, right? And then we'll, you know, we'll put it in 10 years, whatever, you know, it has to be done ethically. But that's how you
SPEAKER_06: get everyone around you. But kind of invalidates crypto then.
SPEAKER_04: No, I mean, the blockchain would be actually a way to do licensing, right? Yeah, that would be an actual use case. I would make sense. Smart contracts. Yeah, yeah. Hey, when you think about AI, and you're kind of in a dreamy state, what do you what do you what do you think's gonna happen here? This has been moving at a crazy pace for a year, but you've been in it for your whole career. What do you know, based on what you
SPEAKER_04: know, on the pace, right back to my question about the pace, the pace is obviously accelerating. Now you got a lot of people using these tools making all kinds of edge cases. And then you got every company with unlimited or seemingly unlimited resources making their own chipsets, you know, in the amount being invested in this means it's only going to get faster. And the velocity is going to get faster because more people participating more money participating and the AI getting better and then the abusing the AI to make AI better, which has always been Elon and other folks concerned that AI will make AI better. We're kind of on the cusp of AI making AI better. Yeah. And so what do you think about the pace?
SPEAKER_06: Yeah, I think the pace is not going to slow down from here, because we have entered a time where the data for all the different modalities that humans understand, are there like video, audio, you know, like, and it's only gonna get get more and more. So this is not slowing, basically, we have reached the, you know, we see human civilization, how curved they were. And yeah, I think we are there. It's like it is going to accelerate the inflection point, thought classic hockey
SPEAKER_04: stick, but probably like a hockey stick we've never seen before. And that's an in fairness, shout out to my guy, Tim urban. In fairness, that's why the government wants to slow this down, because it thinks that that inflection point could happen any second. Yeah, when does AI start making AI better? Like, literally, somebody says to Claude, you know, hey, look at your or Falcon, somebody trains Falcon to look at Claude's code base and pulls up their open source, whatever. I don't know if it's open source, but you get the idea. Maybe it would be Falcon looks at, you know, what other open source project and says, make that better. And then it actually contributes without a human being involved and actually does make it better. That's the main point. That's the main thing. So
SPEAKER_06: right now AI is making AI better. A lot of the open source model is actually collecting data from GPT for cloud, etc. You know, like, and making the models better. However, AI doing that itself, the entire loop all the way from training, fine tuning and all that. I think we are we probably may not want that we always still want human in the loop. At least when would
SPEAKER_04: it be possible to think to set an AI? Technically, yeah,
SPEAKER_06: technically, one could one could build that already. But that hasn't somebody that may not be the perfect thing. Because when you're training a model, you have to look into so many different things. Is it even going towards convergence? Or is it making the model more stupid? Yeah. And there's like, like, are you cleaning the right data? How many GPU is it using? Did a run fail job fail or whatnot? Yeah, definitely requires, you know, humans right now. And even if you set up a system, it's not guaranteed to succeed. Yeah, like, and who's going to add more new data, like you can you can make the loop of AI making it a bit better. But like, it ultimately matters, like, where is the data coming from? You know, like the new data, the new data,
SPEAKER_04: is synthetic data a thing? Or is it just a buzzword? That like this AI can make data that would then train the next AI like, oh, make a bunch of images and then train it on the images. That doesn't make a lot of logical sense to me. But yeah, so not a thing.
SPEAKER_06: So so I think the because you have already added the data to in this case, right? Like it may help the model in some cases, where it may be stuck on some optimization situation. So to give it the necessary noise or the necessary variance in the data, synthetic data may be helpful. But you primarily want more and more new data, and different modalities of data, right? Like videos, and that require new. Let's put this out. Let me tell you an idea. This is an idea
SPEAKER_04: shout out to the team over at Cora. What if Cora look at the
SPEAKER_04: experts with the most highest ratings, you know, with the highest rating, so they look at the experts with the highest ratings. And then they say, Hey, AI wants to talk to you about your expertise. Would you for 100 bucks an hour for 100 hours? answer questions with this AI? Just sit there and answer questions. And you'll go up the leaderboard and we'll send you some cash. But you say to the AI, what do you know about? You know, poetry? What do you know about history? Yeah. And then it goes and asks 100 history experts what they know about history. And then it says, Okay, this history expert told me this, this one has this thing. So you could actually have AI interviewing humans to find out what it doesn't know. That would work. Yeah, yeah, yeah. You basically want human
SPEAKER_06: feedback, especially for the corner cases, the gold data are the corner case data. Gold. Yeah. So like the corner case data where things are not working, or where you would want human feedback. That's the gold data. And, you know, like, if there's an incentive system, that's what I'm saying, like, you basically want Amazon Mechanical Turk at scale. Yes, you know, yes. And but the Yeah, it's mechanical Turk talking to
SPEAKER_04: experts, not low wage. Yes, exactly. Who are saying, is this a hot dog or not? Yeah, you want the person who's the greatest chef in the world who will tell you how to make the hot dog and then 100 other things that kind of where it gets super interesting. So somebody could make an AI mechanical Turk type thing or core could do it, where it identifies the experts, and then just lets AI talk to them constantly, and then pays them for their knowledge. Wow. Then all of the startups that are starting, let's say, legal
SPEAKER_06: startups that are basically that they're actually legal or, yeah, you think they're hiring lawyers to answer
SPEAKER_04: questions? Yeah. Yeah. Yeah, yeah, yeah. Not just giving it to potential customers and saying, hey, give it a feedback loop. They're explicitly going to lawyers and say, Hey, did he get this answer? Correct? Right. Same with healthcare startups,
SPEAKER_06: and you know, on top of LLM, someone or that's what they're doing. Um, yeah, and what's crazy about that is the people
SPEAKER_04: who are being asked to train it are training, they're basically training the replacements. Yeah, they're training the replacement.
SPEAKER_04: No, like, literally, this happened in America. They, there was a movie about it. So moving the 80s, it was a comedy. And it had the guy who played Michael Keaton. I think it might have been Michael Keaton. It was about factories. And I think it was like on a couple different contexts. But the idea was like, Oh, the Americans are going to go to Mexico or Japan and that they would teach them how they did what they did. And they were like, Wait a second, we're teaching them, they're automating it, whatever. And then we're losing our jobs. They're taking our jobs. And there was like, gung ho, maybe it was the name of it. I can't remember. It was a crazy, crazy movie. Yeah, but I think net net, it'll be positive in the
SPEAKER_06: sense that for the most part, you still would want the experts in the room. They may take out the redundant stuff. But I don't see at least in the next 10 years that they lawyers or doctors would be invalidated completely by the AI. That's that's like at least 30 years away. Yeah. Yeah. Anything
SPEAKER_04: you're worried about right now with AI?
SPEAKER_06: I think the regulation part I have to check. I don't know if it affects our startup. But that's one thing. Yeah, that's
SPEAKER_04: the goal is for you not to understand it. You have to hire a lawyer to tell you. Yeah, yeah. The other thing would be
SPEAKER_06: the funding market is bad. Thankfully for AI. It isn't terrible, but who knows what was gonna happen. So there's that. Yeah, yeah. All right. Listen, you've been an amazing guest.
SPEAKER_04: Come back on in like six months or a year. And let's just chop it up and talk about other AI stuff. You'd be good for an AI roundtable. Yeah, I like the fact that you just answer the questions and don't try to be like super correct about the answer. Like this stuff is dangerous. Yeah, it's also inspiring. Absolutely better and better. I mean, I this multi modal thing and chat GPT for is really mind blowing. Like, I just took a picture of a medical document. And I was like, hey, transcribe this. And then I was like, Oh, and give me some advice on it. It's like, you're a doctor. Give me a second opinion. And it does. It's kind of nuts. You know, it's like, you're gonna, yeah, you're gonna be like at the doctor, the doctor is gonna like be looking at your x ray, you take out your phone, you put on chat GPT for and you hear the doctor and you take a picture of it. And then it's gonna like basically check that the doctor got this right or give feedback. And it's gonna be nuts next year. It's coming next year.
SPEAKER_06: Yes, yes. Yeah, we hope like, yeah, once we once we have our plan is that these characters today that you just saw, yeah, they can actually see the virtual world with scene metadata, you know, because it's an engine. Yeah, our goal is that they can actually take in the vision input. Oh, you're gonna give them a vision. Oh, that's so brilliant.
SPEAKER_04: So you're gonna give them virtual eyes. So they see the perspective that the player would say, apply GPT for. Yeah, they learn from what they see. Yes. This is why like, they
SPEAKER_04: might want to stay alive. And they might want to try and hack the game to stay alive. That was by the way, another great 80s film. War Games. I'm giving you all the stuff you missed. War Games. Another great one. Shall we play a game? You've never seen War Games? No, I have to I hit has to be on my list now.
SPEAKER_06: Okay. Oh my god, you're gonna love it. Oh my god. Oh, I see
SPEAKER_04: that alley shield. Sheedy. Oh my god, you're gonna fall in love with a alley sheedy. Like she was like, see, everybody, every nerd, guys like dream girlfriend, Ally sheedy. I see. My oldest was oldest was Blade Runner. And obviously, I was
SPEAKER_06: like free guy free guy is like exactly what we are doing. But you guys exactly forgot to bring up shadow free guy, which is the
SPEAKER_04: NPC trying to break out and then be in the real world. Incredible. Of course, there's no way to do that right now. Yeah, yeah. No, War Games. Pretty great. verticalized AI, becoming sent to me and senti and basically you're gonna love it. Alright, everybody. We'll see you next time on this week. So bye bye.
