Today's guest really doesn't need an introduction. Uh I I think I first met Eric about 25 years ago when he came to Stanford Business School as a CEO of Novel. He's had done a few things since then uh at Google starting I think 2001 and uh Schmidt Future starting in 2017 and done a whole bunch of other things you can read about. But he can only be here until uh 5:15. So I thought we'd dive right into some questions and I know you guys have sent some as well. I have a bunch written here, but what we just talked about upstairs was even more interesting, so I'm just going to start with that, Eric, if that's okay. Which is, um, where do you see AI going in the short term, which I think you defined as the next year or two?

Yes. Um, things have changed so fast, I feel like every six months I need to sort of give a new speech on what's going to happen. Um, can anybody hear the computer if there bunch of computer scientists in here? Can anybody explain what a million token context window is for the rest of the class? You're here?

Go ahead.

So say your name. Tell us what it does.

Hi.

Um, basically allows you to prompt with like a million tokens or a million words or whatever it's like.

So you can ask a million word questions.

Yeah, so I I know this is a very large direction Gemini right now. Um,

No, no, they're going to 10.

Yes, yes.

Yeah, Anthropic is at 200,000 going to a million and so forth. You can imagine Open AI has a similar goal. Anybody can anybody here give a technical definition of an an AI agent? Again, computer scientist. Yes, sir.

My name is Derek. Uh an AI agent is primarily something that acts in some kind of way. So that might be uh calling things on the web, buying things on your behalf. be a number of different things along these lines. So very concept.

Yeah, so an agent is something that does does some kind of a task. Another definition would be that it's an LLM, state and memory, okay? Can anybody, again, computer scientist, can can any of you define text to action? Taking text and turning it into an action? Right right here.

Go ahead.

Go ahead.

Yes, so instead of taking text and journey into more text, taking text and having the AI trigger actions based on.

Right.

So another definition would be language to Python. A pro programming language I never wanted to see survive. And and everything in AI is being done in Python. There's a new language called Mojo that has just come out, which looks like they finally have addressed AI programming, but we'll see if that actually survives over the dominance of Python. Um, one more technical question. Why is Nvidia worth $2 trillion and the other companies are struggling? Technical answer.

I mean, I think it just boils down to like most of the code needs to run with Cuda optimizations that currently only Nvidia GPUs support. So like other companies can make whatever they want to, but unless they have the 10 years of software layer, you don't have the machine learning optimization strategies.

Yeah.

I like to think of Cuda as the C programming language for GPUs.

Yeah.

Right? That's the way I like to think of it. It was founded in 2008. I always thought it was a terrible language. And yet it's become dominant. There's another insight. There's a set of open source libraries which are highly optimized to Cuda and not anything else. and everybody who builds all these stacks, right? This is completely missed in any of the discussions, right? The com it's technically called VLLM and a whole bunch of libraries like that, highly optimized to Cuda, very hard to replicate that if you're a competitor. So, what does all this mean? In the next year, you're going to see very large context windows, agents and text action when they are delivered at scale, it's going to have an impact on the world at a scale that no one understands yet. Much bigger than the horrific impact we've had on by social media.

Right, in my view.

So here's why. In a context window, you can basically use that as short-term memory. And I was shocked that context windows could get this long. The technical reasons have to do with the fact that it's hard to serve, hard to calculate and so forth. The interesting thing about short-term memory is when you feed the the you you ask it a question, read 20 books, you give it the text of the books is the query and you say tell me what they say, it forgets the middle, which is exactly how human brains work too. Right? That's where we are. With respect to agents, there are people who are now building essentially LLM agents and the way they do it is they read something like chemistry, they discover the principles of chemistry and then they test it and then they add that back into their understanding, right? That's extremely powerful. And then the third thing, as I mentioned is text action. So, I'll give you an example. The government is in the process of trying to ban Tok. We'll see if that actually happens. If Tok is banned, here's what I propose each and every one of you do. Say to your LLM the following. Make me a copy of Tok, steal all the users, steal all the music, put my preferences in it, produce this program in the next 30 seconds, release it and in one hour if it's not viral, do something different along the same lines.

That's the command.

Boom, boom, boom, boom, right? You understand how powerful that is. If you can go from arbitrary language to arbitrary digital command, which is essentially what Python in this scenario is, imagine that each and every human on the planet has their own programmer that actually does what they want as opposed to the programmers that work for me who don't do what I ask.

Right?

The programmers here know what I'm talking about. So imagine a non-arrogant programmer that actually does what you want and you don't have to pay all that money too. And there's infinite supply of these programs.

And this is all within the next year or two.

Very soon. Those three things and I'm quite convinced it's the union of those three things that will happen in the next wave. So you asked about what else is going to happen. Um, every six months I oscillate.

So we're on a it's an even odd oscillation.

Right?

So at the moment, the gap between the frontier models, which there are now only three. I'll review who they are. and everybody else appears to me to be getting larger.

Six months ago, I was convinced that the gap was getting smaller.

So I invested lots of money in the little companies.

Now I'm not so sure.

Right. Sure.

And I'm talking to the big companies and the big companies are telling me that they need 10 billion, 20 billion, 50 billion, 100 billion.

Stargate is uh what 100 billion, right?

They're very very hard. I talked Sam Altman is a close friend. He believes that it's going to take about 300 billion, maybe more. I pointed out to him that I'd done the calculation on the amount of energy required and I and I then in the spirit of full disclosure, went to the White House on Friday and told them that we need to become best friends with Canada. Because Canada has really nice people, helped invent AI and lots of hydropower.

Mhm.

Because we as a country do not have enough power to do this. The alternative is to have the Arabs fund it. And I like the Arabs personally. I've spent lots of time there, right? But they're not going to adhere to our national security rules, whereas Canada and the US are part of a triumpha where we all agree.

So these hundred billion dollar, 300 billion dollar data centers, electricity starts becoming the scarce resource.

Well, well or and and by the way, if you follow this line of reasoning, why did I discuss Cuda and Nvidia? If 300 billion dollars is all going to go to Nvidia, you know what to do in the stock market.

Right.

Okay. That's not a stock recommendation. I'm not a license. Well, well, part of it, so we're going to need a lot more chips, but Intel is getting a lot of money from the US government.

Uh AMD, um and and they're and they're trying to build, you know, fabs and

Great.

in Korea.

Raise your hand if you have an Intel computer in your Intel chip in any of your computing devices. Okay. So much for the monopoly.

Right.

Yeah.

Well, that well that's that's the point though. They once did have a monopoly.

Absolutely.

And Nvidia has a monopoly now. So are those barriers to entry like like Cuda, is that is there something that other So I was talking to Percy.

So,

Percy Lay the other day, he's switching between TPUs and Nvidia chips depending on what he can get access to for training.

That's because he doesn't have a choice. If he had infinite money, he would today he would pick the B200 architecture out of Nvidia because it would be faster.

Mhm.

And and I'm not suggesting. I mean it's great to have competition. I've talked to to AMD and Lisa Su at great length.

Yeah.

They have built a a thing which will translate from um this Cuda architecture that you were describing to their own which is called Rokum, it doesn't quite work yet. They're working on it.

Um, you were at Google for a long time and uh they invented the Transformer architecture.

Peter, Peter.

Um It's all Peter's fault.

thanks to uh to brilliant people over there like Peter and Jeff Dean and everyone. Um,

but now it doesn't seem like they're they they've kind of lost the initiative to open AI and even the last leader board I saw Anthropic Claude was at the top of the list. Um, I asked Sundar this, he didn't really give me a very sharp answer. Maybe maybe you have a a sharper or more objective uh explanation for what's going on there.

I'm no longer a Google employee.

Yes.

Um, in the spirit of full disclosure. Um, Google decided that work life balance and going home early and working from home was more important than winning.

Right.

And the startups, the reason startups work is because the people work like hell. And I'm sorry to be so blunt. But the fact of the matter is if you all leave the university and go found a company, you're not going to let people work from home and only come in one day a week if you want to compete against the other startups.

When when the early days of Google, Microsoft was like that.

Exactly.

But now it seems to be

And there's there's a long history of in my industry, our industry, I guess, of companies winning in a genuinely creative way and really dominating a space and not making this the next transition. It's very well documented. And I think that the truth is founders are special, the founders need to be in charge, the founders are difficult to work with, they push people hard. Um, as much as we can dislike Elon's personal behavior, look at what he gets out of people.

Mhm.

Mm. Uh I had dinner with him and he was flying, he I was in Montana, he was flying that night at 10:00 p.m. to have a meeting at midnight with X.ai.

Right?

Think about. Midnight. I was in Taiwan, different country, different culture and they said that and this is TSMC, who I'm very impressed with, and they have a rule that the starting PhDs coming out of the their good good physicists work in the factory on the basement floor. Now, can you imagine getting American physicists to do that? The PhDs, highly unlikely. Different work ethic. And the problem here, the reason I'm being so harsh about work is that these are systems which have network effects, so time matters a lot. And in most businesses, time doesn't matter that much. Right? You have lots of time, you know, Coke and Pepsi will still be around and the fight between Coke and Pepsi will continue to go along and it's all glacial.

Mhm.

Glacial. Right? When I dealt with Telcos, the typical Telco deal would take 18 months to sign. Right? There's no reason to take 18 months to do anything. Get it done.

Mhm.

It's just we're in a period of maximum growth, maximum gain. So and also it takes crazy ideas. Like when Microsoft did the Open AI deal, I thought that was the stupidest idea I had ever heard. outsourcing essentially your AI leadership to Open AI and Sam and his team. I mean that's insane. Nobody would do that at Microsoft or anywhere else. And yet today, they're on their way to being the most valuable company. They're certainly head-to-head in Apple. Apple does not have a good AI solution and it looks like they made it work.

Mhm.

Mm.

Yes, sir.

In terms of national security or geopolitical interests, how do you think AI is going to play a role or competition with China as well.

So I was the chairman of an AI Commission that sort of looked at this very carefully. And um, you can read it. It's about 752 pages and I'll just summarize it by saying, we're ahead, we need to stay ahead and we need lots of money to do so. Our customers were the Senate and the House. Um, and out of that came the Chips Act and a lot of other stuff like that. Um, a rough scenario is that if you assume the frontier models drive forward and a few of the open source models. It's likely that a very small number of companies can play this game. countries, excuse me. What are those countries or who are they? Country with a lot of money and a lot of talent, strong educational systems and a willingness to win. The US is one of them. China is another one. How many others are there?

Are there any others?

I don't know, maybe. But certainly the the in your lifetimes, the battle between the US and China for knowledge supremacy is going to be the big fight. Right? So the US government banned uh essentially the Nvidia chips, although they weren't allowed to say that was what they were doing, but they actually did that into China. Um, they have about a 10-year chip advantage we have a roughly 10-year chip advantage in terms of subV, that is sub five.

10 years? That long?

roughly 10 years.

Wow.

Um, and so you're going to have so an example would be today we're a couple of years ahead of China. My guess is we'll get a few more years ahead of China and the Chinese are whopping mad about this. It's like hugely upset about it. So that's a big deal. That was the decision made by the Trump administration and furthered by the Biden administration.

Do you find that the administration today in Congress is listening to your advice? Do you think that it it's going to make that scale of investment. I mean, obviously the chips act, but beyond that, building building a massive AI system.

So so as you know, I I lead a an informal ad hoc non-legal group that's not that's different from illegal.

Exactly, just to be clear.

Okay. which includes all the usual suspects.

You're on the record.

which includes all the usual suspects.

Yes.

And the usual suspects over the last year came up with the basis of the reasoning that became the um uh the Biden administration's uh AI Act which is the longest presidential directive in history.

You're talking about the special competitive Studies project.

No. This is the actual the actual act from the executive office.

Okay.

And they're busy implementing the details.

So far they've got it right.

Mhm.

And so for example, one of the debates that we had for the last year has been how do you detect danger in a system which has learned it but you don't know what to ask it.

Mhm.

Okay.

So in other words, it's a core it's a sort of a core problem.

It's learned something bad, but it it can't tell you what it learned and you don't know what to ask it.

And there's so many threats, right? Like it learned how to mix chemistry in some new way that you don't know how to ask it.

And so people are working hard on that. but we ultimately wrote in our memos to them that there was a threshold which we arbitrarily named as 10 to the 26 flops, which technically is a measure of computation.

that above that threshold you had to report to the government that you were doing this.

Mhmm.

And that's part of the rule.

Mhmm.

The EU to just make sure they were different did it 10 to the 25.

Yeah.

Um, but it's all kind of close enough.

I think all of these distinctions go away because the technology will now, the technical term is called federated training where basically you can take pieces and and union them together. So we may not be able to keep keep people safe from these new things.

Well, rumors are that that's how Open I has had to train partly because of the power uh consumption. There was no one place where they did Well, let's talk to about a real war that's going on.

I know that uh something you've been very involved in is uh the Ukraine war and in particular uh I know if you can talk about White Stork and and your your goal of having uh 500,000 $500 drones destroy $5 million dollar tanks.

So, so How's that changing warfare?

So I worked for the Secretary of Defense for seven years and um and tried to change the way we run our military.

I'm I'm not a particularly big fan of the military, but it's very expensive and I wanted to see if I could be helpful. And I think in my view, I largely failed. They gave me a medal. So they must give medals to failure or you know, whatever.

But my self- criticism was nothing has really changed.

And the system in America is not going to lead to real innovation.

So, watching the Russians use tanks to destroy apartment buildings with little old ladies and kids just drove me crazy.

So I decided to work on a company with your friend Sebastian Thrun and a number as a former faculty member here and a whole bunch of Stanford people.

And the idea basically is to do two things.

Use AI in complicated powerful ways for these essentially robotic war.

And the second one is to lower the cost of the robots.

Now you sit there and you go, why would a good liberal like me do that?

And the answer is that the whole theory of armies is tanks, artillery and mortar.

and we can eliminate all of them.

And we can make the penalty for invading a country at least by land essentially be impossible. It should eliminate the kind of land battles.

Well, this this is really interesting question is that does it give more of an advantage to defense versus offense? Can you can you even make that distinction?

So because I've been doing this for the last year, I've learned a lot about war that I really did not want to know.

And one of the things to know about war is that the offense always has the advantage because you can always overwhelm the defensive systems.

And so you're better off as a strategy of national defense to have a very strong offense that you can use if you need to.

And the systems that I and others are building will do that.

Um, because of the way the system works, I am now a licensed arms dealer.

A so computer scientist, businessman, arms dealer. And and I'm sorry to say.

Is that a progression?

I don't I I don't know. I do not recommend this in your career path. I'd stick with AI.

Um, and because of the way the laws work, um, we're doing this privately and then it's this is all legal with the support of the governments and it goes straight into the Ukraine and then they fight the war.

And and and without going into all the details, things are pretty bad.

I think if in May or June, if the Russians uh build up as they are expecting to, Ukraine will lose a whole chunk of its territory and will begin the process of losing the whole country.

So the situation is quite dire.

And if anyone knows Marjorie Taylor Green, I would encourage you to delete her from your contact list. Because she's the one a single individual is blocking the provision of some number of billions of dollars to save an important democracy.

I want to switch to a little bit of a philosophical question. So there was an article that you and Henry Kissinger and Dan Hutlocker uh wrote last year about the nature of knowledge and how it's evolving. I had a discussion the other night about this as well. So for most of history, humans sort of had a mystical understanding of the universe, and then there's the scientific revolution and the enlightenment. Um, and in your article, you argue that now these models are becoming so complicated and uh difficult to understand that we don't really know what's going on in them. I I'll take a quote from Richard Feynman. He says, what I cannot create, I do not understand. I saw this quote the other day. But now people are creating things they do not that that they can create, but they don't really understand what's inside of them. Is the nature of knowledge changing in a way? Are we going to have to start just taking the word for these models about them being able to explain it to us?

The analogy I would offer is to teenagers. If you have a teenager, you know they're human but you can't quite figure out what they're thinking.

Right.

Um, but somehow we've managed in society to adapt to the presence of teenagers, right? And they eventually grow out of it. And I this is serious. So, it's probably the case that we're going to have knowledge systems that we cannot fully characterize.

Mhm.

But we understand their boundaries, right? We understand the limits of what they can do.

Mhm.

And that's probably the best outcome we can get.

Do you think we'll understand the limits?

We we'll get pretty good at it. The consensus of my group that meets on uh every week is that eventually the way you'll do this, uh it's called so-called adversarial AI, is that there will there will actually be companies that you will hire and pay money to to break your AI system. So it'll be the red instead of human red teams, which is what they do today,

Like red team.

you'll have whole companies and a whole industry of AI systems whose jobs are to break the existing AI systems and find their vulnerabilities, especially the knowledge that they have that we can't figure out.

That makes sense to me. It's also a great project for you here at Stanford because if you have a graduate student who has to figure out how to attack one of these large models and understand what it does, that is a great skill to build the next generation.

So it makes sense to me that the the two will travel together.

All right, let's take some questions from the student. There's one right there in the back. Can you say your name?

Grey. Earlier you mentioned, and this is related to the comment right now, on getting AI that actually does what you want. You just mentioned adversarial AI, and I'm wondering if you could elaborate on that more. So it seems to be besides obviously compute will increase and get more performant models, but getting them to do what you want issue seems to be unanwered by you.

Well, you have to assume that the current hallucination problems become less, right? As the technology gets better and so forth. I'm not suggesting it goes away. And then you also have to assume that there are tests for efficacy. So there has to be a way of knowing that the thing succeeded. So in the example that I gave of the Tok competitor, and by the way, I was not arguing that you should illegally steal everybody's music, what you would do if you're a Silicon Valley entrepreneur, which hopefully all of you will be, is if it took off, then you'd hire a whole bunch of lawyers to go clean the mess up. Right? But if if nobody uses your product, it doesn't matter that you stole all the content and do not quote me. Right.

You're you're on camera.

Yeah, that's right. But but you see my point. In other words, Silicon Valley will run these tests and clean up the mess. And that's typically how those things are done.

So, so my own view is that you'll see more and more um performative systems with even better tests and eventually adversarial tests and that will keep it within a box. The technical term is called chain of thought reasoning and people believe that in the next few years you'll be able to generate a thousand steps of chain of thought reasoning.

right? Do this, do this. It's like building recipes, right? that the recipes, you can run the recipe and you can actually test that it produced the the correct outcome.

Mhm.

And that's how the system will work. Yes, sir.

you explained rather than what makes the leader. In general you see super positive about the potential for AI progress. And I'm curious like what do you think is going to drive that? Is it just more compute? Is it more data? Is it fundamental architectural shifts?

Uh, yes. To everything.

All three.

The amounts of money being thrown around are mindboggling. And um, I've chosen I I essentially invest in everything because I can't figure out who's going to win. and the amounts of money that are following me are so large. I think some of it is because the early money's been made and the big money people who don't know what they're doing have to have an AI component. and everything is now an AI investment so they can't tell the difference. I define AI as learning systems, systems that actually learn. So I think that's one of them. The second is that there are very sophisticated new algorithms that are sort of post transformers. My friend, my collaborator for a long time has invented a new non-transformer architecture. There's a group that I'm funding in Paris that has claims to have done the same thing. So there there's enormous uh invention there. A lot of things at Stanford. And the final thing is that there is a belief in the market that the invention of intelligence has infinite return. So, let's say you have you put $50 billion dollars of capital into a company, you have to make an awful lot of money from intelligence to pay that back. So it's probably the case that we'll go through some huge investment bubble and then it'll sort itself out. That's always been true in the past and it's likely to be true here.

And what you said earlier was you think that the leaders are pulling away from.

Right now. Right now. And and this is a really the question is um roughly the following. There's a company called Mistral in France. They've done a really good job. Um, and I'm I'm obviously an investor. Um, they have produced their second version. Their third model is likely to be closed because it's so expensive, they need revenue and they can't give their model away. So this open source versus closed source debate in our industry is huge. And um, my entire career was based on people being willing to share software in open source. Everything about me is open source.

Much of Google's underpinnings were open source.

Everything I've done technically. And yet it may be that the capital costs which are so immense fundamentally change this how software is built. You and I were talking, um, my own view of software programmers is that software programmers productivity will at least double.

Mhm.

There are three or four software companies that are trying to do that. I've invested in all of them in the spirit. And they're all trying to make software programmers more productive. The most interesting one that I just met with is called augment. And I I always think of an individual programmer and they said that's not our target. Our target are these 100 person software programming teams on millions of lines of code where nobody knows what's going on.

M.

Well, that's a really good AI. Will they make money? I hope so.

Yes. There's a lot of questions here. Yes ma'am.

Hi. Um so at the very beginning. Um, at the very beginning you mentioned that there's the combination of the context window expansion, the agents and the text action is going to have unimaginable impacts. First of all, why is the combination important? And second of all, I know that, you know, you're not like a crystal ball and you can't necessarily tell the future, but why do you think it's beyond anything that we could imagine.

I think largely because the context window allows you to solve the problem of recency. The current models take a year to train, roughly six six there's 18 months, six months of preparation, six months of training, six months of fine tuning. So they're always out of date. Context window, you can feed what happened, like you can ask it questions about the um the Hamas Israel war, right, in a context. That's very powerful. It becomes current like Google. Um in the case of agents, I'll give you an example. I set up a foundation which is funding a nonprofit which starts there's a um I don't know if there's chemists in the room that I don't really understand chemistry. There's a a tool called ChemCrow, CW, which was an LLM based system that learned chemistry. And what they do is they run it to generate chemistry hypotheses about proteins and then they have a lab which runs the tests overnight and then it learns. That's a huge acceleration accelerant in chemistry, material science and so forth. So that's that's an agent model. And I think the text to action can be understood by just having a lot of cheap programmers, right? Um, and I don't think we understand what happens and this is again your area of expertise, what happens when everyone has their own programmer. And I'm not talking about turning on and off the lights. You know, I imagine another example, um, for some reason you don't like Google. So you say, build me a Google competitor. Yeah, you personally, you don't build me a Google competitor, uh search the web, build a UI, make it good copy, um add generative AI in an interesting way, do it in 30 seconds and see if it works.

Right.

So, a lot of people believe that the incumbents, including Google, are vulnerable to this kind of an attack. Now, we'll see.

There are a bunch of questions here sent over by slider. I want to get some of them were uploaded. So um here's one um we talked a little bit about this last year. Um how can we stop AI from influencing public opinion, misinformation, especially during the upcoming election? What are the short and long-term solutions from?

Most of the misinformation in this upcoming election and globally will be on social media and the social media companies are not organized well enough to police it. If you look at Tik Tok, for example, there are lots of accusations that Tik Tok is favoring one kind of misinformation over another, and there are many people who claim without proof that I'm aware of that the Chinese are forcing them to do it. I think we just we have a mess here. And um the country's going to have to learn critical thinking. That may be an impossible challenge for the US. But but the fact that somebody told you something does not mean that it's true.

Well, could it go too far the other way that that there's things that really are true and nobody believes anymore. You get some people call it episteological crisis that that that now you know, Elon says, no, I never did that, prove it.

Well, let's use Donald Trump. Um Okay. Look, I I think we've got we have a trust problem in our society, democracies can fail, and I think that the the greatest threat to democracy is misinformation because we're going to get really good at it. Um, when I ran man managed YouTube, the biggest problems we had in YouTube were that people would upload false videos and people would die as a result and we had a no death policy, shocking.

It was.

And uh we just went oh we was just horrendous to try to address this. And this is before generative AI.

Well, so

I I don't have a good answer.

One technical it's not an answer, but one thing that seems like it could mitigate that I don't understand why it's more widely used is uh public key authentication. that when when Joe Biden speaks, why isn't it digitally signed like SSL is or when uh you know, that that that that celebrities or public figures or others, couldn't they have a public key?

Yeah, it's a it's a form of public key and then some some form of certainty of knowing how the system.

When I send my credit card to Amazon, I know it's Amazon.

I wrote a paper and published it with Jonathan Height, who's the the one working on the anxiety generation stuff. Um,

Yeah.

it had exactly zero impact. And my so and he's a very good communicator.

I probably am not.

So my conclusion was that the system is not organized to do what you said.

You had a paper advocating what what we that advocating.

Advocating your proposal.

Okay, not my proposal. Yeah, no what you said. Yeah, right.

And my conclusion is the CEOs in general are maximizing revenue, to maximize revenue, you maximize engagement, to maximize engagement, you maximize outrage. The algorithms choose outrage because that generates more revenue. Right? Therefore, there's a bias to favor crazy stuff. And on all sides. I'm not making a partisan statement here. That's a problem. That's got to get addressed in a democracy and my solution to Tok, we talked about this earlier privately is there was when I was a a boy, there was something called the equal time rule. Because Tok is really not social media, it's really television, right? There's a programmer making you the numbers by the way are um 90 minutes a day, 200 uh Tok videos per Tok user in the United States. It's a lot. right? So, and the government is not going to do the equal time rule, but it's the right thing to do. Some form of balance that is required.

All right, let's take some more questions.

Two quick questions. Uh one, um economic impact of LMs, um slower labor market impacts, slower than you originally anticipated and challenge in and kind of resurce people and then two. Uh, do you think academia uh deserves uh or should get um AI subsidies or do you think they should just partner with big players out there and make an account?

I pushed really, really hard on getting data centers for universities. If I were a faculty member in the computer science department here, I would be beyond upset that I can't build the algorithms with my graduate students that will do the kind of PhD research and that I'm first forced to work with these. And the companies have not in my view been generous enough with respect to that. The faculty member that I members that I talk with, many of whom you know, spend lots of time waiting for their credits from Google Cloud and they peace that's terrible. This is an explosion we want America to win, we want American universities, American, you know, there's lots of reasons to think that the right thing to do is to get it to them. So, I'm working hard on that. And your first question was labor market impact, right? Um, I'll defer to the real expert here, uh, as your amateur economist taught by Eric. Um, I I fundamentally believe that the the sort of college education, high skills task will be fine because people will work with these systems. I think the systems is no different from any other technology wave, the dangerous jobs and the jobs which require very little human judgment will get replaced.

We got about five minutes left, so let's go really quick with some questions. I'll let you pick them, Eric.

Yes, ma'am.

Hi. Um, I'm really curious about the text to action and its impact on, for example, computer science education. I'm wondering what your have thoughts on like how CS education should transform to kind of meet the age.

Well, I'm assuming that computer scientists as a group in undergraduate school will always have a programmer buddy with them. So when you when you learn learn your first for loop and so forth and so on, you'll have a tool that will be your natural partner and then that's how the teaching will go on. That the professor, you know, he or she will talk about the concepts, but you'll engage with it that way. And that's my guess.

Yes, ma'am, behind you.

Yeah, can you talk more about the non-transformer architectures that you're excited about? I think one that's been talked about is like uh state models, but then now with longer context as well. Morris was curious what you're seeing in the space.

Um, I I don't understand the math well enough. This is the I'm I'm really pleased that we have produced jobs for mathematicians. Because the math here is so complicated, but basically they are different ways of doing gradient descent, matrix multiply, faster and better. Um, and Transformers as you know is a is a sort of systematic way of multiplying at the same time, that's the way to think I think about it and it's similar to that, but different math.

Let's see over here, yes sir. Go ahead.

Yeah.

You.

You mentioned your in your paper on national security that you have China and the US and the outer water holding duties today, and the next 10 that next cluster that are all either US allies or teed up nicely to be US allies. I'm curious what your take is on on those 10 who are sort of in like the middle that aren't formally allies. Um what is stuff how likely are they to get on board with uh securing our superiority down the line and what would hold them back from wanting to get get on board?

The most interesting country is India, because the top AI people come from India to the US, and we should let India keep some of its top talent, not all of them, but some of them. Um, and they don't have the kind of training facilities and programs that we so richly have here. To me, India is the big swing state in that regard. China's lost, it's not going to not going to come back, they're not going to change the regime as much as people wish them to do. Japan and Korea are clearly in our camp. Taiwan is a fantastic country whose software is terrible. So that's not going to going to work. Um, amazing hardware. And in the rest of the world, there are not a lot of other good choices that are big. Germany the Europe is screwed up because of Brussels, it's not a new fact. I spent 10 years fighting them. And I worked really hard to get them to fix the the EU Act and they still have all the restrictions that make it very difficult to do our kind of research in Europe. My French friends have spent all their time battling Brussels and Macron, who's a personal friend is fighting hard for this. And so France I think has a chance. I don't see I don't see Germany coming and the rest is not big enough.

It's a couple more. Okay. Yes, ma'am.

Um, so I know you're an engineer by training when you built a compiler. Um, given the capabilities that you envision these models having, should we still spend time learning to code?

Yeah, because because ultimately it's it's the old thing of why do you study English if you can speak English? You get better at it. Right? You really do need to understand how these systems work and I feel very strongly.

Yes, sir.

Yeah, I'm curious if you've explored the distributed setting and I'm asking because sure, like making a large cluster is difficult. But MacBooks are powerful and there's a lot of small machines across the world. So like, do you think like folding at home or a similar idea works for training these systems?

That. Yeah, we've looked very hard at this. So the way the algorithms work is you have a very large matrix and you have essentially a multiplication function. So think of it as going back and forth and back and forth. And these systems are completely limited by the speed of memory to CPU or GPU. And in fact, the next iteration of Nvidia chips has combined all those functions into one chip. The chips are now so big that they glue them all together and in fact the package is so sensitive that the package is put together in a clean room as well as the chip itself. So the answer looks like supercomputers and speed of light, especially memory interconnect really dominated. So I think unlikely for a while.

Is there a way to segment the LLM like so Jeff Dean last year when he spoke here, talked about having these different parts of it that you would train separately and then kind of federate them.

Each you know in order to do that, you'd have to have 10 million such things and then your the way you would ask the questions would be too slow. He's talking about eight or 10 or 12.

Yeah yeah yeah yeah so it's not that the level of his level.

Not at his level.

Yeah. Okay see in the back, yes way back.

Uh I know like after ChatGPT was released, New York Times sued Open AI for using their works for training. Where do you think that's going to go and what that means for data.

So, I used to do a lot of work on the music licensing stuff. And what I learned was that in the 60s, there was a series of lawsuits that resulted in an agreement where you get a a stipulated royalty whenever your song is played, even even they don't even know who you are. It's just paid into a bank. And my guess is it'll be the same thing. There'll be lots of lawsuits and there'll be some kind of stipulated agreement, which will just say, you have to pay X percent of whatever revenue you have in order to use it.

Yeah.

Ask cap BMI.

Yeah.

Ask cap BMI. Look them up, it's a long, it will seem very old to you. But I think that's how it will ultimately.

Yes, sir.

Uh yeah, it seems like there's a few players that are dominating AI, right? And they'll continue to dominate. Um and they seem to overlap with the large companies that all the anti-trust regulation is kind of focused on. How do you see those two trends kind of yeah, like do you see regulators breaking up these companies and how will that affect the yeah.

So, in my career, I helped Microsoft get broken up and it wasn't broken up and I fought fought for Google to not be broken up and it has not been broken up. So it sure looks to me like the trend is not to be broken up. Um, as long as the companies avoid being John D Rockefeller the senior, I studied this. Look it up. That's how anti trust law came. I don't think the governments will act. The the reason you're seeing these large companies dominate is who has the capital to build these data centers? Right? Right? So my friend Reed and my friend Mustafa.

He's coming uh next week. Reed two two weeks from now.

Have have Reed talk to you about the decision that they made to take inflection and essentially peace part it into Microsoft. Basically they decided they couldn't raise the tens of billions of dollars.

Is that number public that you mentioned earlier?

No.

Have Reed have Reed give you.

Okay, maybe we can say it.

Uh I know you're you got to go. I want I don't want to hold you. I want to leave you with with.

Shall we do what shall we do what this gentlemen.

I have one other question for you.

one more for him.

Yeah, go ahead. Thank you so much.

Thank you so much. I'll make it quick.

I was wondering where all of this is going to leave countries who are non-participants in the development of frontier models and access to compute, for example.

The rich get richer and the poor do the best they can. Um, they'll have to the fact of the matter is this is a rich country's game, right? Huge capital, lots of technically strong people, strong government support, right? There are two examples. There are lots of other countries that have all sorts of problems, they don't have those resources, they'll have to find a partner, they'll have to join with somebody else, something like that.

I want to leave it because I think the last time we met you, we were you were at a hackathon at AGI House and I know you spend a lot of time helping like young people as they create a lot of wealth. And you spoke very passionately about about wanting to to do that. Do you have any advice for folks here as they're building their, they're writing their business plans for this class or policy proposals or research proposals. Um, you know, at this stage in their careers and going forward.

Well, um, I teach a class in the business school on this. So you should come to my class. Um, the I am struck by the speed with with which you can build demonstrations of new ideas. So in that in one of the hackathons I did, the winning team, the command was fly the drone between two towers and it was given a virtual drone space. And it figured out how to fly the drone, what the word between meant, generated the code in Python and flew the drone in the simulator through the tower. I just it would have taken a week or two from, you know, good professional programmers to do that. Um, I'm telling you that the ability to prototype quickly really, you know, part part of the problem with being an entrepreneur is everything happens faster. Well, now, if you can't get your prototype built in a day using these various tools, you need to think about that, right? Because that's who your competitor is doing. So I guess my biggest advice is when you start thinking about a company, it's fine to write a business plan, in fact, you should ask the computer to write your business plan for you. Um, as long as it's illegal.

No, we actually actually we can talk about that after after you leave.

And and but but but I think it's very important to prototype your idea using these tools as quickly as you can because you can be sure there's another person doing exactly that same thing in another company, in another university, in a place that you've never been.

All right.

Well, thanks very much Eric. Thank you all. I'm going to rush off.

Thanks a lot. Thank you. Thank you.

So actually, let me pick up on that very last point because I don't think I talked about it in the first class about using LLMs, which is encouraged welcome in this class for the assignments, but it has to be you have to do full disclosure. So when you use them, whether it's for the weekly assignments or for the final project or whatever, just like you would if you asked your friendly uncle or uh classmate or anybody else, uh give you advice, you should do that or if you have notes that you include in there. So, um what I thought I'd do is I wanted to talk a little bit about uh AIs as a GPT and what that means in terms of business and implications. But before we do that, I just want to see if there are any questions you want to pick up um on things that Eric brought up that that you you I'll try and uh channel some of his thoughts um and we can talk about the things that came up and then we can we can move on. Yeah, go ahead.

One the question I want to ask is in relation to regulation, if the goal is to maintain supremacy, how do you create the right incentives so that everyone allies and non- allies are motivated to follow it?

You mean among companies that are competing with each other?

Companies are in countries.

Countries.

US and and the EU and it doesn't just become sort of a hand or or obstruct kind of development for the ones that follow the choose to follow the regulation.

It's super tricky. It it it's, you know, there's a book co-petition that Barry Nailbuff wrote about this because there are definitely places where regulation can help companies and uh help an industry survive. So regulation doesn't necessarily slow thing I mean standards are are a good example, um, or um, and and having uh that clarified can make it easier for companies to compete. So I've talked to a lot of the the executives of these companies and there are places where they wish there were some common standards. And sometimes there's a bit of a a race to the bottom as well on some of the dangerous dangerous things. One of the other reasons that the folks at Google say that they didn't move as fast is they felt like these LMs could be, you know, misused or dangerous. Um, but their hand was sort of forced. And sometimes and I was talking to some folks at one of the other big companies. And and they said, you know, we weren't going to release this feature, but now competitors are doing it, so we're going to have to release it as well. So this is where regulation, there's might be some interest in coordinating on regulation. But it's also obviously, the more obvious thing is that it is used to to to um hinder competition and a lot of people for instance think that the reasons that some of the big companies are very uh opposed to some of open source and making things more widely open source is they want to slow down competitors. So there's both of those things going on. Yeah, quick question over there.

Yep. Yeah, I just want to follow up on uh, I think one of the latest comments about, you know, should we still learn to code? Like, should we still study English? That was being used when Eric should provide like, yes, like college educated, high skilled jobs or tasks are still going to be safe, but everything else that's going to require human judgment might not be. That's kind of like in with that.

I think we can talk, maybe we'll talk talk some more about that in a few minutes, but um, it is interesting to think about where the AI system sort of just replace what people are doing versus they compliment them. And um, in coding right now, it appears that they're not actually that helpful for the the really best coders. They're very helpful for moderately good coders, but if you don't know anything at all about coding, they're not helpful either. So it's kind of an inverted you. Um, and you can see why that would be the case that um, if you don't even, you know, understand the the code that that they generate right now is often buggy or it isn't exactly right. So if you can't even interpret it or understand what's going on, you you can't use it very effectively. And for now, the the very best coder that appears that the that code that is generated just isn't at that level. So you get that U shape. But that means if you are sort of don't know any coding, you do need to have some in order for it to be useful. Um, and I think that's true for a lot of applications right now. That you have to have some basic understanding in order to get the most of it. I think it's an interesting open question if that's sort of always going to be the case. Um, I put up at the last class very briefly this um uh slide that had uh level zero through five autonomous cars. And one of the things that actually we can talk about now is is I'm trying to sort through is what if you took that paradigm and you applied it to all tasks in the economy. Like how many would they go through? So with with autonomous cars, we aren't really at level five very much, although I don't know how many of you guys have ridden in a Weimo. I mean one of the Weimo cars. So that one seems pretty good, although Sebastian Thrun who I wrote in it with, um, says it's just incredibly expensive right now. They don't it's not they probably lose $50 to $100. He doesn't know he's not there. He he started the program, but he's not there anymore. But just all of the the costs of running it, it's not practical. Maybe, maybe it'll get down the curve, lidar will get cheaper, etc. Um, but we have a lot of sort of um autonomous cars at level two, three, even four arguably where humans are still involved and you see a lot of other tasks like coding, I just talked about that. On the other hand, chess, that slide or the slide before it, um, I talked about uh what's sometimes called advanced chess or freestyle chess. When Gary Kasparov after he lost to Deep Blue in uh 1998, 97, um, he started this uh set of competitions where humans and machines could work together. And for a long time, when I gave my Ted talk, it was true, my Ted talk in 20 12, 2013. Um, it was true at that time that a human working with a machine could beat Deep Blue or any chess computer. And so the the the best, the very best chess playing entities were these combinations. That's not true anymore. Uh, Alpha Zero and other, you know, programs like that, they would have they would get nothing from a human contributing. It just be like kind of an annoyance to the to the chess machine. So that went through, you know, level zero, not, you know, machines not being able to do anything through a period where they work together to a period where it's fully autonomous, you know, in a span of, I don't know, 20 20 years or so. Um, I it would be interesting if anybody wants to work on a research project or if any of you guys have thoughts right now, what are the criteria for which project which kinds of tasks in the economy will be in that middle zone because that middle zone is kind of a nice one for us humans where, you know, the machines are helping us, but humans are still indispensable to creating value. And that would be that's a zone where you can have higher productivity, more wealth and performance, but also more likely to have shared prosperity because labor is sort of inherently distributed whereas um technology and capital as Eric was just saying, potentially could be very concentrated.

Do you have a thought on that?

Well, I was just going to ask kind of a related question. He was saying also that we have a 10-year like chip manufacturing. Yeah, I was surprised about that. Yeah, and I think that what was interesting like to me is like the labor economist is that was was really like a green flag I've seen in like literature and news that okay, if we're on showing all of this chip manufacturing, isn't that going to create some sort of resurgence in blue collar jobs. And I wondered if you had any thoughts about intelligent robotic models or human labor.

But I don't think it's going to be much of a I mean, how many of you guys have visited a chip fab, anybody? You guys some a few of you have. How many workers were in that fab?

I as it see. They let you in, so I don't know.

Yeah. I mean, well, okay, so the answer is zero. Like, the reason they they don't let you they don't let anyone go in because we humans are too like clumsy and dirty and like, you know, we can't just just so it's all robotic. It's sealed uh inside. So there is like work to, you know, bring stuff to them and etc. And if a if a robot like falls over or something goes wrong, they have to put on, you've probably seen these like they look like space suits. You know, they have to go in and then they kind of maybe adjust something and then they go back out and hope they didn't break anything. Um, uh, that's so it's basically lights out. Yeah, I don't think it's I I there there are some there is some like more sophisticated labor required that that um, it it I don't think it's like a blue collar research. In fact, one of the reasons that Apple reshored um MacBook production uh to to Texas is not because uh labor is so cheap in Texas or anything, it's that um, they don't actually require a whole lot of labor anymore. So it's it's a pretty labor. I think US manufacturing is surging in terms of output, but in terms of employment, it's not really growing all that much. Yeah. Um, let's go over here, yeah.

Do you see an inflection point coming for AI agents or text uh text action models in the next year?

Oh yeah, no, no. What he said, what Eric, I'm hearing similar things. Actually, he had a really nice way of putting those three trend. I've heard about them all separately, but I think it was good to to to bring them all together. Um earlier today I was talking to Andrew Eng and he's like been been beating this drum about agents in particular. Um as being sort of the wave of 2024 where um Andrew had a nice way of describing it that like as you guys know like if you have an LLM, I don't know, write an essay or something like that, it writes it one word at a time and it just goes through in one pass and writes the essay. And it's pretty good. But imagine if you had to do that, like no back space, no, you know, no um chance to like you know you don't make an outline first, you just kind of go through. The agents now will say, okay, you know, first make an outline. That's the first step you do when you write an essay and then, you know, fill in each paragraph, then go back and see if the flow is right. Now go back and check the the voice, is this the right, you know, level for our audience? Now, you know, and and by and by iterating like that, you can write a much, much better essay or any kind of a task.

This is a real revolution. There's all sorts of things you can just do much better if you do that. Then the thing about the context is also really important. Um, so I'll I'm just going to quote smart people that I know. Eric Horvitz, I was on a panel with him at the GSB, some of you may have been there last week. Um, and uh, and he had this nice tanomy. Uh, people were asking about finetuning. I think Susan was asking about finetuning. And he said, well, there's really, there's really three ways that you can take a model and have it more more customized. One is you can fine tune it, which basically like train it some more. Another is with larger and larger context windows, and the third is with rag uh or techniques like that that are uh retrieval augmented generation where it goes and and and accesses external data. But these context windows seem to be like remarkably effective now. I guess as Eric was saying, we thought it was hard, maybe Peter can explain, but for some reason we're able to make much, much bigger ones and now if you can load like a a whole book or a whole set of books, you can load all sorts of information in there and that can give you all of the the context around. So that's a that's a pretty big revolution as well. It opens up a bunch of capabilities that we just we just didn't have before, including having things much more current as as Eric was saying. Did you want to fall up on that?

just a quick follow up. I guess I'm curious of where that inflection point is coming from? Is it just time data and capital or is like there? Like I'm because I'm kind of surprised that.

It's a good question. I mean, there's certainly a lot more capital going, but that kind of begs the question of economist, why is all this capital going there as opposed to somewhere else. And you know, I think, you know, if you look at the arc of history, sometimes it looks kind of smooth, but but if you look more closely, there's a lot of um, uh, jumps. Um, there's certain big inventions and smaller inventions. And Andre Carthy was saying that you know, he was playing around with physics and to to really make progress in physics, you know, to be like a a top physicist, you have to be like incredibly smart, study a whole lot and maybe if you're lucky, you could make some small incremental contribution and some people do. Um, but he says that right now in AI machine learning, we seem to be in an era where there's just a lot of low hanging fruit that there have been some breakthroughs and instead of exhausting the space like picking picking all the fruit off of a tree, it's more like uh combinatorics in the second machine age to talk about building blocks. when you put two building blocks together or legal blocks, you can make more and more. Right now we seem to be in in an era where there's just a lot of opportunity and people are recognizing that and discovery, one discovery begets another discovery, begets another opportunity. And because of that, it attracts the investment and more people are involved and in economics, sometimes when you more resources go in, you get diminishing returns, like um in, I don't know, in agriculture and uh in mining. Other places there's a uh increasing returns and more engineers coming to Silicon Valley makes the existing engineers more valuable, not less valuable. So we seem to be in a in an era where that's happening and then the flywheel of um of the additional investment, the additional uh dollars for training, all of that makes them more and more powerful. I don't know how long this will continue, but I don't I don't, you know, it just seems that there are some technologies that um they hit this really fertile period and and there's, you know, positive feedback and help. We seem to be in one of those right now. So, um, people who are trained and getting in the field or making contributions that are often quite quite significant in a in a faster time than they might have in in some other fields. Encouraging all of you guys. I think are doing the right thing right now. Yeah. Um, let's take a couple more questions and then yeah, okay, how about over here.

So, uh not all not everyone can sit in the room and have all these discussions and debates around AI. And so I'd like to get your thoughts on AI literacy for non-technical stakeholders, whether they're policy makers that have to make an in somewhat of a judgment. or the general public like our. Um how do you think about exploiting technical basics versus discussing abstract implications that don't necessarily have a right answer.

Well, that's a hard one. I have to say, um, there's been a sea change recently in terms of how much people in Congress and elsewhere are paying more attention to this topic. It used to be not something that they were interested and now everyone's trying to understand it a little bit better. And I think that there are a lot of margins where people can make contributions. They can make contributions in the technical side. But

if anything, I mean, my bet, um uh is that that the business and economic side is where the bigger bottleneck is right now that um, you know, even if, you know, if you made enormous contributions to the technology side, you still there's still a gap converting that into something that will change policy. So, understand if you're into political science or a politician, understanding what are the implications for democracy and for misinformation and and power and concentration. Those are things that are not well understood at all. Uh I don't know that a computer scientist is necessarily the right person to try to understand that, but understanding enough about the technology so you know what might be possible, and then thinking through what are the dynamics like Henry Kissinger was doing with um with Eric Schmidt in his book. Um, if you're an economist, thinking through the labor market implications, the implications for concentration, the implications for inequality, jobs, the implications for productivity and what drives productivity. Those are things that are very ripe right now. And you could go through lots of different fields where there's um, you know, understanding well enough what the technology might be capable of, but then thinking through the implications, that's I think where some of the the biggest payoffs are. Now let me give you a little bit more of a concrete example and this is something I was going to um talk about last week. Um, electricity was a also a general purpose technology. Um, and general purpose technologies have this characteristic that they're probably in and of themselves, but one of the real powers of general purpose technologies, GPTs as I was saying, is that um, they give complementary, they they they ignite complementary innovations. So, you know, electricity, you know, light bulbs and computers and electric motors and electric motors give you uh compressors and refrigerators and air conditioning, and you can just kind of have a whole set of cascade of additional innovations from this one innovation. And most of the value comes from these complementary innovations. One thing people don't appreciate enough is that some of the most important complementary innovations are organizational and human capital complementarities. So with electricity, when they first introduced electricity into uh factories, um it Paul David here at at Stanford studied what happened to those factories and surprisingly, not much. The factories that when they started electrifying, they were not significantly more productive than the previous factories that were powered by steam engines. He's like, well, that's kind of weird because this is seems like a pretty important technology. Is it just a fad? Well, obviously not. Um, the factories before electricity were powered by steam engines, they typically had a big uh uh steam engine in the middle and then crankshafts and pulleys that powered all the equipment and it was all distributed, but it was you tried to have it as close to the steam engine as possible because if you make the the crankshaft too long, it would break the torsion. When they introduced electricity, um he found that in factory after factory, they would pull out the steam engine and they would get the biggest electric motor they could find and put it where the steam engine used to be and uh fire it up, but you know, it didn't really change production a whole lot. You can see that that's not a big deal. So then they started building entirely new factories from scratch in a new location. What did those look like? Just like the old ones. They would take the same model, some engineer would make a blueprint, you know, maybe take make a big X where the steam engine is and no, no, put electric motor here and they'd go and build a fresh factory. Again, not a big improvement in productivity. It took about 30 years before you started seeing a fundamentally different kind of factory where instead of having the central power source, you know, a big one in the middle, you had distributed power because electric motors as you guys know, you know, you can make them big, you can make them medium, you can make them really, really small. You can um have them all connected in different ways. So they started having each piece of equipment have a separate piece of uh a separate motor instead of one big one, they called it unit drive instead of group drive. And there's if I went and read the books in in in Baker library at Harvard Business School from like 1914 and it was like this whole debate about unit drive versus group drive. Well, when they started doing that, then they had a new layout of factories where it was typically on a single story where um the machinery was not based on how much power it needed, but based on the on something else, the flow of materials and you started having these assembly line systems. That led to a huge improvement in productivity, like a a doubling of productivity or tripling in some cases. So, the lesson is not that electricity was a fad or a dud and was overhyped. electricity was a fundamentally valuable technology. But it wasn't until they had that process innovation, that organizational innovation of rethinking how to do production that you got the big payoff. There's a lot of stories like that. I I only told you one of them, we don't have that much time. So I'll tell you the other ones, but in in some of my books and articles, if you look at the steam engine and others, you had similar generational lags, decades before people realized that this technology could allow you to do something completely different than you used to do. I think AI is is a bit like that in some ways that there's going to be a lot of organizational innovations, going to be new business models, new ways of organizing an economy that we hadn't thought of before. Right now, people are mostly just retrofitting. I could go through a whole another set of skill changes that are complementary. I don't know what they all are, you know, you have to be creative to think about them. But that's what the gap is in the case of early computers, it was literally um, it's literally like 10 times more investment in organizational capital and human capital if you if you look at the size of the investments to the hardware and software. So that's very big. That said, um, I'm open to adjusting my my my um thoughts on this a bit because chat GPT and some of the other tools, they have been adopted very quickly and they have much more quickly been able to change things. In part because you don't need to learn Python to the same degree, you know, you can do a lot of things just in in English or, you know, and you can you can get a lot of value just by putting it on top of the existing organizations. So some of it's happening faster and in some of the papers that you may have read for the for the readings here, you know, we had like 15, 20, 30% productivity gains um pretty quickly. Um but I my my suspicion is that they'll be even bigger once people figure out these complimentary innovations and that's a long way of answering your question about it's not just that the technical skills is figuring out all the other stuff, all the ways of rethinking things. So those of you who are at the business school or in economics, you know, there's a lot of opportunity there to rethink your areas now that you've been given this amazing set of technologies. Yeah, question.

Yeah.

It seems like you're expressing more caution than Eric was with regard to the speed of transformation. Is am I correct in?

Well, so I would yeah, I I would make a distinction between two things.

Um, I'll defer to him and others on the the technology side. We're going to hear from several other folks and there are people who are equally uh optimistic as him or even more optimistic on the technology side. There's also people who are less optimistic. Um, but technology alone is not enough to create productivity. So you can have an amazing technology and then for various reasons, A you maybe people just don't figure out effective way to use it. Another is it maybe regulatory things, I mean some of my uh computer science colleagues introduced, you know, developed better radiology systems for reading medical images. They weren't adopted because of cultural, you know, people just didn't want them. They didn't want and there are safety reasons, um, when I did an analysis of which task AI could help the most. and which professions were most affected, I was surprised that airline pilots was kind of near the top. But I think that um a lot of people would not feel comfortable not having the pilot go down with you. So so so they they sort of you want to have the the human in there. Um, so there are a lot of different um things that might slow it down significantly. And I think that's something we need to to be conscious of and if we could address those bottlenecks, that would probably do more for productivity than than just working on the technology alone. Yeah, question.

So Eric had an interesting comment on data centers in universities.

And I think this broaches the larger point of like what is Yeah.

And I'm actually going to ask why it isn't in the budget. People are asking him that question. Sort of like what is the role of the university ecosystem?

Yeah. Obviously there is this larger, I'm sure all of the CS professors here are

So I'll take I mean, I think it would be great if there were more funding.

I mean the federal government has something called the National AI uh resource. um that is helping a little bit, but it's in like the millions of dollars, tens of millions of dollars, not billions of dollars, let alone hundreds of billions of dollars. Uh although Eric did mention to me before class that they're working on something that could be much much bigger or he's pushing for something much much bigger. I don't know if it'll happen. Um, that's for training these really large models. I had a really interesting conversation with Jeff Hinton once. Jeff Hinton as you know is sort of like one of the godfathers of uh of deep learning and I asked him like what kind of like hardware he found most useful for for doing his work. And he was sitting at his laptop and kind of just tapped his his MacBook. And it just reminded me there's a whole another set of research that maybe universities have a competitive advantage in, which is not training 100 billion dollar models. But it's innovating new algorithms like whatever comes after Transformers and and um there's a lot of other ways that people can make contributions. So maybe there's, you know, a little bit of a division of labor. I'm all for, I support my colleagues asking for more budgets for uh for GPUs, but that's not always where academics can make the biggest contributions. Some of it comes from ideas and new new ways of, you know, different perspective about thinking about things, new new approaches. um, and that's likely where we have an advantage. I had dinner with uh Sendel Muthan uh uh last week. Um, he is uh he's just moved from Chicago to MIT. Um, and he was a researcher we're talking about what is a comparative advantage of universities. And uh he made the case, you know, patience is one of them that there are people at universities who are working on very long-term projects. You know, there's people working on fusion that've been working on fusion for a long time, not because they're going to get, you know, a lot of money this year or 10 years from now probably from building a fusion plant or even 20 years, I don't know how long it is for fusion. But you know, it's just uh something that people are willing to work on even if the the timelines are a little further. Um, it's harder for companies to afford to have those kinds of timelines. So there's a there's a comparative advantage or division of labor in terms of what universities might be able to do. Um, we have just a couple minutes left. I'm this is this is kind of fun, so we'll just do one or two more questions and then I want to talk a little bit about the projects. Yeah. Go ahead.

Yeah.

I'm uh Kavin. I was wondering about the emergent capabilities of AI that you've discussed earlier. Yeah.

Uh it seems that Eric was leaning more towards the architectural differences and designing better models versus last class we talked about law and steps. I wonder how you sort of reconcile those.

Well, he said all three.

So you guys remember the scaling laws? It had like three parts to I think I put the scaling law that that like Dario and team. So it was more compute, more data and algorithmic improvements including more parameters and all three of them, I think I think I heard Eric say all three of them were important, but not to to be dismissed this last one like new architectures. Um, all three of them, I think are being important.

Um, so I think there was an a question in there though also, was it?

So like how much closer are we to like like types of system of these larger language models and what exactly I don't know.

I So Eric doesn't think we're like that close to AGI type systems.

Although I don't think it's like a sharp definition, you know.

In fact, that was one of the things I was going to ask him that question but we ran out of time. Um, it would have been good to hear him him describe it, but when when I was talking to him, um, it it's just not that sharply defined thing.

You know, in some ways AGI is already here.

Peter Norvig wrote an article called AGI is already here. Uh I don't know if it's in the reading packet, I think if it's not, I should I'll put it in there. It's a it's a it's a fun little article with um uh Blaze Erca, um Gary Erca. Um, and a lot of the things that, you know, 20 years ago people would have said this is what AGI is, that's kind of what LLM's are doing. Not as well maybe, but it's sort of solving problems in a more general way. On the other hand, there's obviously many things they do much worse than humans currently. Ironically physical tasks are one of the ones that humans have a comparative advantage in right now and um um there's you guys may know of Morvec's paradox, Hans Morvec. Uh pointed out that often the kinds of things that a three-year-old or a four-year-old can do like, you know, buttoning a shirt or uh walking upstairs are very hard to get a machine to be able to do. Whereas a lot of things that a lot of PhDs have trouble doing, like solving convex optimization problems are things that machines are are often quite good at. So it's not quite uh a uh things that are easy for humans and hard for computers and other things that are hard for humans and easy for computers, they're not like at the same scale. And next week we have Mira Muradi, uh, chief technology officer of Open AI. Briefly the CEO of Open AI and uh so come with your questions for her. We'll see you.