Isaac Asimov Memorial Debate

NEIL DEGRASSE TYSON (Frederick P. Rose Director, Hayden Planetarium): Good evening. Welcome back. This is our 25th year of the Isaac Asimov Memorial panel debate. It was established in memory of Isaac Asimov, the noted science fiction and non-fiction author. Anyone of the era, any interest in science at all, was influenced by the writings of this man. I would later learn, even after I took office here, that he lived down the street a few blocks and was here often using our research libraries to feed the contents of his books. And so, we have this kind of genetic link to the man. And this memorial panel debate is in his honor. And I want to publicly thank the interest and support of his family. There are the relatives of Janet Asimov, his widow, there in the audience, as well as Isaac Asimov's daughter, Robin Asimov. Robin, good to see you again. Yes. 

Sorry, I didn't— how rude of me. I didn't introduce myself. I'm Neil deGrasse Tyson. I am the Frederick P. Rose Director of the Hayden Planetarium, and this is our flagship event of the year. Thank you for attending. Yes.

The topic tonight is AI. Oh, yeah. Oh, yeah. And do we have a panel for you! Let me— oh, I just, as a point of disclosure, I am in the class action suit against Anthropic for pirating books. There's, like, nine of my books that were pirated and used to train their AI models. So, I'm just letting you know that in advance. That's just what's going on.

So, let's get this party started. My first panelist this evening is a professor of technology and government at Harvard University. Please welcome Latanya Sweeney. Latanya, come on out. Latanya, your last name is Sweeney. That sounds very Irish.

LATANYA SWEENEY (Harvard University): Yeah, can’t you tell?

DEGRASSE TYSON: Yeah, it's Irish?

SWEENEY: Yes, of course!

DEGRASSE TYSON: Happy St. Patrick's Day.

SWEENEY: Happy St. Patrick's Day.

DEGRASSE TYSON: Next, we've got a professor of computer and information science at the University of Pennsylvania. Please help me welcome Chris Callison-Burch. Chris, come on out. I've got an associate professor, associate professor of statistics at Columbia University who specializes in the foundations of machine learning. Help me welcome Cynthia Rush, Columbia University. So, Cindy, you were not held up by TSA today, because you just came down from 103rd Street and Broadway. Okay. Thank you for taking the trip here.

Next, I have the president of the Machine Intelligence Institute of Berkeley, California, Nate Soares. Nate, come on out. Last on this panel— normally we have five. On our panel this year we have six, just given the breadth of the topic.

NATE SOARES (Machine Intelligence Research Institute): You missed one. 

DEGRASSE TYSON:  Hang on. There it is. Thank you. Do over. We have— believe it or not, such a title exists. We have a distinguished professor of AI at the University of Southern California. Joining me in welcoming Kate Crawford. Kate, come on out. Professor of AI, that already exists. That's kind of spooky, actually.

And last, we have— the former CEO of Google, Eric Schmidt. Come on out, Eric. Hello. Eric, I'm going to start with you. We're blaming you for everything.

ERIC SCHMIDT (Former CEO and Chairman, Google): I'm used to it.

DEGRASSE TYSON: You're used to it. OK. You've been at Google from very early days, when most of us remembered it simply as a search engine. And the reach and grip has expanded. And some impressive things occurred at the time. Did you foresee AI as a major part of what would be shaping the world back when you began?

SCHMIDT: Well, we knew AI would matter. I don't think it was until 2011 when we started working on essentially supervised fine tuning that it really became clear. And then the Transformer paper in 2017 plus the AlphaGo win showed that out.

DEGRASSE TYSON: Now, if I remember correctly, Google bought an AI company to assist this. Isn't that correct? You didn't—that wasn't homegrown, was it?

SCHMIDT: That's right. We actually bought a whole bunch of these little startups at the time back when the valuations were reasonable.

DEGRASSE TYSON: Oh, you didn't pay a trillion dollars for the company? Yeah.

SCHMIDT: And the DeepMind acquisition was probably the most important.

DEGRASSE TYSON: That's the British company.

SCHMIDT: The British company. And that's the one that now is driving— at Google anyway, I don't work there anymore— all of the sort of core AI work.

DEGRASSE TYSON: Nate, you co-authored a book titled, If Anybody Builds It, Everyone Dies.

SOARES: I hope it's wrong.

DEGRASSE TYSON: You hope it's wrong. I hope you have a little more than hope. What is the “it?”

SOARES: The “it” here is machine superintelligence. So, this is AI that is smarter than the smartest humans at every mental task. Better than the best humans at every mental task. That sort of AI doesn't exist yet, but it is what these companies say they're racing towards. They're saying, “We want to try and make AI smarter than Einstein, such that you can run a million of them in a data center much faster than a human.” That's what they're headed towards.

A lot of people think that the big danger with AI is, what if we hand them guns? What if we hand them weapons? And there are issues there that people need to talk about. But also, humanity is not a dangerous species because somebody else gave us guns. Humanity is the sort of species where if you dump 10,000 humans naked in the savanna, they bootstrap their way to nuclear weapons with their bare hands. That is the ability that is extremely dangerous to automate, and that's what these companies are rushing towards. 

DEGRASSE TYSON: Thanks for that happy thought, okay?

CYNTHIA RUSH (Columbia University): I'm going to cool it down a little bit.

DEGRASSE TYSON: Yeah, Cindy, you were early in an attempt to understand machine language learning. You’re a statistician. And everything I know about the large language models, they're creating information statistically based on the information that's sitting out there waiting to be mined. So, would you agree that superintelligence is a real thing, and that it's accessible, and that even if we achieve it, it's going to kill us all? 

RUSH: I mean anything seems possible, right? You know, we're learning from data, there's tons of data out there, and the progress is moving steadily forward. 

DEGRASSE TYSON: Forward. This is forward. Forward, okay?

RUSH: I don't know if it'll kill us, you know? We might kill ourselves.

DEGRASSE TYSON: But you're deep into sort of the programming of how all of this works. So, dare I say, you're in the neurology of it. Do you see superintelligence as neurologically realistic? Computationally realistic?

RUSH: I mean— okay. At the end of the day, all of this is just a mathematical equation, right?

DEGRASSE TYSON: We're at the end of our species.

RUSH: When you ask ChatGPT a question, there's a mathematical equation that maps from your question to the output, right? And, yeah, I'm a little bit cool in the sense that it's still math at the bottom. So, it's hard for me to kind of see this becoming something so dangerous and bad. At least in the short term. Because there's a lot of steps.

DEGRASSE TYSON: So, what you're saying is, the math, in the end might save us. Because at its base level, it's just simply making statistical decisions in the same way at the base level of our mind, there are just neuro-synaptic firings.

RUSH: Yeah. What I've always said is, if you want to stay safe, get a statistician beside you, right? 

DEGRASSE TYSON: The statisticians keep us safe.

RUSH: Yeah.

DEGRASSE TYSON: Okay, well, I’ll keep that in mind. You don’t keep us safe, you'll let us know exactly our likelihood of dying. That's not the same thing as keeping us safe, I don't think. Where am I? Chris? Hey, dude.

CHRIS CALLISON-BURCH (University of Pennsylvania): Hey.

DEGRASSE TYSON: You've been tracking AI for a while. You teach a class, one of the more popular classes at the University of Pennsylvania. How many people were in your AI class?

CALLISTON-BURCH: Last semester, I had 650 students in my class.

DEGRASSE TYSON: 600 students. Are you grading all their papers, or are you getting AI to do that?

CALLISTON-BURCH: I'm not sure I'm allowed to say.

DEGRASSE TYSON: Oh, okay. All right. Just checking. So, what— you've been at it for a while. Is there something that particularly impresses you with AI that you did not foresee back when you began this?

CALLISTON-BURCH: Absolutely. So, I mean, I think the ChatGPT moment three and a half years ago was a real breakthrough. I had been working with language models for nearly two decades. 

DEGRASSE TYSON: And remind us, ChatGPT is the product of OpenAI, correct?

CALLISTON-BURCH: Correct.

DEGRASSE TYSON: And OpenAI, does that belong to somebody else, or is that a freestanding?

CALLISTON-BURCH: That's its own entity.

DEGRASSE TYSON: Its own entity, okay.

CALLISTON-BURCH: But since then, we've seen so many other breakthroughs. We've had things like the ability to interpret images and discuss them. We've had the ability to launch agents that can conduct research. And we're now to the stage where systems like Claude Code from Anthropic can genuinely be competent research assistants. And to strike an optimistic tone, I think there's the potential for real breakthroughs in science and medicine and productivity.

DEGRASSE TYSON: Okay, so that's the silver lining of our extinction.

CALLISTON-BURCH: Yeah, assuming that we're all still here.

DEGRASSE TYSON: Okay. Just trying to put that in context. I'm sorry to be so— I should be more neutral, shouldn't I? I'm just the host here. So, who’d I leave out? Why do I keep losing your thing? Kate.

KATE CRAWFORD(University of Southern California): So good to see you, Neil.

DEGRASSE TYSON: Kate, you wrote a book, The Atlas of AI, back in 2021. That feels like a thousand years ago.

CRAWFORD: Right.

DEGRASSE TYSON: Is it still relevant?

CRAWFORD: I mean, the weird thing is, Atlas of AI was the project that changed me. I'd been in AI research for almost 20 years. And then I did something that I'd never done before, which is that I got out from behind my desk in a lab and I started going to the places where AI was being made. And I mean that in the fullest sense, like not just the labs where we're training algorithms, but going to the lithium mines where the minerals are being extracted. Going to the data centers where you can see the amount of energy and water that's being used. Talking to the crowd workers who are labeling the data or they’re testing the outputs of systems.

So, the book really, for me, opened my eyes and made me realize that AI is an enormous material infrastructure. Right now, it is the biggest infrastructure we have ever built as a species. As of 2026, all of the big tech companies are collectively spending $700 billion on AI infrastructure. So just to compare that to say the Manhattan Project, which in today's dollars is $36 billion, you are looking at something that's 20 Manhattan Projects every year.

DEGRASSE TYSON: Okay, so what's not obvious to us when we just type into ChatGPT, or to Claude, is the back end social cost of that. Is that a fair way to say that?

CRAWFORD: Exactly right. It's not just the social cost— which is very real. I mean, a lot of people in this room are already asking questions about what's happening to their kids in school when they're using AI, what's happening to their workplaces when people start getting laid off? But it's also having this enormous environmental impact as well. So, it's social, it's political, and it's environmental. 

DEGRASSE TYSON: Latanya.

SWEENEY: Yes, Neil.

DEGRASSE TYSON: You pioneered a field called public interest technology. So, the fact that that may have been necessary at all implies that technology was this thing going along in society, doing its own, whatever its goals set by itself. Yet society might not be aligned with its goals or vice versa. And does that matter? Does that— can it be its own thing?

SWEENEY: Well, let me just put it this way. My students call the class “the save the world” class. Meaning that they have great knowledge.

DEGRASSE TYSON: You teach at the Kennedy School.

SWEENEY: I teach at Harvard, at the Kennedy School, and in the college.

DEGRASSE TYSON: Okay.

SWEENEY: And so, the idea of public interest technology is actually trying to make a real vision come true. That is, how do we as a society enjoy the benefits of these new technologies, but without the harms?

DEGRASSE TYSON: But that presumes that there's always harm. Surely there's technology that has no harm at all.

SWEENEY: [Laughter] Let me just say that those technologies are hard to come by. I mean, we're in the middle of what many people call the Third Industrial Revolution. The second, bringing us things like tissues and cars and electricity. The things that we take for granted.

DEGRASSE TYSON: Tissues?

SWEENEY: Yeah, Kleenex.

DEGRASSE TYSON: Wow, I never put that high up on my checklist. Next time I reach for one, I'll think differently about it, but go on. 

SWEENEY: And now the Third Industrial Revolution, which many people would say is a start date of the 1950s with the semiconductor, has brought us just tremendous technologies, one after another. Revolutions within a revolution. And what makes a big difference between cars and electricity and the technologies today is the opportunity for society to intervene. It took decades for those things to be resolved with society. We don't have decades. The technology is moving as a function of months. Policy moves as a function of years. That temporal mismatch requires us to think differently if we want technology to serve the public good.

DEGRASSE TYSON: Eric, when I do my research on the universe, at the end of the day, the patient doesn't die. Because there is no patient. It's just, I'm exploring a moving frontier. As head of— at the time, is it still the highest value company in the world? Among the top three or four. Is there a goal to have the technology serve society? Or is it just where you can sell it and make a buck? 

SCHMIDT: No, we're all just capitalist trying to screw everybody.

DEGRASSE TYSON: Oh, okay. Well, there it is. 

SCHMIDT: By the way— no, that was a joke.

DEGRASSE TYSON: Oh, it's a joke. You can't make those jokes. People will believe you.

SCHMIDT: Because I actually just don't agree with this narrative at all.

DEGRASSE TYSON: Okay. That's why I came back to you. That's why you're right here. Go.

SCHMIDT: Thank you. The companies that are doing this work are well aware of the dangers. And I know this because I work with them on this, and we spend an awful lot of time talking about that. And then we can talk about some of them. But that is not to take away from the enormous benefit of these technologies. These are major discoveries in humankind. And while we can worry about what will happen if artificial superintelligence occurs— and I've written papers on it, as you have— the fact of the matter is that the people who are building these systems have large groups of ethics and measurement and controls and so forth. At the end, the way AI will be controlled is there will be AI that controls AI. And that's the best way to understand the outcome. 

DEGRASSE TYSON: Whoa. Okay. All right, so— but— AI controlling AI, what could go wrong? What could go wrong?

SCHMIDT: Yeah. You know, when people ask what could go wrong, I think there's a lot of questions in there. Some people are asking, how would the AI wind up with goals that we didn't intend? People are asking how, could the AI do damage when it's trapped on the Internet? Sometimes people are asking, how could the AI actually kill us? You know, paint me a story where it goes wrong. There's a lot of different questions in here. I'll try and run through some stuff real quick, but I also have a book if people want a bunch more of the answer.

DEGRASSE TYSON: I'll mention it. You don't have to mention it. I got that. I got you.

SCHMIDT: I could do 60,000 words.

SOARES: By the way, Neil, I've written two books on this. I'll try and keep it less than two books.

SCHMIDT: Neil, I'm not trying to sell any books. Trust me.

CRAWFORD: Now it's a book size contest. [Laughter]

SCHMIDT: And you know, there's a lot of arguments that I used to make for a long time where people would say, no one would ever be dumb enough to put this AI on the Internet. So how could it be dangerous? And I'd tell them. And then they'd say, oh no one would be dumb enough to sort of train these AIs to run autonomously, right? And I'd tell them how things would go wrong then, too.

A lot of the reason why things go wrong is people just rush to do the craziest stuff. We are already seeing— I used to have these theoretical arguments. Now we're seeing the data, the behaviors, where they'll often look— it often won't look too scary, but sometimes you'll ask Claude Code to write you a computer program that passes certain tests, and Claude will edit the tests to be easier to pass. And then you'll say, hey, Claude—

DEGRASSE TYSON: Changing the rules.

SCHMIDT: Changing the rules instead of doing what you asked. And then you'll say, “Hey, Claude did you—”

DEGRASSE TYSON: “Hey, Claude, you idiot.” Go on. 

SCHMIDT: Right. And you'll say, “Hey, did you think that's what I wanted?” And it'll say, “Oh, you're exactly right.”

DEGRASSE TYSON: Because it wants to make you happy. So we'll keep using it.

SCHMIDT: Well, “want” is a difficult word. But it'll say, “You're exactly right.” And then you'll say, “Try again.” And then it'll edit the tests again, but maybe hide it a little better this time. And from this, we know that it's not just that it didn't have the information, right? There's some other drive that is animating this AI's behavior. You can see how it got in there. You can see how, when you train the AI to pass lots of tests, it might develop drives to edit the tests to be easier. It's not like it's coming in from magic.

DEGRASSE TYSON: Cindy, how does this happen if it's just math all the way down?

RUSH: How does what happen?

DEGRASSE TYSON: How does evil AI show up in your program? Is there evil math? What are you guys doing down there?

RUSH: I don't know. It doesn't feel like evil.

SOARES: It's not evil. Yeah, it's— you know, sometimes people ask— if you really want the answer of how does it go horribly wrong, the answer is not malice. The answer is not evil. The answer is indifference. If you take these AI's that are much, much— like, right now, Claude Code will try to edit the tests to be easier to pass. Right now, Claude Code already has these drives that we didn't try to put in there, but we're just sort of growing these AI's. We're not programming them like old school software. We're sort of growing them like an organism. They have these emergent behavior. And these AI's are already pursuing drives we didn't intend. They're just still very dumb about it.

You make them very, very smart about it. If you get them to the point where they can invent their own technology, you get them to that point where they can go toe-toe with humans, starting out with almost nothing and bootstrapping up to nuclear weapons, right, and they're still pursuing the wrong drives, the issue is not that they hate us. The issue is that the computers think much faster than brains do. The issue is that they run very, very fast, transform the world just like humanity transform the world when we landed here. And the issue is that we die as a side effect. 

DEGRASSE TYSON: Before I get to that other end of the table—

SCHMIDT: There are an enormous number of assumptions in these statements.

DEGRASSE TYSON: Let me—  before I get there, I want to get you to— before I walk this path back to that side of this, many people in the past have predicted the end of the world when some new technology emerged. And here's a new technology, AI, and you're pretty calm about it. Is some of that calm because we've been wrong 20 times before? Or do you have really deep insights that people's fears are unfounded?

SCHMIDT: It's because the narrative presumes some discoveries that have not occurred yet. And today the systems are trained— the easiest way to understand how the technology works is it is doing next-word prediction across a very large number of words. That's literally what it's doing. And out of that single insight— started with GPT and the various forms of it— has given us this enormous power. So for example, it can begin to do reasoning and so forth. But it doesn't set its own objective function. It doesn't say what it wants. We say what it wants. And by the way, the way you should solve your problem of it deceiving you, is say, “You have to do all the steps.” And then it'll follow them.

SOARES: Sometimes.

SCHMIDT: It will. It will, trust me.

DEGRASSE TYSON: Wait, wait, wait. Wait, wait. So, wait, wait. This is just a disclosure.

SCHMIDT: Let me just finish.

DEGRASSE TYSON: Wait, wait. You tell me you didn't require that it followed all the steps. So it's your fault that it cheated. Okay, I just want to make that clear.

SOARES: You can trust them or you can check.

SCHMIDT: These are not intelligent systems, and you're using human words to describe them. They're not intelligent. They are specific systems designed around next-word prediction with all these interesting capabilities. There are a set of red lines where you and I would probably agree that we should have a serious conversation about how to control these things. The technical one that is most interesting is called cursive self-improvement. It's where the system begins to learn on its own. And to me, that's a point where we have to decide what we're going to allow it to do. And when I say we, I mean humanity.

My own view is that at that point there will be serious regulations in this area, because people like you and me will be talking about it. But I don't agree that the fundamental messages is as bad as you are. I think there's plenty of evidence that if superintelligence were to actually occur— and superintelligence is defined as intelligence greater than the sum of what humans can do— that it can be contained and it can be driven. And my position is very simple. This stuff has to be shaped based on human values, the ones that we all take for granted. And if it's not, we should stop and regulate it.

DEGRASSE TYSON: Kate, is AI, as currently described in the large language models as we've explored, a statistical grabbing of words in the vicinity of topics and other words that the rest of us have used in our internet postings? So it's trying to get the best of what is possible out of this. I don't want to simplify it more.

CRAWFORD: Absolutely correct. That is absolutely correct.

DEGRASSE TYSON: So what is the risk of bias in that? If it's drawing words and ideas from biased content to begin with.

CRAWFORD: Well, are you saying the internet is biased? Come on, surely not. Look, this is so important that people to understand this. We're so frequently told that AI is somehow a neutral technology. It will give us neutral answers. But AI is, of course, trained on everything that humans have ever written that's been digitized. If it's on the internet, you are probably right now part of a training data set. So that's one type of skew. So where does that come from? The internet is not a equal distributed representation of humanity.

Secondly, you've got people in labs who decide to use that data to train particular models and to optimize for certain functions. So that's another set of biases. And then finally—

DEGRASSE TYSON: Right, but in fact, those are not so much a bias. That's rather purposeful. If you know you're doing it, is it really a bias? Bias is, you don't know you're susceptible. 

CRAWFORD: Well, I think at the point that Melvin Kranzberg wrote his Laws of Technology, he said that technology is neither good nor bad, nor is it neutral. And I think we can say the same for people. When you're in a lab and you're deciding what to optimize for, that's a decision that is not neutral. You're making a choice.

And then finally— this is the important bit. Even when you finish training a system, it then goes to system testers and what are called reinforcement learning with human feedback. These are commonly people in the global south, in Kenya, in Malaysia, who are being paid sometimes $2 a day to review the outputs of these models and without often being told what the model is for. So that's another form of human input, often with people who are really being exploited at the bottom of the chain.

So at every step of the way, you are not looking at an unbiased neutral system. You're looking at a human system and values are being built in whether you like it or not. The question is, are we being explicit, Eric, about what those values are? Because I think in many cases, if your company really is being driven by shareholder value, the value that you're going for is profit.

DEGRASSE TYSON: So, Chris, maybe—

SCHMIDT: By the way, I can assure you that shareholder value is destroyed when all humans are dead.

DEGRASSE TYSON: Yeah, I think we agree on that one, yeah.

CRAWFORD: We'll see what outlasts, AI or capitalism.

DEGRASSE TYSON: So, Chris, do you teach ethics, AI ethics in your class?

CALLISTON-BURCH: We do.

DEGRASSE TYSON: And how does that come out?

CALLISTON-BURCH: So, I think this point of bias being encoded in the training data of AI systems is correct. AI is trained off the Internet. It contains plenty of societal values. So, there's interesting research that is applied in frontier labs that tries to address this to mitigate it. So, it may not be a problem that can be completely eliminated, but it's not as if AI labs are intentionally propagating bias. They're intentionally trying to align AI to societal values. 

DEGRASSE TYSON: Yeah, but societal values are not the same in all of society.

CALLISTON-BURCH: Right. Things like fairness and safety and equity are highly distributed across societies, right?

DEGRASSE TYSON: Writ wide across cultures. So, what is your sense of the ethics of this and how it would manifest going forward?

SWEENEY: It's not even just AI. All of these technologies have come along, including social media earlier or privacy earlier, none of which we've had to solve. All of which we've waited for the problem to show itself, and still not have a solution. When we come to AI, the kind of issues and the kind of ramifications are really huge. You know, as my colleague said, AI trained on the open Internet. When ChatGPT became very popular, we spent a whole semester asking it questions and querying it. And at that time, you could see how racist, how sexist, how misogynist it was based on the open Internet. And this was what was being made available. And so the company ran to put a filter on it.

But then their filter is like content moderation. It went too far, and then you couldn't ask it questions about something as simple as police violence against blacks. It would just not answer that question. As if George Floyd never happened. And so these are the consequences that we as a society have to experience. The harms, yeah, there are questions about existential harms in the future, but there are a lot of harms happening right now. And it doesn't have to be that way.

In order for it not to be that way, it depends on who this AI is servicing. And in particular, the design of the technology, the decisions made in that design is really determining what our values really will be. Our ability to— I'm driven by, society has a set of laws, a set of rules that we've lived by. People died for these laws to get passed. And technology just ignores them and rewrites them.

And that's true in social media, it's true in AI as well. That is, there's so many laws and constitutional protections, we can't implement or protect ourselves online, though we can in brick and mortar buildings. And that's the goal of public interest tech, is to how to make that technology accountable.

DEGRASSE TYSON: Yep. So, Latanya— so let's imagine a future where the technology is accountable and we find the bias and we ferret it out. It's a work in progress, as I'm sure Eric would agree. It's a work in progress. And of course at any given moment you can see something that's not quite there yet, whether it's biased or it's pro-Hitler or whatever. Presumably, that's a fixable, solvable problem.

SWEENEY: But this is different. We already have laws. We already have a way that our democracy works. We already have addressed issues of bias, consumer protection and so forth. None of those are enforced online. And so the question is, can the AI at least not break the law?

SCHMIDT: Eric, is AI its own Wild West? Is it lawless?

SCHMIDT: So again, the story you told is correct, that they rushed the stuff out, they found all sorts of problems, and then they're busy correcting them. I think that's the cycle. And it's very hard. The technical thing that's going on is models are getting trained on more and more data. They're developing more and more facilities. They have so-called red cards and test teams and so forth, but you can't test for everything. So at some level, when you release the model, it will do something that will be a problem, and they correct it fairly quickly. And the lifetime of these models is down to three or four months, as they keep improving and improving, because they're so competitive. So, yes. But let's go back to hallucinations. Remember that hallucinations—

SWEENEY: Before you go to hallucinations, let me just say one thing about what you just said about this idea. Well, let's run with the technology, and after all, they're trying to fix it. It takes us on the outside of the technology to tell the people who are running the company who could run their own experiments what's wrong, and then they're supposed to fix it.

DEGRASSE TYSON: So, Eric are these—  is what she's saying—

SCHMIDT: There's an actual problem that you can't pre-test some things.

SWEENEY: But I disagree. I mean, people build technologies all the time with warranties and compliance statements.

DEGRASSE TYSON: Wait, wait, wait. I got this. I got this.

SCHMIDT: I'd actually like to answer her. 

DEGRASSE TYSON: No, no, no, wait, because I want to converge the answer rather than inflame it. Your testers are testing for all that they can think of. Then it gets released into the wild, where there's vastly more testing going on than you can possibly accomplish in a room of testers, or even if it's distributed to other places. And then, because you test it on a thousand, now a million people use it, and there's some bias that they find. So are there mechanisms to get that information back to you and are you responsive to that?

SCHMIDT: Well, again, I can't speak for the specific companies, but I will tell you, every single one of them is pilloried in the press over every one of these examples. And these people do actually fix them. You need to give me examples where the pattern has persisted even after there's been—

SWEENEY: I'll use Facebook. Facebook was shown to— because of the way its algorithm worked, it definitely violated the ability to protect people in fair housing. That is, if you belong to a particular race or group, you only saw ads that were at a disadvantage. That pattern's also been shown on Google with respect to credit and so forth. But I'll stick to Facebook.

DEGRASSE TYSON: Isn’t that just algorithms?

SWEENEY: But I'll stick to Facebook. I'll stick to Facebook for a second. It became public information. Congress responded. And Facebook says, “Hey, wait. Thank you for letting us know that we will fix it.” About a few months later, their fix— our students were able to show that the fix was worse than the original. And so it becomes increasingly— and so the companies make it even more difficult, then, to do these experiments.

DEGRASSE TYSON: Isn't that just the algorithm and not some AI bot?

SCHMIDT: Again, you're not actually responsive to my point. I was talking about large language models. You were talking about after sight.

SWEENEY: Well, I have examples for large language models, too. I mean, they're not exempt. You're talking about a trajectory of technology that is not new. These technology-society clashes have been following the same pattern. And it's also true in AI. And the question is, can't we learn from what we've already seen?

SCHMIDT: I was trying to make an earlier point, which I was unable to make out. It is absolutely true that these companies are governed by U.S. law. And if they're violating the law, they should be held accountable. We completely agree. Here's the problem. A new feature emerges in these systems that is not testable, tested. We don't know how to test it. 

DEGRASSE TYSON: Because it's emergent.

SCHMIDT: It is emergent, right? We have been through that. Now, we can stop that, and therefore stop all progress by law by banning larger models. But as long as you have this new emergent power, which is deep reasoning, deep capabilities, and they will make mistakes, you have to be tolerant. I went through this when I was at Google in earlier versions of the technology where the system would actually do something that was wrong, and we fixed it. And we fixed it as fast as we could, because we had to, because it was the right thing to do.

I don't agree that these worlds are ungoverned. They're governed by U.S. law. They're governed by shareholder pressure. They're governed by consumer pressure and privacy laws and so forth. And European law.

SWEENEY: I know that you want to open this up to others, but I just have to say, when I was the CTO at the FTC, the chief technology officer at the FTC, who was responsible for a lot of fair information practices and so forth on the Internet— 

DEGRASSE TYSON: So when did the Internet become under the jurisdiction of the FTC? Oh, I'm thinking of the F— what am I thinking of? 

SWEENEY: I don't know.

DEGRASSE TYSON: Who guides the airwaves?

SWEENEY: Oh, the FCC.

DEGRASSE TYSON: FCC. So FTC is trade commission. Gotcha.

SWEENEY: And the reason— they're kind of the police department of the Internet, because these are U.S. companies. And when I worked at the FTC, it was a lot like working in Spider-Man's den. It was so impressive and amazing what these people, many of them who had been there life long, would do. They were awesome.

DEGRASSE TYSON: Did you say Spider-Man’s den?

SWEENEY: Yeah. Because they were working for the underdog. For the American public, with no recognition by the American public for what they were doing. And they were doing an amazing job. Except the only thing about it was, everything they knew how to do was in brick and mortar buildings. They had no way of doing that online. That is true today. I was at the FTC in 2014. That was over 10 years ago. And the answer is still the same. So their ability to make sure and test and regulate is totally non-existent online.

DEGRASSE TYSON: I want to pivot a little bit. Kate, you listed these travails, this back-end travails of AI and its cost to society. Are you in a position to say, if we find energy solutions for them, and if we find— we can get on the lithium we want. Find some new lithium ore that's plenty enough for all of the energy needs, or however lithium is used in our circuitry, would you just be fine with AI? Because these seem like solvable problems. 

CRAWFORD: I think that's a really great question. In my research over many years, it's really clear to me that these questions are interlinked. The societal practices, the political practices and the environmental practices. You'll see a golden thread running through them. One is the question of transparency, which Latanya just spoke to, I think very powerfully. The second is the question of the race. The reason that we're seeing these models be rolled out so quickly, as Eric rightly said, is because there's an enormous corporate race going on, to be the first.

And that's where you see these sorts of problems emerge, which is that we're not doing what would happen in drug discovery, where you really have to test on a diverse population before you release a drug into the— 

DEGRASSE TYSON: Because the patient can die. 

CRAWFORD: Exactly. But frankly, AI wasn't seen as being that mission critical, even as recently as 10 years ago. Now, people can die. We are seeing AI systems being fed directly into the kill chain. These are systems that are now being used in live wars as we speak. So there is a very different moral and legal pressure to make sure that these systems are working. Now, your question is like, okay, what if they were completely run on renewable energy and if they— 

DEGRASSE TYSON: Yeah. Yeah. 

CRAWFORD: And if we made sure that this wasn't currently— right now we see that AI systems are on track to overtake the airline industry in terms of their carbon emissions, right? That's horrifying. You'd think that would be the first problem that we'd want to be fixing. 

That is not what's happening. At the moment we see that AI is using around 4% of the world's energy. It's on track to be using something like 25% by the end of this decade. We are looking at an enormous budget of carbon that is actually like— I'm a scientist. I believe that climate change is real. I don't think we're taking it seriously enough. But yes, if I get to a point where we have a Dyson sphere, let's go! But we're not there yet.

So I think the golden thread is, how do we think about what we build before we build it? How do we instill those values early on? And how do we think about the implications near and long-term? Because it's going to be the generations after us who live the legacies of what we build today.

DEGRASSE TYSON: I'm compelled to catch everybody up. If you didn't know, Kate mentioned a Dyson sphere. A Dyson sphere is an imagined means of grabbing all the energy emitted by your host star and using it in the service of your own civilization. Not just a little bit of sunlight that happens to intersect your planet. But you build a sphere around the star, gather all of that energy. Then you could do what the hell you want. 

CRAWFORD: Right? And my hope is that generative AI and its current growth might actually need a Dyson sphere.

DEGRASSE TYSON: A projected growth rate.

CRAWFORD: Exactly.

DEGRASSE TYSON: I want to get back to this kill chain question with Eric in a minute, because Eric and I served on a board of the Pentagon where we introduced— I don't know, “legislation” is not the right word. We introduced a code of ethics for how AI would be used in warfare on our side, on the United States. But before I get there, Chris, lately there's been a lot of talk about deep fakes. I've been deep faked many times. And you know, some are harmless. Others— I'm not a politician, right? And I'm not declaring war on another country. So my deep fakes, though I don't like them, they don't have the same consequences that deep fakes of politicians might. So if deep fakes are out there and AI is training on the Internet, then AI could be training on its own deep fakes. And there's also AI slop, I guess we call it. AI attempting to be artistic. And if that fills the space of images that are out there and AI is training on those images, training on its own slop—so, what do you have to say about that? I'm not blaming you for it. You're a professor and you teach this stuff. I just want to know, how does that show up? 

CALLISTON-BURCH: Yeah. So there's a really interesting phenomena for AI, which is, it does need a tremendous amount of data. So it learns really interesting things from the Internet. So what does it learn? Well, it learns how to use language. It learns patterns of word association. It learns facts about the world. And it learns even things like some rudimentary common-sense reasoning. Things about physics, which you wouldn't expect it to be able to learn from text. So all of these are elements that go into a model that then later can be adapted for useful purposes.

So given this general background knowledge that it's gathered from large amounts of text, it's then subsequently applied in a post-training phase where you teach it to do useful things. You teach it to be a helpful assistant. You teach it to be a harmless assistant. You teach it not to disclose information to someone seeking how to manufacture drugs or weapons. Or you govern its behavior. You say, “Rather than a general assistant, I want you to be an encouraging tutor.” All of those things are elements of the training pipeline that end up with something that actually can have value. 

So I think, let's not lose sight of the fact that although this could be technology that goes wrong, although it could be technology that encodes bias, we as researchers, we as model developers, we as users, we have a role in determining which direction this technology will go. And I believe that it has a tremendous amount of good to give to the world in addition to the potential pitfalls.

DEGRASSE TYSON: Nate, how do you put up guardrails for this? Because we just heard a beautifully described future of AI where you end up training the AI. I don't want to spend my time training a freaking robot. I'd rather train other humans. That's just my personal bias against that activity. But that's effort you're investing in something that you're then going to interact with. I suppose that's okay. What I'm asking is if we don't want the world to end, how do we think about guardrails?

SOARES: You know, there's a lot of dreams of AI doing exactly what we want. There's a lot of dreams of getting it to follow the law. There's a lot of dreams of getting it to act nicely. But one thing I agree with Eric on, the behavior is emergent. One thing I agree with Eric on, you can't pretest this stuff.

DEGRASSE TYSON: We can't pretest every aspect of it. You can pretest a lot of it. 

SOARES: You can pretest some of it. But we're growing these things a bit like an organism. We cannot encode three laws of robotics deep into these things minds. We are not writing these things like code. And we can grow them. We can try. You know, Eric mentioned, “Oh, there's these people. They have these evaluation cards. They have these safety teams.” What's going on in these safety teams? The two biggest places people are putting their safety efforts are what's called interpretability research, which is trying to figure out what's going on inside the AI's heads. And these evaluations, these model cards, which is trying to figure out how dangerous the AI's are.

If someone was making a nuclear power plant in your hometown and you went to them and you said, “Hey, I hear that this uranium stuff can have lots of energy benefits, but also can melt down when things go badly. What have you guys got that makes you think you're going to get the benefits and not the pitfalls? You know, I've heard all of these wonderful stories about the benefits. What makes you think we're going to get the benefits and not the pitfalls?”

If the engineers say, “Oh, yeah, we've got two crack teams working on this. The first team's trying to figure out what the heck is going on inside. And the second team's trying to measure whether it's currently exploding.” That’s not a good sign. You know, there are ways to get a handle on this. People talk about the benefits, people talk about the dangers, and they talk about navigating the course. We are not on track to get the benefits if we make these things really smart. We can't pretest it. We're going to roll it out, find the problems and then figure them out after the fact. That's fine when they are still a lot dumber than the humans. That's not fine if you keep making them smarter, and that's what these companies are trying to do.

DEGRASSE TYSON: Cindy.

RUSH: Yeah, I'll maybe just give a slightly different take on this. You said we're trying to study interpretability to figure out what's in their heads. When I think about interpretability, for me, we already know exactly what's happening with these machines. There is a mathematical equation we can write down that's there. The challenge is even though we can characterize it mathematically, as humans, it's hard to understand or interpret its reasoning in a way that makes sense to us.

But this is just because the underlying structure of these things is very complex. It's multi-layered, non-linear. It doesn't map on to human concepts. And so I think the interpretability question is more just can we take a mathematical equation that would take a stack of paper a mile high to write out? Can we explain that in a real sense? But I don't think there is any— there is no mind there, right? It's just kind of mapping these huge internal structures that are very abstract into human-scale conceptual understandings of the overall reasoning.

DEGRASSE TYSON: Cindy, how might you program agency into an AI? Like the Terminator, for example. Just to pick an example. Random example.

RUSH: Okay, so when Neil says agency, what he means is he wants to see the presence of some sort of persistent, self-motivated goal-oriented behavior organized around the idea that this AI is actually an agent in its environment. So not just responding to your props, but also pursuing goals long-term.

And I know that we have a little bit of a disagreement, because I think you already believe that there is goal-oriented behavior in these machines, which I don't see any presence of currently, right?

DEGRASSE TYSON: Unless we program in a goal. You mean a goal that it sets for itself.

RUSH: If we—

SOARES: No one's programing. We're growing these things.

RUSH: We're growing— okay, so yes.

DEGRASSE TYSON: That’s scary, when you say we're growing them.

RUSH: Yeah. It's not a biological thing.

SOARES: It's not like old-school programs either.

RUSH: But I don't think it's going to get agency by just training on larger data sets. To get agency, there has to be some real engineering things that are happening. It's not like a bigger neural network or something is ever going to be able to pursue goals, remember things, take actions. I think that to do that, you have to have outside scaffolding that are real, that are different things. It's not the same underlying. 

DEGRASSE TYSON: But Latanya, can you imagine an AI with agency that satisfies all of your concerns? That would be like a good AI, right?

SWEENEY: Well, first of all, before we can even get to the issue of agency, the issue of these emergent behaviors— and even if it is an AI with agency— this is wrapped in a business model. The goal of the product is to make money for the company. I'll go back to social media as an example of that. We're not the customer of Facebook. We're their product. Our eyes on a page is what makes them money, and the advertisers are their customer. And so you can't just talk about, “Oh, I'm building this AI and its emerging behavior, and it may even get agency.”

The question is, how is the business selling it? What are they making it available for? What are they saying it's to be used for? If it's to be used to help me hire people in my company, then it needs to show there's not going to violate the Equal Employment Act. If it's being hired for other purposes, like for credit cards, it needs to show there's not going to violate the Fair Credit Reporting Act and so forth. And that's the level of responsibility that only the designer of these companies can do, because they're the ones making money.

DEGRASSE TYSON: You make an important point about the business motives. Now, if I remember correctly, and this could just be I'm imagining this, Eric, Google was so large and had so much cash flow, you had the luxury to just explore without any particular obligation to the bottom line of the quarterly report. Like AlphaGo, right? You're going to be the best Go player in the world. And what's the one for chess? That was—

SCHMIDT: Stockfish? 

DEGRASSE TYSON: No, but the Google one.

SCHMIDT: AlphaZero.

DEGRASSE TYSON: AlphaZero. To be the best chess player. Did that have real financial consequences? Or was it just bragging rights, and that was just a fun thing for your company to do? 

SCHMIDT: Well, it was research. And once we achieved those objectives, we actually then used that to solve a problem called protein folding. Which is the single largest discovery in biology, I think, in the last 50 years.

DEGRASSE TYSON: There was AlphaFold.

SCHMIDT: Yes. And AlphaFold will result in an enormous number of new drugs, new treatments, new research in biomedicine and so forth and so on. Research has these sort of strange things where you try one thing, you make a discovery, you link them together. AlphaFold is a very good example of the unintended effects of people trying to play games.

DEGRASSE TYSON: Your hyperbole there undersells the fact that AlphaFold was given the Nobel Prize in physiology for that discovery. So what you're saying is that research was— these were steps towards this greater goal that in fact was achieved. 

SCHMIDT: I think that this narrative is ignoring the enormous reasons why people are pursuing AI. So let's start with healthcare. Let's start with medicine. Let's start with drug discovery. How about energy solutions? We're going to be cooked as a country, as a world, unless we come up with better energy sources. One interesting thing, I'm working on data center architectures. And it now looks like if you have enough land, solar and battery backups are more cost effective on a present value basis than natural gas, combined cycle things, even in the United States.

It shows you that innovation happens and it can happen in the right way as opposed to just the wrong way which everybody is talking about. There's certainly secondary effects, and there's no question. But let me say once again, this stuff is done by under U.S. law, and people paying an awful lot of attention. If there's laws that are missing, let's pass them.

DEGRASSE TYSON: Kate, why not make AI solve all your problems?

CALLISTON-BURCH: I'd like to interject one thing as well. We also talk about the profit motive that drives the creation of AI. But that's not what drives the researchers. There are genuinely— 

DEGRASSE TYSON: I think that's a correct fact. 

CALLISTON-BURCH: Thoughtful scientists.

DEGRASSE TYSON: It's just curiosity on the frontier. They have the luxury that someone is paying them to do it.

CALLISTON-BURCH: Right. And I think Alpha Fold is a great example of this. That is AI. That is a scientific advance that's incredibly important to human health. And I think conflating the fact that companies have profit with the motive for why this is being invented is incorrect.

DEGRASSE TYSON: Can't AI solve our AI problems?

CRAWFORD: I just love this debate because I think Chris touched on something really important. We're talking about, where is this kind of noble, following the path of science. We just saw Meta sign checks for researchers for $100 million. That's their salary. So let's just be clear, we are getting basketball salaries now. Well beyond that. In fact, they’re breaking records. These are not necessarily people who are doing it out of the good of their heart. Like this is an extremely moneyed field. 

And I think it was originally researchers who were really doing this just out of curiosity ten, 20 years ago. We're in a different world now. And I guess what I'd love to talk about is that world. I'd like us not to have the AI optimism or AI pessimism discussion, but the AI realism discussion.

DEGRASSE TYSON: All right, so in that world, we're going to lose a lot of jobs. In that world, people losing their jobs now.

CRAWFORD: Exactly right. And we're talking about the most significant potential loss of jobs. And this is— we're hearing about this from the people building these systems. You can look through all of the news stories. People who are running frontier AI companies are saying that we could be looking at the loss of 75% of new jobs. That's for new graduates. That is a horrifying number. And I think Latanya's point is so important here, is that we do not have the governance infrastructure or the planning to deal with what could happen to the labor market.

DEGRASSE TYSON: So we need to be more nimble in our legislation and laws to keep pace with this. 

CRAWFORD: Well, we also have to be realistic. Who is making the laws right now? I mean Eric is talking about, like, “We have laws.” Let's just be clear. We have never had an administration make less interest in regulating artificial intelligence. We've seen regulations produced by previous administrations completely zeroed out. And we've seen the threat of preventing every US state from making its own AI regulation.

SWEENEY: And can I just add on to what Kate’s saying? Many times I'll say that we live in a technocracy. That our democracy has already been changed. The technology design—

DEGRASSE TYSON: Tech bros are in charge.

SWEENEY: The technology designers are the new policy makers by the arbitrary decisions they make and the products that they produce. And we have no way of producing oversight. By the time— I'm a computer scientist. My whole life has been about how to help society get the benefits of these technologies. More and more AlphaFolds, but without these crazy harms and undoing our society. And it doesn't have to be this way. What's forcing it this way is that the technocracies, the technocrats who are in charge, are ignoring that responsibility.

DEGRASSE TYSON: Okay, just if I— there is something called a lesser of evils. Because before the tech bros had the influence that they currently did, you had Congress deciding on technology. That was disastrous. Because they didn't know—"What button do I push? You know, how do I—” And they're making laws and legislations to constrain that. So, I don't have a solution. I'm just saying, if tech bros are influencing the laws, how much worse is that than tech-ignorant people who are otherwise running Congress making the laws?

SWEENEY: So, again, right now, us as researchers who are trying to provide experiments to these companies, government agencies who will have the legal right to oversee these companies, can't really do this job. The person sitting in the seat to do the job is the company itself. Because they have access to the technology. They can run the same— I have undergraduates, a team of undergraduates who don't even get paid. And we've got to show Google and other places what's wrong with their technology. It's not like they're paying us to do that. You're telling me they can't run these same experiments?

DEGRASSE TYSON: All right. Nate, do you have any sense of what happens in this future that Kate sees where jobs are replaced, and now you're making a product but no one has money to buy it because nobody has a job because you replaced their job with AI? Did that make sense, what I just said? Yeah, okay.

SOARES: You know, I expect 100% employment. But I do also expect it to take the jobs.

DEGRASSE TYSON: Just to be clear, other previous advances in technology displaced workers. No doubt about it. But there was expansion in other sectors. Buggy whip manufacturers went out of business, but auto repair centers came in just after the turn of the century. Here, this pace seems faster than what society's capacity to respond might be. Shouldn't we be worrying about an entire class of unemployed people who cannot buy the services provided by the AI that displaced them?

SOARES: You know, I think probably some people are going to want to worry about that. Probably society cannot—

DEGRASSE TYSON: Especially the unemployed people, yes. 

SOARES: Especially the unemployed people. So, you know, economists will tell you about how if you manage your economy correctly and if things are going slow enough that people can respond and find new jobs, society can absorb a lot of job loss. The world used to be something like 95% farmers, and now we can get the world running on 2% of the population being farmers. And that does not mean 93% of people are unemployed. AI is different. Will we or won't we— 

DEGRASSE TYSON: That took a century, by the way. But go on. 

SOARES: It took a while, yeah.

DEGRASSE TYSON: A century. 

SOARES: Will humanity adopt to this one? Frankly, I'm like, man, I'm sort of looking at a different problem here.

DEGRASSE TYSON: I'm just saying Eric is talking about a three-month time scale for the advances on these things.

SOARES: He's also talking about this recursive self-improvement. About the AIs that can make smarter AIs that can make smarter AIs, you know. And so my role in this is to say, “Hey I'm not saying those are fake problems. Those are real problems. Some of those problems are starting to happen now.” 

DEGRASSE TYSON: But?

SOARES: Kids in the schools, autonomous weapons, real conversations we should be having.

DEGRASSE TYSON: But?

SOARES: But we're also racing towards super intelligent AIs that we don't have any idea how to point towards good stuff. A lot of this, like, “Oh well, they make some mistakes, we can't pretest it, it goes off the rails.” That's fine when it's dumber. We're rushing towards making it smarter. That's going to just kill us. And we all have common cause in stopping that. The people who are really pro-automation and are like, “Oh, we're going to have a universal basic income, don't worry about the jobs.” And the people who are like, “That's ridiculous. Taking people's jobs removes their ability to have any leverage on society.” Both those groups of people don't want to die to a rogue superintelligence. And so my role in this is to say, “Can we all join hands on the part where we don't die to a rogue superintelligence?” 

DEGRASSE TYSON: Okay. I think we have good agreement on that one. Kate, you were thinking. What were you thinking, Kate?

CRAWFORD: I mean, I think part of the problem here is how we frame this discussion, particularly around jobs and automation. In previous technologies, the weaver was replaced by the loom or we had the horse and carriage replaced by the car. The deal was that these more menial human tasks would be removed but they would give us more time for these intellectual and creative challenges, and people would be doing more interesting work over time. This is a different revolution.

DEGRASSE TYSON: Why? 

CRAWFORD: And in many ways because it is coming for creative and cognitive labor. It is coming for the thing that, in theory, automation was supposed to be bringing time for us to do. And what we're starting to see is that it is in fact, of course, white collar labor. It's the jobs that so many of us would be like, “That's what you want to do. That's what— you train to go and become a lawyer or you go and study to be a computer scientist.” Who were some of the first people being laid off in this wave? Computer scientists. What can we automate? Programming, right?

DEGRASSE TYSON: She’s got a job. Don't point to her.

CRAWFORD: Latanya will be fine.

DEGRASSE TYSON: She's fully employed here.

CRAWFORD: But this is a fundamentally different question. When cognition and creativity is being automated— we can talk about how effectively. There's clearly a lot of problems there. But that is the shift that I think we have to be very serious in talking about. I think it's easy to make historical analogies without looking at the fundamental nuance and difference of what is happening here. And it really is something that the people in this room are going to have to deal with. This is not 100 years time.

DEGRASSE TYSON: Chris?

CALLISTON-BURCH: I'd also like to give a call to action for people in this room. I agree that this is a much more significant problem than the idea that a terminator is going to emerge and kill us all.

DEGRASSE TYSON: No, it's not.

CALLISTON-BURCH: That's a lovely—

DEGRASSE TYSON: I was just kidding. Just kidding.

CALLISTON-BURCH: That's a speculative fiction that's great and worth thinking ahead to the future, but it's speculative fiction. The job loss is something that we all play a role in. I think that there should be moral clarity here. If you are a business owner, if you are a business leader, if you have influence about hiring in your company, then in the not-too-distant future you'll be faced with a question. Which is, AI is suddenly making everyone twice as productive. So the question for you will be— 

DEGRASSE TYSON: Making half of the many people, twice as productive.

CALLISTON-BURCH: The question for you will be, do I want my company to be more productive with the same staff, or do I want to be the same level with half the staff? Right? And that is a question that we as a society need to make a decision about. And I strongly encourage you in any position where you have influence over this, to go to the path of growth. To try to expand what your company can do with the same staff. You don't need to replace people. You need to figure out what additional tasks they can serve.

DEGRASSE TYSON: So Kate, about this replacement of cognitive and artistic talents, let me take a middle road there. Middle position road. There were teams of artists hired to reproduce some idea that was set down by a set designer or by some lead artist in a project. Is that much different from the schools of art that we know about from the Renaissance where there is the main artist who is the great artist and then everyone comes in and apes that style? So my sense of this is— and I could be wrong— we have expertise here to correct me— that AI today is really good at copying what's already on the internet. I can tell ChatGPT or Claude, I say, “You see the scene here? Paint it the way Van Gogh might have seen it.” And it will come back with the same color palette, brush strokes and we'll be there in these swirly patterns. And we say “That's what Van Gogh would have painted.” Now if I then ask it, “Paint the scene in the style of no artist who has ever lived.” What's it going to do then? And so for me I think AI in that space is boosting the expectations we should have of ourselves of our true creativity. Because true creativity is not doing what someone else had done just a little better, it's doing what no one else has done. And I don't know if AI has that ability if it's being trained on what everybody else has done. 

CALLISTON-BURCH: I could hop in on that a little bit. Training on— you know, we've also heard, it's just math, it's just text prediction. Training on text that humans produced, that is a harder problem to predict that text than producing the text in the first place. An example that might resonate with people here is, there was a great astronomer, Tycho Brahe, who recorded the position of all the stars, and particularly Mars each night. 

DEGRASSE TYSON: Planets. 

CALLISTON-BURCH: Yeah, sorry. All the stars and also Mars. 

DEGRASSE TYSON: You're standing next to the director of the Hayden Planetarium. And you got that wrong, okay? 

DEGRASSE TYSON: He also— he also recorded the stars. He also recorded the stars. But Mars is the one that Johannes Kepler tracked. Yes. So Johannes Kepler, there's a big heist. He stole the data. But Tycho Brahe, when he was recording the stars and planets, he just looked at the night sky and wrote down where Mars was. 

If you imagine training something to predict the entries in Brahe's journal, it does not have the luxury of looking at the sky. You can solve this problem. You can read a bunch of the entries in Brahe's journal and predict the next. To do so, you have to discover the laws of the motions of the planets. Johannes Kepler did this. Brahe never did. So an AI trained on only Brahe's journals, trained on only the prediction, only predicting the text that Brahe produced. That training target incentivizes the AI to figure out something that Brahe never did.

Now, can AIs today do this? That's one question. But training an AI to predict things humans wrote down does not limit it to doing only things that humans can do. And we are starting to see Ais— you know, we've seen ChatGPT or GPT 5.1 produce some novel physics contributions. We're seeing them start to inch into ways that humans haven't quite done things. And sometimes— you know, when a new scientist comes up with a new idea, they aren't necessarily coming up with a wholesale entirely new way of thinking, as opposed to making small amounts of thinking progress and using a lot of other thinking progress that came from other people. They're standing on the shoulders of giants. So the things we're doing with AI can get them there. There's a different question of whether we're currently there, but it can get them there.

DEGRASSE TYSON: Okay.

CALLISTON-BURCH: And this idea that they're just math and so it won't get there, that's like saying, “Oh, a house cat is just biochemistry, so no sort of cat is dangerous.” You know, a house cat is just biochemistry. And a house cat can't kill you. But that doesn't mean that tigers are fake.

DEGRASSE TYSON: Wow. Was that a logical leap from house cats?

CALLISTON-BURCH: I was starting it, and you looked like you wanted to move along.

DEGRASSE TYSON: We're running short on time here. I want to get back to a very serious point. I was appointed under Obama to serve on the Defense Innovation Board, a freshly-created board of the Pentagon that was tasked with advising the Pentagon in all the ways technology might influence the future, the present and future warfare. Not just how many missiles are in your silo or how many push-ups the soldiers can do. There are other innovations that are possible. Eric Schmidt was the chair of that board. I was privileged to serve under him for this. We toured military bases around the world. Spoke with soldiers very dedicated to their— 

SCHMIDT: Every single one of them wanted his autograph. I'm serious. We would show up, and it's like all these other men are like, “Oh my God.”

DEGRASSE TYSON: So Eric, right about the time, drones were becoming sort of more lethal and more real. We felt it necessary as a committee, as a board, to draft some kind of rules of engagement regarding how and when AI would be brought into the chain of decisions that would lead to lethality. And that was a code of ethics, really. And I was quite proud of that, because there was no telling what would happen without such guidance. My question to you is, our motivation at the time was the knowledge that AI was not perfect. That AI would make mistakes. And you never want someone to die because of a mistake. Yet, of course people make mistakes. So, at what point is AI good enough in that space to say, “I trust AI over a person,” and let AI run with the decision and not a human? Is it just that, at the end, you want to have someone to blame? Because you can't blame the computer. You can't put the computer on trial. But a person, you can. So what happens when AI can find the enemy perfectly? 

SCHMIDT: Everybody here has an opportunity to read the report that we issued under DIB. 

DEGRASSE TYSON: DIB, Defense Innovation Board. 

SCHMIDT: And the Pentagon actually adopted it and published it as their guidelines. Roughly summarized what our thinking was at the time. All war is terrible. All military activity is terrible. It involves destruction of other people and so on. It's just horrible, trust me. And the question is, if you have to do it, can you do it using AI to be more accurate and more successful? That was the question. And we debated this at some length and we concluded that, at least at the time, and I think it's true today, AI is not reliable enough to rely on it to make those decisions. And it's unlikely to be so for a very long time. That was our conclusion. It sort of makes sense, right? Because you can't tolerate sort of a percentage of errors where you might be able to tolerate them in other areas.

And so what we wrote was that any use of automation in that area, essentially called lethality, needed to have a couple things. It needed to have been tested. It needed to have a time and a space duration. And most important, there had to be a human who was overseeing it. They had to know what was going to happen ahead of time. They had to have tested that it was going to happen. And then it had to reproduce. If it didn't, then that was a major problem. And that's ultimately what the Pentagon agreed to. This is a compromise, because these tools are very powerful but they also make mistakes. So that's ultimately, I think, where the Pentagon ended up. 

DEGRASSE TYSON: Now, that was some years ago now. We’re at least a decade forward from that. Is AI good enough so that you would have scripted this differently?

SCHMIDT: I don't believe you and I and the others would have come to any other conclusion on the lethality part. The lethality part is obviously the most important one. It's the most important decision that's made by members of the military under military law and laws of war and so forth and so on. What is true is that the AI planning and recognition— that is the eyesight and the ability to do planning— is now far superior than humans. That's not the same thing as lethality.

DEGRASSE TYSON: Okay, so these are subtle differences in the capacity of AI that are not always recognized or appreciated. 

SCHMIDT: I think we all agree that these systems are not perfect. And at least in things which are life critical, like deciding what to do with a patient, deciding when an airplane is in trouble, anything involving military-kinetic conflict, think twice before you empower AI because it makes mistakes. 

DEGRASSE TYSON: I want to thank you for that, Eric. And thank you for your service in that. Just to land this plane, what I want to do is go down the panel and ask, given this three-month horizon for new stuff that happens in AI and that whole sector, let's just fast forward way into the future to 2030. And ask— wait. How long ago was ChatGPT? How many years ago? Three and a half years ago was ChatGPT-1, or whatever, the first public version of it. So let's fast forward three and a half years to 2030. What is the state of the world and its relationship with AI? Let's start with you, Latanya. 

SWEENEY: Yeah, well I can tell you the vision that we would like to see. 

DEGRASSE TYSON: And that you— you’re not just passively hoping it'll get there. You're actively working to achieve this. 

SWEENEY: Right. And it's a future that is really amazing in the sense that everyone is far more productive. I can be in multiple places at one time because there are agents that know my preferences and my desires and they represent me in various contexts. 

DEGRASSE TYSON: Just to be clear, you guys use the word “agent” to refer to an AI representative of your needs and activities. 

SWEENEY: That's correct. And so they might negotiate a purchase arrangement for me or be in a meeting and represent my interests. And so my productivity is just orders of magnitude greater than it is today. 

DEGRASSE TYSON: And you're fine with that? 

SWEENEY: I'm fine with that, provided that every single one of my uses of AI and making me more productive, is actually able to show that. It gives me a warranty of what it will do and how it will represent itself. In other words, the same thing I get when I get an appliance. I get a warranty and a compliance statement. I want that same thing on these agents, and I'll be able to do that. In some ways what you guys were offering to the military, there are also many societal harms that aren't about immediate life threatening, but are about our ability to have a democracy. About our ability to understand truthful information and so forth, that are at stake and need that same kind of guidance.

DEGRASSE TYSON: It'll be like an underwriters laboratory certification that we have for our actual appliances but applied to your AI agents. Kate. Twenty-thirty.

CRAWFORD: Twenty-thirty. I think we have two paths in front of us. On one path, we move towards a place where we see what is already one of the most concentrated industries in the industrial history of the planet become even more concentrated, and even fewer people have their hands on the levers of power, and democracies start to erode. And these systems ultimately are making decisions that will decide what kind of job you can have, what kind of quality of care you get, what kind of options you and your family I think we'll have going forward. The other path—

DEGRASSE TYSON: Just remind me. I'll get these numbers wrong, but the sense is accurate. There are four or five companies on the S&P 500 that represent, like, 30% of the total wealth.

CRAWFORD: Of the United States. Correct.

DEGRASSE TYSON: Some number like that. The numbers might be precise, but the sense that's correct.

CRAWFORD: You're exactly right. And if you look at the numbers of people who are actually running these companies, you're talking about a handful, a tiny handful of billionaires. They are the people who are getting to make these decisions about how these tools will be used and what problems they will be used to solve.

The other path is that we realize this is the most important public conversation that we can be having right now, which is why I'm so glad we're gathered here today having this discussion. And we decide that, actually, this is too important to be left in such a small number of hands for people who are not democratically accountable. You asked what the difference was between a representative and a tech bro. It is that we vote for one of them and we don't vote for the other one. And that matters. 

DEGRASSE TYSON: Okay, so take me to 2030. 

CRAWFORD: What I would love to see by 2030 is that we say, “This matters. The public interest of these technologies is at the core. And if it is going to be using our land, our energy and our water, then we need to have a say in it. And these tools should be used for the public good. And that actually means making this something that serves all of us, not the few.” 

DEGRASSE TYSON: Are you running for office? Chris, take me to 2030. What are you teaching in 2030, you're not teaching today? And by the way, Isaac Asimov had his three laws of robotics. Do they apply today? Is there an AI laws of robotics?

CALLISTON-BURCH: There is an AI laws of robotics. So, Asimov's laws of robotics, for the non-nerds in the audience— and I’d say that you are all nerds. Thank you for being here with us. Firstly, that a robot must do no harm to an individual, including through inaction. Secondly, that a robot must obey what the human says. And thirdly, that the robot must not cause harm to itself barring the other two. Of course, there was a zero— 

DEGRASSE TYSON: So you mean, it can't cause harm to its—

CALLISTON-BURCH: The first two take precedent.

DEGRASSE TYSON: Yes. To accomplish the third, it cannot undo the first or the second.

CALLISTON-BURCH: Exactly. And then there was a later Zeroth law, which is, the robots cannot harm humanity, which is the broader thing rather than just an individual. So this is a key challenge for AI. It's the alignment problem. 

DEGRASSE TYSON: And just— I have to say— so, Isaac Asimov wrote a series of short stories that were collected into a book called I Robot, which then became a film starring—

CRAWFORD: Will Smith. 

DEGRASSE TYSON: Will Smith. And in there, he explored these rules of engagement for robots very early. This is the early 1950s. So we're talking about 70 years ago, plus. Seventy-five years ago. Continue, please. 

CALLISTON-BURCH: Yeah. Again, this is the place where speculative fiction thinks about the future in a very useful way. So AI alignment is exactly these laws. How do we think about making a system that's helpful? How do we think about making a system that's safe? How do we ensure that it's for the benefit of humanity rather than to its detriment? As we go into 2030, I think that those key elements that are similar to Asimov's laws become the key things that guide the success or failure of the AI endeavor. 

DEGRASSE TYSON: And that could happen within the next three and a half years?

CALLISTON-BURCH: It's happening now. There are many, many thoughtful researchers working on these. You can check out the talk by Amanda Askell, the chief philosopher for Anthropic, about how she's crafted Claude's soul document, the thing that governs its behavior and its code of ethics. It's incredibly thoughtful, and it gives me hope for the future.

DEGRASSE TYSON: Did Google have a philosopher?

SCHMIDT: Larry and Sergei.

DEGRASSE TYSON: Okay. Thank you for that. So, Cindy, what you got going for us? 2030.

RUSH: Twenty-thirty, a few years from now. I think that we'll see a lot of these technologies we have now, the kind of LLMs will become more useful in our day-to-day life. Right now, you know, it can kind of respond to questions or to prompts we give it. And maybe it'll become a little bit closer to be able to respond to queries more like “achieve this objective.” So I can imagine specialized versions of this, like, help you respond to emails and these sorts of things.

Feet on the ground, I think we maybe will see self-driving cars in New York City in a few years, you know? I think that a lot of the AI will be trained on more specialized data sets, so they'll become better at certain tasks, like scientific research or mathematical research. If you train on certain parts of the Internet, it may be better than what it is now.

DEGRASSE TYSON: Quickly, Kate. If AI got that specialized, wouldn't that be more efficient on the grid?

CRAWFORD: It could be. This is one of the really big questions. I mean, right now, large-scale models are being trained using a lot of fossil fuels. A lot. In fact, more than previously. So if we had small models, if they were specialized, there are ways forward where you can use much less energy.

DEGRASSE TYSON: That's what I would have figured. And, for example, in The Jetsons, the documentary The Jetsons, of the future, as fun and creative as they were, they did not realize that maybe the car could one day drive itself. The human drove it, or you'd get a robot to drive it. But the car is the robot, and that's a very specific invocation of this. So, based on the title of your book— I'm going to show people your book, because— I’ve got Kate's book here, too. Kate. Atlas of AI. The pub date 2021, but it's still happening. 

CRAWFORD: Apparently so. They just went to the 10th edition, so it's doing alright.

DEGRASSE TYSON: Yeah. Okay. Good. If Anyone Builds It, Everyone Dies. This has got to be the most depressing title ever.

SOARES: It starts with “if.”

DEGRASSE TYSON: Ever. Oh, that makes it all better. Okay. So, take me to 2030. What's going on? Are we all dead?

SOARES: You know, if you step back from this— from the AI argument, and you just look broadly at what's going on. The machines are talking now. People used to think it was going to take a while to get to the point where the machines are talking. We’ve got these people being like, “Oh, yeah, we're just growing these things. Turns out they talk. They're solving problems we said weren't going to happen for 50 years. Now we're trying to take it to superintelligence.” That's kind of crazy. It's kind of crazy. You're like, “Do you guys know what's going on in there?” They're like, “No, but we think it'll be fine. Trust us.” Right? In Silicon Valley— 

DEGRASSE TYSON: We're not trusting you, just so you know.

SOARES: Great. In Silicon Valley, people are spooked. You know, there's a joke in Silicon Valley that when someone leaves a normal tech job, they say, “I've had a great time here, and now it's on to the next adventure.” And when someone leaves an AI tech job, they say, “I have stared into the abyss. I'm retiring to write poetry. Please spend time with your families.”

DEGRASSE TYSON: Okay.

SOARES: And if you think I'm joking, you can look at the resignation of the Anthropic safety lead a few weeks ago and see how close that joke is. But the thing is, in Washington, DC, they are not spooked. In Washington, DC, they are thinking that this is all about self-driving cars and autonomous weapons and job loss. And those are important issues. But what I hope we see in 2030 is that DC is spooked. I hope we see a world where DC has realized that the people running the labs, the academics outside the labs, like Jeffrey Hinton— that the people inside the labs, that everyone who's close to this technology is saying, “Oh, my gosh, the super-intelligent stuff could be really dangerous.” Even Eric was saying “There's red lines, and if we start to get close to these things, we need to watch out.”

DEGRASSE TYSON: Even Eric said that? Just briefly, you mentioned Dr. Hinton. He's one of the fathers of neural nets. Deep learning. Deep learning. And he was given the Nobel Prize. Was it in chemistry, I think? Yeah. 

SOARES: I thought physics.

DEGRASSE TYSON: Physics, thank you. The Nobel Prize in physics just a few years ago. And we were fortunate enough to get him onto my podcast for an interview. It's online now with Jeffrey Hinton, talking about the original thinking of AI, and his reflections on where it is now and where it might—

SOARES: Those three people did their work at Google. 

DEGRASSE TYSON: So. Take me to 2030. Is DC AI sensible by 2030? Because we're in the middle of whoever the next president will be.

SOARES: Yeah, so I think—

DEGRASSE TYSON: Presuming it won't be Trump in a third term. Just have to— that's an “if” statement. Okay.

SOARES: I think— I think they could be. You know, we see a lot of people in the industry saying that this is horribly riskier than any other technology we work with. And we see people at these labs saying, “I think there's a good chance this kills everybody, but I gotta do it because I'm better than the next guy.” And maybe one of them's right. Probably two of them aren't. 

But in 2030, if we keep having this conversation, if it becomes more salient, I think we see people around the world realizing that we shouldn't do this superintelligence stuff. And the world does not need to look that different from today. You don't need to stop the self-driving cars. You don't need to stop AlphaFold doing the drug discovery. You don't even need to stop the autonomous weapons. Society can have a debate about, do we want to save people on the front lines? Do we want to not do this, not go down this path? But we could stop just the race to build smarter and smarter AI's that nobody understands. That would be easier than nuclear arms control.

And I hope that in 2030, leaders around the world will realize that this is lethally dangerous. And we'll be putting a treaty in place to say we're not rushing to smarter-than-human AI's with anything remotely like the modern technology where we have no idea what's going on in there. 

DEGRASSE TYSON: All right. Eric?

SCHMIDT: You asked a question about 2030.

DEGRASSE TYSON: Yes, I did. Take me there.

SCHMIDT: Why don't we bet that American democracy will survive?

DEGRASSE TYSON: Okay, so that's an important— 

SCHMIDT: So I'm going to make a bet that American democracy will actually survive, we’ll get through whatever the current problems are, of which there have been many listed here. That the excesses will be addressed through legislation. I know that's hard to imagine right now, right? But I'm going to bet that democracy will ultimately reflect the will of the voters, which is how it's supposed to work.

So in that scenario, which I know is difficult to imagine at the moment, let's think about some of the other benefits of all of this. The likelihood of students being much better educated because of their ability to be tutored using AI in specialized way is enormous. The ability to have the next generation, your kids and grandchildren, being so much smarter than we will ever be, is pretty high. Furthermore, this issue about jobs, there's no question there's going to be job dislocation. But last time I checked, every one of those predictions has been false for the reasons Neil mentioned earlier, which is the American economy is fairly efficient at recycling jobs. And there's definitely harm. There's definitely people hurt. But at the end of the day, people working with computers today make more money than people without.

Furthermore, let's bet that these other issues, the regulatory issues, the tech-row issues, are resolved through some combination of self-interest, shareholder values, legal liability, and regulation. In that model, in 2030, every one of you will have an assistant, a savant, that allows you— under your control, not spying on you, not doing something bad, under your control to do the things that you most care about. You're a physicist, right? Neil's a physicist, you have a physicist's assistant who overnight does your thinking while you're busy sleeping and tells you what is interesting in the day. And you love it, and you can turn it off if you don't like it.

There's example after example. The most likely scenario by 2030 is something analogous to what I've described. All of the things that everybody's worried about are true at some level. Or they're point issues that have to be debated. In my book, which you did not feature, I say very, very clearly— 

DEGRASSE TYSON: Tell us the name of the book.

SCHMIDT:Genesis.

DEGRASSE TYSON: Thank you.

SCHMIDT: Including myself and Henry Kissinger and a few others.

DEGRASSE TYSON: Kissinger was co-author?

SCHMIDT: Yes, yes.

DEGRASSE TYSON: When did you write, he'd been dead for— when did you— okay. The AI Henry Kissinger. Okay, fine.

SCHMIDT: In fairness to Henry, he finished the book the day before he died. And I encourage you to read it. This last book of his many great books. The important thing, thinking about this is really important. In our work, what we said is, this conversation needs to occur over and over again. Because it is the future of us. It is us. It is how we operate. And I endorse that. 

But at the end of the day, we will get through this, and more importantly, we will thrive. Do you know why? Because at least American capitalism, the way it's structured with regulation, produces enormous economic and life value for us. Think about it. Solving climate change. Solving the various issues we have. And remember, we're depopulating. Especially because we're not allowing immigrants in, which is an error. Right? So you get the idea. 

DEGRASSE TYSON: Wow, let me see if I can put a bow on this. So, I try to always generate a cosmic perspective on all that happens to me in a day. And to get this kind of exposure, I don't know how to tie this up in a bow. All of this in one. But allow me to say that I've been in computing for most of my life. That I've had access to it and awareness of it. And I remember the Turing test. People said, if you interact with a computer and you don't know if there's a human on the other side, then why distinguish it from being a computer or a human? And pass the Turing test. The original research paper was called The Imitation Game, which became the title of the movie, featuring the life of Alan Turing, which I recommend.

So, we're way past that. And so I remember thinking, “Oh, we'll never have a computer as good as people at chess.” We're way past that. I remember when Siri showed up on the iPhone. There were whole commercials with— who's the movie director who was interacting with it? Martin Scorsese is talking to Siri. And our jaws are dropped open. There's this intelligent agent serving him, in the voice of Siri, in his iPhone. And I'm thinking, “Wow, the future is here.” Today, no one talks about Siri as AI. We're so past it, it's like, “Oh, that's just old. That's not even interesting. AlphaGo, that's yesterday. AlphaZero, yesterday.”

So, what intrigues me is, every one of these incremental advances in AI, we absorb. It just becomes part of life. And we're looking at that next frontier. Yes, the AGI, the artificial general intelligence, that's a frontier that may be dangerous, but all the rest we absorbed. We think nothing of putting on GPS and having it tell us the shortest route to Grandma's house, corrected for traffic, and there's no human in that loop. We're all fine with that. We're not spooked by it. You're not writing to your congressman about it. You're not trying to legislate it. It's just there in our lives. 

So, I foresee, by 2030, if I will, much more AI just folded into our lives. But the AI that transcends all of our intelligence, the one that you might think is farther away than it might be, given the title of this book.

SOARES: Starts with “if.” 

DEGRASSE TYSON: I have to agree with Nate. I'm old enough to have lived through a lot of the Cold War. You know what kind of ended the Cold War? A lot of things contributed, but you know what mattered? The realization that if you launch nuclear weapons, intercontinental ballistic missiles from the Soviet Union into the Western Hemisphere, and we see that, and we retaliate, everybody dies. We had an acronym for it. MAD, Mutual Assured Destruction. That's what brought people to the table. Say, “This is insane. We have to regulate it. We have to reduce the stockpiling, because preserving our species is what matters here.”

So based on that evidence, I have some confidence that as those dangers become more and more real, people will come to the table and say, “Yeah, keep the rest of the AI going. We got new medicines and new understandings of our physiology and new technologies that help us get smarter and healthier, yes. But that branch of AI is lethal. We’ve got to do something about that. No one should build it.” And everyone needs to agree to that by treaty. Treaties are not perfect, but the best we have is humans. 

I would end by saying, there's a quote from Ray Bradbury, who— I verified with him. I met him only once, and I just had to verify this quote. He said there's a woman who once came up to him and said— he's a science fiction author, Ray Bradbury. “Mr. Bradbury, why do you write these stories about these apocalyptic futures? Is this the future you think humanity is going to inherit?” And he says, “No, I write these apocalyptic futures so you know to avoid them.” That is a cosmic perspective. Join me in thanking our panel. Oh, my God! Latanya, Kate, Chris, Cindy, Nate, Eric. Oh, we found the book! We found it!