Overview
Prof. James Evans, a University of Chicago sociologist and data scientist, believes we’re training AI to think too much like humans—and it’s holding science back.

In this episode, Evans shares how our current models risk narrowing scientific exploration rather than expanding it, and explains why he’s pushing for AIs that think differently from us—what he calls “cognitive aliens.” Could these “alien minds” help us unlock hidden breakthroughs? And what would it take to build them?

Related
AI Could Be Making Scientists Less Creative — Gizmodo
NSF awards $20 million to build AI models that predict scientific discoveries and technological advancements — UChicago News
Transcript
Paul Rand: Imagine this, an AI prediction algorithm so sophisticated that it doesn’t just guess the next big scientific breakthrough. It actually knows who will discover it.

James Evans: The capacity to simulate collective outcomes is dramatically improving.

Paul Rand: Like Minority Report, but instead of stopping crimes, we’re spotting scientific genius.

James Evans: In fact, I’m in the process of trying to build a little AI anthropologist for the lab to build a lab, build a model of the lab to go interview people. It’ll sit in lab meetings. It’ll see what people are reading. Maybe it’ll make suggestions about who can connect with what information environment.

Paul Rand: That’s James Evans, a University of Chicago sociologist, data scientist, and Google researcher. Way before ChatGPT became a thing, he was already predicting scientific breakthroughs and exactly who’d be behind them.

James Evans: I definitely have predicted within universities that I know things that people are working on, and they are steeped in those space of possible combinable elements, and so it seems typically totally naturally for them. They’re like, “Oh, yeah.” That’s one of 50 things that we’re studying. And I’m thinking to myself, well, that’s the one that’s going to...

Paul Rand: But lately, Evans is thinking even bigger. He’s not only using AI to study how we do science, but studying how AI could change the way science is actually done.

James Evans: And I think we’ve seen from scientific research both ways in which machine learning and more recently transform-based models and chat is influencing for good and ill scientific discourse and advance.

Paul Rand: Evans is onto something crucial here. AI’s potential for science is huge, but it also carries risks. And the one thing that keeps him up at night the most is that we’re missing out on mind-expanding possibilities because of a dangerous assumption about what AI should be.

James Evans: I think that the whole idea of artificial intelligence, as coined by John McCarthy and inspired by Alan Turing was built on a notion of intelligence, which is the human subject, either as an interlocutor or as a co-thinker. And so we’re really going to try do what an intelligent being does. We all agree what an intelligent being is, we’re going to try to replicate and capture that.

Paul Rand: Just look at how we judge AI’s success. We reward AIs for acting human. Almost every benchmark pushes them to think like a person, to perform with human-like intelligence. But what if that’s the wrong goal? What if we need an entirely different kind of intelligence instead?

James Evans: The Industrial Revolution, it’s not about replicating the human capacity. It’s about massive complementarities building things that humans couldn’t do, they couldn’t conceive of.

Paul Rand: And that’s exactly what Evans is working on now, novel intelligences that don’t think like us at all.

James Evans: I think cognitive aliens that have memory and cognition and sensation that expands or crosses levels of analysis that we don’t experience and we can’t experience, I think these are the kinds of things that are going to have the potential to really complement human discovery in novel ways.

Paul Rand: Of course, building these alien minds means totally rethinking the kind of data that we feed AI. Because right now, we’re basically stuck. Everything is language. Human language.

James Evans: I think large language models are precisely the kinds of models that don’t have a novel sensory capacity, so I think that models of the future are going to be much broader. They’re going to have both increased sensory capacity as well as increased articulative capacity.

Paul Rand: From the University of Chicago Podcast Network, welcome to Big Brains, where we explore the groundbreaking research and discoveries that are transforming our world. I’m your host, Paul Rand. Join me as we meet the minds behind the breakthroughs. On today’s episode, Rethinking How We Build AI Scientists.

The University of Chicago Leadership and Society Initiative guides accomplished executive leaders in transitioning from their long-standing careers into purposeful encore chapters of leadership for society. The initiative is currently accepting candidates for its second cohort of fellows. Your next chapter matters for you and for society. Learn more about this unique fellowship experience at leadforsociety.uchicago.edu.

Over the past few decades, James Evans has been tracking a disturbing trend. When he and his team designed a model using 1.8 billion citations from about 90 million academic papers, they saw that despite the overall amount of research increasing, the number of novel breakthrough discoveries has actually been decreasing.

Portrait of James Evans
James Evans
James Evans: Science is populated by disciplines. Disciplines are the marriage of a set of problems or opportunities and a set of methods that we believe are the most fruitful ways of pursuing those opportunities. It’s organized by often the most senior researchers in these departments who are basically claiming what the future most fruitful methods are going to be to solve problems, but these are methods that were fruitful in the past.

And so there is an institutionalized diminishing marginal returns that those methods can yield for these disciplines. Why? Because people, when they pick out these methods, they typically point them at the most fruitful possible place. And so the next problem that they solve with them, they can’t do as well on average, and so you have a situation where the system potentially slows down. And we can see this also demographically. As science gets older, as scientists reach about year 10 in their publishing career, then they start systematically criticizing the work that they cite, and that work that they criticize disproportionately is produced by young scientists.

So, one of the things that some economist colleagues identify in a wonderfully interesting paper called Science Advance One Funeral at a Time, they look at in the only field that they could look at this, because it’s the only field that’s large enough to look at it, but in biomedicine, they looked at early deaths, unexpected deaths from car accidents, from aggressive cancers, et cetera. And when people die in these subfields that they identify across the biomedical space, then those deaths are associated with a burst of innovation inside those fields.

Paul Rand: So, how do we solve this slowdown in scientific progress? Enter AI.

James Evans: So, AI can be designed to not be pressured and controlled by those demographic and careerist forces,

Paul Rand: But how we design AI matters immensely. In a recent study, Evans’s team showed that the way we’re currently using AI might actually narrow scientific fields instead of expanding them.

James Evans: So, one of the things our team did was we just looked in the natural sciences at the use of AI over the course of the last 25 years. And one of the things that we see systematically is that the researchers who use AI demonstrably in their work, they do better. So, they have better careers. They get tenure 2.3 years early. They get more citations. But if you look at the subjects that end up getting anointed, it’s actually shrinking.

Paul Rand: Today’s AIs are only as good as the data we feed them. So, scientists who want to use AI will naturally gravitate toward the areas in which we have enough data to build useful AIs.

James Evans: We gravitate to the places where there’s the most data already available to us. Now, why did we generate that data historically? Well, because we believe that understanding it and predicting its function, we could do better. We could have greater control of the world, greater understanding of the world. So, these are already our best bets, even without AI. And now we’ve thrown AI, and because deep neural networks require enormous data to learn all of those parameters, it takes the biggest, best data that we have. This is why we need the full web basically to learn a chatbot that’s plausible.

So, that means that we’re killing off or finishing fields that already existed, and we’re not yet using AI to generate new fields. So, there’s a whole host of places where there is... So, across scales of analysis, for example, what’s the relationship between low-level particle interaction and high-level biochemical interaction or even, I’m a social scientist, high-level sociological or ecological or cosmological interaction between high-level entities? And these questions, we haven’t deeply explored them because the signal from the lower level seems so sparse relative to the higher-level question that we haven’t even gathered the data. What people tend to be doing is to bring AI into the places that they’re already looking, and it accelerates science in that space, and then we’ve over-farmed the region.

Paul Rand: Yep, very good analogy.

James Evans: We need to move on.

Paul Rand: And this narrowing makes science more predictable, and not necessarily in a good way. In fact, Evans’s AI models have become remarkably adept at forecasting not just what new discoveries are coming, but actually who’s going to make them.

James Evans: So, we build these landscapes of how it is that ideas are combined across the scientific literature in the past and who’s combined them. And we do this by basically creating these random walks that move through the literature. They jump from properties to papers with the property, to people who write the paper, to other papers by that people, by materials on those new papers, by other papers with that material. And if you run these random walks, they end up simulating the distribution of discoveries that are made as a function of experience that, I’ve studied A and I’ve studied B, so I can combine them. Or conversation, that I study A and I’ve worked with you, and you study B, and so A and B can be combined in this way.

So, if you run these random walks and you effectively embed them or learn them with these large transformer-style models, then you can do a very strong job of predicting what’s going to be discovered by people in the future and who’s going to discover it, as individuals and labs.

Paul Rand: As you talk about that, there clearly is some excitement and an upside to that. But maybe back to this idea, is that actually problematic? And if it’s problematic, is it because the breakthroughs that are really going to happen are not going to happen on a predictive model? They’re going to be things that were not expected to be discovered or created that actually create the biggest insights?

James Evans: Well, to be clear, so the reason we’re building this, what I call a digital double of the scientific landscape, is so that we can avoid it.

Paul Rand: So, how do we avoid it? Evans believes the answer is in creating AIs to think in ways completely different from how humans think, but how do you build such an AI?

James Evans: So, for example, we as a result of these models of what it is that people are going to discover, we use this to build other AIs, which I think of as cultural aliens.

Paul Rand: If you can build an AI to understand the entire scientific landscape so completely that it can predict future discoveries, then you could also train that model to avoid all of those predicted pathways.

James Evans: We can’t build an alien until we have a model of actual scientists in the actual scientific system doing... We wouldn’t even know what an alien looks like if we don’t know what the big hairy beast of this scientific system looks like. The predictions that these human-avoiding aliens generate in the space of both physical energy-related materials, health-related materials, they end up lining up really powerfully against even first principles models of the properties inside these spaces. And so now, we’re also working with labs to test.

And the reason is because of this diminishing marginal returns of theory. If you have a bunch of people in a part of the space, then they publish the things that are successful. They don’t publish the things that aren’t successful. So, there’s a whole bunch of what I’ll call negative knowledge, which is hidden there. And these empty spaces, these borderlands, there are very few people. Scientists don’t want to die in the forest alone if they’re unsuccessful. And so there’s a ton of low-hanging fruit that can be harvested by these kinds of human-avoiding models.

Paul Rand: Okay, that’s really awesome. And I love this explanation. So, almost in some ways, the alien models are helping skip what humans may find on their own, even just in time, and either make it come a much faster in the example that you just gave or actually put together combinations that very likely under current thinking would not have been created. They’re putting different models together, different fields together, different discoveries together to come up with something that may never have been discovered.

James Evans: Yes. Yeah.

Paul Rand: Okay. And what is helping us move that area forward? And what is the things that are limiting our ability to move that forward?

James Evans: It requires us to think about AI not just as a cognitive device, but also as a sensory device. We need to have AI out there generating new data, searching new spaces so that it’s building explanations. They’re not just competing with existing explanations, but that are generating explanations in places where we have no explanations.

Paul Rand: But what exactly does that mean to think of AI not as a cognitive device, but as a sensory device? Well, that’s after the break.

How can we improve communications at work? Why did McKinsey’s former CEO go to prison? How irrational are we really, according to Chicago Booth’s Richard Thaler and Harvard’s Steven Pinker? And are stock actually efficient? The Chicago Booth Review Podcast addresses the big questions in business, policy, and markets with insights from the world’s leading academic researchers. We bring you groundbreaking research in a clear and straightforward way. Find the Chicago Booth Review Podcast wherever you get your podcasts.

In order to create truly alien AIs, Evan says we’ll need to devise novel ideas about what kinds of data we use and how we get that data into the AI. The biggest shift will be moving towards new forms of sensory data. And when he says AI needs new senses, he means literally new ways to perceive the world beyond human limitations.

James Evans: I think it comes down to this idea that von Uexküll, who was a German physiologist of the early 20th century, described as the Umwelt. So, this is the life world, the sensory world of an animal. And his example was the tick. The tick basically measures a certain kind of acid that comes out of mammals, presence or absence of hairs that come out of mammal skin, and a certain level of heat that corresponds to basically the kinds of mammals that they like to zap blood from. And the idea is that’s their sensory world. That’s the senses that they need to basically act on. And every animal has basically a sensory world, a sensorium, a world of vision and hearing and smell and proprioception, where we understand our internal selves, which allows us to act intelligently in that field.

Imagine superhuman senses. So, think about the discovery, for example, that dogs can smell Parkinson’s disease years before it can be diagnosed by other human tests. So, there’s a whole space of sensory capacities that are not part of our sensorium, but are nevertheless there. Creatures detect things that we can’t detect, vibrational sense, electric sense. These were historical senses that we may have had in our evolutionary lineage that we’ve lost because we’ve superseded them with other kinds of things.

And I would say this is similar to Yann LeCun, who has criticized that language AI can only get us so far. An example in the context of radiology is that we’ve got these radiological artifacts that we’ve created for human interpretation and consumption over the course of the last 100 years, from X-rays to MRIs. What can it squeeze out of those human-focused generations or artifacts that humans haven’t discovered in the past? And it turns out they can... Yes, they can identify and see things that we couldn’t see.

But we generated those artifacts, we flattened those artifacts, we reduced the dimension of those artifacts for human understanding and capacity. So, how do we start opening those up again? How do we start putting the biophysics to work to explore the space of possible representations that will maximize our ability for these novel intelligences, these alternative intelligences to diagnose, to treat, to provide effective prognoses of future healing potential, et cetera?

And so right now, we have increasingly flexible intelligent cognition, but we just got to plug in a data source. So, it’s like we basically just plug in the videos that we’ve generated from some completely other space for some completely other purpose. We don’t evolve them to maximize their capacity to act in a specific space like evolution does. And if we look at the history of the evolution of science, we systematically see the big advances in understanding come from new technologies of sensation, so new measurements of otherwise or previously inaccessible wavelengths or particle sizes, or whatever, this discriminatory capacity.

And so if we really want AI to allow science to see farther, we need to not just intensively apply it to the sense data that we’ve already gotten. If we want it to correspond with any of the evolution of scientific advance, we’re going to have to use it to actively learn what to look for to build new telescopes, microscopes, and sensory devices across the space.

Paul Rand: Okay, so does that push us to think that we can have a future where the science is not necessarily done by humans, but the thinking, and maybe this is back to the complementary nature, of the philosophy of science is going to be where the humans play a role and the scientific discovery work is going to be done more by some of these large language models?

James Evans: I think that these models will enable us to do is when the models are tuned to see the scientific frontier the same way that the full distribution of individuals see it, and when surprising papers or patents or products emerge, surprising which is we didn’t expect them to come, they happened as a result of some unexpected combination of sensation and theory that made this thing or made this thing visible, then we can put those into the model. And because of the rich internal geometry of the model, we can say this improbable thing becomes extremely probable. What are all the other things in the geometry of the model that were improbable five minutes ago that are now much, much more probable?

I think that will be a unique and novel capability for government, for industrial funding of technology. So, it’s like, what are all the entailments? If this particular material has this energy level, if this particular therapeutic works against this undruggable cancer, what are all the other things that now are very likely to be true as a result of that thing being true? I think that will be, I think, the potential to build superhuman cognitive alien thought partners. And I think these thought partners, I personally want to be expanded by them, but I also want the whole system of the social sciences, the natural sciences to be expanded by them.

Paul Rand: So, what would it mean for AI to revolutionize the social sciences? It’s easy to picture AI breakthroughs in STEM fields. For example, DeepMind’s AlphaFold can predict new protein structures. But human society has hidden forces too, and AI might help us discover those invisible patterns that drive our social world.

James Evans: When it sees us engaging in discussion or activity, it’s likely that there’s a whole bunch of things that are going on, competitive things, arousing things, connecting things that are just invisible, that are actually driving the way in which conversation that have nothing to do with the way we experience things about these patterns. And so human intelligent capacity, so much of it’s social, everything from being able to have the whites of our eyes clear that many other animals don’t have that allow us to see who’s looking at who’s saying what, who’s attending to who. We have so many things that allow us to be aware of the pack, what the collective is thinking.

Paul Rand: And this is exactly why AIs shouldn’t be limited to a purely human point of view. If we only train them to notice what we already know, they’ll never show us what we don’t know. In other words, an AI that only sees the world as humans do won’t have the capacity to reveal the things that we can’t yet perceive and what that could mean for our often difficult interactions between partners, friends, political divides, and international conflicts.

James Evans: I think one of the things that’s missed by thinking about just AI in STEM fields is that we’re looking at and improving our capacity to control a world out there. But the real human capacity is not controlling the world. It’s controlling each other. It’s actually building massive states that facilitate population growth and retention and decrease violence and disease through collective action. So, if we’re able to unlock the ability to, not wage war, but wage peace, wage collaboration at larger scales, of course there will be agents that are trying to use these tools to sow division, to sow chaos and, in the midst of that chaos, to gain power.

But at the same time, because those things aren’t necessarily going to happen, we have to build our humanities and social science capacity to wage collaboration to identify and become less susceptible to the infectiousness of bad, selfish, and other action for us to be able to, even in this moment, for example, garner collective commitment to scientific advance.

There was this interesting paper this last fall that showed how AI could decrease the likelihood that people believe in conspiracy theories of various types, and that those new beliefs or new uncertainties about those theories actually persisted over time. I think that, especially because people are going to use these in self-focused ways, their capacity to actually build cross-conversation between polarized groups and to engage and minimizing the effect of conflict is absolutely critical. Governments and other agencies interested in peacemaking and peacekeeping have to be in the Red Queen race, where we’re using it to wage peace rather than war. Absolutely.

I think one thing that’s exciting about artificial intelligence for sociology, political science, economics, history, anthropology, is the ability to spin up digital doubles of human agents inside this space. Historically, I thought that first we need to construct an agent out of these systems, a persona that corresponds to an individual, and then maybe if we can get that 90% accurate, maybe if we have them interact with each other, maybe we’ll get those interactions predictive 75% of the time. And then maybe if they’re interacting as groups, we’ll predict them at 50% accuracy that there’ll be some degradation from the integrity of an individual agent that we’re predicting their opinions to to predicting a whole field of interacting agents.

And it looks like, with increasing work of my group and others groups, that this may be reversed. And this is something we see in ecology and evolution. In ecology, the more stable predictions come from the more complex interactions between component parts. And so I actually think that there is enormous potential for looking at collective behavior in a predictive way using simulated digital doubles of humans in these systems. And I think this is increasingly being used and generated by both tech firms, but also by computational social scientists to build, in some cases, simulations of hundreds of thousands of agents, and growing.

Paul Rand: So, did you see this as actually having a role coming up in potential policy implications?

James Evans: Absolutely.

Paul Rand: Yeah.

James Evans: Absolutely. Yeah, if you think about what are the implications of this or that policy on voting or on segregation in neighborhoods or of polarization or of government or non-governmental support, this allows us to spin up worlds in which we can test things with really highly resolved, not just, again, individual agents, but systems of agents that may in fact even be more robust than our models of individual agents. And so it’s exciting. It’s also dangerous because who else is going to be doing this? Authoritarian governments are going to be playing and replaying their messages. These are technologies that have the potential to be used for good and evil.

Paul Rand: Well, it’s really fascinating. Even in the communication space, which I’m in, there are groups that are working on what they’re calling is synthetic audiences, where they can program it and say, “Look how different messages have been received by different audiences.” And if you say X, Y, or Z, this is likely how that’s going to resonate. And so that ability in many, many, many different fields is shaping the implications of the choices that we’re making.

James Evans: For example, we see when we put agents together that have the distribution of, for example, political preferences that we observe in society at large, we see them polarizing at roughly the same rates that we see, and for roughly the same reasons. For elite bias, for self-confirmation biases, for all these different kinds of things.

I think how are agents likely to be used in the future? Well, individuals will have agents that are working on their behalf. That’s one of the ways. And of course, wealthier corporations and individuals will have more agents that are contracting for them, that are prospecting for relevant information, that are making deals on their behalf, et cetera. And so I think one concern about the ways in which basically human biases are built into these models is that it could accelerate the way in which individuals cluster and clump into segregating groups. And so I think that’s one big concern. I think thinking about how it is that platforms can build systems that facilitate broader collaboration and conversation versus individuals having untrammeled access to build agents that are trustworthy to themselves, but that then accelerate the degree to which they’re filtering their information is problematic.

We had a paper just a couple of months ago at Nature Communications where we showed that online systems like Twitter’s Community Notes, now X’s Community Notes, that gradually made information accessible to users about whether or not others perceived their information as misinformation or disinformation was actually much more successful than just letting individuals as vigilantes shoot it at each other’s information online. Because individuals were not only rude and sometimes wrong, but they resulted in the people who were being chastised retreating further into the information bubbles. So, they were actually exploring novel political and civil information when they were basically identified as promulgating misinformation.

And so I think that it’s going to be a Red Queen race where individuals are using agents in ways that are likely going to lead to greater echo chambers and discriminated and segregated groups. And I think that platforms are going to have the burden to facilitate exchanges and arbitrage and conversation that brings the civil square together again. And I think those are two forces that are both going to be at work.

Matt Hodapp: Big Brains is a production of the University of Chicago Podcast Network. We’re sponsored by the Graham School. Are you a lifelong learner with an insatiable curiosity? Access more than 50 open enrollment courses every quarter. Learn more at graham.uchicago.edu/bigbrains.

If you like what you heard on our podcast, please leave us a rating and review. The show is hosted by Paul M. Rand and produced by Lea Ceasrine and me, Matt Hodapp. Thanks for listening.

1000-1099 Miscellaneous