In conversation: Anna Goldsworthy on "The God We Made: The Threat and Promise of AI"

Liam Carroll

I just want to start by saying thank you so much for such a wonderful piece of work. I have been telling absolutely everybody that I know that this Quarterly Essay is pretty much my favourite thing that I've seen written about AI. I think that it is a really earnest and beautifully written inquiry into what is going on in AI right now. It takes it very seriously, and it really confronts us with some big questions about what it even means to be human, which I'll ask you about later.

Thank you very much. And I just thought I'd start by asking you to read out a passage from towards the end of the essay. And then I'll ask you something after that. Thank you.

Anna Goldsworthy

Liam, thank you for your kind words. The issue is not just plagiarism or copyright violation or even the depletion of AI resources. The issue is, quite simply, everything. We are at a critical moment and we need to address it collectively. Otherwise, the thinking, such as it is, will be done for us by Silicon Valley. It is a moment that will test our much-vaunted adaptability more than any other moment in history. We will need to grapple with issues of work and meaning and social justice and human survival.

We will need to determine whether there is still a place for us in this world, alongside these new and old gods. It might be a short moment, or it might be a longer one. It will probably not feel like enough time. It never does.

Liam Carroll

Thank you very much for reading that. It's a really beautiful passage. I just wanted to start by asking — as someone who I gather was an outsider to this space a few years ago, to then write that passage right there — what was that journey, and what brought you to write exactly that?

Anna Goldsworthy

I sort of touched on this in the last session. Basically, my thinking was prompted by conversations, and specifically arguments, I was having with my two sons around the dinner table, one of whom is currently sitting in the front row here. So he's responsible for it. And almost through contagion, I think because of my children's engagement with this tech, I watched the way they dealt with it — the very specific way they would use prompts, the kind of Socratic dialogue that it engendered. And that Socratic dialogue was then brought to me.

I felt I was almost dealing with Claude sometimes in a second-hand way, as an interlocutor. But what was fascinating to me — and I'm fully mindful of all the ways in which we're at risk of giving away cognitive capacity and so on to the tech — was the flip side of it. When it's used to scaffold thinking instead of substitute for thinking, I sort of could witness firsthand the way in which it could be used as, I suppose, a staircase to better thought. And I think essentially it was the kind of Socratic dialogues that my children,

particularly Reuben here, were having with Claude — and that Socratic dialogue that's just been such a part of our life every night, sitting around the table at dinner — that kind of precipitated this essay, and also provided me with a lot of the information and kind of gave me an education. So, yeah, thanks.

Liam Carroll

There is a lot of mention throughout the essay of catastrophic risk. I think you are quite dispassionate in your observations of what's going on in the field — I would say in a very positive way. And I wonder what you make of the discourse around catastrophic risks due to AI. Because obviously there are a lot of people in this audience who are very interested in that.

Anna Goldsworthy

Actually, I think the first time I began to feel very alarmed about this was back during the pandemic, and I read a book that has probably been quite a seminal text for me. It's The Precipice. And Toby Ord, as many of you will know, was a philosopher of existential risk at Oxford University. And what scared me about that was, he says that over the course of any given century, the likelihood of a serious existential event would be 1 in 10,000. And that would occur because of something that happens to the natural world,

such as a meteorite hitting Earth or the sun burning out or whatever. But over the course of this century, he elevates those odds — or I guess reduces them, depending on how you look at it — to Russian roulette odds: there's a 1 in 6 chance of an existential event befalling humanity, which could mean absolute destruction of the entire species. Or it could mean we find ourselves in some unrecoverable situation. And during the pandemic, in which we were already somewhat attuned to the notion of existential risks, he explained where he thought those risks were situated.

And alongside the usual suspects — runaway climate change, nuclear risk, pandemics — above those he nominated unaligned artificial intelligence as an even more significant risk than those. So that terrified me. And it made me start to feel that we very much were living on borrowed time. And that feeling has remained with me. It sort of inflects everything slightly differently. You have a real sense of contingency, but also you look at it and think, well, what can we do about this? What can I do about this, as an artist

and a writer and a performer? And I guess I arrived at the position where the best thing to do about it was just try to really get into the public conversation and raise awareness, because obviously people in this room understand, but I'm just constantly struck by how many people have elected, perhaps, to bury their heads in the sand about the risks of this technology.

Liam Carroll

If we step back for a moment from the risk of a rogue superintelligence and just think about the need to learn to live alongside very powerful systems — you write in the essay that we have already surrendered so much of ourselves to the algorithm, in reference to social media, and that we have already suffered a kind of self-colonisation by smartphone. How worried are you that something similar is going to happen with AI?

Anna Goldsworthy

If you look at how we dealt with smartphones, and particularly social media, and the really significant lag between pretty much giving screens to all of our children, and then later on becoming fully mindful of the mental health effects, and later again actually doing some things about it and starting to address it from a policy perspective — that doesn't necessarily engender hope. On the other hand, perhaps we can learn from that and say that we need to act with much more alacrity in this situation. Whether or not we have the capacity to.

Democracy is a bit of a slow moving machine, but I think the more we can communicate the urgency of these conversations and the necessity for them to transcend particular silos, and they've got to be cross-disciplinary. We need to bring a philosophical lens to it, a sociological lens to it, a humanistic lens to it, a political science lens to it. We need to be able to have these conversations, and we need to be able to have them at pace.

Liam Carroll

If we then turn more towards individuals, something I get quite worried about is individuals essentially handing over their own agency — as you say, self-colonising. And there are very structural problems that lead to this self-colonisation by smartphone. But there's also a sense of individuals doing that themselves, and a risk of us gradually disempowering ourselves, I suppose. I'm curious: what do you think individuals can be doing to preserve their own sense of agency? And what do you think a healthy coexistence looks like?

Anna Goldsworthy

That's a really good and big question. I was struck by the fact that I had an interview with Sam Altman, in which he was identifying the risks of AI, and alongside the usual suspects that have been canvassed here today, like the existential risk and the future of work and so on, he said he's actually alarmed about another thing, which is essentially the abdication of will or the capacity to make decisions. And we see this already in a lot of young people, not only young people using ChatGPT or Claude to script conversations with boyfriends or girlfriends or landlords,

but he said, what happens when the president of the United States can't make a decision without running it by an AI, or knows that he or she can't make a good enough decision without running it by an AI? Now, a lot of people might say, well, given the current president of the United States, that can only be an improvement. But I think it's a bigger question, which is: how much do we wish to surrender? What we're talking about here is our agency. And that's something to look at collectively.

But as you so rightfully say, it's also something I think we need to address in our own lives. And we've already seen this. I can't read a map to save myself. I never could, but Google Maps has made that even worse. And we've surrendered our capacity for arithmetic, often because of our reliance on calculators. And the argument might be that's not a problem, because it frees up more mental space to do other superior, more lofty things. And yet I don't think there's that much evidence that suggests that it does.

So in terms of our own life, I suspect we possibly need to start cultivating friction. In the same way that the industrial revolution freed a lot of people from the need to do physical labour, and that coincided with the advent of weight training and personal fitness, quite possibly we're all going to need to address our brains more consciously towards problem solving, towards difficulty. Or the option, I guess, is we just all hook ourselves up into catheters and surrender into virtual reality, blissful parallel universes and just surrender the whole project of existence to the machines.

Liam Carroll

That's very well said. You write very beautifully throughout the essay about the question of meaning, and meaning in the face of AI, and this line that I love is: one of the many ironies of AI is that the least human thing we have ever made is forcing us to look more deeply at ourselves than any of our other inventions. So supposing that we do develop something like AGI — whatever definition you want to take of that — that can functionally do nearly all of the things that us humans usually do, but better, faster, cheaper, etc., then what?

In the words of Keynes, how do we live wisely and agreeably and well?

Anna Goldsworthy

That's the million-dollar question. And what remains for us? There are a couple of places we can look to for examples. One, as Keynes identifies, is the independently wealthy, who haven't always distinguished themselves with the decisions that they make about how they choose to live their lives. But retirees are another category. What are the ingredients of a good retirement, if we're all going to essentially have this enforced retirement? The evidence is pretty clear. It's about having a sense of meaningful activity. It's about relationship. We're not, presumably.

Or maybe we are — that's another risk I address in the essay, again, outsourcing relationship to the AIs, that human connection is meaningful embodiment. It's meaningful process being a creature of time. I think art is suddenly even more meaningful. Certainly for me personally, I had a recital last night with a beautiful tenor in Adelaide, of some songs by Schubert and piano works by Schubert. And somehow, in the shadow of all of this, those very human activities of making art together in real time, for me, have an even more kind

of incandescent value. It's about being in a room with other people, having conversations, I think. Again, it's about leaning into our humanity. The answer is not going to be the same for everybody. But there are certain things that we do know that human beings flourish around, and a really big part of that is it's relational.