Practical steps towards democracy fit for a world with AI

Luke Thorburn

So I've got two goals for this talk. On sort of an ordinary level and a meta level. The ordinary level is that I'm going to try and make the case for deliberative processes as having a role in governance of AI, right. And then the second is more meta to sort of model a way of thinking about governance strategy and design, which I think is more broadly applicable. So happy to talk about any of them. As a brief introduction I'm Australian, I'm from Melbourne. I've recently finished a PhD in London.

I'm now living in Boston where I split my time doing a postdoc at MIT with Michiel Bakker, and half of my time as a research fellow with the AI and Democracy Foundation. Most of my research, historically, has been on societal impacts of recommender systems, particularly with respect to, conflict risks, risks of conflict escalation. But I also do a lot of tech governance in general. And I'm fortunate to have been involved in some fairly high level deliberative processes, including as part of peace processes, in particular in the lead up to the ceasefire in Israel and Palestine last year.

Just briefly about the AI and Democracy Foundation as well, we're I think the only org that sits at the intersection of the sort of deliberative democracy community, or whole ecosystem and the AI policy community. Broadly, we think of our work in three buckets. There's figuring out what governance mechanisms are needed with respect to democratic, influence over how AI goes, and then there's sort of a supply and demand side. The supply side is ensuring that democratic infrastructure and mechanisms are good enough to meet the moment. And then the demand side is trying to ensure that the case is being made sufficiently clearly that these mechanisms are needed and will be adopted

where they are. So there's sort of an outline for the talk on the left that I'm just going to work through it. To start with I'm going to briefly review the types of governance challenges that come up in AI governance. And I suspect much of this will be familiar to many of you so I'm not going to dwell on it too thoroughly. But broadly, there's four buckets I want to flag. The first is power concentration. So there's risks that a small number of companies or countries in which those companies are located might end up controlling the majority of our infrastructure that underpins economic activity, state capacity or influence over the public sphere.

There’s incentives to make context available to AI systems, which creates surveillance infrastructure which can be abused. And there's the emergence of automated organisations or the organisations with a much higher proportion of automated labor to the number of humans. And this, reduces the effectiveness of a lot of the implicit guardrails that sort of govern our behaviour at the moment. At the moment, if you want to have a large impact on the world, you generally need to convince a lot of other people to help you do that.

And that's going to be less the case going forward. There's a bucket of risks related to coordination and collective action. These include risks related to race dynamics, whether those are real or perceived, leading to corner cutting on, risk mitigation measures. There's risks of proliferation of bio and cyber capabilities. And all of these are particularly acute if the nature of the risks is such that you need a minimum floor of safeguards that are effectively implemented everywhere. In game theory, this is called a minimum effort game or a weakest link game.

Which means you need sort of complete coordination amongst all the relevant actors (which is particularly hard to achieve). There's a cluster of risks related to disempowerment. So if economic activity becomes increasingly driven by AI systems rather than humans, governments might become increasingly dependent on capital rather than human constituents for tax revenue. Information and culture might be increasingly driven by AI systems that mediate those phenomena. And as we use AI systems our capabilities might atrophy in various ways, our beliefs might become less accurate, our value judgments might shift away from those we actually hold, and our actions might become misaligned with our values.

I'm not saying this is certain, but it's a risk. And then finally, there's risks associated with having a degraded information environment, As AI undermines, at least in the short term, the frictions that currently signal effort, authenticity, accuracy, sincerity, proof of humanity, which we use to navigate the world and to make governance decisions collectively. And it also means our ability to form common knowledge. It's increasingly less the case that, just because I have some sort of textual or audio visual evidence for something, I can't trust that you will look at that and understand that it's true in the same way that I know it to be true

or believe it to be true. And, there's a large body of literature on the importance of common knowledge of that sort for coordination. So those are some governance risks just to have in the back of your mind. What are we to do about all these risks? I think there's lots of concrete things that might come to mind to help address some of them. I think it's fair to say there's a huge amount of uncertainty about what kinds of institutional structures might be necessary to provide anywhere near a sort of comprehensive response to these risks.

And there's a lot of uncertainty about which risks might manifest or be more or less significant. That is a general sort of strategic posture. I think it makes sense that the nature of the risks is relatively more certain than the types of investments we want to make to address those risks. And even more certain again than the sort of stable institutional structures, if we ever get to those that might provide some sort of equilibria that's manageable. And so I think the responsible thing to do or the strategic thing to do in this context is to invest in flexible governance building blocks that can be used across many different future institutions or possible futures institutions across many different scenarios, and preserve optionality

in how we govern AI. In that context, I'm going to, in this talk, make the case for one particular building block, representative deliberative processes. Some of you might be familiar with these, but I'm just going to give a brief overview for those who aren't. So you can think of these processes as combining a population microcosm with an information microcosm. And by that I mean, the sort of prototypical example of these takes a random sample of the population who are going to be subject to some policy or decision, and places them in a context where they have the time and resources to learn from each other and deliberate with each other.

And also hear from experts and stakeholders and come to a considered deliberative decision about what should be done in that context, how risks and tradeoffs should be drawn. These take many different forms. They vary in terms of whether the process is offline or in person, whether it's online or some hybrid of the two. The number of people involved, the duration of the process ranging from a few hours to months where people come together every other weekend for an extended period. They vary in the output shape of the process.

These processes are quite flexible. They can produce budgets or recommendations or sort of reviews of some ballot proposal. And I'll talk about some of those later in the talk. And they also vary in terms of how the process is structured, which is called the process design. And there's people who are skilled in designing processes like these so that they're good (and I'll say a bit more in a second about what makes them good). Such processes have a long history. Versions have been used in ancient Athens and Renaissance Italy and Venice.

But more recently, there's been, over a thousand that have been used. They're used by governments of all levels. They're also used by companies including banks and tech companies and public utilities. And, they're fairly well established. I think it's fair to say that they're variable in quality at the moment, but we do have in some places in the world the capacity to run them really well. And it's mainly at the moment, a dissemination problem and capacity building problem to expand that capacity to other parts of the world and, parallelising.

What makes a deliberative process good? A good process designer and a good facilitator will ensure things like: the participants are informed about the topic, that they are able to substantively deliberate about what should be done, the process should be accessible to all participants, the that the process is not disproportionately influenced by any small subset of the participants, that the process actually does what it was asked to do (it delivers on its remit and that the process), and the outputs of the process are actually feasible or implementable.

A common failure mode, if they're not done well, is that a government might ask for a set of recommendations and they're all far too expensive or they can't be done at the same time, and it just becomes a sort of wish list of what people might want. But the participants didn't really grapple with the tradeoffs involved. But that's avoidable with good process design. So if you think about deliberative processes as a kind of governance module, this goes to my broader point about how to think about governance strategy in general.

But deliberative processes as a module have particular affordances that allow you to do things in the world. And so this is a sort of incomplete and imperfectly formulated list of some of the things that deliberative democracy enables you to do. But for example, it enables you (sometimes, if they're done well) to navigate through gridlock by creating a sort of forcing function for making a decision, diffusing the responsibility for who is credited with that decision, creating a mandate for the ultimate decision maker to act and to move forward (or if these are politicians providing sort of political cover for them to do something).

And there's a whole bunch of other, sort of enablers there. This is an example for deliberative democracy. You can imagine any other governance mechanism that you can think of would have its own set of affordances that it enables you to do. And to some extent, I think the task before us is to find what combinations of governance mechanisms, appropriate or have the right set of affordances for addressing the natures of the risks that we and governance challenges that we face. But to flesh this out a bit more for deliberative

processes, I'm going to give three examples of what this might look like. And before I do that, I'll just qualify briefly: this is very realpolitik or instrumental as a sort of motivation for using some sort of democratic process. You might credibly desire democracy for much more intrinsic or normative reasons. But the examples I give here are going to be very instrumental in terms of this is a way of using democracy that allows you to avoid certain outcomes. And, sort of regardless of the intrinsic desirability of making a decision democratically.

So the first is tied to the and the affordance of enabling action. So organisations may struggle to take action for multiple reasons (and by organisations here I'm including governments). And those reasons can include the action being costly for decision makers. They might be punished by certain stakeholders in some form for taking that action. While they don't themselves bear the costs of inaction (so their incentives are misaligned), there might be bureaucracy that makes it harder to take action than they had sort of willpower or desire for change can overcome.

And the benefits of taking action can often be speculative, which means generating buy-in and momentum can be difficult. So to make this very concrete, say, for example, (and I'm not taking a stance on this, but I'm using this as an example of a sort of governance design) you believe that on merit, Australia should build more data centres (as Emily was talking about) but the blocker for that at the moment is that Australian copyright law doesn't allow training frontier models in Australian data centres. My understanding, and I haven't been following this super closely,

but my understanding of the situation is more or less that there's a few large copyright holders who would not be happy if copyright law was “weakened” to allow this to be possible. And those copyright holders are essentially the media, and so, that's one reason why risk averse politicians would not make this decision. And but it's also a policy area where regular people don't really understand the nuance and the long term strategic implications, potentially. So if that's the situation, using a deliberative process might enable you to move forward with this.

If the government ran a national Citizens Assembly on AI and copyright. If the case for doing this was strong enough, it's a pretty high chance that the Citizens Assembly would decide to adopt to build more data centres in some form with maybe some rules about compensation and so on. And if the copyright holders were included as stakeholder inputs to this process, it would be hard for them to attack the legitimacy of that decision. And it would create a mandate for the government to move forward. That's a very concrete example.

But I'm making a more general point that when there's gridlock over some important decision, using a citizens assembly can help you move forward more quickly. Kim was talking this morning about the tension between speed and good governance. I think there's a misconception that democracy or making decisions democratically in general can be slow. I think gridlock can also be extremely slow. And using mechanisms like citizens assemblies can be a way in which we can, in some settings, actually move faster on making important decisions. And it's plausible, as we adapt to AI, there's going to be a lot of sort of big decisions like this that will need to be made in fairly quick succession.

The second example I'm going to use is the affordance of preserving optionality. So policy decisions made unilaterally or with insufficient consultation can cause politicisation, or some form of conflict of that sort, which reduces the space of possible future policy actions. So I think of climate policy, the pandemic, and including not only but sometimes by poisoning the well, where some decision was made too quickly by some particular political actor or decision maker which makes that type of intervention untouchable for a period of time. And this politicisation also makes nuanced policymaking in that area more difficult (which is perhaps a variant of the first point).

A concrete example. This plays out at all levels. It plays out in national politics. It also plays out in businesses and organisations and schools and universities who are making decisions about how AI is to be used or can be used within those organisations. And so the example I want to give here is some work that we have been doing funded by the Advanced Research and Invention Agency in the UK on using deliberative processes to produce formal safety specifications that an AI system must meet to be compatible with risk tolerances and normative trade off so that is deemed acceptable to a given population.

So this is a very poor diagram and the text is probably too small. But the essential idea is you have a deliberative process. And the thing that it produces (within a given policy area) is a formal statement of what trade offs and risk tolerances are acceptable for a model to have or an AI system to have. And this is stated with sufficient concreteness that you can formally check whether an AI system meets those criteria or not. So we think that this is quite a promising direction because standards based governance, specification based governance is such an established paradigm in high stakes technology domains.

But to the point I was making earlier about preserving optionality: if we can run these processes sufficiently reliably, this is a mechanism by which communities can make decisions for themselves about what standards an AI system needs to meet before they are subject to its decisions, before it's deployed in that context, and so on. The third example I want to give is avoiding drift, and I apologise in advance that this might be a little more US centric than the other two, but robustly aligning organisations with the public interest purpose is quite difficult.

If there are opportunities to gain significant profit or wield significant power, purpose tends to drift, usually due to inarticulacy in and how the purpose is measured or operationalised. So a canonical example of this would be recommender systems and social media. Facebook's mission was to connect people or words to that effect. But the way they operationalise that in the metrics they were optimising for (watch time, dwell time and so on) is a form of imperfect articulacy of that goal. Another cause of drift is self scoring by those responsible for pursuing the purpose or some form of capture by external actors.

And in the context of frontier AI labs (which all have these sort of public interest purpose corporate structures). If you talk to the people who design those structures, they're very frank that we don't know how to do this in a way that's super robust in the long term, or immune to capture. And so one thing I've been spending a lot of my time working on is the potential for deliberative processes to be used as part of public interest corporate governance structures or steward ownership structures, to provide a sort of a form of decision making about what constitutes the public benefit that the organisation needs to align to.

But it's also very hard to capture because the people making that decision are randomly selected. They don't have any significant personal stake in the decision or opportunity to gain power from deciding in a certain way. So that's another research direction which I think is promising. Feel free to come and talk to me about that. And finally, I want to briefly make the case that this general direction of increasing public agency over how AI goes is something that Australia is quite well placed to lead in. It would be quite a cheap thing for Australia to lead in in the whole AI policy discourse.

But you can if you imagine democratic infrastructure feeding into three levels of AI governance: alignment of AI systems, governance of AI organisations, AI policymaking at a regulatory level. We could, if we wanted to, advance all of these fronts. Briefly, we have the capacity to do so (particularly in the context of deliberative processes). We have some of the best deliberative practitioners in the world. You probably don't recognise these organisations, but in the deliberative democracy ecosystem, they're very well regarded. There's the largest concentration in the world of excellent deliberative practitioners is probably in Europe.

Australia might be second. I don't think that's a stretch to say. The US is underserved in this respect. The there's precedents in Australia. So, for example, all local governments in Victoria are required, and have been for quite a few years, to run deliberative processes on a regular basis. Utility companies in Australia are also required to do this kind of citizen engagement as part of their pricing strategies and how they prioritise infrastructure investment and so on. So there's experience in that respect. And finally, I think there's demand for it politically.

These are just two examples chosen for contrast. I'm not going to get into the specific details of these proposals. They're different to the kind deliberative processes that I've been talking about. But both of these parties on opposite ends of the political spectrum are arguably pushing for some more participatory forms of governance – and that could be built on. I'll finish there. These are some of my collaborators on this work. Would love any questions. Thanks for your attention.