AI & National Security for Middle Powers

As Greg mentioned, I'm Doctor Austin Wyatt. I'm the research leader, responsible AI and military adaptation, for RAND Australia, which is the Australian subsidiary of the RAND Corporation, which is a major think tank in the US and the UK. So I'll do the standard disclaimer. Everything I say today is in a personal capacity as a researcher. It's not necessarily reflective of the views of the RAND Corporation or the Australian or US Department of Defense. And I can see there's a few people in the room who I've been following in this space for a long time, so if I screw up, I'm sure you'll hear about it in the Q&A.

So on we go. Thank you very much to the conference presenters as well. Most of our discussion when we talk about AI, and particularly when we talk about [inaudible] AI and the civilian sphere, focuses on frontier models and focuses on the next big thing that's going to come out. It's focusing on our race towards artificial general intelligence, etc. And there's nothing necessarily wrong with that, but from the perspective of someone that works in the more defence and national security space, it does create a potential issue, which is that as we focus on these frontier models, we run the risk of not paying enough attention to the risks and potential benefits of outdated models, which, even though they're not as capable, they are increasingly capable.

And as we've seen, when the frontier lab decides that they've got their next big thing, they move all their resources off the last one and on to the next one. And so increasingly, we're seeing these models come out, get released into the world, get copied, get played with, get broken, with less [inaudible] engagement at the government level or at the corporate level than these frontier models. And for someone like myself who has done work in disruptive innovation in the military space, this concerns me, because we've seen with prior innovations — or we're starting to see — adopters are adopting good enough capability at a lower price point.

So the conventional narrative for why we need to be pushing really hard for advanced autonomous weapons systems and military applications of AI is a similar one to what we hear in the commercial space, in that there's this idea that if we don't push for these technologies, we're going to be outgunned, we're going to be outmanoeuvred. There's a myth around that. It's almost like unilateral disarmament, the idea being that if our systems are operating under meaningful human control and the adversaries aren't, that we are somehow never going to catch up in military terms.

And having heard that a lot in my discussions, I wanted to sort of push back on that. And this conference presentation comes out of a paper that's forthcoming in a special issue. Well, I have to wait to get the reviewer comments back. So maybe you'll hear some of them— ones you'll hear today, but I'll hear them in writing, which is always fun. And it's similar to the logic here in the commercial space: we've got to compete, we've got to be the best. Push, push, push, push, push, push.

Don't worry about ethics. And the problem I have is that I think that we're missing the fact that especially for middle powers, there is actually benefit in responsible AI and there's benefit in these ethical restrictions. And part of the conversation I wanted to have today is not only how can we push, when we don't own the tech stack, for ethical restrictions and responsible AI restrictions to be embedded in these technologies, but also how it can be conceptualised as an enabler, as a strategic advantage, not as a restraint.

So why is this important to be dealing with now, and particularly in our region? Well, technology doesn't wait for policy, right. There's been discussions ongoing in this international community for well over a decade now, and that's just the current batch of AI discussions. As I'm sure you all know, we've had AI discussions for a very long time in very different versions. But we are seeing that even though we haven't had the progress that we would have wanted to see towards binding international regulation, the technology is accelerating and is continuing on, and so is adoption.

We're seeing this in our region. So at least seven of the ASEAN states have sophisticated remote or semi-autonomous platforms in development or in use, and particularly in the naval space. So Thailand, Indonesia, Singapore, for example, they all have established indigenous production capability for remote operated — granted, UAS — capability. Vietnam and Malaysia are trailing behind them. And we're seeing this democratisation of precision effect happening obviously in the Ukraine, but we're also seeing it elsewhere as we go through. And when we see, for example, like we saw with the Iran US conflict, where a drone measured in costs of tens of thousands of dollars is able to cripple a global economy or saturate a multi-billion dollar warship in theatre, then that's something we need to be dealing with.

So the problem for us is that we don't own the technology stack. And there's a number of arguments around it by going after a fully sovereign technology stack — for example, the South Koreans, and I'll talk about their experience in a second. But even if we can't, if we assume that we can't get there, and I think there's a good argument that we can't necessarily get there across the whole stack, there are ways to get to sovereign agency. And that's one of the things I want to talk about, the challenges that we are a technology taker in this respect.

And we think about the pillars underpinning AI, or Paul Scharre's four battlegrounds of AI, for example, around models and data and compute infrastructure, human talent — we don't have dominance in any of those. We have ability to compete in some of them, and we do lack those resources. So for example, look at Australia. We have a high reliance on US military technology. Something along the lines of 83% of our military imports come from the US, which obviously creates a level of reliance that we need to be aware of.

I've left my notes behind and that's never a good sign. Anyway, we've also got the fact that we need to deal with— so we're the 13th largest economy, fourth largest importer of US arms, but we're the 17th largest arms exporter. Which means that the typical way that people go after trying to build an indigenous capability in the military space, subsidising with exports, is really hard. In Australia's case, we're starting from a low floor, a low start point, and this creates some potential risks around things like path dependency.

So I wanted to look at this idea of sovereignty without the stack, which is obviously a very simple and catchy way of saying something that is actually very complicated, but I'm going to stick with it. So there's two potential avenues that I talked about in my paper from an individual state perspective for going after this problem. The first is about trying to generate strategic control over critical nodes, essentially trying to arbitrage our way through. If we can't get to full independence, then let's see if we can find that balance.

So complete dependence leaves us vulnerable, therefore we need to apply the logic of arbitrage. We need to deliberately allocate resources, limited as they are, to identifying where in the supply chain we can create advantage and leverage, [inaudible] and we can then use that to create assured access to the rest of the supply chain. Right. Obviously that's really easy to say and really hard to do in practice, but that's why I'm an academic and not a policymaker. So for example, South Korea. They announced with a lot of fanfare and a lot of money that they were going to go for a full sovereign tech stack in Korea.

Where they've sort of landed is more of what they're calling a cooperative technology stack. You can see in there that building technology in their sovereign capabilities, they've identified that they can do. They're trying to go after compute. They've known that they have currently got this stranglehold on memory chip manufacture, and they're leveraging both of those to stand up decent capabilities in other areas of the technology stack, but really focusing on building that leverage up. And they're cooperating with allies in other parts. That's that second track.

And you see it with things like the US Korea dialogue on AI standards. You see it with Stargate and SK involvement with some of the frontier labs, for example. This is the other approach, which is what we have sort of looked at in Australia, which is essentially trying to distribute assurance around the device as opposed to within the device, so to speak. And this is what they call a system of control. Originally we saw it presented by the Australian government in 2019. But essentially we're arguing that if you put it into a regulatory framework, then you can limit the unknown risk, because you can't see into the quote unquote black box.

Right? If you can't control the algorithm and you can't control the data, weights and things, then the argument goes, you can put in place structures around it that limit that risk. So political, legal — for example, the requirement that a system has to go through an Article 36 legal review process, that's the international humanitarian law assessment, before it can enter into service. In that regard, there are some technical things we can do, but also around operational doctrine and training — training our humans to understand better the technology, to understand better the potential in its environments that it's going to be placed in, in order to essentially try and limit the left and right of arc, or the harm capability of that system if it operates in a way that we weren't anticipating it to.

So that's one approach. Obviously, there are those issues with all of these, but that's another approach that I've seen. But another idea is to work in coalition, now, as a middle power. We all love coalition except in practice, but we try. So as an individual, it's hard for us to go against a frontier lab, for example, or a US government, and get them to move their decision-making process, whether that's geopolitically or market based. But if we were to gather together among a small group — so, for example, I just looked here at Australia, South Korea, Japan and Singapore.

Just between these four states, we account for about 20% of US arms exports. So that's a decent market leverage, right? The challenge then becomes, how can you convert this into leverage on a military producer, on an exporter, on a government? Part of that is either leveraging that through procurement rules, saying you can only sell into one of these four militaries if your system meets this criteria that we've established, this technical benchmark or this test or this requirement — but that's the market power way of doing it.

The other thing I talked about in the paper was around how we could leverage our credibility, let's say, in the international rules-based system such as it exists, to try and leverage these countries into abiding by norms and rules as part of gaining access to these systems, because, again, we operate in cooperation with the US. And when we look at how our military planning is done, it's almost always in collaboration, in coalition operation, with the US. But it's the same from their end. They're not planning to fight a conflict alone if they don't have to.

And if we give them an opportunity to bring us along, right, we can put conditions on that. And this is part of the argument here, that if you can combine your leverage, then we go for that. Now, it's important— all this sounds really wonderful. It's hard, and we've seen that. So even amongst the closest allies — and I think you could say AUKUS is probably one of the closest aligned groups, and present company anyway — we see those differences. So let's look at AUKUS and let's look at military AI.

So within the AUKUS countries, there are multiple definitions. There's at least six, by my last count, definitions of military AI, what it means. And the Article 36 review processes don't line up either. Australia has one, the UK has one, the US has four, [inaudible] one for each service branch. And then even that doesn't work sometimes. So you can see how there would be friction even among the AUKUS partners, even around something as limited as, for example, a collaborative combat aircraft. Never mind when you try and bring it larger.

We've seen with, for example, the European Union, where there are a bunch of problems with trying to do anything in cooperation. So part of what I've said here is that one of the things that we should try and do around that is to hold our own domestic firms to those same standards, but be willing to limit the scope of that agreement away from things that commonly cause disruption and disagreement within coalition partners — things like work share, IP sharing, export markets. These are all things that will have a political cost, particularly for a government like Australia's, to accept, because you are leaving money on the table and domestic industry will be very cross.

But I feel like that is one way to try and firm up such an approach. I now just want to talk about— since we've talked about how we can protect our agency, I want to talk about this other argument that I was making at the beginning, about leveraging AI to gain military advantage, not just to prevent risk. So this is my pet project of sorts. And if you've heard me talk before, you've heard me talk about middle power offsets, because I wrote a book on it and I talk about it all the time, and people still listen, which is great.

And captive audience. But essentially what we're arguing here is that unlike when a great power goes into an offset strategy, a middle power strategy is smaller in scale and smaller in purpose. Smaller scale, because we don't have the resources to compete on something like, let's build our own aircraft carrier, or let's build a massive arsenal of carrier killer missiles, for example. But it's also narrower in purpose. Our purpose is to achieve regional advantage, regional deterrence. It's not to become a global hegemon. And so it focuses here around things like a focus on asymmetric deterrence, prioritisation and arbitrage.

That's a really hard one, particularly for Western democracies, on picking losers. Everyone wants to work. Picking winners will be even worse, picking losers. And then organisational and doctrinal agility. Because I argue, as we move past the frontier of these models, and particularly in AI and autonomy, but in other things as well, the entry barriers drop. And when the entry barriers drop to any military capability, the advantage doesn't come from the technology anymore. From a military perspective, it comes from your ability to be superior in operational deployment and the operational praxis — how you use that technology to obtain advantage.

You have to be, as an institution— you require a greater level of agility and a greater ability to regenerate that novel approach to using things in combat than an adversary. That gets into a military adaptation literature. It's hard to talk about at length when you're not listening to me talk about this, and then we get here. So, responsible AI as a strategic enabler. So we come down to this argument that we heard before, that I mentioned before, around the point that if we have human meaningful control and the adversary doesn't, then we'll lose because they have operated at machine speed.

That only works if you agree with all the technology. Maybe it only works if you don't have human machine teaming, because humans, as we've seen in a number of the research papers that have been done particularly recently, they require the ability to have that right level of trust to avoid risks on either side. If you over-trust the system, you end up in automation bias. Everybody knows about that here. Obviously, that's a big problem when it comes to military decisions. But also under trust. If you under trust that system, what we see is soldiers just don't deploy with it.

Right. Or if it's a decision support system, they don't trust it. And that defeats the whole purpose. And so you need to be able to find that right level of— calibrated trust is just what I'm calling it here, but it's got a lot of different names. It allows you to give these operators, these humans, the ability to trust that these systems are operating the way they're supposed to. And it's not a coincidence that the things that lead to the right level of trust also just happen to be the things that are responsible AI principles, right?

Explainability, transparency, reliability — these are all things that help find that right level of trust. And training: you can actually help your soldiers and sailors and airmen get used to the technology, even if it's not top tier technology. Experiment with it, learn with it, and figure out where the best place to put their trust level is. It's also obviously pretty important for interoperability. The risk, for example, of a decision support system trained on data that is derived from a US military, for example, is not necessarily going to reflect the conditions, the attributes or the equipment of an Australian equivalent.

The obvious example of this is if you look at the data used for anti-submarine operations, for example, in Europe versus the Pacific, it's just completely non applicable. Right. The environmental conditions are too different. You see the same thing with the Five Eyes, for example. In conclusion, because I've seen the big blue thing that says five minutes, I'll stop talking. I think that the main thing here is that we shouldn't be waiting for frontier labs and great powers to be making the decision on what we do with military applications of AI.

We certainly shouldn't be waiting for them to take steps to ensure that we can get the military advantage out of this technology, without opening ourselves up to some of the potential risks involved here, both from a human perspective and from a psychological perspective and an ethical perspective. But I don't think we have to. And I think the three things I'm really pointing to here is, whether we operate alone or in tandem, leveraging our procurement, our dollars, to influence these would are, at the end of the day, mercantile operators.

Operational advantage via finding that right balance of teaming, finding that right balance of trust, and having that experimentation which is going to be so important to building that capability on the basis of how we use the technology. The only way you get there is your operators are good at it. And then around leveraging our ability to use normative leadership. It's not going to be easy, though. As I said, we will need the political will to transcend the fact that there is inertia here. We'll have to go faster than we typically like to, and there will be institutional resistance to some of this stuff.

And there needs to be willingness to accept political critique domestically, because it's not— I don't think it's appropriate. I'll leave you this anecdote. And I said before that I'm opinions of Austin, not the Defence Force. But if you look at how we talked about Ghost Bat, which is a collaborative combat aircraft — it was going to be purely surveillance, purely surveillance, purely surveillance. Oh, look, we fired a rocket at Woomera, right? That's not the way to go about getting the population onside with effective and responsible uses of AI in the military.

We need to be having these conversations with the public, with our Defence Force personnel, early, and accepting the political critique that comes with that, because that's the only way we can leverage our normative power to push these labs towards things that we think are important in the responsible AI space. Thank you.

Greg Sadler

Thanks. We probably have time for one question. So I think there's general interest in this idea of middle power coalitions. So I suppose the question is about what that looks like in the context of AI and supply chains, as opposed to military questions. So who could Australia be forming a coalition with around compute supply chains or other AI? Who are obvious allies here?

Austin Wyatt

I think the one that immediately comes to mind is the South Koreans. That said, I think that you could also look to Europe and then, to a lesser extent, to the Southeast Asian nations. I think that we're going to have to go along with this application nations and diplomatic partnership anyway, so we may as well try and bring them along and leverage their ability. Indonesia, for example, is one of the highest levels of commercial adoption of AI in the business community in the world.

Greg Sadler

Okay, well, that's probably time. So thank you everyone for coming. And we'll give another round of applause to—