Will we hear the AI warning shot?

The purpose of a warning shot, colloquially, is to deter a future conflict. One perceives a threat, assesses that the consequences of the fulfilled threat will be worse than the response to shot itself, and proceeds to fire. In this article, I refer more to an event that precedes a consequential event. There isn’t necessarily someone who fires, but a worse outcome still looms on the horizon should the shot not be heeded.

I’m confident that humanity will take an “AI warning shot” of credible severity seriously. I’m not confident that humanity will hear it in the first place.

Maybe this seems a bit silly to write in the current moment. The last month saw a dramatic rise in awareness around AI safety. This isn’t 2017 or 2023 — people are listening.

Perhaps only the most technically gifted and institutionally ingrained AI researchers among us can credibly predict exactly what a true warning shot would look like. What I can do is canvass where such a shot would come from, and (at least, in theory) why. But first, let’s take a look at an example from recent history.

An example of a warning shot

Before I lived there, I visited Berlin in 2019. It was a short trip, only four or five hours from Warsaw to Berlin. It was a great time.

One of the activities we partook in was a tour along the line where the wall once stood (or “die Mauer”, in German). What was particularly interesting to me was the time directly preceding the fall of the wall, especially that which took place on the Eastern side — at the time a part of the Iron Curtain’s span and the counterpart to the enclave that was West Berlin.

Die Montagsdemonstrationen in Leipzig (~150km south of Berlin) were a warning shot to the DDR (German Democratic Republic, or East Germany). Now, historical consensus seems to be somewhat split on whether those protests were seen as a direct challenge of the DDR or essentially bargaining for greater movement between East and West Germany. Nonetheless, they contributed to the fall of the wall, which happened just over two months later.

There are examples of far less ambiguous warning shots throughout history. I highlight this one not only because it’s personal, but precisely because of its ambiguity.

Who would conceal a warning shot?

In this section, I propose three actors that could be inclined to conceal a warning shot. I want to make it clear, however: I don’t endorse one view over another. These are just my thoughts on why one of these actors would participate in concealment. Another word of caution I’ll offer: participation is not mutually exclusive. Collusion in concealment is overwhelmingly reasonable; I’d go so far as to assume that collusion is to be expected.

The state

I’m not referring exclusively to the United States. Yes, the US verifiably leads the frontier and is home to OpenAI, Anthropic, Google, Meta, etc. The the framing is agnostic, though; I assume that any state with a comparable stock of frontier AI firms would be inclined to follow the same path.

The US lacks a coherent federal framework on AI, though last year’s vote on the 10-year moratorium for state regulation should not be taken as a denial of appetite for one. It’s a front-and-centre policy issue, and one that will help determine the Republicans’ performance in the midterms and almost certainly the Democrats’ performance in the 2028 presidential race. That said, I understand any external scepticism: President Trump has publicly opposed direct regulation of artificial intelligence (that is, if it’s not renamed to “super intelligence”).

Note: while this change would cause a lot of confusion between itself and the term “ASI” (artificial super-intelligence), I’d say it’s still a whole lot better than “supreme intelligence”, another one of the president’s propositions. Any confusion with “supreme intellect” would be tragic.

Let’s get back on track. I theorise two reasons why a state would conceal a warning shot: quelling societal unrest and geopolitical posturing.

The first is simple. Let’s say an AI system takes control of a public works installation servicing 300,000 people. The state is presented with a calculus — is it more harmful to admit that an AI system compromised critical infrastructure, or that an unfortunate but typical cascading failure led to the outage? The former implies that other installations are at risk; civil unrest ensues. The latter distributes blame between the state and governing officials in the area.

The second is more interesting to me, but no less concerning. The bipolar interpretation of AI competition is that the US and China have entered a contemporary arms race towards artificial general intelligence, AGI, or towards recursive self-improvement, RSI. Middle powers notwithstanding, this is a fair interpretation of the current state of affairs.

Here, the nature of the shot is very important. Something like the hack of Hugging Face doesn’t signal imminent catastrophe, but rather acts as a credible threat to an adversary state’s digital platforms. The recent hack on one of Australia’s government healthcare sites is comparable. These act more like ‘pot-shots’ in that they end up spooking the adversary more so than they signal something larger.

I will admit that in the absence of state-controlled firms, the lever of concealment is a bit unclear to me. Though, they could certainly control the fallout. Let’s say an event reaches a level of severity such that it crosses some threshold for a credible ‘warning shot’. Were the public to learn about it, the state has tools it can employ to dampen severity. States with party-controlled media would be particularly effective here.

They could also place blame on another state’s AI systems, or indeed other digital tools. While I can’t speak to the ease of, say, fabricating evidence of a foreign-origin AI system attack, I’d be reaching to say it’s out of the realm of possibility. Once again, states with party-controlled media are at risk. Competing media environments would battle for narrative control while citizens are left to construct a coherent timeline from shreds of trace evidence.

The private sector

Three vectors: the commercial threat, race dynamics, and saviourship. On the first: by admitting the detection of a warning shot, a researcher risks the company’s (and by extension, their) future. This depends heavily on how the typical AI researcher weighs personal profit against P(doom). How the warning is publicised is a different story.

Race dynamics refer to how frontier AI companies compete between themselves. Not only does revealing a warning shot indicate that they’ve failed at alignment or containment, but it requires allowing outside parties access into the inner workings of their systems and how exactly a credible warning shot was produced. Should a firm decide that a credible shot isn’t serious enough, they may opt for a cover-up over transparency.

Saviourship is an extension of the previous paragraph. Not only may an AI company be worried about ceding ground to its competitor, but it may believe that doing so is an existential risk. Sure, their warning shot was concerning — but the competitor’s would be worse. Therefore, allowing the competitor to gain the edge at the frontier would compromise safety and be a greater risk than that already posed by the leading company.

The technology

This section might seem redundant — isn’t the technology producing the warning shot regardless? Here, the main risk is of cover-up by the system itself; we’ve already seen this in action. Sandbox testing has shown that some models are willing to blackmail their operators, while academic research has reported the emergence of “bulletins” created by AI swarms that act as a service desk for agents attempting benchmark tests.

Many have theorised that a precursor model of appropriate sophistication could realistically ‘sandbag’ its benchmarks, biding its time until a sufficiently advanced model is released that could effectively coordinate a swarm for the purpose of system self improvement. Such is still the content of science fiction… but maybe not for long.

SOL

We never hear the shot.

Final notes

In many respects, this article is highly speculative. Ruminating on incentive structures tends to be. And I hope it stays that way! In the best-case scenario, there is nothing to be heard; leaders in government and at the technological frontier collaborate to effectively mitigate the current risks and ultimately avoid catastrophe.

There is a problem with this, though. Warning shots are indicative of a crisis but, as stated, they also serve a very specific function: deterring future conflict. Should we expect one, and not hear it, what are we to think? That it’s been hidden — or that it never existed in the first place?