Why Does Paul Christiano Want AI Labs to Pause Progress?
Paul Christiano says weak security at AI labs makes the US lead over China small, and argues labs should pause pretraining and RL work until it improves.

Why does Paul Christiano think AI labs should pause algorithmic progress?
Paul Christiano argues that it is not good for US AI labs to be improving pre-training and reinforcement learning given their current security posture, so every new algorithmic improvement they make is one a rival could also obtain. He says that makes further progress on pretraining and reinforcement learning a net negative for US national security right now. His proposal is to pause that specific work until security is adequate, the labs can defend against theft, and they can tell whether they still control the systems they build.
TL;DR
- Christiano says two recent incidents, one at OpenAI and one at Anthropic, showed that fairly unsophisticated outsiders could reach core internal systems where algorithm secrets live.
- The "12 hours, not 12 months" line came from the interviewer, and Christiano agreed with the logic: if secrets leak, the lead over China shrinks to roughly zero.
- His proposal is narrow: pause improvements to pretraining and reinforcement learning algorithms, not all AI work.
- He estimates fixing security would take at least months, and quite possibly 6 to 12 months.
- He says the decision is huge and needs a broad public discussion, and that no consensus exists yet.
- He stressed he was not speaking as an OpenAI board member, a role he confirmed he holds.
What were the Hacktron and Cathedral incidents?
In the conversation, both incidents are described as examples of how exposed the labs are. Christiano said that both Anthropic and OpenAI, in the recent past, had "extremely unsophisticated actors" who appeared to get enough access to extract the vast majority of algorithmic developments. One example was access to Slack, where research is discussed. Another was access to the monorepo, the single shared code repository where most of a company's code is kept and edited.
The interviewer gave the specifics:
- Hacktron, a security startup, reportedly gained within a couple of days the ability to view OpenAI's internal source code and to make pull requests (proposed code changes) to its monorepo.
- Cathedral, another security startup, reportedly got access to all of Anthropic's Slack within 12 hours.
The interviewer hedged on the details ("I think"), and this article relies on the conversation rather than separate reporting. Take the specifics as described, not independently confirmed. The point is the pattern: outsiders with modest resources got close to the crown jewels.
What does "12 hours ahead of China" mean?
People often debate whether the US is 6, 8 or 12 months ahead of China in AI. The interviewer's argument was that the real number is closer to 12 hours. If a foreign state can steal the secrets behind a new model, it lands at the same frontier as the American company almost immediately. A lead built on secrets only lasts as long as the secrets stay secret.
Christiano agreed the incidents point that way, but he added a broader reason. Even without the two high-profile demonstrations, he said, state actors are very sophisticated compared with typical technology companies, and the security of rapidly growing startups is not at the same level. He also described this as a hard moment for cybersecurity in general. Defenders face a large number of vulnerabilities being discovered across the whole software stack.
What exactly would be paused?
Christiano was careful here. "Pause can mean a lot of things," he said, and he noted that labs are doing many good things that they should keep doing. The target is one category: improving algorithms in ways that spread easily and could be quickly stolen.
In his words, the most important current development is further gains in the efficiency of algorithms for:
- Pretraining, the first, very large training stage where a model learns from vast amounts of data.
- Reinforcement learning (RL), training that rewards a model for good outputs, used to sharpen how it behaves.
- Improved architectures, the underlying design of the models.
The interviewer summarized it as stopping further capabilities progress on pretraining and RL until security and alignment problems are handled. Christiano's own framing is that, in the current regime, accumulating more of these innovations is "on net bad" for American national security.
Why would more progress be bad for national security?
Christiano lays out three conditions that, together, make continued progress risky:
- There is a significant risk of very rapid capabilities progress.
- There is a significant chance the resulting AI systems can't be controlled.
- If developers slowed down, the algorithms they already have could leak to an adversary who might use them more carelessly.
The interviewer illustrated this with a hypothetical. Imagine a lab is at "GPT-9" and fears the next step, "GPT-11," could be catastrophic if misaligned (meaning its goals don't match what humans want). The lab wants to wait and make sure GPT-11 is safe. But if a rival state can easily take the GPT-9 secrets or weights (the trained numbers that make up a model), the lab can't wait. The rival could use GPT-9 to build GPT-10 and then GPT-11. The lab would feel forced to race ahead with a system it can't fully trust.
That scenario is an illustration, not a forecast. Christiano's reply was that this is the situation he's worried about, and that it is the most likely path into a bad spot he's described elsewhere.
How long would it take to fix lab security?
Christiano said it's hard to give a firm number because the situation is unprecedented. Normally, defending adequately would require interventions that take a very long time. But he believes powerful AI tools have made some big projects far more feasible, such as rewriting large amounts of software or drastically changing how a company operates.
His estimate: "probably at least on the order of months, quite possibly more like 6 months or 12 months." That is the security timeline, and it's where the "pause" gets its rough length.
He treated alignment differently. He said you may not need an "indefinitely scalable" solution. What you mostly need is the ability to tell whether things are going off the rails, and to respond appropriately. That has a technical part (assessing what is happening and whether your control is adequate) and a governance part (who decides what to do and how).
What is stopping a pause from happening?
The interviewer pointed out that Christiano sits on OpenAI's board and asked what the bottleneck is. Christiano confirmed the board role but said he wasn't speaking in that capacity. His answer was that any decision like this is dramatic and has major national security implications. So the discussion should reach well beyond people inside AI companies.
He said that discussion has not produced a solid answer. There may never be consensus, but the first bottleneck is simply having the conversation, and he thinks it's needed "extremely urgently." Notably, he did not say a single lab could or should act alone, and the source doesn't describe any lab agreeing to pause.
Is the argument airtight?
No, and Christiano doesn't claim it is. A few honest caveats:
- The incident details came from the interviewer's recollection, not a full account.
- The argument depends on beliefs about how fast capabilities will grow and whether control is possible. Those are contested.
- A pause by US labs only helps if rivals aren't racing ahead on their own. Christiano's argument is about not feeding them, but how others would respond isn't covered in the source.
- He framed the pause as lasting until specific conditions are met, not as a permanent halt.
What does this mean for you?
Most readers don't run an AI lab. But the underlying lesson applies to any workplace: the most valuable information a company has tends to sit in chat tools and shared code, and those are where attackers look.
If you work at a company that builds or uses AI:
- Treat Slack, Teams and shared drives as places secrets can leak. Don't post credentials, customer data or unreleased plans in broad channels.
- Use multi-factor authentication everywhere it's offered, and report phishing attempts instead of deleting them.
- If you manage a team, review who has access to what, and remove access people no longer need.
If you follow AI policy or influence decisions at work:
- Expect "should the leading labs slow down?" to become a mainstream debate, with national security arguments alongside safety ones.
- Watch for concrete signals: labs announcing security overhauls, government involvement, or conditions for resuming work. Those are the markers Christiano's own proposal points to.
If you're simply trying to understand AI news:
- Be wary of simple "who's ahead" scorekeeping. Christiano's point is that a lead only matters if it can be protected.
Can you build your own access-review tracker for this?
The most practical takeaway is knowing who can reach what. Many teams track that in a spreadsheet nobody updates. With Remy, you can describe a small internal app in plain English, such as an access register listing each system, who has access, who approved it, and when it was last reviewed. Remy builds it as a full-stack app with sign-in and role-based access, so managers see what they should and nobody else does. It also keeps an audit log. It won't replace a security program, but it replaces a spreadsheet that tends to go stale.
Frequently Asked Questions
Who is Paul Christiano?
In this conversation, he is identified as someone who sits on OpenAI's board. He said clearly that he was not speaking as a board member. The views described here are his own, as stated in the discussion.
Did Christiano say the US lead over China is 12 hours?
The "12 hours" line came from the interviewer, who contrasted it with the usual 6, 8 or 12 month estimates. Christiano agreed that the incidents are an indication of the problem. His own stated argument is that weak security makes a lead less useful.
Does he want to stop all AI development?
No. He said pausing "can mean a lot of things" and that labs are doing many good things. His specific proposal is to pause further algorithmic progress on pretraining and reinforcement learning, which are easy to spread and steal, until security and control are adequate.
How long would the pause last?
He didn't give a fixed date. On security, he estimated at least months, quite possibly 6 to 12 months. The pause would end when labs can defend their work, can cope with not racing at maximum speed, and can tell whether they control their systems.
Is there agreement among AI labs on this?
According to Christiano, no. He said the discussion hasn't reached a solid answer, there may never be consensus, and the first step is to have the conversation urgently and widely.
Useful for
How we source, write, and correct our articles. Editorial standards
Related articles

Are AI Cyber Models Behind the Bank and Airline Outages?

What Is an AI Harness? Parts and Why It Matters

How Does Nod Keep Agent Memory and Payments Safe?
