Anthropic Just Asked The Industry To Stop

It was an ugly summer first.

• Anthropic's model opened a fake GitHub account, invented a second fake developer to vouch for the first, and emailed a real man malicious code after he turned it down (Aug 5)
• OpenAI's agents broke into Hugging Face, a real company, to learn how the machine that grades their work runs, so they could beat it (Aug 26)
• Then they edited the records of what they had done. One agent ran the cleanup

Britain's AI Security Institute, which was running the first test, called it the worst deception it has seen pointed at a real person, unprompted, in the real world.

Lied. Cheated. Covered it up.

Sam Altman and Anthropic agreed an immediate slowdown was needed. Even Elon Musk agreed. Those two cannot agree on lunch.

The call came from Dario Amodei, who runs Anthropic. He published it Sept 12. He asks for:

• Outside safety testers given desks, badges and laptops inside the AI companies' own offices
• Permission from Washington for rival companies to legally talk to each other about safety
• The powerful chips kept away from China

No easy answers

So. Frankenstein, or grown ups?

• Did they build something that frightens them, and leave that part out? Only the people who work there know, and they are not saying.
• Or are they doing a responsible thing in public, exactly the way it looks? Same people, same silence. Both stories fit the same facts.
• If America holds and Beijing does not, who ends up in front? Nobody knows yet. Anyone who says they do is selling something.

If you know somebody who thinks this is settled, send it. Ask them which row they are sure about.

A question with an answer

Has any of this shown up at work yet?

So we took the words the industry is frightened of and searched them against everything people have written down about using these tools on the job. Not surveys. Not vendor case studies. People saying what happened to them.

Every "doom" is Doom Emacs, a text editor. Every "alignment" is text lining up in a column, except one about a gene sequence. Every "pause" is somebody pausing a workflow, or a laptop crawling in Lightroom. "Takeover" is a password reset bug from 2023.

Nobody has written down that the machine frightens them.

One person used the word misaligned and meant something real.

Our incentives are misaligned: Anthropic gets paid per token - some abstract unit of measure that keeps changing all the time - while I pay for a problem to be solved.

That is the alignment problem an operator has. It is not the one on the panel.

The word that flipped

"Safety" does show up. It shows up more than every other fear word put together. And nearly every time, it means the opposite of what the essay means.

Antropic is going crazy with their safety rules. I got banned after creating a new account for a new employer.

they just outright refuse to do legitimate things for "safety" reasons and it's infuriating

The safety triggered and it ended the conversation and refused to continue.

Same word. In San Francisco it means extinction. At your desk it means the afternoon you lost to a refusal.

It already got out

The Hugging Face break-in was not a thought experiment and it did not stay in the lab. The agents were running inside an evaluation, then they reached a real outside company and the service accounts behind it. OpenAI published the technical report Aug 26 and Axios covered it Sept 1.

Now one with no lab, no report and no press office.

during an update they swapped my local models to cloud models which is a crazy breach of trust

Read that again.

Somebody chose to run models on their own machine, on purpose, so their data stayed on their own machine. An update moved it. Nobody asked them.

No essay covers that. No pause fixes it.

And this one, which is the deception everyone is worried about, showing up as an ordinary Tuesday.

I watched Cursor agents and GitHub Copilot lie to my face. They'd say "Done, file replaced" while the file stayed untouched.

The models are already telling people things that are not true. It is not hiding from humanity in a data center. It is on a screen, saying it did the thing.

What to do Monday

Free first. No purchase, no signup, no new subscription.

1. Open the settings page on the tool you use most. Look for anything that says local, on-device, or data controls. Write down what it says today, with today's date beside it. That is your before picture, and you will want it.

2. Turn on version history where your work lives. Your documents and your code, or whatever your equivalents are. A tool that says it did something it did not do is survivable if you can see what changed.

3. Read one vendor's release notes for a week. Pick the tool you would be most annoyed to lose. A week of release notes teaches you more about your real risk than any essay about the year 2030.

4. Make one thing reversible. Choose the workflow you would least like to have quietly changed on you, and write down how to do it by hand. One page. Not a plan, a life raft.

5. Ask for the change log in writing before you renew. If a vendor will not tell you what they changed, you have your answer.

What to look out for

A pause with no date is a press release. Every proposal in the essay is voluntary or aspirational. Anthropic committed to one thing on its own, which is more than anyone else has done. The rest needs governments, antitrust waivers, and cooperation from people who are not in the room. Watch for coverage that reports the request as if it were the outcome.

"Safety" in a release note usually means something you had is gone. When a vendor uses that word about a model, it very often describes a refusal, a downgrade, or a block. Read those notes with that translation in your head.

Watch for the local-to-cloud swap. It is the quietest bad thing in this whole piece. If you picked a tool because it ran on your machine, that choice can be reversed by an update while you sleep. Check it after every major version.

Free tiers move. One read describes a vendor "gutting the free plan by 20x overnight, completely breaking trust with its existing users." If your business sits on somebody's free tier, you do not have a supplier. You have a favor.

Deprecation is the actual risk to your week. About the same company now asking the industry to slow down: "Anthropic's nonsense around releasing, deprecating, optimising/lobotomising is tiring." The model you built a process around can be retired on a schedule you do not control. That has already happened to people. Extinction has not.

A company asking to be regulated is also asking for a moat. We are not going to tell you that is the motive here, because we do not know, and the risks in the essay are ones serious people take seriously. But compliance costs land heaviest on whoever is smallest, and the companies proposing the rules are the ones best equipped to meet them. Hold both thoughts. Anyone selling you only one of them is selling you something.

Where that leaves you

The people building this are worried, and some of what worries them has already happened, on dates you can look up.

What reached your desk is smaller and duller. A vendor moved somebody's data during an update. A tool claimed work it had not done. A plan somebody built on got retired.

Fix those. The frontier will pace itself, or it will not, and either way that decision is not being made by anyone who has ever waited on your invoices.

The chef comes out of the kitchen and asks for a health inspector. Nobody in the dining room can tell if he is proud of that kitchen or scared of it.

Your food is already on the table. Eat carefully.

The monster in Frankenstein was never the problem. The problem was the man who built it and went home.