Agency vs Inference: LLMs, or AI as some like to call them, are token generators. For a model to “break containment“, “send an email” or perform any other action outside of generating a token, it needs a functional bridge (APIs, tool-calling, or a shell environment) to perform these actions. If it “escaped“, that means a human either misconfigured or deliberately enabled it to perform said task.
Anthropic’s Mythos model, which without doubt is impressive, found a 27 year old bug in OpenBSD for example. But that doesn’t mean it should be thought of as being “too powerful” for public use. This is both a fear mongering tactic and a way to boost company value right before the IPO. It is massive compute infrastructure running heuristic analysis, something which static analysis tools have done for years, just with far less “natural language” flair and computing resources.
There is a long history of “regulatory capture” carried out by many large companies. By claiming their tech is a digital nuclear weapon, they justify high valuations and lobby for laws that make it impossible for smaller, open-source competitors to exist. If it’s “too dangerous” for us (peasants), only the giant corporations get to keep it.
Reading Dario’s posts as an engineer is not only painful, it feels like they’ve rebranded “we gave our agentic framework too many permissions” as “the AI is becoming sentient and dangerous” to juice up the hype right before their IPO.
When an AI CEO pivots from talking about parameters to talking about “consciousness” or “digital souls” they aren’t talking to engineers; they’re talking to the FCA/SEC and venture capitalists.
- The power of linguistics: By calling their LLM “sentient” or “conscious,” they move the conversation from Product Liability and Copyright Infringement, to Existential Risk. It’s much easier to debate “AI alignment” with a terrified politician (that likely wouldn’t be able to tell you the difference between a PDF and Python script) than it is to explain to a judge why they scraped a private codebase or copyrighted content without a license.
- The compliance rug pull: If Anthropic or OpenAI can get laws passed that require every model to undergo £50M “safety audits” or “consciousness certifications” they effectively execute a Distributed Denial of Service (DDoS) attack on Open Source. A dev in their bunker (bedroom/garage) can’t afford a compliance department.
- Automated plagiarism: If we strip the “soul” or “powerful” narrative away it reveals the true reality: these are high-dimensional lossy compressors of human effort. The “intelligence” is just the statistical average of billions of human-hours of writing and coding, repackaged as a proprietary product.
We should be proposing regulating the data acquisition rather than the “vibe” of the output; this is exactly what the industry giants are terrified of. If they had to prove a legal chain of custody for every terabyte in their training set said “miracle” would suddenly look like a massive liability on a balance sheet.
Simply put they have essentially carried out “Money Laundering for Data“. They take “dirty” (unlicensed) data, run it through a transformer, and out comes “clean” (proprietary) weights.
Our focus should be looking at a “Data Provenance Act” that mandates a transparent audit trail for training sets which would be quickly collapse the “too powerful to release” marketing bubble.