OpenAI's New Policy Arm Says Power Concentration, Not Rogue AI, Is the Long-Run Risk

OpenAI launched AI Futures on August 20, run by a newly formed Strategic Futures team, built around one question: how does a free society keep its structure intact — individual rights, distributed agency — as AI systems become genuinely transformative. The inaugural post reaches back to James Madison’s warning about “parchment barriers,” the idea that a written constitution alone can’t restrain concentrated power without institutions built to enforce it. OpenAI’s argument is that power-concentration risk, not any single technical failure mode, is the most serious long-run category of AI safety risk.

The mechanism they’re worried about is specific: political power has always depended on the cooperation of human labor — soldiers who follow orders, bureaucrats who administer, taxpayers who fund. Autonomous systems and machine intelligence could let a state project force and collect revenue without needing that broad human buy-in, which is the bargain that currently gives people and institutions a seat at the table even under governments that aren’t fully democratic. Erode that bargain and you erode the leverage ordinary people have.

This isn’t happening in isolation. Anthropic hired Tino Cuéllar — a former California Supreme Court justice and past president of the Carnegie Endowment for International Peace — as its first Chief Global Affairs Officer on August 4, a role built to manage government relationships as AI governance tensions intensify. Weeks earlier, Dario Amodei staked out Anthropic’s own position on a related axis: he rejected calls to ban open-weights models outright, but named authoritarian governments building more powerful models for military dominance or domestic repression as one of exactly two national-security concerns he considers real.

Two frontier labs are running parallel tracks on the same worry — one publishing philosophy, one hiring for it and drawing policy lines around export controls and mandatory safety testing. For consulting engagements advising boards on AI governance, that convergence is worth naming: labs are treating political-economy risk as seriously as technical risk, and clients should expect that framing in regulatory conversations before it shows up in a product release.