Infrastructure for AI Agents
The world needs external protocols and systems that shape how agents interact with institutions and other actors.
by Seth Lazar, Alan Chan, Kevin Wei, Sihao Huang, Nitarshan Rajkumar, Elija Perrier, Gillian K. Hadfield, and Markus Anderljung
published by on May 10, 2025
Transactions in Machine Learning Research
More work from Carnegie
Artificial IntelligenceAs artificial intelligence (AI) changes how people around the world live and work, new frontiers for international collaboration, competition, and conflict are opening. AI can, for example, improve (or detract) from international cyber stability, optimize (or bias) cloud-based services, or guide the targeting of biotechnology toward great discoveries (or terrible abuses). Carnegie partners with governments, industry, academia, and civil society to anticipate and mitigate the international security challenges from AI. By confronting both the short-term (2-5 years) and medium-term (5-10 years) challenges, we hope to mitigate the most urgent risks of AI while laying the groundwork for addressing its slower and subtler effects.
Digital Technology, Strategic Adaptation, and the Outcomes of Twenty-First Century Armed ConflictIf digital technology is truly transforming the nature of armed conflict, why aren’t these advances leading to decisive victories?
- Nate Allen
The California Report on Frontier AI PolicyThe innovations emerging at the frontier of artificial intelligence are poised to create historic opportunities for humanity but also raise complex policy challenges. As the epicenter of global AI innovation, California has a unique opportunity to continue supporting developments in frontier AI while addressing substantial risks that could have far-reaching consequences for the state and beyond.
- +20
- Rishi Bommasani,
- Scott Singer,
- Ruth Appel,
- Sarah Cen,
- A. Feder Cooper,
- Elena Cryst,
- Lindsey Gailmard,
- Ian Klaus,
- Meredith Lee,
- Inioluwa Raji,
- Anka Reuel,
- Drew Spence,
- Alexander Wan,
- Angelina Wang,
- Daniel Zhang,
- Daniel Ho,
- Percy Liang,
- Dawn Song,
- Joseph Gonzalez,
- Jonathan Zittrain,
- Jennifer Chayes,
- Mariano-Florentino (Tino) Cuéllar,
- Li Fei-Fei
The Joint California Policy Working Group on AI Frontier Models- How Some of China’s Top AI Thinkers Built Their Own AI Safety Institute
The emergence of the China AI Safety and Development Association (CnAISDA) is a pivotal moment for China’s frontier AI governance. How it navigates substantial domestic challenges and growing geopolitical tensions will shape conversations on frontier AI risks in China and abroad.
- Scott Singer,
- Karson Elmgren,
- Oliver Guest
Rethinking EU Digital Policies: From Tech Sovereignty to Tech CitizenshipThe EU’s pursuit of tech sovereignty has often sidelined the role of democracy in the digital sphere. The union should adopt a tech citizenship strategy that promotes citizen engagement, democratic innovation, and accountability.
