Holden Karnofsky is a visiting scholar at Carnegie California. His research focuses on international security risks from advances in artificial intelligence: what the most imminent risks are, how to prepare, and possible early warnings (e.g. from AI capability evaluations).
Holden previously served as co-founder and CEO (and later co-CEO) of Open Philanthropy. Open Philanthropy has been one of the largest philanthropic funders of both AI risk reduction and biosecurity and pandemic preparedness since 2015. It also works in a number of other areas including global health R&D (including work toward universal flu and syphilis vaccines, hepatitis B cures and malaria gene drives), land use reform (it was the first institutional funder of the YIMBY movement), and farm animal welfare (where its grantees have won thousands of commitments for improved animal treatment).
Prior to that, Holden co-founded and served as co-Executive Director of GiveWell, whose public charity recommendations direct hundreds of millions of dollars per year.
He is married to the president of Anthropic, an AI company, and has financial exposure to both Anthropic and OpenAI via his spouse.
AI is a rapidly developing industry where pragmatism and dynamism are key. An approach prioritizing early release and iteration may be the best hope to reduce risk at a satisfactory pace.
Six possible future AI capabilities that could deserve advance preparation and pre-commitments, in order to avoid catastrophic risks.
If-then commitments are an emerging framework for preparing for risks from AI without unnecessarily slowing the development of new technology. The more attention and interest there is in these commitments, the faster a mature framework can progress.
