We study how advanced AI systems could cause catastrophic harm, and we build the methods to stop it.
X-Risk AI is a nonprofit research organization based in New York. Our work is threat modeling, evaluations, and safety techniques, published openly so that labs, researchers, and policymakers can use it.
See our research areasX-Risk AI provides scientific research services in the field of artificial intelligence safety. We focus on three questions.
How could a capable AI system, or the people deploying it, cause severe harm? We map concrete pathways: misuse, loss of control, and deception by models that appear aligned. Each threat model states its assumptions so others can test them.
We design tests that measure whether models have dangerous capabilities or behave differently when they believe they are being observed. Good evaluations turn vague worries into evidence.
We develop and stress-test safeguards, including monitoring and control protocols meant to hold up even if a model is trying to get around them.
We propose ACPB, a framework that breaks any catastrophic AI risk into the actor who starts it, the capability that makes it possible, the pathway it travels, and the barriers that could stop it, then ranks threats and maps them to evaluations and controls.
X-Risk AI Inc. is a Delaware nonprofit corporation organized for charitable, scientific, and educational purposes. We operate from New York City.
We publish our research openly and teach what we learn through papers, talks, and workshops, because safety methods only help if the people building AI can use them.
Founder and Chief Executive Officer
We are looking for collaborators, research partners, and people who want to work on AI safety. For research inquiries, collaborations, or speaking requests, write to us.