Resources
Start here
- Video
AI Just Became Humanity’s Biggest Threat
Kurzgesagt · 2026
(opens in a new tab) - Videos
AI in Context
80,000 Hours
(opens in a new tab) - Guide
agi.fyi: Make Sense of the AI Situation
Lightcone Infrastructure · 2026
(opens in a new tab) - Podcast
We Can’t Lose Control of A.I.
The Ezra Klein Show · 2026
(opens in a new tab) - Article
Why AI risks are the world’s most pressing problems
80,000 Hours · 2026
(opens in a new tab)
Where this is heading
Alignment difficulty
- Series
AGI safety from first principles
Richard Ngo · 2020
(opens in a new tab) - Paper
The Alignment Problem from a Deep Learning Perspective
Richard Ngo, Lawrence Chan, Sören Mindermann · 2022
(opens in a new tab) - Article
The Persona Selection Model: Why AI Assistants might Behave like Humans
Sam Marks, Jack Lindsey, Chris Olah (Anthropic) · 2026
(opens in a new tab) - Article
From shortcuts to sabotage: natural emergent misalignment from reward hacking
Anthropic · 2025
(opens in a new tab) - Article
Reward Hacking in Reinforcement Learning
Lilian Weng · 2024
(opens in a new tab)
Power concentration and better futures
- Paper
AI-Enabled Coups: How a Small Group Could Use AI to Seize Power
Tom Davidson, Lukas Finnveden, Rose Hadshar (Forethought) · 2025
(opens in a new tab) - Paper
Preparing for the Intelligence Explosion
Will MacAskill, Fin Moorhouse (Forethought) · 2025
(opens in a new tab) - Series
Better Futures
Will MacAskill (Forethought) · 2025
(opens in a new tab)
Keep up
Next steps
Fellowships and programmes, roughly in order of how much background they need.
New to AI safety
- OAISI Intro Fellowship OAISI
- BlueDot Impact coursesCourse · Online, cohort-based
- 80,000 Hours career advisingAdvising · 1:1 calls
- ML4GoodBootcamp · In-person bootcamps
- Global Challenges ProjectWorkshop
Know the basics, want a first project
- Apart Research hackathonsHackathon · Weekend research sprints
- SPARPart-time, remote, mentored research
- FIG (Future Impact Group)Part-time, remote
- MARSCambridge AI Safety Hub
- AI Safety CampRemote research teams
- Talos FellowshipEU AI policy
- ARBOx Bootcamp · OAISI
Some research experience
- ARENABootcamp · ML engineering skills, London
- ERA CambridgeSummer research fellowship
- Pivotal Research Fellowship
- LASR LabsTechnical research, London
- PIBBSS FellowshipPrinciples of Intelligence · for researchers from non-CS fields
High context
- MATS
- Anthropic Fellows ProgramAnthropic
- CLR Summer Research FellowshipCenter on Long-Term Risk · s-risk focused
- GovAI Summer and Winter FellowshipsGovernance
- IAPS AI Policy FellowshipPolicy
- Astra FellowshipConstellation
- Horizon FellowshipUS policy · US only
- Tarbell FellowshipAI journalism
For generalists
- Gen StreamKairos · London, 1 week
- Generator ResidencyResidency · Kairos & Constellation · Berkeley, 3 months
- MATS Founding & Field-Building trackFor founders and field-builders
Get involved
If you’re new and interested in learning more about Oxford’s AI safety community, we’d love to have you sign up for our mailing list or join us at an upcoming event.
If you have questions about AI safety or OAISI, you’re also welcome to reach out — we’re happy to meet for a coffee, go for a walk, or have a quick call.