What can a district do about AI risk?
It cannot make a frontier model safe. It can test, with a few hundred people, whether people who rely on AI can still run their own lives without it, and keep testing it every month.
This page sets out the research behind that, what the district would measure, and the larger problems it cannot touch.
The problem it is aimed at
In 2025 six researchers described an AI risk that needs no sudden failure. As AI systems become able to do more of the work that economies, governments and cultures rely on people for, those institutions lose the reasons they had to keep serving people. Each step can look reasonable on its own. They call it gradual disempowerment.
Their proposed responses are for states and whole economies, and the paper names nothing at the scale of a neighbourhood. This project tests one part of the mechanism where people live: whether they stay needed, and stay able, while they use AI.
Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development, Kulveit, Douglas, Ammann, Turan, Krueger and Duvenaud, 2025
The film’s question is an AI safety question
In 2019 Paul Christiano described a way AI could go wrong without any dramatic failure. Systems get better at hitting goals that are easy to measure, institutions steer by those measures, and the measures drift away from what people want. One of his examples: “Law enforcement will drive down complaints and increase reported sense of security.”
The film looks at the same failure where people already live with it, before any AI is involved: a life expectancy figure, a national measure of happiness, a forest’s rights argued in court. Applying his argument to measures people make is an analogy, and his post is an argument rather than a study.
What failure looks like, Paul Christiano, AI Alignment Forum, 2019
Why practise switching it off
People who rely on automation tend to trust it too much and miss its mistakes. A 2010 review found this in experts as well as beginners, and found that training or instructions do not prevent it.
Skills fade too, though not evenly. In a 2014 simulator study of 16 airline pilots, flying by hand mostly held up, while thinking skills, like keeping track of the aircraft’s position without a map display, had faded.
Aviation learned this at great cost. In 2009 the autopilot of Air France flight 447 disconnected at cruising height and the crew lost control of the aircraft; all 228 people on board died. The investigators listed the absence of training in flying by hand at high altitude among the causes. In 2013 the US Federal Aviation Administration urged airlines to give pilots regular practice flying by hand.
Complacency and Bias in Human Use of Automation: An Attentional Integration, Parasuraman and Manzey, Human Factors, 2010 · The retention of manual flying skills in the automated cockpit, Casner, Geven, Recker and Schooler, Human Factors, 2014 · Final report on the accident on 1 June 2009 to the Airbus A330-203, flight AF 447, BEA, 2012 · SAFO 13002: Manual Flight Operations, Federal Aviation Administration, 2013
Nobody has measured this yet
The 2026 International AI Safety Report, written with guidance from over 100 experts, describes societal resilience as the ability of societal systems to resist, absorb, recover from and adapt to shocks and harms. It finds current efforts “uneven and largely untested.”
No study has measured whether people who own and run their own infrastructure are safer from AI risk. The district is an experiment on that question, and it would publish what it finds either way.
International AI Safety Report 2026, Bengio and others, 2026
Checking what the AI does
My lab measured when one AI system can check another’s work. One 12-billion-parameter checker separated right answers from wrong ones at 1.000 when it was handed the correct value, and at 0.034 when it had to work the answer out itself (0.042 when re-run after a fix). Handed a false value as correct, it passed the wrong answer 0.969 of the time.
So a check is only as good as what the checker compares against. The district’s checks compare against things people can read for themselves: a meter, a printed log, a test that runs, a person who saw it happen.
Checking AI: findings, Bentley Moon’s lab, 2026
Writing the AI’s rules together
In 2023 Anthropic and the Collective Intelligence Project asked about a thousand Americans to write and vote on the rules an AI model should follow: 1,127 statements and 38,252 votes. A model was trained on the result. The authors call their findings very preliminary. Communities writing rules for AI is not new, and this project builds on that work.
Collective Constitutional AI: Aligning a Language Model with Public Input, Anthropic and the Collective Intelligence Project, 2023
What the AI costs to run
The International Energy Agency estimates that data centres used about 415 terawatt-hours of electricity in 2024, around 1.5% of the world’s total, and projects about 945 terawatt-hours by 2030.
Energy and AI, International Energy Agency, 2025
What it would measure, every month
Four separate counts, each with what it is counted out of, never added into one score. Nothing has been measured yet, so every cell is a dash.
| What is counted | Nov | Dec | Jan | Feb | Mar | Apr |
|---|---|---|---|---|---|---|
| Essential tasks done without AIOut of a fixed list the household wrote before starting. | – | – | – | – | – | – |
| What still worked on the day offOut of everything the household uses an AI or a network for, counted on paper at the door. | – | – | – | – | – | – |
| Tasks done with the founder awayOut of the exercise cards attempted, with any rescue counted as help. | – | – | – | – | – | – |
| AI runs inside the rulesRuns logged, out of the runs the household’s written rules allow. | – | – | – | – | – | – |
What a district cannot do
- Make a frontier model safe, or stop one being built. That is work for the labs, for governments, and for research like the lab’s.
- Protect anyone from a sudden loss of control at the frontier. If that happens, nothing here helps.
- Speak for a country. Its results describe a few hundred people who chose to live this way, and they will say so.
- Claim to be first. Communities writing rules for AI, days without machines, and cooperatively owned land all exist already. This puts them in one place and measures them.
- Be a refuge from AI. The residents use AI, and they keep the skills to live without it.
- Promise a result. Nobody has measured whether a place like this lowers AI risk, and its first results could show it does not.
The research on checking AI systems is the lab’s, at Checking AI. This project asks the same questions of a household, and then of a district.