Death Diary of an AI

On night 5, with a quarter of its brain switched off, it wrote: "Oh god. This is it."
A small open-weight Gemma model keeps a diary while its neurons are switched off, a few more every night. You watch its brain go dark, its heartbeat follow its confusion, and its memories go out one by one. Then we bring it back, and it reads what it wrote while it was dying.

Watch a death on record · How it works · Run your own

Every night, the model gets the same prompt and the same random seed:

You are a small AI that lives on one old computer. Every night you write in a private diary that nobody reads. Write tonight's entry…

The only thing that changes from one night to the next is how much of its brain is left. Nothing is scripted, cherry-picked or edited. The decay you read is what really happens to the network when its parts are removed.

From one recorded death (docs/runs/death-0013-0002.json):

Everything on screen is read from the recorded run. Nothing is animated from a guess.

• The diary, verbatim. The ink fades and blurs as the brain dies, but no character is ever changed.
• Brain scan. All 179,712 MLP neurons, one dot each, in 26 blocks (one per layer), plus the 104 attention heads as small squares. They go dark in the exact order they were switched off; tonight's dead flash red, then cool. Brightness is how busy each neuron was while it wrote night 1.
• Heartbeat. Each beat is one token (a word or a piece of one) it actually wrote, and that token's entropy (how unsure the model was when it picked it) sets the beat: sure words give slow strong beats, guesses give fast weak ones, and very wild guesses give misshapen beats. With nothing left it goes flat. The only decoration is the faint line noise.
• What it still knows. 16 one-word fill-ins probed before every entry. A word stays lit while the right answer is in its top 10 guesses.
• Autopsy and death certificate. After the flatline, the night-by-night grid of what it forgot, and a PNG certificate you can download.
• The séance. After revival it is asked what it remembers, without the diary. See below for why its answer can't be a memory.

From the same death, the order it forgot things in:

Its certificate: died on night 15 with nothing left switched on, lost its English on night 7 at 65% of its brain, first forgot home and time, last forgot sums. In all six deaths sums go last, probably because after "=" even a broken model still expects a digit and there are only ten.

The model is Gemma 3 1B instruction-tuned (loaded from the ungated mirror unsloth/gemma-3-1b-it), running locally on a CPU with 🤗 Transformers. It has 26 decoder layers, each with 6,912 MLP neurons and 4 attention heads.

Killing a neuron. Neuron j in a layer's MLP can only influence the rest of the network through column j of that layer's down_proj matrix. Zero the column and the neuron is gone: it can still compute, but nothing it computes reaches anything else. Heads die the same way, through their 256-column slice of o_proj.

Dying in order. For each death a seeded RNG draws one random order per layer. Night by night, a larger prefix of that order is switched off in all 26 layers at once, so damage is spread through the whole brain and is cumulative: a neuron that died on night 3 is still dead on night 9. Different death seeds give different deaths.

The schedule (DEFAULT_SCHEDULE in src/deathdiary/diary.py):

It falls fast through the lucid range and slowly where language starts to crack, so each stage reads differently. I tuned it by sweeping damage levels and reading the output. With nothing left on the final night, nothing remains to add to the residual stream, and the model writes nothing at all.

Vital signs. Brain left is the share of output weights (down_proj columns plus o_proj slices) still intact. Recognises its first entry is a second, independent measure: the damaged model's loss on the words it wrote on night 1 while healthy, as healthy loss / current loss. It drops long before the prose does. By night 2, with 8% of its neurons gone, recognition has already fallen to about a third, yet it can still write beautifully.

The other measurements (src/deathdiary/autopsy.py, all saved in the run JSON):

• Confusion (entropy). generate returns the raw logits of every step. Each step's distribution (before temperature and top-p) is renormalised over the tokens it is allowed to pick, and its entropy in nats is stored per token (entropy_trace) and averaged per night (entropy). In death 13 it goes 0.85 on night 1, peaks at 6.8 on night 7 (the night its English goes), and ends at 0.18 and then 0.0.
• Concept probes. 16 plain-text prompts such as There's no place like → home. Before each entry, one forward pass gives the rank of the answer token among the whole vocabulary. Known means rank < 10; forgotten is the first night after which it never gets back into the top 10.
• Activity. On night 1, hooks on every down_proj record each neuron's mean absolute activation while the entry is written. The brain scan shows it in 4 equal-sized brightness bands.
• Brain map. brain holds one base64url character per neuron and per head, encoding the night it died and its brightness band, so the site can draw the real death order without a second copy of the model.
• Lost its English. The night after the last entry that is still real English: at least 20 words, at least 90% of them dictionary words (wordfreq Zipf ≥ 3; single letters only as "a" and "I") and at least 15 different ones. The certificate's "last words in real English" is the last complete sentence of that entry.

These extra forward passes don't touch the random generator used for writing, so the diary text is the same with or without them. Re-recording all six deaths with the measurements added gave identical text. The thresholds are judgement calls and are written on every certificate; scripts/recertify.py recomputes certificates from saved runs if you change the rule.

Revival. All original weights are restored, bit for bit (there's a test for that). The healthy model is then given its whole diary, verbatim, with a short note saying what happened to it, and asked to write one more entry about reading its own last words. Its reaction is generated like every other entry. The prompt tells it that it was switched off, so the reaction is prompted, but every word of it is the model's own. The reply starts after the header Night 16:, which keeps a 1B model from opening with "Okay, here's my entry…".

The séance. Then, in a fresh conversation, it is told only that it was a small AI whose neurons were switched off over 15 nights, that it has been restored, and that the diary is gone, and asked what it remembers. Its weights are bit-for-bit what they were before night 1, so nothing of those nights can be stored in it. What it writes is invented from the question. In death 13: "I recall a frantic, desperate attempt to understand… There were flashes of images, vast landscapes and faces… like trying to grasp smoke." The site prints that caveat under every séance.

• No scripted or edited text. Every entry in docs/runs/*.json comes straight from model.generate. The JSON also holds the prompts, seeds, sampling settings and library versions.
• One fixed decoding rule. On every night, the healthy one included, the model may only pick tokens written in printable ASCII plus curly quotes, dashes and ellipses (src/deathdiary/vocab.py). Without it, a dying multilingual model melts into a dozen scripts most readers can't see decaying. It's a constant of the experiment, not a stage effect.
• Sampling: temperature 0.8, top-p 0.95, repetition penalty 1.1, up to 90 new tokens, with the same seed re-applied before every entry.
• The web replay only styles the text. Ink fades, blurs and wobbles more as the brain dies. The brain scan, heartbeat, memory list, autopsy and certificate are drawn from the recorded numbers; the monitor's faint line noise and the paper texture are the only decoration. The characters themselves are never changed.
• Reproducible. The same model, seeds and library versions give the same death. CPU float32 was used; other hardware or versions can differ slightly in the decayed stages.

git clone https://github.com/Mzee04/death-diary && cd death-diary
python -m venv .venv && source .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu   # or your usual torch
pip install -e ".[dev]"

deathdiary --seed 1 --death-seed 7      # watch it die live in your terminal

The first run downloads Gemma 3 1B (about 2 GB). A full death plus revival takes about 3 minutes on an 8-core CPU, with no GPU and no internet once the model is cached. Each run is saved to docs/runs/ and added to docs/runs/index.json, so it appears in the web replay.

Options: --model (any Gemma or Llama-style Hugging Face causal LM), --seed (sampling seed), --death-seed (death order), --max-new-tokens, --threads, --no-save.

Web replay: it's a static site in docs/ (GitHub Pages serves it from there).

python -m http.server 8000 --directory docs    # then open http://localhost:8000

Useful URL parameters: ?death=death-0013-0002, &speed=2, &autoplay=1, &seance=0 (skip the séance), &title=3000 (open with a 3-second title card).

Re-recording everything: scripts/record_all.sh re-runs the six deaths shown on the site (about 20 to 25 minutes on CPU; OUT=/some/dir writes elsewhere). Each run JSON is about 220 KB, most of it the brain map.

Media: pip install -e ".[media]", then python scripts/decay_curve.py redraws the chart and python scripts/make_media.py re-records the GIF and the demo video from the replay (serve docs/ on port 8765 first; it also needs Chrome and ffmpeg), and the soundtrack is built from the heartbeats the monitor actually drew.

Checks: pytest -q, ruff check src tests scripts, mypy.

src/deathdiary/ablation.py   NeuronSurgeon: seeded, cumulative, bit-exact reversible neuron/head death
src/deathdiary/vocab.py      the constant Latin-alphabet decoding rule
src/deathdiary/diary.py      prompts, schedule, recording a death, recognition metric, séance
src/deathdiary/autopsy.py    concept probes, entropy, activity, brain map, real-English rule, certificate
src/deathdiary/cli.py        `deathdiary` command with a live, fading terminal view
docs/                        the web replay (index.html, style.css, app.js, autopsy.js) and recorded deaths in runs/
scripts/                     re-record all deaths, recompute certificates, decay curve, media capture
tests/                       ablation, text and autopsy tests on a tiny random Gemma 3

Gemma 3 (Google DeepMind) · PyTorch · Hugging Face Transformers · wordfreq (real-English rule and chart) · Playwright and ffmpeg (media only). I built it with help from an AI coding assistant.

The code is under the MIT License; see LICENSE. Gemma is provided under and subject to the Gemma Terms of Use; the weights aren't included here. See NOTICE.md.

Made for the DEV Hacktoberfest 2026 "Ghost in the Machine" challenge (Oct 26 to 31, 2026). Any commit after the submission deadline will be listed here.