This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Every account you forget about is a ghost in the machine: it still has your email, maybe your address, maybe your card, and it keeps haunting servers long after you stopped visiting. Twelve years of signups later, I wanted to find mine and lay them to rest, without handing my inbox to anyone.

Unhaunt reads only the headers of my Gmail signup emails, finds the services I have accounts with, checks them against public breach records, finds out how each company lets you delete an account, and sends a deletion request from my Gmail, but only after I sign in to Auth0 again to approve it.

The short version:

• 215 services found in 793 signup emails (12 years), in 2 minutes. 4 were breached after I signed up.
• On 50 services I labelled by hand, the lazy rule ("no email in 2 years = ghost") was 84% right. My local open model, Gemma 4 E2B, was 54% right and would have deleted 23 accounts I still use. The lazy rule would have deleted 6. Neither gets to decide: I mark every ghost, and every send needs a fresh Auth0 sign-in.
• Finding how to delete: Ultramarine and Gemma tie at 57% on 14 checked services (guessing privacy@domain: 21%), but every quote Ultramarine gave was really on the company's page (Gemma: 80%). The whole run cost $0.00265005, and the account balance at /v1/usage dropped by exactly 2,650,050 nanodollars: the same number, to the last digit.
• Of the 6 accounts I most wanted gone, only 1 could be deleted by email. The rest hide behind web forms, dead sites, or no route at all. 3 real deletion requests sent (that one, plus two other ghosts whose own sites list an email address), each after its own fresh Auth0 sign-in, 9–14 seconds from "confirm" to "sent".

The census on my real inbox, in the app's redacted view (?redact=1): no names, only ids, dates and outcomes.

Demo

The repo runs on your own inbox in about 10 minutes of setup (Auth0 tenant, Google OAuth client, ollama pull gemma4:e2b-it-qat): see the README. My own data stays on my laptop (data/ is gitignored), so there's no hosted demo of my inbox. The numbers in this post come from one real run on my inbox; npm run eval recomputes them from that run's cached outputs, which stay on my laptop because they contain my inbox metadata.

Code

github.com/klee1611/unhaunt · MIT. npm run eval scores any run against your own labels. Tests: npm test (58). Evidence: docs/evidence.md, failures: docs/failures.md.

The account and ghost labels are mine. The deletion-route answer key was built by the agent, blind to the models, and is marked as such. Every sign-in that approved a send was mine.

How I Built It

1. Census. Gmail is reached through Auth0 Token Vault: the app never sees a Google password or refresh token, only a short-lived token for headers (From, Subject, Date). Bodies are never fetched. Senders are grouped by company domain (English and Chinese signup subjects).
2. Classify. Gemma 4 E2B, running on my 8 GB M1 MacBook through Ollama, sorts real accounts from newsletters and suggests ghosts. My inbox never leaves the laptop except to Google itself.
3. Breaches. Each domain is joined against the free public Have I Been Pwned breach list. No email lookups.
4. How to delete it. Ultramarine reads the company's public privacy pages and returns the route with a quote. Code checks the quote is really on the page, and an email address is only used if it sits inside that verified quote, on the company's own domain. Sites that block scripts are read from the Wayback Machine.
5. Approve and send. I mark the ghosts. For each deletion request, Unhaunt sends me to sign in to Auth0 again (prompt=login, max_age=0); the server sends only if that sign-in happened after I asked and within 2 minutes. One sign-in, one email. Token Vault supplies the Gmail token for the send.

Each box maps to a file in the repo. Two Auth0 moments: Token Vault lends a read token for the census and a send token only after the fresh sign-in.

The numbers

Is this a real account? (50 services, frozen with a fixed seed before any tuning, labelled by hand)

Ghost, or still in use?

How do I delete it? (20 services; 6 excluded because the site was down or geo-blocked; answer key built blind by a second agent reading each company's own pages before seeing any model output)

What surprised me, and what failed

• My AI lost to the lazy rule. 13 of the 23 accounts Gemma wanted to delete had emailed me this year. In 18 of 23, its reason was some form of "only welcome / verification emails". For a service that had emailed me five days earlier, Gemma wrote: "The only email is a welcome email from a long time ago." The census only collects signup-type subjects as evidence, and Gemma judged those subjects instead of the last-email date in the same prompt. I didn't tune it on the test set: that would make the number meaningless. It just never decides.
• Thinking cost 28× the time (one probe, and the first call also loaded the model). With reasoning on, one classification took 34 s and 549 tokens. With reasoning_effort: "none": 1.2 s, 22 tokens, same answer. That made 430 calls feasible (19 minutes).
• Gemma's confidence is always 0.5, 0.9 or 1. Useless as a signal, so the post doesn't show it.
• Bots aren't welcome at the graveyard. Canva, Adobe and Fiverr return HTTP 403 even on their homepage to scripts. The fix: read the newest Wayback Machine copy, dated, so an old policy is marked as old.
• Cloudflare hid the address from both models. One privacy page states its deletion email plainly in a browser; the HTML says [email protected]. Ultramarine answered "email" with a verified quote, but no address, so my guard had nothing to keep.
• My own guard cost Ultramarine a point. It found the right address, but its quote stopped just before it, so the "address must be inside the verified quote" rule dropped it. It failed safe: a lost right answer, never an email to an unverified address.
• Google's Gmail quota is 6,000 units/minute per user for new projects, not the 15,000 in the docs, and googleapis bundles its own HTTP error class, so my instanceof retry never fired. Both cost me a run.

Limits

• One person's inbox. 50 labelled services, 14 scored deletion routes: small samples.
• The deletion-route answer key was built by an AI agent reading company pages, blind to the models; not checked by hand by me (no time before the deadline): treat those numbers as indicative.
• Service names are redacted.
• A bug I found too late to re-run: the census gave every service the same email count (201, from Gmail's unreliable resultSizeEstimate), and that number was in Gemma's prompt. The ghost comparison carries that caveat; the count is fixed in the code now.
• You can't run it on your own Gmail through my deployment: Google only allows test users for unverified apps. It runs locally on yours. This is designed for the privacy concern.

Privacy and taking it all back

Disconnect Gmail revokes the grant at Google and then deletes the Token Vault connection (Auth0 notes that deleting the connection alone doesn't revoke the provider's tokens). npm run forget -- --yes deletes everything stored locally. The repo's docs/privacy-and-revocation.md lists what is stored where and a kill-switch order. A security audit (docs/security.md) fixed SSRF from forged sender domains, DNS rebinding, CSRF between localhost ports, and recipient-parsing tricks, double sends; 58 tests.

What's next

Detect the companies' confirmation replies and mark accounts as laid to rest; an in-browser model (no install); Outlook / IMAP; Google app verification so others can use it.

Why Does Open Innovation Matter?

• My inbox is the most private dataset I own. 793 email headers over 12 years say who I bank with, what I bought, where I studied. An open model on my laptop read all of them; no AI provider saw any of it. The only things that left: Gmail API calls (to Google), public privacy pages (to Ultramarine), and the deletion emails I approved.
• It costs nothing to run again. 430 local calls, $0. Re-scoring everything is free: model outputs are cached.
• I could look inside it. Turning thinking off, seeing that confidence is meaningless, and tracing why it wanted to delete live accounts all came from running the model myself.
• Where open lost: at finding deletion routes, the open model tied Ultramarine on correctness but made claims its quotes didn't support 20% of the time. For the step that can send an email, I used the model whose every quote checked out, and I still check the quote in code.

The Spirit of It

Día de los Muertos is about remembering what came before and letting it rest. The census reached back to signup emails from 2014: services I had forgotten I ever trusted with my email address, four of them breached after I signed up. Then the local model suggested deleting 23 accounts I still use, which is exactly why this tool never acts alone. On Halloween night, three deletion requests left my outbox, each one only after I signed in again to say: yes, let this one rest.

Prize Categories

Best Use of Ultramarine.

• Its job: read a company's public privacy and help pages and return how to delete an account, as self_serve / email / form / unknown, with one sentence quoted from the page as proof.
• Every claim checked in code: 100% of Ultramarine's quotes were found word for word on the page (the local model: 80%). An email address is only used if it sits inside that verified quote, on the company's own domain.
• Accuracy: 8 / 14 correct routes against a blind answer key (local model 8 / 14, guessing privacy@domain 3 / 14). With a page actually fetched: 6 / 10. One of its misses was my guard dropping a correct address its quote didn't contain; another was Cloudflare hiding the address as [email protected].
• A quote it gave, checked on the page. Here Cloudflare had hidden the address from both models:

Cost, reconciled to the nanodollar, twice:

Best Use of Auth0 for AI Agents.

• Token Vault brokers every Gmail call, reading headers and sending, through Connected Accounts with just two Gmail scopes. The agent never holds a Google credential.
• Human approval by Auth0: each send requires a fresh Auth0 sign-in made after the request and within 2 minutes; one sign-in unlocks one send. 3 real deletion requests sent this way to real companies, plus one correctly refused when the sign-in came 24 minutes after the request.

• Real data, real effects: my own Gmail (793 signup emails, 12 years), not test fixtures. Revocation is real too (see Privacy and taking it all back).
• Honest gap: phone-push approval (CIBA + Guardian) needs a paid Auth0 plan, so the default is a fresh sign-in; the CIBA path ships in the code behind APPROVAL_MODE=ciba.
• Revocation: Disconnect Gmail revokes at Google, then deletes the Token Vault connected account via the My Account API.

Best Use of Gemma. Gemma 4 E2B on an 8 GB M1 classified 215 services in 430 local calls (2.7 s each, thinking off), and it's where the story's twist came from.