Skip to content

fix(threads): clean persisted records safely on thread deletion - #5535

Merged
WillemJiang merged 8 commits into
bytedance:mainfrom
jamespud:fix/thread-delete-persistence-cleanup
Sep 18, 2026
Merged

WillemJiang merged 8 commits into
bytedance:mainfrom
jamespud:fix/thread-delete-persistence-cleanup

Conversation

@jamespud

Copy link
Copy Markdown
Contributor

Fixes #5530
Supersedes #2803

Why

DELETE /api/threads/{id} removed the thread directory, the checkpoints, the
browser session and the threads_meta row, but left the thread's persisted
records behind: run_events (the user-visible conversation feed), historical
runs rows and feedback all survived. Two consequences, both reachable on
current main:

  1. GET /api/threads/{id}/messages still answers after the delete. The route is
    owner-checked but read-style (require_existing=False), and
    ThreadMetaRepository.check_access returns True for a missing meta row, so
    a deleted thread's feed stays readable.
  2. POST /api/threads accepts a client-chosen thread_id. Re-creating the id
    immediately inherits the deleted thread's messages.

The persistence half of this was identified in #2803 (still open, last touched
2026-06-11). This PR rebases that scope onto the current durable thread-deletion
lifecycle and closes the event-store contract gap tracked in #5530.

What changed

Event store (#5530). DbRunEventStore writers already serialize per thread:
the in-process _get_write_lock() plus, on PostgreSQL,
pg_advisory_xact_lock(hashtext(thread_id)) taken inside _max_seq_for_thread().
Deletion bypassed both, so a deletion could return while a concurrently admitted
writer still committed a row for the deleted thread — resurrection after delete.
The advisory lock is now an explicit primitive
(_acquire_thread_mutation_fence), and delete_by_thread / delete_by_run take
both fences. JsonlRunEventStore already serialized deletion through
_run_mutation(); the two durable stores now expose the same mutation contract.

Thread deletion cleanup (#2803 scope). _delete_thread_data_with_reservation
now also deletes historical runs, run events and feedback, all inside the
existing RunManager.reserve_thread_operation(kind=delete) reservation, all
owner-scoped with one get_effective_user_id() value, and all best-effort so the
HTTP success contract is unchanged.

Reservations stay reserved. RunRepository.delete_by_thread /
MemoryRunStore.delete_by_thread delete only operation_kind == "run" rows.
The runs table also holds durable thread-operation reservations
(checkpoint_write, artifact_write, branch, delete); a blanket
DELETE ... WHERE thread_id = ? would have removed the very reservation
protecting the in-flight delete and dropped the cross-worker exclusion. This is
the one place this PR deliberately diverges from #2803's implementation, and it
has its own regression test.

Deliberate non-goals. Bulk run cleanup does not bump the run-change clock
(matching the existing single-run RunRepository.delete(), and keeping thread
cleanup away from run_change_clock — the table #5516/#5517 were about). No
thread-incarnation/generation fencing, no sandbox or MCP teardown, no JSONL
queued-mutation change, no migration.

Verification

RED → GREEN per behaviour change:

  • RED before the fence: 6 failing tests (deletion completes while a writer holds
    the thread lock; deletion's first statement is the DELETE, not the advisory
    lock). GREEN: tests/test_db_event_store_lock_lifecycle.py +
    tests/test_run_event_store.py → 95 passed, plus 50 passed across the adjacent
    event-store suites.
  • RED before the bulk cleanup: 5 failing tests (AttributeError). GREEN:
    tests/test_run_repository.py → 70 passed, including the mandatory
    reservation-preservation case.
  • RED before feedback cleanup: 2 failing tests. GREEN:
    tests/test_feedback.py → 30 passed.
  • RED before the router wiring: 3 failing tests (stores never called). GREEN:
    344 passed across test_threads_router.py, test_run_repository.py,
    test_feedback.py, test_run_event_store.py,
    test_db_event_store_lock_lifecycle.py, test_jsonl_event_store_async_io.py.
  • ruff check + ruff format --check clean; git diff --check clean.

Notes for reviewers

  • delete_by_thread/delete_by_run now take an optional owner scope on every
    event-store backend. The memory and JSONL stores are not user-scoped, so they
    accept the parameter for interface parity and ignore it — the same convention
    their read methods already use — which lets the Gateway call deletion
    uniformly. Third-party RunStore implementations without bulk cleanup are
    skipped through a capability check.
  • Test-suite environment note for this branch: tests/test_browser_automation.py
    and tests/test_browserless_client.py require Playwright and reachable
    browserless/example.com endpoints; they fail or hang in a sandbox without
    those, independently of this change.

@jamespud
jamespud marked this pull request as ready for review September 18, 2026 06:53
@github-actions github-actions Bot added area:backend Gateway / runtime / core backend under backend/ area:docs Documentation and Markdown only needs-validation Touches front/back contract surface; needs real-path validation risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/XL PR changes 700+ lines labels Sep 18, 2026

@willem-bd willem-bd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 56d4ee9. I ran the six changed test files in a worktree at the exact head SHA with import paths pinned to the PR sources: all 344 pass, and the new fence tests in test_db_event_store_lock_lifecycle.py fail against the merge-base (4889f61) sources, confirming the RED→GREEN claims. The deletion critical section (per-thread asyncio lock → session → advisory fence) now matches the writer path exactly, owner scoping is applied consistently across runs/events/feedback/meta from a single resolved identity, and the operation-kind reservation-preservation semantics in the bulk run delete are the right call and are pinned by tests. One consistency gap worth addressing (inline): the event-store cleanup call is not capability-gated the way the run-store call is, so a store implementing the old delete signature would skip event deletion silently at debug level. Non-blocking.

Comment thread backend/app/gateway/routers/threads.py Resolved
@jamespud
jamespud requested a review from willem-bd September 18, 2026 10:06
@WillemJiang WillemJiang added this to the 2.1.0 milestone Sep 18, 2026
@WillemJiang
WillemJiang merged commit 3776f6f into bytedance:main Sep 18, 2026
19 checks passed
@WillemJiang WillemJiang modified the milestones: 2.1.0, 2.2.0 Sep 22, 2026
vieenrose pushed a commit to vieenrose/deer-flow that referenced this pull request Sep 25, 2026
…dance#5535)

* fix(events): serialize DB deletion with thread writers

* fix(runs): delete thread history without dropping reservations

* fix(feedback): support owner-scoped thread cleanup

* fix(threads): clean persisted records on deletion

* fix(threads): correct the feedback cleanup rationale

* test(runs): drop the wall-clock probe from the in-flight delete test

* docs: record the thread-delete and event-store fence contracts

* fix(threads): preserve legacy event-store delete compatibility
ethan-jiang-1 pushed a commit to ethan-jiang-1/deer-flow that referenced this pull request Sep 25, 2026
锚点 bytedance#6 (v2.1.0-rc0, 769589e) → bytedance#7 (v2.1.0, 345f08b),10 commits / 90 files。
注意 v2.1.0 tag 打在 2.1.x-dev 上,与 upstream/main 已在 rc0 处分叉(main 另有 189 commits 待 sync bytedance#8)。

源码侧实质变更 → digest 校正:
- 0025_repair_run_change_seq(bytedance#5517/bytedance#5516):修复被"插队"的 0023_run_change_seq
  留下的 schema 空洞;补注册 RunChangeClockRow / UserPreferenceRow
- 线程删除全量清理(bytedance#5535):DELETE /api/threads/{id} 持 durable delete reservation,
  依次清理文件系统/checkpoints/历史 run(只删 operation_kind="run")/events/feedback/thread_meta
- 事件存储变更串行域 + owner-scoped 删除签名(bytedance#5535)
- CI:6 个 workflow push 触发改 *-dev 通配,skill-review-ci 显式钉 2.1.x-dev(bytedance#5765)
- frontend:Projects 侧栏嵌套 SidebarMenu 加 w-auto(bytedance#5681/bytedance#5682)
- 文档:subagents 11 页手册 / extensions 10 页手册,root AGENTS.md 精简

同时校正文档侧漂移:AGENTS.md 行号引用与计数、仓库行数/方法数、
_faq_on_digested 14 处基线标注 rc0→v2.1.0。

SYNC.md / SYNC_LOG.md bytedance#7 / UPDATE_PLAN_7.md 已同步记录。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:backend Gateway / runtime / core backend under backend/ area:docs Documentation and Markdown only needs-validation Touches front/back contract surface; needs real-path validation risk:high High risk: backend API, agents, sandbox, auth, deps, CI size/XL PR changes 700+ lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] DbRunEventStore deletion bypasses the per-thread mutation fence used by event writers

3 participants