Skip to content

Add context manager support to ConnectionPool - #4167

Merged
petyaslavova merged 3 commits into
redis:masterfrom
sean-kim05:fix/connection-pool-context-manager
Jul 9, 2026
Merged

petyaslavova merged 3 commits into
redis:masterfrom
sean-kim05:fix/connection-pool-context-manager

Conversation

@sean-kim05

@sean-kim05 sean-kim05 commented Jul 4, 2026 •

Copy link
Copy Markdown
Contributor

Description of change

Adds context manager support to ConnectionPool / BlockingConnectionPool (and the async equivalents in redis.asyncio), so a pool's lifecycle can be managed explicitly and safely:

with ConnectionPool.from_url(url) as pool:
    r = Redis(connection_pool=pool)
async with ConnectionPool.from_url(url) as pool:
    r = Redis(connection_pool=pool)

Motivation β€” issue #3669 (Redis.from_pool(...) is not thread safe).

Redis.from_pool() sets auto_close_connection_pool = True, so when the client is closed or garbage-collected it calls connection_pool.disconnect(), which defaults to inuse_connections=True and disconnects every connection in the pool β€” including connections currently checked out by other clients that share the same pool object. In a threaded server (e.g. FastAPI running sync routes in a threadpool) where a client is created per request from a shared pool, one request's client being GC'd tears down connections other in-flight requests are still using, producing errors like ValueError: I/O operation on closed file mid-healthcheck. The async client hits the same problem via aclose().

There isn't currently a first-class, safe idiom for sharing one pool across multiple clients. This PR adds the context-manager protocol the issue author suggested, and clarifies the from_pool() docstrings (sync + async) to document that the returned client takes exclusive ownership of the pool and that the pool must not be shared β€” pointing users at the context manager instead.

This change is purely additive: no existing behavior changes.

Pull Request check-list

  • Do tests and lints pass with this change? (ruff check, ruff format --check, and the new tests pass locally)
  • Do the CI tests pass with this change?
  • Is the new or changed code fully tested? (new test_pool_context_manager in sync and async pool test suites)
  • Is a documentation update included? (expanded from_pool docstrings)
  • Is there an example added to the examples folder (if applicable)? (n/a β€” docstring example included)

Closes #3669


Note

Low Risk
Additive API and documentation only; existing pool/client behavior is unchanged aside from optional context-manager cleanup.

Overview
ConnectionPool lifecycle: Sync and async ConnectionPool (including blocking variants via inheritance) now support with / async with, exiting by calling close() / aclose() and disconnecting pooled connectionsβ€”including those still checked out in the new tests.

Safer shared-pool guidance: Redis.from_pool docstrings (sync and asyncio) now state that the client owns the pool and must not be shared across multiple clients, and point to Redis(connection_pool=pool) plus explicit pool managementβ€”documented with a ConnectionPool.from_url context-manager exampleβ€”for sharing one pool.

Tests: New test_pool_context_manager cases in sync and async pool test suites verify disconnect-on-exit behavior.

Reviewed by Cursor Bugbot for commit b96e110. Bugbot is set up for automated code reviews on this repo. Configure here.

Support using ConnectionPool (and BlockingConnectionPool, plus the async
equivalents) as a context manager so the pool's lifecycle can be managed
explicitly and safely:

    with ConnectionPool.from_url(url) as pool:
        r = Redis(connection_pool=pool)

This gives users a thread-safe idiom for sharing a single pool across
multiple clients, avoiding the footgun in Redis.from_pool(), which takes
exclusive ownership of the pool and disconnects all of its connections --
including ones in use by other clients/threads -- when the owning client
is closed or garbage-collected (issue redis#3669).

Also clarifies the from_pool() docstrings (sync and async) to document
that the pool must not be shared, and points to the context manager as
the supported way to share a pool.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@petyaslavova

Copy link
Copy Markdown
Collaborator

Hi @sean-kim05, thank you for your contribution! I'll provide a review shortly.

@petyaslavova petyaslavova left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution β€” the context-manager support and the from_pool docstring clarification are both welcome, and the sync/async parity is done correctly. Before merging, please address these points:

  1. BlockingConnectionPool coverage: it inherits the new context-manager methods in both stacks, but there's no test for it. Please add a test_pool_context_manager to TestBlockingConnectionPool in tests/test_connection_pool.py and the async equivalent in tests/test_asyncio/test_connection_pool.py.
  2. Return type of __enter__ / __aenter__: they're annotated as "ConnectionPool", so with BlockingConnectionPool(...) as pool: types pool as the base class. Please use a Self-style annotation via a TypeVar bound to ConnectionPool (the async client does this with _RedisT in redis/asyncio/client.py), keeping sync and async annotations identical.
  3. Docstring accuracy: in the new from_pool docstrings, please also state explicitly that the plain constructor, Redis(connection_pool=pool), does not take ownership of the pool β€” that's what makes the recommended example safe, and it's worth spelling out for readers.

With those covered, this should be ready for another review.

…r__ with Self

Address review on redis#4167:
- add test_pool_context_manager for BlockingConnectionPool (sync + async)
- annotate __enter__/__aenter__ return type via the _CP TypeVar so
  subclasses type as themselves, not the ConnectionPool base
- note in from_pool docstrings that Redis(connection_pool=pool) does not
  take ownership of the pool
@sean-kim05

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Addressed all three:

  1. Added test_pool_context_manager to TestBlockingConnectionPool in both the sync (tests/test_connection_pool.py) and async (tests/test_asyncio/test_connection_pool.py) suites.
  2. __enter__/__aenter__ now use the existing _CP TypeVar (bound to ConnectionPool), so subclasses type as themselves rather than the base class β€” identical in sync and async.
  3. The from_pool docstrings now state explicitly that the plain Redis(connection_pool=pool) constructor does not take ownership of the pool, which is what makes the shared-pool context-manager example safe.

ruff check / ruff format --check are clean and the pool tests pass locally. Ready for another look.

@petyaslavova petyaslavova left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@petyaslavova
petyaslavova merged commit 1029559 into redis:master Jul 9, 2026
727 of 731 checks passed
@petyaslavova petyaslavova added the maintenance Maintenance (CI, Releases, etc) label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Maintenance (CI, Releases, etc)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Redis.from_pool(...) is not thread safe

2 participants