Skip to content

Add rule for context manager iterator annotations (UP052) - #29000

Merged
charliermarsh merged 4 commits into
mainfrom
charlie/contextmanager-generator-annotations
Sep 30, 2026
Merged

charliermarsh merged 4 commits into
mainfrom
charlie/contextmanager-generator-annotations

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

The contextmanager and asynccontextmanager decorators require generators, which support methods not guaranteed by the Iterator and AsyncIterator protocols. Typeshed deprecates the overloads that accept these iterator annotations.

Add the preview rule context-manager-iterator (UP052) to identify these annotations and offer a fix. For example:

from collections.abc import Iterator
from contextlib import contextmanager

@contextmanager
def example() -> Iterator[int]:
    yield 1

The fix replaces Iterator[int] with Generator[int, None, None]. For an async context manager, it replaces AsyncIterator[int] with AsyncGenerator[int, None]. On Python 3.13 and later, it omits the default type arguments and uses Generator[int] or AsyncGenerator[int]. If a synchronous generator returns a value, the fix uses object as its return type.

The fix is unsafe because it changes runtime annotations and may introduce an import. The rule checks functions whose innermost decorator is a context manager and whose body contains a yield or yield from.

@astral-sh-bot

astral-sh-bot Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

ruff-ecosystem results

Linter (stable)

βœ… ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+342 -0 violations, +0 -0 fixes in 27 projects; 31 projects unchanged)

aiven/aiven-client (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ tests/test_cli.py:964:39: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

apache/airflow (+11 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview --select ALL

+ airflow-core/src/airflow/jobs/triggerer_job_runner.py:713:30: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ devel-common/src/sphinx_exts/docs_build/docs_builder.py:111:29: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ devel-common/src/sphinx_exts/docs_build/docs_builder.py:124:39: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ devel-common/src/sphinx_exts/docs_build/docs_builder.py:134:47: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ devel-common/src/sphinx_exts/docs_build/docs_builder.py:146:37: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ devel-common/src/tests_common/test_utils/team.py:32:95: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ providers/common/ai/src/airflow/providers/common/ai/observability.py:176:67: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ providers/common/ai/src/airflow/providers/common/ai/sandbox/opensandbox.py:73:6: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ providers/common/ai/src/airflow/providers/common/ai/skills.py:216:51: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ providers/standard/src/airflow/providers/standard/hooks/subprocess.py:33:50: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
... 1 additional changes omitted for project

apache/superset (+43 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview --select ALL

+ superset/commands/deletion_retention/purge_cascade.py:72:62: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/config.py:2390:6: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/distributed_lock/__init__.py:33:6: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/extensions/context.py:103:51: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/extensions/context.py:122:46: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/models/helpers.py:1568:65: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/tasks/ambient_context.py:65:38: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/tasks/async_queries.py:221:67: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/tasks/heartbeat.py:73:6: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ superset/tasks/locks.py:55:34: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
... 33 additional changes omitted for project

bokeh/bokeh (+6 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview --select ALL

+ src/bokeh/io/util.py:137:19: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/bokeh/io/util.py:216:67: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/integration/server_e2e/test_examples.py:88:34: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/support/plugins/managed_server_loop.py:62:89: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/support/util/env.py:41:62: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/support/util/filesystem.py:159:34: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

ibis-project/ibis (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ ibis/backends/trino/__init__.py:115:10: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

langchain-ai/langchain (+26 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ libs/core/langchain_core/utils/utils.py:74:46: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/core/tests/unit_tests/language_models/chat_models/test_base.py:384:35: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/core/tests/unit_tests/prompts/test_loading.py:27:41: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/langchain_v1/tests/integration_tests/mcp/test_protocol_eras.py:75:27: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/langchain_v1/tests/unit_tests/agents/conftest_checkpointer.py:10:31: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/langchain_v1/tests/unit_tests/agents/conftest_checkpointer.py:15:41: context-manager-generator Use `AsyncGenerator` instead of `AsyncIterator` for a context manager
+ libs/langchain_v1/tests/unit_tests/agents/conftest_checkpointer.py:21:31: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/langchain_v1/tests/unit_tests/agents/conftest_checkpointer.py:27:33: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/langchain_v1/tests/unit_tests/agents/conftest_checkpointer.py:33:38: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ libs/langchain_v1/tests/unit_tests/agents/conftest_checkpointer.py:39:38: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
... 16 additional changes omitted for project

latchbio/latch (+4 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ src/latch/account.py:178:59: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/latch/registry/project.py:147:59: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/latch/registry/table.py:378:59: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/latch_cli/centromere/utils.py:33:42: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

mhammond/pywin32 (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ win32/test/test_win32crypt.py:59:69: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

milvus-io/pymilvus (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ pymilvus/client/bloom_filter.py:426:29: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ pymilvus/client/roaring_filter.py:465:29: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

prefecthq/prefect (+8 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ src/integrations/prefect-kubernetes/prefect_kubernetes/observer.py:118:33: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/prefect/deployments/steps/core.py:75:68: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/prefect/server/services/db_vacuum.py:102:6: context-manager-generator Use `AsyncGenerator` instead of `AsyncIterator` for a context manager
+ src/prefect/server/utilities/worker_channel_cleanup.py:58:10: context-manager-generator Use `AsyncGenerator` instead of `AsyncIterator` for a context manager
+ src/prefect/testing/standard_test_suites/worker_cleanup_queue.py:66:14: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/cli/test_start_server.py:75:6: context-manager-generator Use `AsyncGenerator` instead of `AsyncIterator` for a context manager
+ tests/server/database/test_migrations.py:44:6: context-manager-generator Use `AsyncGenerator` instead of `AsyncIterator` for a context manager
+ tests/server/orchestration/api/test_worker_channel_cleanup_dispatcher.py:50:52: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

pypa/pip (+17 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ src/pip/_internal/cli/base_command.py:86:70: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/pip/_internal/cli/index_command.py:184:70: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/pip/_internal/commands/install.py:361:70: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ src/pip/_internal/commands/list.py:140:70: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/lib/__init__.py:1130:63: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/lib/compat.py:16:30: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/lib/configuration_helpers.py:45:41: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/lib/filesystem.py:26:43: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/lib/server.py:123:47: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ tests/lib/server.py:247:31: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
... 7 additional changes omitted for project

pypa/setuptools (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ setuptools/build_meta.py:84:23: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ setuptools/wheel.py:69:30: context-manager-generator Use `Generator` instead of `Iterator` for a context manager

python/mypy (+54 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ misc/upload-pypi.py:90:20: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/binder.py:452:46: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/binder.py:627:36: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/build.py:3081:60: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/checker.py:1445:52: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/checker.py:3984:58: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/checker.py:7919:37: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/checker.py:8089:10: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/checker.py:970:58: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
+ mypy/checker_shared.py:368:48: context-manager-generator Use `Generator` instead of `Iterator` for a context manager
... 44 additional changes omitted for project

... Truncated remaining completed project reports due to GitHub comment length restrictions

Changes by rule (2 rules affected)

code total + violation - violation + fix - fix
context-manager-generator 341 341 0 0 0
UP052 1 1 0 0 0

@charliermarsh charliermarsh changed the title Add rule for context manager generator annotations (RUF079) Add rule for context manager generator annotations (UP052) Sep 30, 2026
@charliermarsh charliermarsh added rule Implementing or modifying a lint rule preview Related to preview-mode features labels Sep 30, 2026
@charliermarsh
charliermarsh requested a review from ntBre September 30, 2026 01:09
@charliermarsh
charliermarsh marked this pull request as ready for review September 30, 2026 01:09

@ntBre ntBre left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment on lines +41 to +43
/// @contextmanager
/// def example() -> Generator[int, None, None]:
/// yield 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we/should we generate the shorter Generator[int] form on 3.13+? It also seems okay to leave it to unnecessary-default-type-args (UP043), though.

Comment on lines +183 to +196
let return_type = if visitor.returns_value {
"object"
} else {
"None"
};
let (object_edit, return_type) = if visitor.returns_value {
checker.importer().get_or_import_builtin_symbol(
return_type,
function.start(),
checker.semantic(),
)?
} else {
(None, return_type.to_string())
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This first return_type binding seems kind of redundant, if I understand correctly:

Suggested change
let return_type = if visitor.returns_value {
"object"
} else {
"None"
};
let (object_edit, return_type) = if visitor.returns_value {
checker.importer().get_or_import_builtin_symbol(
return_type,
function.start(),
checker.semantic(),
)?
} else {
(None, return_type.to_string())
};
let (object_edit, return_type) = if visitor.returns_value {
checker.importer().get_or_import_builtin_symbol(
"object",
function.start(),
checker.semantic(),
)?
} else {
(None, "None".to_string())
};

/// - [Python documentation: `contextlib.asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
#[derive(ViolationMetadata)]
#[violation_metadata(preview_since = "NEXT_RUFF_VERSION", category = Category::Suspicious)]
pub(crate) struct ContextManagerGenerator {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should this be ContextManagerIterator since that's the issue being flagged in the initial code?

}

#[test]
fn context_manager_generator_fixes() -> Result<()> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we just use a regular snapshot here? Or maybe mdtests?

@charliermarsh
charliermarsh force-pushed the charlie/contextmanager-generator-annotations branch from 108e3dc to 0fa1944 Compare September 30, 2026 14:27
@charliermarsh charliermarsh changed the title Add rule for context manager generator annotations (UP052) Add rule for context manager iterator annotations (UP052) Sep 30, 2026
@charliermarsh
charliermarsh enabled auto-merge (squash) September 30, 2026 14:28
@charliermarsh
charliermarsh merged commit a4e7c20 into main Sep 30, 2026
50 checks passed
@charliermarsh
charliermarsh deleted the charlie/contextmanager-generator-annotations branch September 30, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preview Related to preview-mode features rule Implementing or modifying a lint rule

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants