Skip to content

Fix GeneratorExp in LocalPythonExecutor to return Generator instead of list - #1650

Merged
albertvillanova merged 3 commits into
huggingface:mainfrom
albertvillanova:fix-generator-exp
Aug 6, 2025
Merged

albertvillanova merged 3 commits into
huggingface:mainfrom
albertvillanova:fix-generator-exp

Conversation

@albertvillanova

@albertvillanova albertvillanova commented Aug 6, 2025 •

Copy link
Copy Markdown
Member

This PR fixes a bug in the LocalPythonExecutor where ast.GeneratorExp nodes were incorrectly returning lists instead of proper Python generators.

Previously, generator expressions like (x for x in range(10)) would be evaluated eagerly and return a list, which breaks the expected lazy evaluation semantics of Python generators and could cause memory issues with large sequences.

Changes made:

  • Fix evaluate_generatorexp function: Modified the implementation to return an actual Python generator object instead of a list
  • Maintaine lazy evaluation: Generator expressions now properly implement lazy evaluation as expected in Python
  • Preserve functionality: All existing logic for handling comprehension generators, conditions, and variable scoping remains intact

Tests:

  • Add regression tests to verify generator expressions return proper generator objects

This fix ensures that the LocalPythonExecutor correctly handles generator expressions according to Python's standard behavior, maintaining both performance and semantic correctness.

@albertvillanova
albertvillanova marked this pull request as ready for review August 6, 2025 16:36
@albertvillanova
albertvillanova merged commit f7b4a33 into huggingface:main Aug 6, 2025
@albertvillanova
albertvillanova deleted the fix-generator-exp branch August 6, 2025 16:38
maxwellfet928 added a commit to maxwellfet928/smolagents that referenced this pull request Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant