This page is for version Latest · v1.4.0 (26.09) (default).
For other versions, use one of these documentation indexes:
- Latest · v1.4.0 (26.09) (default): https://docs.nvidia.com/nemo/curator/latest/llms.txt
- Main · preview: https://docs.nvidia.com/nemo/curator/main/llms.txt
- 26.09 · v1.4.0: https://docs.nvidia.com/nemo/curator/v26.09/llms.txt
- 26.07 · v1.3.0: https://docs.nvidia.com/nemo/curator/v26.07/llms.txt
- 26.04 · v1.2.0: https://docs.nvidia.com/nemo/curator/v26.04/llms.txt
- 26.02 · v1.1.0: https://docs.nvidia.com/nemo/curator/v26.02/llms.txt
- 25.09 · v1.0.0: https://docs.nvidia.com/nemo/curator/v25.09/llms.txt
Browse these references to configure distributed execution, memory use, GPU processing, checkpoints, workers, and containers across NeMo Curator modalities.
Infrastructure Components
Memory Management
Optimize memory usage when processing large datasets partitioning batching monitoring
GPU Acceleration
Use NVIDIA GPUs to accelerate data processing cuda rmm performance
Resumable Processing
Continue interrupted operations across large datasets checkpoints recovery batching
Stage Worker Sizing
Configure fixed and autoscaling worker pools across Ray Data, Xenna, and Ray Actor Pool backends workers autoscaling stage-overrides
Container Environments
Build your own NeMo Curator container and configure its dependencies, build arguments, and runtime environment. docker uv environments
