[lbliii]

What changed

• add a complete, source-verified TranslationStage parameter reference covering inputs, segmentation, inference, FAITH evaluation, output, and resume controls
• document validation rules and interactions among output modes, score merging, message reconstruction, dry runs, and health checks
• explain skip/restore semantics, including how pre-translated rows interact with FAITH filtering and output formatting
• add built-in Google, AWS, and NMT backend configuration details
• add a non-LLM translation example that uses a separate LLM for FAITH scoring
• document custom translation and FAITH prompt schemas, placeholders, escaping, and dry-run behavior

Why

The translation controls introduced in #2038 were not fully represented in the user guide. Users had to inspect implementation details to discover defaults, constraints, backend requirements, and the behavior of resumed translation runs.

Impact

Users can configure and validate experimental translation pipelines directly from the guide, including mixed-provider translation and quality evaluation workflows. This is a documentation-only change; runtime behavior is unchanged.

Validation

• cd fern && npm run check — passes with 0 errors
• cd fern && fern docs broken-links — changed page has no broken links; the command reports 22 pre-existing errors in older API-reference pages
• git diff --check — passes

Closes #2126

Parent workstream: #2118

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[greptile-apps]

Greptile Summary

This PR adds a comprehensive TranslationStage parameter reference to the translation guide, covering input/segmentation, inference, FAITH evaluation, output/resume controls, built-in backend configuration, custom prompts, and dry-run usage. All parameter defaults, types, and validation rules were cross-checked against the pipeline source and match the implementation.

• Adds complete TranslationStage parameter tables with verified defaults and constraint descriptions, skip/restore semantics, and output-mode interaction rules.
• Adds a non-LLM translation example (Google + separate FAITH scorer), built-in backend config table, and custom prompt/dry-run reference.
• One documentation gap: the FAITH custom prompt section lists {source_text} and {translated_text} as available in "FAITH templates" without clarifying these are only valid in the user: section — placing them in the system: section raises a KeyError at runtime.

Confidence Score: 4/5

This is a documentation-only change with no runtime impact; the one issue found is a misleading claim about FAITH prompt placeholder availability that could cause users of advanced custom FAITH prompts to produce a broken configuration.

All parameter defaults, validation rules, and behavioral descriptions were traced to implementation and are accurate. The single gap is in the custom FAITH prompt section, where listing {source_text} and {translated_text} as available in 'FAITH templates' without distinguishing system vs. user sections could lead a user who writes a custom system prompt with those variables to encounter a KeyError at runtime.

fern/versions/main/pages/curate-text/process-data/language-management/translation.mdx — the Custom Prompts section needs a clarification that {source_text} and {translated_text} are only available in the user: section of a custom FAITH prompt.

Important Files Changed

Filename	Overview
fern/versions/main/pages/curate-text/process-data/language-management/translation.mdx	Documentation-only addition covering TranslationStage parameter reference, backend configs, custom prompts, and skip/restore semantics — all verified against source. One documentation error: FAITH custom prompt docs list {source_text}/{translated_text} as available in "FAITH templates" without clarifying they are only available in the user: section, not the system: section.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant U as User
    participant TS as TranslationStage
    participant SK as SkipExistingTranslationsStage
    participant SEG as SegmentationStage
    participant TR as SegmentTranslationStage
    participant FE as FaithEvalFilter
    participant RE as ReassemblyStage
    participant FT as FaithThresholdFilterStage
    participant RS as RestoreSkippedRowsStage
    participant FO as FormatTranslationOutputStage
    participant MF as MergeFaithScoresStage

    U->>TS: DocumentBatch (1 row/doc)
    alt "skip_translated=True"
        TS->>SK: filter pre-translated rows into _metadata
    end
    TS->>SEG: explode docs into segments (1 row/segment)
    SEG->>TR: _seg_segments column
    TR-->>TR: LLM/Google/AWS/NMT translate
    TR->>FE: _translated column
    note over FE: only when enable_faith_eval=True
    FE-->>FE: "score segments (filter_enabled=False)"
    FE->>RE: "faith_* columns per segment"
    RE->>FT: collapse to 1 row/doc, aggregate faith_avg
    note over FT: only when filter_enabled=True
    FT-->>FT: "drop rows where faith_avg < threshold"
    alt "skip_translated=True"
        FT->>RS: restore pre-translated rows with defaults
    end
    RS->>FO: apply output_mode (replaced/raw/both)
    note over FO: only when output_mode != replaced or reconstruct_messages=True
    FO->>MF: build translation_metadata, translated_messages
    note over MF: only when merge_scores=True
    MF-->>U: DocumentBatch with final columns

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant U as User
    participant TS as TranslationStage
    participant SK as SkipExistingTranslationsStage
    participant SEG as SegmentationStage
    participant TR as SegmentTranslationStage
    participant FE as FaithEvalFilter
    participant RE as ReassemblyStage
    participant FT as FaithThresholdFilterStage
    participant RS as RestoreSkippedRowsStage
    participant FO as FormatTranslationOutputStage
    participant MF as MergeFaithScoresStage

    U->>TS: DocumentBatch (1 row/doc)
    alt "skip_translated=True"
        TS->>SK: filter pre-translated rows into _metadata
    end
    TS->>SEG: explode docs into segments (1 row/segment)
    SEG->>TR: _seg_segments column
    TR-->>TR: LLM/Google/AWS/NMT translate
    TR->>FE: _translated column
    note over FE: only when enable_faith_eval=True
    FE-->>FE: "score segments (filter_enabled=False)"
    FE->>RE: "faith_* columns per segment"
    RE->>FT: collapse to 1 row/doc, aggregate faith_avg
    note over FT: only when filter_enabled=True
    FT-->>FT: "drop rows where faith_avg < threshold"
    alt "skip_translated=True"
        FT->>RS: restore pre-translated rows with defaults
    end
    RS->>FO: apply output_mode (replaced/raw/both)
    note over FO: only when output_mode != replaced or reconstruct_messages=True
    FO->>MF: build translation_metadata, translated_messages
    note over MF: only when merge_scores=True
    MF-->>U: DocumentBatch with final columns

Loading

_{Reviews (6): Last reviewed commit: "Merge branch 'main' into codex/docs-tran..." | Re-trigger Greptile}

[greptile-apps]

The guidance correctly says not to combine output_mode="raw" with reconstruct_messages, but doesn't explain the concrete consequence. In FormatTranslationOutputStage.process, output_field is dropped from the DataFrame before _build_translated_messages is called, so every translated_messages entry ends up as an empty string. Describing that failure mode here will save users from a confusing silent-empty result.

Suggested change

	- `output_mode="raw"` removes `output_field` after building `translation_metadata`. Do not combine raw mode with message reconstruction; use `"both"` when you need metadata and `translated_messages`.
	- `output_mode="raw"` removes `output_field` after building `translation_metadata`. Do not combine raw mode with message reconstruction: `output_field` is dropped before `_build_translated_messages` runs, so every `translated_messages` entry will be an empty string. Use `output_mode="both"` when you need metadata and `translated_messages`.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-codex-docs-translation-reference.docs.buildwithfern.com/nemo/curator

Here are the markdown pages you've updated:

• nemo/curator/main/curate-text/process-data/language-management/translation

[lbliii]

@ayushdg friendly ping on this docs PR for the #2118 release-docs workstream — when you have a moment, could you take a look at the translation configuration reference?

Docs preview: https://nvidia-preview-codex-docs-translation-reference.docs.buildwithfern.com/nemo/curator

Thanks!

[lbliii]

@sarahyurick, since you approved the underlying translation-controls implementation in #2038, could you please review this configuration reference when you have a moment? Thank you!

[sarahyurick]

I don't know so much about this pipeline, but just from eyeballing these changes look fine to me.

[lbliii]

/ok to test e9d0c7e