translate: Developer Reference

Status: Current Last updated: 2026-09-15 20:20 EDT

Implementation guide for the translate command. For user-facing documentation, see User Guide: translate.

Implementation map

Local submissions (auto-daemon or loopback --server) use paths_mode=true as of 2026-04-14: the CLI posts source/output path lists instead of CHAT bytes. See Submission Modes.

No result cache

Translations are not cached. The utterance cache holds audio evidence only (forced alignment, UTR ASR, Rev.AI transcripts and speaker evidence); text NLP caching was removed because re-running warm inference cost less than the cache lookups. Every translate run calls the worker.

Worker IPC: translate task

request item (rendered from that utterance's TranslationSource; the source and
target languages travel on the request envelope, not on each item):
{ "text": "bonjour le monde." }

TranslationResultV2 items, one per request item:
{ "kind": "translated", "raw_translation": "Hello world.", "engine": "googletrans-v1" }
{ "kind": "blank_input" }
{ "kind": "failed", "error": "Translation failed: ..." }

The PyO3 bridge parses each host item through the Rust wire type; an item that does not parse becomes that item’s failed outcome.

What reaches the engine

TranslationSource::render is the only place source text is built, and three closed rules decide its content, each a match with no catch-all:

RENDERED_TERMINATORS lists every string the terminator rule can emit, and TranslationText::admit refuses a translation equal to one of them, so an engine echoing the punctuation batchalign3 sent cannot become a %xtra tier.

Engine identity and provenance

The engine comes from the results, not from the worker’s capability report. translate.rs admits each item into AdmittedTranslation (Translated { text, engine } or BlankInput), and the batch pipeline stamps each file with [fc-ba3 translate | engine=... ; lang=... | ...], naming the distinct engines on the translations that file applied, joined with + in text order (result_named_provenance with ResultNamedCommand::Translate, the builder coref shares; it cannot fail). A file where nothing was translated carries no stamp and the run says why (TextStamp::NotStamped). Because nothing is read from the report, a translate job is never refused for a worker that has not named its translation engine; that pre-dispatch refusal was removed.

Pre-validation gate

translate requires CHAT Level 1.

Idempotency

inject_translation (in talkbank-transform::translate) calls replace_or_add_tier, which overwrites any existing %xtra tier on the utterance. Re-running translate on a file that already has %xtra tiers re-translates and replaces them. This diverges from BA2, which guarded with if i.translation: continue and preserved the first translation.

Engine selection precedence

TranslateOptions::effective_translate_engine mirrors AlignOptions::effective_fa_engine and BenchmarkOptions::effective_asr_engine. From highest priority to lowest:

1. common.engine_overrides.translate: set by --engine-overrides '{"translate":"<engine>"}'.
2. TranslateOptions.translate_engine: TranslateEngineName: set by --translate-engine google|tencent|aliyun|nllb|seamless. Defaults to Google via default_translate_engine().

There is deliberately no server.yaml knob for engine selection. Translation engine is a policy choice, not a host fact, and policy belongs at the invocation site (CLI flag or shell alias), never in a config file. See the no-config-junk principle in book/src/batchalign/user-guide/commands/translate.md.

The worker pool key includes the resolved translate engine (dispatch_engine_overrides_json always emits a translate entry). Google, Tencent, Aliyun, Seamless, and NLLB workers are not interchangeable, so they end up in separate pools.

Tencent backend specifics

The Tencent loader reuses the shared read_asr_config() helper at batchalign/inference/languages/cantonese/_common.py:77, which prefers BATCHALIGN_TENCENT_{ID,KEY,REGION} environment variables (injected by the Rust control plane at worker spawn) and falls back to ~/.batchalign.ini [asr] section:

• engine.tencent.id → TencentSecretId
• engine.tencent.key → TencentSecretKey
• engine.tencent.region → TencentRegion

These are the same CAM credentials used by the Tencent ASR backend (the [asr] section name is historical; the SecretId/SecretKey pair authorizes any product the CAM user has permission for). The user must have tmt:TextTranslate policy attached (e.g., QcloudTMTFullAccess), and the TMT product must be “opened” at the Tencent Cloud account level, both are root-account / admin actions on the Tencent side.

Rate-limit handling: the inference closure in batchalign/inference/translate.py sleeps 0.2 s per item when the backend is Tencent (5 QPS standard free-tier limit on TextTranslate). This is the analogue of the existing 1.5 s per-item sleep for Google.

Language-code handling: _ISO_639_3_TO_TENCENT_LANG (in batchalign/worker/_model_loading/translation.py) maps the ISO-639-3 codes BA3 emits to Tencent’s ISO-639-1 codes (spa→es, cmn→zh, etc.). Unmapped source languages raise a clear ValueError recommending --translate-engine nllb. Tencent does NOT list yue→en in its supported pairs, Cantonese requests are rejected at the table lookup, not at the API call.

Empty SourceText would be rejected by the Tencent API with a typed InvalidParameter error. The loader short-circuits empty input (returns the empty string) so a stray empty utterance doesn’t surface as a SDK exception that looks like a credentials problem.

Aliyun backend specifics

The Aliyun loader (_load_aliyun_translate) uses the same shared read_asr_config() helper, with credentials drawn from the BATCHALIGN_ALIYUN_AK_{ID,SECRET} environment variables (injected by the Rust control plane at worker spawn) or the ~/.batchalign.ini [asr] section:

• engine.aliyun.ak_id → Aliyun Access Key ID
• engine.aliyun.ak_secret → Aliyun Access Key Secret

These are the same access-key pair used by the Aliyun NLS ASR backend. Aliyun MT does NOT need the ak_appkey field that NLS ASR consumes, that key authorizes the WebSocket speech service, not the REST translation service.

Region is pinned to cn-hangzhou (_ALIYUN_MT_REGION in translation.py). Aliyun MT exposes a single global endpoint at mt.aliyuncs.com across every supported region, so the AcsClient region only affects request signing, there is no cn-hangzhou vs us-west-1 quality / availability split. If region-pinning becomes a deployment concern later, promote to a config-driven override.

SDK package: aliyun-python-sdk-alimt (pinned at >=3.2.0 in pyproject.toml). The loader uses the v20181012 General Translation endpoint via TranslateGeneralRequest with FormatType="text" and Scene="general" (both promoted to module-level constants _ALIYUN_MT_FORMAT_TYPE / _ALIYUN_MT_SCENE so the wire shape is visible without grepping for magic strings).

Language-code handling: _ISO_639_3_TO_ALIYUN_LANG (in batchalign/worker/_model_loading/translation.py) maps the ISO-639-3 codes BA3 emits to Aliyun’s ISO-639-1-ish codes (spa→spa, cmn→zh, kor→ko, yue→yue, etc.). The presence of yue is the load-bearing reason this backend exists alongside Tencent, see User Guide: translate for the operator-visible rationale.

Response envelope: Aliyun MT returns a JSON byte payload of the shape {"Code": "200", "Data": {"Translated": "...", "DetectedLanguage": "...", "WordCount": "..."}, "RequestId": "..."}. Non-"200" codes surface as ClientException/ServerException from do_action_with_exception before the loader parses; by the time json.loads runs, Code == "200" is expected.

Empty SourceText short-circuits the same way Tencent does (return empty string before any SDK call) for the same reason, Aliyun treats empty input as an invalid request and would surface a typed SDK exception that looks like a credentials problem.

End-to-end verification: the loader’s SDK call shape is wired against the aliyun-python-sdk-alimt==3.2.0 source. Real-API smoke testing happens at the operator boundary per the user-guide; CI covers the wire shape via the mocked-SDK test in batchalign/tests/pipelines/translate/test_translation_model_loading.py::TestLoadAliyunTranslate.

BA2 → BA3 migration notes

Tier-name clarification. Neither BA2 nor BA3 produces a %tra tier. Both versions emit %xtra. Any other translation-tier name observed in the wild was not written by Batchalign.

Testing

make test
cargo test -p batchalign translate::

Related developer documentation

• Command Flowcharts: translate

This page last changed: 2026-09-16 (commit 197c81e6). The whole book last changed: 2026-09-16 (commit 34d249d8).