Cantonese Language Support

Status: Current Last updated: 2026-09-15 12:12 EDT

User reference for Cantonese (yue) processing in batchalign3, ASR engine options, credentials, retokenize usage, and what to expect from each pipeline stage. For the architecture and rationale (engine dispatch, normalization pipeline, segmenter selection, source-file map), see Cantonese and CJK, Architecture.

Quick Reference

ASR engine options

The default for yue is FunASR/SenseVoice: a local model that empirically outperforms vanilla Whisper-large-v3 by a wide margin on Cantonese child speech (42.8% CER vs 81.9% CER on TalkBank Tier 3 fixtures; see the 2026-05-26 Cantonese ASR benchmark). The default is wired in batchalign/worker/_model_loading/asr.py’s _LANG_DEFAULTS table and applies when no --asr-engine is passed and no Rev.AI key is configured. Alternatives are selected with --asr-engine; run batchalign3 transcribe --help for the full list.

Usage

# Default (FunASR/SenseVoice), no flag needed
batchalign3 transcribe input/ -o output/ --lang yue

# Tencent Cloud ASR (requires credentials)
batchalign3 transcribe input/ -o output/ --lang yue \
  --asr-engine tencent

# Aliyun NLS ASR (requires credentials)
batchalign3 transcribe input/ -o output/ --lang yue \
  --asr-engine aliyun

# Qwen3-ASR (default 1.7B variant; pinned via qwen_model for 0.6B)
batchalign3 transcribe input/ -o output/ --lang yue \
  --asr-engine qwen
batchalign3 transcribe input/ -o output/ --lang yue \
  --asr-engine qwen --engine-overrides '{"qwen_model": "Qwen/Qwen3-ASR-0.6B-hf"}'

# Whisper (explicit opt-in; not recommended for Cantonese)
batchalign3 transcribe input/ -o output/ --lang yue \
  --asr-engine whisper

# Cantonese forced alignment
batchalign3 align input/ -o output/ --lang yue \
  --fa-engine cantonese

Credentials

Cloud engines (Tencent, Aliyun) require credentials in ~/.batchalign.ini:

[asr]
# Tencent Cloud
engine.tencent.id = YOUR_SECRET_ID
engine.tencent.key = YOUR_SECRET_KEY
engine.tencent.region = ap-guangzhou
engine.tencent.bucket = YOUR_COS_BUCKET

# Aliyun NLS
engine.aliyun.ak_id = YOUR_ACCESS_KEY_ID
engine.aliyun.ak_secret = YOUR_ACCESS_KEY_SECRET
engine.aliyun.ak_appkey = YOUR_APPKEY

Missing or empty credentials raise ConfigError with a clear message.

Engine details

Tencent Cloud ASR. Speaker diarization with configurable count. Uploads audio to COS, submits ASR job, polls for results (10-min timeout). Returns pre-segmented words with per-word timestamps and speaker attribution. Automatic COS cleanup after job completes.

Aliyun NLS ASR. Cantonese only (lang=yue required). WebSocket streaming with real-time callbacks. Automatic token refresh (23-hour TTL). WAV format required (16 kHz mono).

FunASR/SenseVoice. Local model, no cloud credentials, no network. Auto model selection: Paraformer or SenseVoice based on availability. VAD built in. Per-character timestamp alignment. Wired as the per-language default for yue via _LANG_DEFAULTS in batchalign/worker/_model_loading/asr.py so a bare batchalign3 transcribe --lang yue ... invocation no longer falls through to Whisper.

Qwen3-ASR. Alibaba’s open-weight Cantonese-capable ASR (qwen-asr Python package, model downloaded from HuggingFace on first use). Two variants are publicly released, 1.7B (default, heavier) and 0.6B (lighter, smaller download). Select the 0.6B variant via --asr-engine qwen --engine-overrides '{"qwen_model": "Qwen/Qwen3-ASR-0.6B-hf"}'. Note the division of labour: the ENGINE is chosen with the flag, while per-engine extras like qwen_model and qwen_device stay in --engine-overrides, which is what that option is actually for. External evaluations report competitive CER on per-utterance Cantonese child speech with the 1.7B variant; TalkBank’s own longer-form Cantonese fixtures show this engine benefits from per-utterance segmentation rather than full-session input.

The Qwen3-ASR worker always pairs the ASR model with Qwen/Qwen3-ForcedAligner-0.6B, Qwen’s companion forced-aligner model. The aligner is loaded at worker bootstrap (~1.2 GB additional download on first use, then cached) so the ASR pipeline emits word-level timestamps the downstream FA stage can consume. This pairing is required, not optional: the qwen-asr library raises ValueError from model.transcribe(..., return_time_stamps=True) whenever the aligner was not supplied at Qwen3ASRModel.from_pretrained(...). Dropping return_time_stamps=True to “fix” that error degrades every downstream %wor tier and is therefore not a permitted shortcut; the only correct configuration is the aligner-paired one.

Cantonese forced alignment. Converts Chinese characters to jyutping romanization (via PyCantonese), strips tone numbers for Wave2Vec compatibility, runs Wave2Vec FA on romanized text, maps word-level timings back to original characters.

Text Normalization

All Cantonese ASR output is automatically normalized regardless of which engine produced it. No configuration. Simplified → Traditional via OpenCC s2hk, then a 31-entry domain replacement table for Cantonese-specific corrections (真系→真係, 中意→鍾意, 系→係, 呀→啊, 松→鬆, …).

Full example: 你真系好吵呀 → 你真係好嘈啊.

Where it runs, and how often. In the Rust server, once per monologue, before any stage splits the words (stage 2d of ASR post-processing). The ASR engines themselves hand back their own characters unchanged, so the transcript reads the same whichever engine produced it. Normalizing per word would lose every multi-character replacement, because these engines report one word per Han character; normalizing twice would undo some of them, because the table maps 繫 to 係 and would turn a converted 聯繫 back into 聯係.

What you would see if it ever could not run. Normalization hands each word back exactly the characters it contributed, which is only sound while the conversion preserves the character count. If it did not, the file is refused with a message naming both counts, rather than producing a transcript whose timings have quietly moved onto different characters. No measured input does this: every Han code point and every entry of OpenCC’s own s2hk dictionaries was checked (191,125 strings, none changed length).

The replacement table was originally written by Chuqiao Song in batchalign2’s replace_cantonese_words() (Python + OpenCC C++). Rebuilt in Rust for batchalign3, no C++ dependency, always available, correct overlapping pattern handling.

Word Segmentation: --retokenize

FunASR/SenseVoice and Whisper output per-character tokens for Cantonese: each character becomes a separate word on the main tier. This makes word counts, MLU, and POS tagging unreliable.

# Morphotag has no --lang flag, the per-file @Languages: header drives
# routing. For Cantonese files (yue), retokenize is the right default.
batchalign3 morphotag --retokenize corpus/ -o output/

This uses PyCantonese’s segment() to group per-character tokens into words before Stanza POS tagging. Cantonese files are detected from each file’s @Languages: yue header, there is no morphotag --lang flag.

Before (per-character):

*CHI:	故 事 係 好 .
%mor:	n|故 n|事 v|係 adj|好 .

After (--retokenize):

*CHI:	故事 係 好 .
%mor:	n|故事 v|係 adj|好 .

Without --retokenize, tokenization is preserved unchanged. A diagnostic warning is emitted when Cantonese input appears per-character:

warn: Cantonese input appears to be per-character tokens (42/50 single-CJK words).
      Consider --retokenize for word-level analysis.

Validation across all 9 TalkBank Cantonese corpora

Word segmentation was tested against all 9 Cantonese corpora in TalkBank (over 737,000 utterances). Multi-character preservation 84-90%, vocabulary coverage 98-100% across MOST, LeeWongLeung, CHCC, EACMC, HKU (CHILDES), MAIN, GlobalTales, and Aphasia HKU. Test: batchalign/tests/languages/cantonese/morphosyntax/test_cantonese_all_corpora.py.

Number Expansion

Cantonese uses traditional Chinese number characters: 5 → 五, 42 → 四十二, 10000 → 一萬 (not 一万). Implemented via num2chinese(n, ChineseScript::Traditional) in Rust. Runs as Stage 4 of ASR post-processing, after the text-normalization stage (2d), so the numerals it writes are already in their final form.

See Number Expansion for the full language table.

Utterance Segmentation

Uses the PolyU BERT model PolyU-AngelChanLab/Cantonese-Utterance-Segmentation. Falls back to punctuation-based splitting if the model is unavailable. The same model is used for transcribe’s pre-CHAT segmentation when --lang yue. See Utterance Segmentation.

Mixed-language morphotag (@s)

In bilingual files, Cantonese-marked words (@s:yue or bare @s resolved to yue) go through the same default-on secondary-language L2 morphotag path as other supported languages. Successful secondary dispatch produces real %mor/%gra; unresolved or unsupported cases still fall back to L2|xxx.

Known limitations

• POS tagging on Cantonese vocabulary. Stanza’s zh model is Mandarin-trained, 佢/佢哋 (he/they) → PROPN, 嘢 (thing) → PUNCT, 唔 (not) → VERB, 係 (is) → VERB. PyCantonese POS override fixes core vocabulary as post-processing but has dictionary gaps on compound nouns, some SFPs, and resultative verbs. See the architecture page for the full rationale and the trained-but-undeployed Cantonese model.
• Word segmentation depends on PyCantonese dictionary. Words not in the dictionary won’t be grouped.
• All four ASR engines produce per-character output for Cantonese, --retokenize is needed for all Cantonese morphotag.
• FunASR CER varies with speech clarity: increases with overlapping/soft/child speech.
• Per-character warning threshold (80%) is empirical-without-basis, not yet validated against real corpus data.
• Daemon warning visibility. The tracing::warn! for per-character input fires in the daemon process, not the CLI. Users may not see it until SSE events or job results surface it.

This page last changed: 2026-09-16 (commit 197c81e6). The whole book last changed: 2026-09-16 (commit 34d249d8).