CHAT Dependent Tier Handling by Command
Status: Current Last updated: 2026-05-20 01:14 EDT
Each command reads and writes different dependent tiers (%mor, %gra, %wor, %xtra).
This determines which parse mode is used at pipeline entry.
Parse modes
- Strict (tree-sitter parse via
talkbank_parser): Rejects the file on ANY parse error. Used when the input is expected to be valid , i.e., output from a previous pipeline stage. - Lenient (
talkbank_transform::parse::parse_lenientat../chatter/crates/talkbank-transform/src/parse.rs:17): Error recovery: keeps parseable content and drops broken tiers. Used at pipeline entry because input may have malformed dependent tiers from legacy CLAN runs or previous batchalign versions.
Where each parse mode is used
| Location | Mode | Why |
|---|---|---|
| Rust server (per-file dispatch) | Lenient | Input CHAT may have broken dep tiers |
| Rust server (post-injection re-parse) | Strict | Engine output should be valid; catch bugs early |
| Rust server (comment insertion) | Strict | Pipeline output should be valid |
Per-command tier handling
| Command | Reads Dep Tiers | Writes Dep Tiers | Notes |
|---|---|---|---|
| morphotag | None (clears %mor/%gra first) | %mor, %gra | clear_morphosyntax() strips existing tiers before processing |
| align | %wor (for UTR) | %wor | Regenerates timing from scratch |
| translate | None | %xtra | Adds translation tier |
| utseg | None | (restructures utterances) | Splits/merges utterance boundaries |
| transcribe | N/A (generates from audio) | All | Creates fresh CHAT |
| opensmile | N/A (media only) | N/A (CSV output) | Analysis only |
Why morphotag clears before processing
The Rust collect_payloads() function
(crates/batchalign-transform/src/morphosyntax/payload.rs:135) skips
utterances that already have a %mor tier (optimization for cache
injection). Without clearing first, files with existing %mor would
be silently round-tripped unchanged. Calling the Rust function
clear_morphosyntax
(crates/batchalign-transform/src/morphosyntax/payload.rs:333) at the
top of process_morphosyntax()
(crates/batchalign/src/morphosyntax/mod.rs:64) ensures all
utterances are reprocessed. Cache hits still work, the cache lookup
happens after clearing but before Stanza runs.
This page last changed: 2026-06-19 (commit c82a6d03). The whole book last changed: 2026-09-16 (commit 34d249d8).