Overlap Encoding: &* and +< Internals

Status: Current Last updated: 2026-09-06 23:14 EDT

AST Representation

&*: OtherSpokenEvent

Model (talkbank-tools): ../chatter/crates/talkbank-model/src/model/content/other_spoken.rs

pub struct OtherSpokenEvent {
    pub speaker: SpeakerCode,     // e.g., "INV"
    pub text: smol_str::SmolStr,  // e.g., "oh_okay_yeah"
    pub span: Span,               // source location (skipped in serde)
}

Appears in two enum locations:

• UtteranceContent::OtherSpokenEvent(OtherSpokenEvent): top-level content
• BracketedItem::OtherSpokenEvent(OtherSpokenEvent): inside groups

Parser (talkbank-tools): ../chatter/crates/talkbank-parser/src/parser/tree_parsing/main_tier/content/

The tree-sitter grammar accepts &* + speaker chars + : + non-whitespace chars.

Serialization: &*SPK:text: roundtrips cleanly via WriteChat.

+<: Linker::LazyOverlapPrecedes

Model (talkbank-tools): ../chatter/crates/talkbank-model/src/model/content/linker.rs

#![allow(unused)]
fn main() {
pub enum Linker {
    LazyOverlapPrecedes,  // +<
    OtherCompletion,      // ++
    QuickUptakeOverlap,   // +^
    // ...
}
}

Stored on TierContent.linkers: TierLinkers (a Vec<Linker> newtype). Linkers appear at the start of an utterance’s content, before words.

Content Walker Behavior

The content walker (for_each_leaf / for_each_leaf_mut) skips OtherSpokenEvent entirely. It is listed in the no-op match arm alongside events, pauses, overlap points, and other non-alignable content:

UtteranceContent::OtherSpokenEvent(_) => {}  // skipped

This means &* content:

• Is not counted in word alignment (Wor, Mor, Pho, Sin domains)
• Does not appear in %wor tier generation
• Is not extracted by collect_fa_words() for forced alignment
• Is not included in the UTR reference word sequence

Two-Pass UTR Strategy

When +< or CA overlap markers (⌊) are present, the alignment pipeline uses a two-pass UTR strategy. See Forced Alignment, UTR for the algorithm details, and the CHAT Data Model content-walker API (walk_overlap_points) in the chatter project.

Key points:

• Pass 1 excludes overlap utterances from the global DP alignment
• Pass 2 recovers their timing from the predecessor’s audio window
• CA overlap markers (position) narrow the pass-2 search window via proportional onset estimation
• Best-of-both fallback compares FA group counts to avoid regression on non-English

Code: crates/batchalign/src/fa/utr.rs and crates/batchalign/src/fa/utr/two_pass.rs

&* → +< Conversion

An experimental convert subcommand transforms &* to separate +< utterances using the typed AST:

1. Walk each utterance’s content (including inside groups).
2. Extract OtherSpokenEvent nodes, recording speaker + text.
3. Remove them from the host utterance.
4. For each extracted event, create a new Utterance with +< linker and words split from the underscore-joined text.
5. Insert after the host utterance.

Edge cases handled

• Multiple &* in one utterance (each becomes its own +< utterance)
• Multi-word &* with underscores (oh_okay_yeah → oh okay yeah)
• &* inside groups (<... &*INV:mhm ...> [//])
• Reverse direction (&*PAR:yeah on INV’s line)
• Host utterances with and without timing bullets
• Host dependent tiers preserved (they were already &*-invisible)

Corpus Statistics

&* (OtherSpokenEvent)

Top words: mhm (~12,500), yeah (~5,500), okay (~3,300), mm (~1,400).

+< (LazyOverlapPrecedes)

Of these, ~131,000 (40%) already have timing bullets.

File Locations

This page last changed: 2026-09-07 (commit 52b853df). The whole book last changed: 2026-09-16 (commit 34d249d8).