kebab/tasks/p8/p8-2-segment-chunker.md

phase, component, task_id, title, status, depends_on, unblocks, contract_source, contract_sections

phase	component	task_id	title	status	depends_on	unblocks	contract_source	contract_sections
P8	kebab-chunk (audio-segment-v1)	p8-2	Audio segment chunker (audio-segment-v1)	planned	p8-1		../../docs/superpowers/specs/2026-04-27-kebab-final-form-design.md	§3.5 Chunk §3.4 SourceSpan::Time §4.2 chunk_id recipe §0 Q3 citation §9 versioning

p8-2 — Audio segment chunker

Goal

Implement Chunker with chunker_version = "audio-segment-v1". Groups consecutive transcript segments into chunks that approach target_tokens while respecting speaker-turn boundaries (when present).

Why now / why this size

Per-medium chunker. Tiny but versioned — chunk_id depends on chunker_version so labeling matters.

Allowed dependencies

• kebab-core
• kebab-config
• serde, serde_json
• blake3 (policy_hash)
• serde-json-canonicalizer
• thiserror

Forbidden dependencies

• kebab-source-fs, kebab-parse-md, kebab-parse-pdf, kebab-parse-image, kebab-parse-audio (consumes via kebab-core only), kebab-normalize, kebab-store-*, kebab-embed*, kebab-search, kebab-llm*, kebab-rag, kebab-tui, kebab-desktop

Inputs

input	type	source
CanonicalDocument containing one AudioRefBlock with Transcript	kebab_core::CanonicalDocument	p8-1
ChunkPolicy	kebab_core::ChunkPolicy	kebab-app

Outputs

output	type	downstream
Vec<Chunk>	kebab_core::Chunk	kebab-store-sqlite, embedders

Public surface (signatures only — no new types)

pub struct AudioSegmentV1Chunker;

impl kebab_core::Chunker for AudioSegmentV1Chunker {
    fn chunker_version(&self) -> kebab_core::ChunkerVersion { kebab_core::ChunkerVersion("audio-segment-v1".into()) }
    fn policy_hash(&self, policy: &kebab_core::ChunkPolicy) -> String;
    fn chunk(&self, doc: &kebab_core::CanonicalDocument, policy: &kebab_core::ChunkPolicy) -> anyhow::Result<Vec<kebab_core::Chunk>>;
}

policy_hash = blake3(canonical_json(policy)) truncated to 16 hex chars.

Behavior contract

• Operates only on documents whose first block is Block::AudioRef with Some(transcript). Other documents → anyhow::Error("AudioSegmentV1Chunker only handles audio docs").
• Iterate transcript.segments (already in chronological order):
  • Greedily group adjacent segments until estimated token budget approaches policy.target_tokens (bytes / 4 proxy on segment text).
  • Force a split when segment[i].speaker != segment[i-1].speaker (only if speaker info present), even if budget not met.
  • No overlap across chunks (audio chunk overlap is rarely useful for retrieval).
• For each emitted chunk:
  • text = segments.iter().map(|s| s.text).join(" ").
  • source_spans = vec![SourceSpan::Time { start_ms: first.start_ms, end_ms: last.end_ms }] (single span covering the whole chunk).
  • heading_path = vec![].
  • block_ids = [audio_ref_block.block_id] (always one block per chunk).
  • token_estimate = byte_len / 4.
• Empty transcript (segments.is_empty()) → Vec::new() (no chunks).
• Speaker label for citation: if all segments in a chunk share a speaker, the chunk's Citation::Time { speaker: Some(...) } (constructed downstream by retrieval) preserves it. This task's responsibility ends at populating source_spans; retrieval-side citation construction reads transcript.segments from DB to attach speaker (or this chunker can serialize speaker into a small extension JSON in chunk.heading_path — chosen approach: leave the speaker propagation to the retriever, NOT the chunker, because including it in chunk_id would couple speakers into chunk_id).
• Determinism: identical Transcript.segments + identical policy → identical chunk_ids and text.

Storage / wire effects

• None.

Test plan

kind	description	fixture / data
unit	5 segments under target → 1 chunk; total span = first.start_ms..last.end_ms	inline
unit	20 segments well above target → multiple chunks, none cross speaker change	inline (with synthetic speakers)
unit	empty transcript → empty Vec	inline
unit	non-audio doc returns error	inline (Markdown-like doc)
determinism	same input → same chunk_ids twice	inline
snapshot	Vec<Chunk> JSON for fixture transcript stable	fixtures/audio/transcript-1.json (constructed)

All tests under cargo test -p kebab-chunk audio.

Definition of Done

• cargo check -p kebab-chunk passes (md-heading-v1 + pdf-page-v1 + audio-segment-v1 all coexist)
• cargo test -p kebab-chunk audio passes
• Snapshot stable across two runs
• No imports outside Allowed dependencies
• PR links design §3.5, §3.4 SourceSpan::Time, §4.2

Out of scope

• Diarization-aware chunking beyond honoring existing speaker boundaries.
• Time-overlap chunks (intentionally not supported in v1).
• Real tokenizer integration (P+ replaces byte proxy across all chunkers).

Risks / notes

• Speaker boundary forcing can create very small chunks if speakers alternate fast (e.g., interview Q/A). Document a policy.min_segments_per_chunk knob (default 1) to optionally suppress force-splits below the floor — implementer's call to add a config knob if metric pressure demands.
• Citation speaker inference at retrieval time needs DB lookup of transcript_segments (or a transcript_segments table — none exists yet). For v1, surface speaker info via the wire Citation::Time.speaker only when the retriever can confidently attach it; otherwise leave None. This task does not block on that decision.
• Bumping chunker_version invalidates downstream embeddings; treat as a versioning event per §9.