feat(ingest): 진행 로그 개선 — 파일명/phase/heartbeat/slowest 요약

OCR/caption 켜진 볼트 ingest 가 중간부터 느릴 때 TTY 진행바가 파일명·phase·
모델·경과시간을 안 보여 "멈춤"처럼 보이던 문제 해결.
- 신규 wire AssetPhase{idx,total,phase,model} + AssetTimings.ocr_ms/caption_ms
  (additive, ingest_progress.v1 유지)
- app: apply_ocr/apply_caption/embed 진입 시 AssetPhase emit + ocr/caption 시간 측정
- cli: TTY 진행바에 현재 파일명 + phase(model) + asset 경과초(heartbeat),
  종료 시 최장 소요 파일 top-5 요약(quiet 여도 출력, --json 미출력)
- wire schema / README / HANDOFF / HOTFIXES 동기화, version 0.26.0 → 0.27.0

검증(리더): clippy 0, kebab-app/cli 61그룹·parse-image/tui 14그룹 0실패(-j8).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-03 10:52:26 +00:00
parent c91ff909ce
commit aeaa18a564
11 changed files with 548 additions and 83 deletions

View File

@@ -96,14 +96,33 @@ pub enum IngestEvent {
/// `idx/total` while its per-chunk phases churn. `chunks` is the chunk
/// count for asset `idx`.
AssetChunked { idx: u32, total: u32, chunks: u32 },
/// v0.26.0 (additive): emitted when an asset enters a *slow* internal
/// phase, so the interactive progress bar can show **which** phase
/// (and which model) is currently running instead of looking frozen.
/// `phase` ∈ {`"ocr"`, `"caption"`, `"embed"`}; short phases
/// (parse / chunk / store) are intentionally *not* emitted to avoid
/// noise. `model` is the model performing the phase — the vision LLM
/// id for `ocr` / `caption`, the embedder `model_id` for `embed`
/// (`None` when the phase runs without a configured model, e.g. embed
/// with no embedder wired). Emitted once per (asset, phase); no
/// throttle needed (low frequency). Wire v1 consumers that predate
/// this variant simply ignore the unknown `asset_phase` kind.
AssetPhase {
idx: u32,
total: u32,
phase: String,
model: Option<String>,
},
/// v0.24.0 (additive): per-phase wall-clock (milliseconds) for asset
/// `idx`, emitted once the asset's markdown pipeline finishes. Lets a
/// user see *where* the time went (parse / chunk / embed / store)
/// without parsing logs. Only the markdown path emits this; the
/// image / PDF paths surface `AssetChunked` but skip phase timing (their
/// phase shapes differ — OCR / caption). `expansion_ms` is retained for
/// wire compatibility but is always 0 since doc-side expansion was
/// removed (HOTFIXES 2026-06-03).
/// `idx`, emitted once the asset's pipeline finishes. Lets a user see
/// *where* the time went (parse / chunk / ocr / caption / embed /
/// store) without parsing logs. The markdown path leaves `ocr_ms` /
/// `caption_ms` at 0 (no image analysis); the image / PDF paths fill
/// them so the slowest-asset summary attributes vision-model time
/// correctly. `expansion_ms` is retained for wire compatibility but is
/// always 0 since doc-side expansion was removed (HOTFIXES 2026-06-03).
/// `ocr_ms` / `caption_ms` (v0.26.0) are additive with serde default 0
/// so pre-v0.26.0 consumers deserialize cleanly.
AssetTimings {
idx: u32,
total: u32,
@@ -112,6 +131,10 @@ pub enum IngestEvent {
expansion_ms: u64,
embed_ms: u64,
store_ms: u64,
#[serde(default)]
ocr_ms: u64,
#[serde(default)]
caption_ms: u64,
},
/// Run finished normally. `counts` is the final aggregate.
Completed { counts: AggregateCounts },
@@ -261,19 +284,23 @@ mod tests {
expansion_ms: 45_000,
embed_ms: 800,
store_ms: 20,
ocr_ms: 1_200,
caption_ms: 3_400,
};
let v = serde_json::to_value(&ev).unwrap();
assert_eq!(
v.get("kind").and_then(|s| s.as_str()),
Some("asset_timings")
);
// All five phase fields are present (plain u64, always serialized).
// All phase fields are present (plain u64, always serialized).
for (field, want) in [
("parse_ms", 12u64),
("chunk_ms", 3),
("expansion_ms", 45_000),
("embed_ms", 800),
("store_ms", 20),
("ocr_ms", 1_200),
("caption_ms", 3_400),
] {
assert_eq!(
v.get(field).and_then(serde_json::Value::as_u64),
@@ -283,6 +310,64 @@ mod tests {
}
}
#[test]
fn asset_timings_ocr_caption_default_to_zero_for_legacy_wire() {
// v0.26.0 additive: a pre-v0.26.0 wire payload omits ocr_ms /
// caption_ms; serde `default` must fill 0 so old producers stay
// compatible.
let legacy = serde_json::json!({
"kind": "asset_timings",
"idx": 1, "total": 1,
"parse_ms": 5, "chunk_ms": 2, "expansion_ms": 0,
"embed_ms": 10, "store_ms": 3
});
let ev: IngestEvent = serde_json::from_value(legacy).unwrap();
match ev {
IngestEvent::AssetTimings {
ocr_ms,
caption_ms,
embed_ms,
..
} => {
assert_eq!(ocr_ms, 0);
assert_eq!(caption_ms, 0);
assert_eq!(embed_ms, 10);
}
other => panic!("unexpected event: {other:?}"),
}
}
#[test]
fn asset_phase_serializes_with_discriminator() {
// v0.26.0 additive variant — `kind` must be snake_case
// `asset_phase`, `phase` is the slow-phase label, `model` the
// model id (nullable).
let ev = IngestEvent::AssetPhase {
idx: 4,
total: 12,
phase: "ocr".into(),
model: Some("gemma4:e4b".into()),
};
let v = serde_json::to_value(&ev).unwrap();
assert_eq!(v.get("kind").and_then(|s| s.as_str()), Some("asset_phase"));
assert_eq!(v.get("idx").and_then(serde_json::Value::as_u64), Some(4));
assert_eq!(v.get("phase").and_then(|s| s.as_str()), Some("ocr"));
assert_eq!(v.get("model").and_then(|s| s.as_str()), Some("gemma4:e4b"));
}
#[test]
fn asset_phase_model_none_serializes_as_null() {
let ev = IngestEvent::AssetPhase {
idx: 1,
total: 1,
phase: "embed".into(),
model: None,
};
let v = serde_json::to_value(&ev).unwrap();
assert_eq!(v.get("phase").and_then(|s| s.as_str()), Some("embed"));
assert!(v.get("model").is_some_and(serde_json::Value::is_null));
}
#[test]
fn ingest_event_completed_has_counts() {
let ev = IngestEvent::Completed {