118 Commits

Author SHA1 Message Date
803d02b68b fix(dogfood): enforce workspace.include in walker (allow-list semantics)
config.workspace.include was completely ignored by the walker — connector.rs
log_scope_include_warning literally said "handled by extractor router" but
no extractor router exists. Dogfooding (PR #142 1B + multi-root corpus
kebab-docs + httpx + zod + lodash) showed user-set include of code+md still
ingested 84 .png + 8 .pdf files.

Fix: walker treats scope.include as an allow-list — empty Vec preserves
backward-compat (all files pass), non-empty requires file path to match at
least one pattern (AND with the existing exclude rules). Removed the
misleading debug log.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 05:15:04 +00:00
16dc02cfa2 chore: bump version 0.8.0 → 0.8.1
dogfood-discovered code_lang/repo filter bug (PR #144) fix lands in
production. patch bump because:
- 1A-1 advertised CLI flags --code-lang / --repo were live but inert
  (SearchFilters fields propagated but never applied to retriever SQL)
- fix restores intended behavior; no new wire surface
- user has dogfooded against httpx + zod + lodash and re-validating
  needs the fixed binary

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 03:35:36 +00:00
44813df052 docs(p10-1b): README/HANDOFF/ARCHITECTURE/SMOKE/INDEX + HOTFIXES; chore: bump version 0.7.0 → 0.8.0
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 01:48:06 +00:00
0c61758931 build(p10-1b): add tree-sitter-python/-typescript/-javascript workspace deps
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 00:28:31 +00:00
80c2d31fb3 docs(p10-1a-2): README/HANDOFF/ARCHITECTURE/SMOKE/INDEX + HOTFIXES; chore: bump version 0.6.0 → 0.7.0
- README: note Rust .rs ingest active (code-rust-ast-v1), update Mermaid parse node + chunker labels, update supported formats note in Quick start and ingest command table; add code citation fields (symbol, code_lang, repo) and filter flags note
- HANDOFF: flip P10 row to note 1A-1  + 1A-2 PR open; add one-liner cross-link to HOTFIXES 2026-05-19 entries
- ARCHITECTURE: add kebab-parse-code node + edge (app → pcode, pcode → ptypes) to Mermaid graph; add directory tree entry; add code parser locked-in decision row (tree-sitter lives parser-side, design §6.3)
- SMOKE: add P10-1A-2 Rust code ingest section (ingest.code config keys, verification steps, known behaviors); add checklist item
- tasks/INDEX.md: flip p10-1A-1 to , update p10-1A-2 to 🟡 PR open
- tasks/p10/INDEX.md: same flips
- tasks/HOTFIXES.md: add two 2026-05-19 dated entries (AST_CHUNK_MAX_LINES constant vs config deviation + SourceType::Code deferred)
- tasks/p10/p10-1a-2-rust-ast-chunker.md: append two HOTFIXES cross-link lines in Risks/notes
- docs/superpowers/specs/2026-04-27-kebab-final-form-design.md §10.1: note p10-1A-2 surface activation
- Cargo.toml: version 0.6.0 → 0.7.0 (dogfooding-ready = minor bump trigger per CLAUDE.md)
- Cargo.lock: regenerated

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 22:48:11 +00:00
580576c2c6 feat(p10-1a-2): wire code ingest dispatch (ingest_one_code_asset)
Add `MediaType::Code("rust")` dispatch arm in `ingest_one_asset`,
`ingest_one_code_asset` fn (faithful mirror of `ingest_one_pdf_asset`),
and `backfill_code_lang` post-processing in `App::search_uncached`.
Integration test `code_ingest_smoke.rs` verifies full pipeline:
ingest `.rs` → Citation::Code hit with lang/symbol/line_start.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 20:14:59 +00:00
c74f8d269e chore(p10-1a-2): sync Cargo.lock for kebab-parse-code deps (Task 6 follow-up)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:36:54 +00:00
5c265bb59f build(p10-1a-2): add tree-sitter + tree-sitter-rust workspace deps
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 15:38:19 +00:00
th-kim0823
40b3ea8408 chore(p10-1a-1): cleanup Task 11 review findings + sync Cargo.lock
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 16:50:55 +09:00
th-kim0823
ff11f81f7f feat(p10-1a-1): kebab-parse-code crate (lang + repo + skip)
Tasks 5-8: new `kebab-parse-code` crate with three infrastructure modules
for the code ingest framework. Ships lang.rs (extension→language identifier
mapping), repo.rs (.git walk-up via gix 0.70 for RepoMeta), and skip.rs
(BUILTIN_BLACKLIST, is_generated_file, is_oversized). 14 integration tests
across three test files, all passing; clippy -D warnings clean.

Note: gix pinned to 0.70 (not 0.83 as originally suggested) because 0.83
fails to compile against Rust 1.94.1 due to non-exhaustive match patterns
in gix-hash. 0.70 resolves cleanly and has identical head_name/head_id API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 15:57:59 +09:00
th-kim0823
c3177561b9 chore: bump version 0.5 → 0.6
v0.6.0 batches RAG quality batch:
- fb-38 score semantics (search_hit.v1 score_kind)
- fb-40 fact-grounded answer (rag-v2 prompt template)
- fb-42 bulk multi-query (kebab search --bulk + mcp__kebab__bulk_search)
- fb-39 eval foundation (precision_at_k_chunk metric)
- fb-39b embedding upgrade (multilingual-e5-large default)

embedding_version cascade triggers minor bump per design §9.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 23:56:51 +09:00
th-kim0823
668e1174cc chore: bump version 0.4 → 0.5
v0.5.0 batches fb-32 (stale doc indicator) + fb-33 (streaming ask)
+ fb-34 (output budget controls) + fb-35 (verbatim fetch) + fb-36
(search filter args) + fb-37 (trace + stats).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 17:04:51 +09:00
th-kim0823
b221686133 chore: sync Cargo.lock for kebab-mcp time dep (fb-36)
PR #127 added time = { workspace = true } to kebab-mcp/Cargo.toml
but Cargo.lock entry was not regenerated before merge. cargo build
on main locally regenerates the +time line under kebab-mcp.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 11:04:17 +09:00
th-kim0823
ebbc3a46ae feat(app): cursor encode/decode for paginated search (fb-34)
Opaque base64(JSON{offset, corpus_revision}). Mismatch or
malformed input returns ErrorV1 with code = stale_cursor.
base64 promoted to workspace dep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 17:49:23 +09:00
th-kim0823
df42d8f621 chore: bump version 0.3.3 → 0.4.0
fb-32 머지로 wire schema 가 search_hit.v1 / citation.v1 의 required
필드를 두 개 (indexed_at, stale) 확장 — additive minor 로 분류했지만
strict validator 입장에서는 한 번 깨진 셈이라 minor bump.

surface 변경 (사용자 도그푸딩 영향):
- 모든 search hit / RAG citation 의 wire JSON 에 indexed_at (RFC3339) +
  stale (bool) 두 필드 추가
- CLI plain 출력 — stale doc 의 doc_path 옆에 [stale] tag (TTY = 노란색)
- TUI Search/Inspect/Ask pane — stale doc 의 doc_path 좌측에 [STALE] 배지
  (Theme::Warning role)
- config.toml [search] stale_threshold_days 신규 (default 30, 0 = 비활성)
- env KEBAB_SEARCH_STALE_THRESHOLD_DAYS

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 12:32:37 +09:00
th-kim0823
15cdc97cae feat(cli): [stale] tag on plain search output (fb-32)
Yellow when TTY, plain when not. JSON path inherits via serde
on the domain type; no CLI-side wire change needed there.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 02:24:54 +09:00
th-kim0823
f4ce6652b2 feat(search/lexical): populate SearchHit.indexed_at (fb-32)
JOIN documents.updated_at. stale defaults to false; App facade
post-processes against config threshold.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:10:20 +09:00
th-kim0823
6b13d8e11f chore: bump version 0.3.2 → 0.3.3 2026-05-07 21:10:23 +09:00
th-kim0823
751377cae8 chore: bump version 0.3.1 → 0.3.2
fb-31 single-file/stdin ingest + MCP ingest tools + docs/mcp-usage.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 18:59:03 +09:00
th-kim0823
5495d96275 chore: bump workspace version 0.3.0 → 0.3.1 — fb-30 release
agent integration MVP (fb-30 MCP server stdio) 머지 후 patch bump.

trigger:
- 신규 CLI surface `kebab mcp` (additive, 기존 명령 동작 무영향)
- new crate `kebab-mcp` (lib only, transitive 만)
- capability flag `mcp_server: false → true` (additive on schema.v1)
- design §10.2 MCP transport 절 추가 (additive subsection)

CLAUDE.md release 규약상 wire schema additive / surface 추가는
minor bump 영역이지만, pre-1.0 (0.x.y) 단계에서는 fb-26~31 group
누적 시까지 0.3.x patch 로 묶고, 큰 jump 시 0.4.0 cut 으로 운용.
fb-30 단독으로 break 없는 additive 라 0.3.1 patch 적정.

후속 fb-26 / 28 / 31 머지 시점에 추가 0.3.x patch 누적, breaking
schema 변경 시 0.4.0 minor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 17:06:45 +09:00
th-kim0823
4a30959fdd feat(kebab-cli): kebab mcp subcommand (fb-30)
Wires kebab_mcp::serve_stdio into kebab-cli. `--config <path>` honored
via the established Config::load pattern.

Updated serve_stdio signature to (Config, Option<PathBuf>) so the doctor
tool's path-aware behavior works correctly via KebabAppState.

Smoke test spawns the binary + sends initialize + initialized +
tools/list over stdin, asserts 4 tools returned. Confirms the MCP
server boots end-to-end via the real binary (rmcp 1.6 has no
in-memory test transport, so this is the only end-to-end assertion).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 16:10:17 +09:00
th-kim0823
2c09ed6af4 🏗️ chore(kebab-mcp): scaffold new crate (fb-30)
Empty lib + serve_stdio entry that bails until Task 3 wires rmcp. Adds
rmcp 1.6 to workspace dependencies (server + macros + transport-io +
schemars features) + tokio multi-thread/io-util/io-std local extensions.

schemars declared as "1" (resolved to 1.2.1) — matches rmcp 1.6's ^1.0
requirement (verified via crates.io /dependencies; plan literal was 0.9
which would conflict). Path-style refs for kebab-app / kebab-config /
kebab-core follow workspace convention.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 15:25:57 +09:00
th-kim0823
1f53930234 🏗️ refactor(kebab-app): promote error_classify → kebab-app::error_wire (fb-30 prep)
fb-30 의 새 crate `kebab-mcp` 가 동일 classify 모듈 사용 — UI crate 끼리
import 는 facade rule 위반이므로 kebab-app 으로 promotion. fb-27 commit
c91228e 의 코드 그대로 이전 (struct + classify + classify_llm + 7 unit
test). reqwest dev-dep 도 함께 이동.

kebab-cli 는 `kebab_app::ErrorV1` / `kebab_app::classify` 로 import 경로
1줄 변경 + wire.rs 의 `&crate::error_classify::ErrorV1` 1줄 교체. 동작
무영향.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 15:13:28 +09:00
50ad7f1f28 Merge pull request 'chore: bump version 0.2.1 → 0.3.0 (fb-27 release)' (#105) from chore/bump-v0.3.0 into main
Reviewed-on: #105
2026-05-07 04:48:53 +00:00
th-kim0823
73f5d73112 chore: bump workspace version 0.2.1 → 0.3.0 — fb-27 release
agent foundation MVP 첫 component (fb-27 introspection + structured
error wire) 머지 후 minor bump. trigger:

- frozen design contract 변경 (§10.1 capability matrix subsection 신설)
- wire schema additive (`schema.v1` + `error.v1` 신규)
- 신규 CLI surface (`kebab schema`)

이전 binary (v0.2.1) 는 새 wire schema / error wire 인지 못 함 — 0.3.0
부터 agent integration (fb-30 MCP, fb-29 daemon, fb-28 readonly/quiet 등)
의 capability discovery + structured error 활용 가능.

후속 fb-26 / 28 / 29 / 30 / 31 머지는 0.3.x patch / 0.4.0 누적 예정.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 13:38:05 +09:00
th-kim0823
c91228e7d5 feat(kebab-cli): error_classify dispatcher + wire helpers (fb-27)
`error_classify::classify` maps anyhow::Error → ErrorV1 wire record by
downcasting to known typed errors (LlmError + ConfigInvalid + NotIndexed
re-exported from kebab_app::error_signal, plus std::io::Error chain).
Generic fallback emits `code: "generic"` with the chain in `details` when
verbose.

wire.rs adds wire_schema (idempotent re-tag, mirrors wire_doctor pattern
since SchemaV1 carries its own schema_version field) and wire_error_v1
(simple tag_object). Tests pin both wrappers + 7 classify code paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 12:11:54 +09:00
th-kim0823
58c06664b8 feat(kebab-config): add ConfigInvalid typed error (fb-27)
Wraps every error path in `Config::from_file` (read failure, TOML parse,
validation) so downstream callers can `downcast_ref::<ConfigInvalid>()`
to build the `error.v1` wire record. kebab-app re-exports the type via
its `error_signal` module.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 11:22:58 +09:00
1409eaae51 chore: bump workspace version 0.2.0 → 0.2.1 — p9-fb-25 release
사용자 도그푸딩 / 실사용 binary 필요로 release bump (CLAUDE.md release
규약 트리거 1).

p9-fb-25 변경사항:

- WorkspaceCfg.include 제거 (옛 config 의 include = [...] silently
  무시 + 단발 deprecation warning).
- IngestItem.warnings 가 Skipped 시 사유 채움 (`unsupported media
  type: .docx` 등).
- IngestReport.skipped_by_extension: BTreeMap<String, u32> 신규
  (additive wire — v1 호환).
- CLI / TUI summary 에 breakdown (`5 skipped: 3 docx, 1 txt`).
- kebab init 헤더 + README 에 지원 형식 (md / png / jpg / pdf) 명시.

caller breaking 없음 (모든 surface 변경 backward-compat). 730
워크스페이스 테스트 통과.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:49:10 +00:00
ce68885d92 chore: bump workspace version 0.1.0 → 0.2.0 for next dev cycle
v0.1.0 release tagged on 2026-05-05 — first official release covering
P0~P4 + P5 + P6~P7 + P9 (UI) 의 도그푸딩 사이클 1회 완성.

이후 도그푸딩 follow-up (p9-fb-21 ~ p9-fb-24) 까지 v0.1.0 에 포함:
- p9-fb-22: TUI input cursor mid-string editing + Ask follow-tail.
- p9-fb-23: incremental ingest (skip unchanged docs).
- p9-fb-24: TUI status/key bar + Library 컬럼 헤더 + PgUp/PgDn.

다음 dev cycle 부터 0.2.0. minor bump rationale (semver pre-1.0):

- Wire schema additive (`IngestReport.unchanged`, `IngestEvent` 의
  `Unchanged` variant) — backward-compat 유지 but 신규 surface.
- 신규 CLI flag (`--force-reingest`).
- 신규 SQLite migration V006 (incremental ingest 컬럼).
- TUI surface 변경 (status bar 항상 노출, Library 헤더 row, PgUp/PgDn,
  cursor 편집).
- IngestOpts struct 도입 (AskOpts 패턴) — 기존 wrapper 보존하므로
  caller breaking 은 없음.

기존 ~723 워크스페이스 테스트 무수정 통과 (CARGO_PKG_VERSION 가 env!
로 동적 — TUI status bar 테스트 자동 추적).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 02:28:20 +00:00
0e408fb1b5 feat(kebab-app + kebab-store-sqlite): p9-fb-19 search LRU cache + corpus_revision
도그푸딩 item 15 — TUI / 같은 process 안에서 동일 query 반복 시 SQLite
FTS + Lance + RRF 재계산이 매번 발생하던 비용 해소. in-process LRU
캐시 + 모노토닉 corpus_revision 카운터로 ingest commit 발생 시 모든
entry 자동 stale.

## 핵심 변경

- **SQLite V004 migration**: `kv (key TEXT PRIMARY KEY, value TEXT)
  STRICT` + `corpus_revision = '0'` seed. 미래의 다른 scalar 도 같은
  테이블에 들어갈 수 있는 generic shape.
- **`SqliteStore::corpus_revision()` / `bump_corpus_revision()`** —
  `UPDATE ... CAST AS INTEGER + 1` atomic. INSERT-OR-IGNORE 도 함께
  실행 (V004 seed 가 무슨 이유로 누락된 케이스 paranoid).
- **`kebab-app::ingest_with_config_cancellable`** — `new + updated > 0`
  시 bump, no-op (skipped-only) reingest 는 cache 보존.
- **`App.search_cache: Option<Mutex<LruCache<SearchCacheKey, Vec<
  SearchHit>>>>`** — `config.search.cache_capacity` (default 256, 0
  비활성). `lru = "0.12"` workspace dep 추가.
- **`SearchCacheKey`** = `query_norm` (NFKC + trim + lowercase) +
  `mode` + `k` + `snippet_chars` + `embedding_version` (vector/hybrid
  만, lexical 은 빈 문자열) + `chunker_version` + `corpus_revision`
  snapshot.
- **`App::search`** rewrite — cache 활성 시 lookup → miss 면 기존
  `search_uncached` 호출 후 put. cache 비활성이거나 lock 실패면
  straight-line.
- **`App::search_uncached`** (rename of pre-fb-19 `search` body) +
  `search_uncached_with_config` facade — CLI `kebab search --no-cache`
  로 진입.
- **`Config.search.cache_capacity: usize`** field, `#[serde(default)]`
  로 기존 config 호환.
- **CLI `--no-cache`** flag — 디버깅용 (CLI 는 매 호출이 새 process
  라 사실상 no-op 이지만 spec 명시 + 향후 long-lived process 호환).
- **frozen design §9 versioning** 표에 `corpus_revision` row 추가
  (기존 `index_version` 라벨과 다른 차원: 라벨은 retrieval 형상,
  corpus_revision 은 ingest commit ack).

## 테스트

- `kebab-store-sqlite` 신규 3 unit (fresh=0, monotonic bump, persist
  across reopen)
- `kebab-app` 신규 4 integration (cached repeat 같은 hits, NFKC 정규화
  로 case/whitespace collapse, --no-cache parity, first ingest bumps
  corpus_revision)
- 워크스페이스 전체 `cargo test --workspace --no-fail-fast -j 1` exit 0
- `cargo clippy --workspace --all-targets -- -D warnings` clean

## 문서

- README `kebab search` 행: 캐시 동작 + `--no-cache` 안내 + corpus_
  revision 무효화 메커니즘
- docs/SMOKE.md `[search]` 절에 `cache_capacity` 라인 추가
- HANDOFF: 2026-05-03 entry
- spec status planned → in_progress

## Out of scope

- patch-and-merge incremental (RRF 정규화 전체 hit set 기준이라 어려움)
- SQLite 영속 cache (P+)
- 다른 process 간 cache 공유 (in-process 만 — corpus_revision 이
  cross-process 무효화는 O(1))

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:01:31 +00:00
702c7c89f7 review(p9-fb-05): 회차 1 nit 반영
- `Config.source_dir` 를 `pub(crate)` 로 좁힘. invariant ("from_file
  / load 만이 정당한 setter") 가 외부 mutation 으로 깨지지 않도록.
  대신 `pub fn source_dir(&self) -> Option<&Path>` (read-only) +
  `pub fn with_source_dir(self, dir) -> Self` (builder) 노출 — 테스트
  / 프로그래마틱 사용은 builder 통과.
- `resolve_workspace_root` 의 `current_dir()` 실패 fallback 에
  `tracing::warn!` 추가. chroot / deleted-cwd / permission 문제로
  cwd 가 안 잡힐 때 silently `./root` 로 떨어지지 않고 로그가 남음.
  `tracing` 을 kebab-config 의 deps 에 추가 (workspace dep).

테스트 27 통과 + 워크스페이스 clippy clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 04:23:16 +00:00
c462dbf6a4 feat(kebab-tui): p9-fb-11 ask answer markdown rendering
도그푸딩 item 9 — TUI Ask 답변 본문이 raw `**bold**` / `# Title` /
` ```code``` ` 그대로 보여 가독성 떨어지던 문제 해소. pulldown-cmark
파싱 → ratatui Span/Line 변환.

## 핵심 변경

- **`kebab-tui::markdown::render(text, &Theme) -> Vec<Line<'static>>`**
  신규. pulldown-cmark = "0.13" (이미 kebab-parse-md 가 사용 중인
  버전) 위에 build.

  inline:
  - `**bold**` / `__bold__` → `Modifier::BOLD`
  - `*italic*` / `_italic_` → `Modifier::ITALIC`
  - `~~strike~~` → `Modifier::CROSSED_OUT`
  - `` `code` `` → `Role::Hint` (DIM 스타일 — 터미널 호환성 위해 bg
    color 보다 안전)
  - `[text](url)` → `Role::CitationMarker` + `Modifier::UNDERLINED`

  block:
  - heading H1/H2 → `Role::Heading` (Cyan + BOLD), H3-H6 → `Role::Title`
    (White + BOLD)
  - bullet list `-`/`*` → `- ` + 깊이별 indent
  - ordered list `1.` → 실제 번호 prefix + indent
  - fenced code block ``` ``` ``` → `  ` indented + `Role::Hint`
  - blockquote `>` → 좌측 `▎` bar (중첩 시 반복) + `Role::Hint`
  - table `| col |` → `| col1 | col2 |` 식 줄, `|` separator 색 강조
  - horizontal rule `---` → `─` × 40

- **streaming 안전성**: 매 frame 재 parse 가 spec — pulldown
  토크나이저가 µs/KB 라 비용 무시. unterminated `**` (사용자가 한창
  입력 중인 inline 가 닫히기 전) 은 pulldown 이 Text 로 처리 →
  literal `**` 그대로 표시 (글자 누락 X).

- **`ask::push_turn_lines` 통합**: grounded 답변에서만 markdown
  렌더 사용. refusal turn (`Role::Warning` override) 와 streaming
  turn (`Role::Hint`) 은 raw 로 두어 role color 시그널이 markdown
  스타일에 묻히지 않도록. body line 들은 `  ` indent 로 transcript
  에서 답변 본문 시각 구분.

- **CLI `kebab ask` 출력은 raw markdown** — 터미널 호환성 + pipe
  처리 시 안정성 위해 (ANSI escape 없이 plain text).

## 테스트 (markdown.rs 14 unit)

- empty input → 빈 라인 1 줄 (caller scroll/measure 안전)
- plain text → 단일 라인 + paragraph blank
- bold / italic / strikethrough / inline code → 해당 modifier 검증
- link → UNDERLINED 검증
- heading H1 → BOLD 텍스트 span
- bullet list `-` / numbered list `1./2.` → prefix 검증
- code fence body → 줄별 `  ` indent 보존
- blockquote → `▎` prefix
- 2x2 table → `|`-separated 줄 검증
- unterminated `**` → 글자 누락 없음 (streaming 안전성 회귀 방지)
- composite (heading + para + list + code) → 문서 순서 보존

기존 75 TUI 테스트 + 신규 14 markdown = 89 통과. clippy clean.

## 문서

- README `kebab tui` 행에 markdown 렌더 안내 + CLI 는 raw 명시
- HANDOFF: 2026-05-03 entry
- spec status planned → in_progress

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 03:33:21 +00:00
fa02a7c68d feat: ingest cooperative cancellation (p9-fb-04)
Ctrl-C / Esc 가 ingest 를 즉시 중단. 현재 in-flight asset 마무리 후
이후 asset 미실행, IngestEvent::Aborted { partial_counts } 발신,
Ok(IngestReport) 정상 반환 (Err 아님). 부분 commit 보존, 다음 ingest
가 idempotent 재개.

신규 facade: kebab-app::ingest_with_config_cancellable(.., progress,
cancel: Option<Arc<AtomicBool>>). 기존 _progress 가 cancel=None
forwarding wrapper. asset loop 시작 boundary 마다 atomic load —
true 면 break + Aborted emit + 정상 종료. Lock 없음.

CLI: ctrlc crate 신규 dep. SIGINT handler 가 첫 신호에 cancel.store(true)
+ stderr hint, 두 번째 신호에 std::process::exit(130) (canonical SIGINT
exit code). install_sigint_cancel() helper 가 Arc<AtomicBool> 반환,
Cmd::Ingest 가 facade 에 전달.

TUI: IngestState 에 cancel: Arc<AtomicBool> field 추가 (회차 1 review
결과의 reshape 정확). start_ingest 가 둘 다 만들어 worker 에 clone
move. cancel_running_ingest(&app) helper — Esc / Ctrl-C 가
ingest 진행 중일 때만 cancel 우선, 그 외에는 quit.

Test:
- 3 facade integration (cancel-before / cancel-mid / no-cancel
  default).
- 3 tui lib unit (cancel_running_ingest no-state / in-flight /
  terminated).

Plan 갱신: p9-fb-04 status planned → in_progress. 머지 후 한 줄
commit 으로 completed flip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 21:36:17 +00:00
e613236d60 feat(cli): kebab ingest progress display (p9-fb-02) + p9-fb-01 status flip
`kebab ingest` 가 진행 상황을 사용자에게 보여주는 두 surface 추가:

- **사람 모드 (TTY)**: indicatif `ProgressBar` on stderr — scan 중에는
  spinner, ScanCompleted 후 bar 로 전환, 매 asset 마다 message 갱신.
- **사람 모드 (non-TTY, CI/pipe)**: indicatif draw target 을 hidden
  으로 두고 stderr 에 한 줄씩 (`ingest: scanning`, `ingest: 1/N path`,
  `ingest: complete (...)`).
- **`--json` 모드**: stderr 비우고 stdout 에 line-delimited
  `ingest_progress.v1` JSON 을 emit. 마지막 줄은 기존
  `ingest_report.v1` 그대로 (외부 wrapper backward-compat).

구현:

- 신규 `crates/kebab-cli/src/progress.rs` — `ProgressMode::{Json,
  Human { tty }}`, `ProgressDisplay` (background thread 가 channel
  drain + 모드별 render), `now_rfc3339` helper. mode 가 무엇이든 ts
  는 wire emit 시점에 stamp.
- `crates/kebab-cli/src/wire.rs` 에 `wire_ingest_progress` 추가.
  serde tag (`kind`) 위에 `schema_version` + `ts` 두 필드 더해 spec
  §2.4a wire shape 완성.
- `Cmd::Ingest` 핸들러: mpsc channel 만들고 background thread 가
  display 돌리는 동안 main 이 `ingest_with_config_progress` 호출.
  ingest 반환 시 Sender drop → display thread 정상 종료. join 후
  최종 ingest_report 출력.
- 새 dep: `indicatif` 0.17 (TTY 전용 진행 바, non-TTY/--json 에서는
  hidden draw target).

Test:

- 3 lib unit (mode resolution + RFC 3339 round-trip).
- 3 integration (--json line-delimited / non-TTY stderr text /
  ts+kind 검증). 16 PASS 전체 회귀 0.

Plan 갱신:

- p9-fb-01: status `in_progress` → `completed` (PR #52 머지 후속).
- p9-fb-02: status `planned` → `in_progress`. 머지 후 별도 한 줄
  commit 으로 `completed` flip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 19:57:02 +00:00
8879bd4de2 chore(p9-fb-06): sync Cargo.lock for tempfile dev-dep + flip spec to completed
Two follow-ups after PR #49 (kebab reset) merged:
- Cargo.lock: kebab-cli's new dev-dep `tempfile` was committed in the
  feature PR but the lockfile entry was not regenerated, leaving main
  with a stale lock. `cargo metadata` regenerates the one-line addition
  to kebab-cli's dependency list.
- tasks/p9/p9-fb-06-data-reset-command.md: status `in_progress` →
  `completed`, per the plan's task 6 commitment to flip in a separate
  one-line commit only after the implementation PR merges.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 18:47:23 +00:00
b6e0ab352f feat(kebab-tui): P9-4 Inspect pane — doc/chunk detail with collapsible sections
Library Enter / Search 'i' 가 Inspect 진입. Doc 또는 Chunk 단일 view 로
metadata / provenance / blocks (doc) 또는 spans / text / embeddings (chunk)
6 section 을 collapsible 로 표시. Esc/q 로 originating pane 으로 복귀.

핵심:
- InspectTarget enum (`Doc(DocumentId) | Chunk(ChunkId)`).
- InspectState 본체 (`app.rs`) — target / doc / chunk / collapsed
  HashSet / scroll / return_to / needs_fetch / loading.
- `src/inspect.rs`:
  - `render_inspect` — target 종류별 render_doc / render_chunk 분기,
    section header 가 collapse marker (▾/▸) 표시. metadata.user JSON
    pretty-printed.
  - `handle_key_inspect`: j/k / Down/Up scroll. PageDown/PageUp 10 row.
    c = toggle all sections (v1 simplification). Esc/q = SwitchPane(return_to).
  - `enter_inspect(state, target, return_to)` helper — Library 와 Search
    공통 entry point.
  - run-loop hook `refresh_inspect` — needs_fetch 면 lazy
    inspect_doc_with_config / inspect_chunk_with_config.
- run.rs: Pane::Inspect arm 이 handle_key_inspect + render_inspect.
  Idle tick 마다 refresh_inspect. SwitchPane(Inspect) lazy init.
- Library: Enter 가 enter_inspect(Doc(selected)) 호출 후 SwitchPane.
- Search: 'i' (plain modifier) 가 enter_inspect(Chunk(selected_hit))
  호출 후 SwitchPane. typing 'i' (\"instance\") 와 충돌 가드.

테스트 12개 (`tests/inspect.rs`, TestBackend) — Esc 가 return_to 사용
/ q 도 동작 / j/k scroll bounds / PgUp PgDn ±10 / c 일괄 toggle / no
target hint / loading / doc view header+metadata+provenance+blocks /
collapse hides body / chunk view text+block_ids / no slot →
SwitchPane(Library) / enter_inspect helper sets fields.

Spec deviation (HOTFIXES `2026-05-02 P9-4`):
- `render_inspect<B: Backend>` generic 제거 (P9-1/2/3 와 동일).
- Search `i` 키 추가 (P9-2 spec 에 없었음, P9-4 retroactive 추가).
- `c` 일괄 collapse — spec 의 \"focus 기반 selective collapse\" 는 P+.

Docs (sync rule):
- README: TUI 행 \"4 패널\" + Quick start 코멘트.
- HANDOFF: 한 줄 요약 + Phase status (P9 3/5 → 4/5) + deviation 한 줄.
- HOTFIXES: P9-4 entry.
- tasks/p9/p9-4 status: completed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 15:41:11 +00:00
43ff4048e8 feat(kebab-tui): P9-1 Ratatui shell + Library pane
새 crate `kebab-tui` 가 §8 facade rule 따라 `kebab-app` 만 import.
Ratatui 0.28 + crossterm 0.28 기반 shell 이 다음을 제공:

- `App` 구조체: config + focus + library + 3 Option sub-state slot
  (search/ask/inspect — p9-2/3/4 가 자기 모듈에서 채우는 parallel-safety
  contract). p9-1 외에 App 정의 손대지 않음.
- `Pane` enum (Library/Search/Ask/Inspect/Jobs).
- `KeyOutcome` (Continue/Quit/SwitchPane/Refresh).
- `LibraryState` + 내부 inner: docs / list_state / filter / filter_edit /
  needs_refresh / loading / pending_g.
- `render_library` (Frame, area, &App) — heading/body, filter overlay
  toggleable, Korean/wide-char 너비는 unicode-width 로 계산.
- `handle_key_library`: j/k/Down/Up 이동, gg/G 끝, f 필터 overlay,
  /=>Search ?=>Ask Enter=>Inspect, q/Esc 종료. error overlay 가 켜
  있으면 어떤 키든 dismiss.
- 필터 overlay: tags_any (CSV) + lang 두 필드, Tab cycle, Enter
  apply→Refresh, Esc cancel.
- `ErrorOverlay`: anyhow chain 캡쳐 후 popup 렌더 (Clear + 빨간 border).
- 터미널 lifecycle: `TuiTerminal` 가 enter raw mode + alt screen,
  Drop 이 종료 시 (panic 포함) restore — 사용자 쉘 깨지지 않게.
- 비동기 없음: facade 호출은 main thread 동기. v1 의 brief hang 수용.

CLI: `kebab tui` 서브커맨드 추가, --config 받아 App::new + run.

테스트 10건 (`tests/library.rs`, TestBackend 사용):
- 빈 library / 3-doc render / q,Esc quit / / Search 전환 / ? Ask 전환
- Enter 빈 list 무동작 / Enter Inspect 전환 / j 이동 (3-step clamp) /
  f 필터 overlay → 입력 → Enter Refresh.

Test seam: `App::populate_library_for_testing` (#[doc(hidden)]) 가
`pub(crate)` inner 를 우회. spec parallel-safety contract 그대로 유지.

Spec deviation (HOTFIXES `2026-05-02 P9-1`):
- `render_library` 의 `<B: Backend>` generic 제거 — ratatui 0.28 의 Frame
  이 backend-agnostic.
- `populate_library_for_testing` 추가 (test seam, 공식 API 아님).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 13:26:24 +00:00
5f3a37cafa feat(kebab-app): P7-3 PDF ingest wiring — kebab ingest 가 PDF 자산도 처리
P7-1 (`PdfTextExtractor`) + P7-2 (`PdfPageV1Chunker`) 의 라이브러리를
`kebab-app::ingest_with_config` 에 와이어링. `kebab-source-fs` 가 이미
`*.pdf` 를 `MediaType::Pdf` 로 분류하던 자산이 이제 검색 가능한 doc 으로
색인됨. P6-4 image wiring 패턴과 평행 — `ingest_one_asset` 에 `MediaType::Pdf`
arm 추가, 새 private fn `ingest_one_pdf_asset` 로 분기.

핵심 동작:
- per-medium chunker 선택: PDF 자산은 `PdfPageV1Chunker` 하드코딩 (compile-time
  match 기반). `config.chunking.chunker_version` 은 markdown 만 represent —
  PDF 는 항상 `pdf-page-v1`. HOTFIXES entry `2026-05-02 P7-3` 에 deviation 기록.
- encrypted PDF / corrupt PDF → `errors+=1` + P7-1 의 `qpdf --decrypt` hint
  를 `IngestItem.error` 에 verbatim 보존.
- 빈/scanned candidate 페이지 → 0 chunk, P7-1 의 `Provenance::Warning` 그대로
  통과. v1 에서는 검색 불가, P+ scanned-PDF OCR fallback 대기.
- determinism stress: extract → chunk 사이 `now()` 추가 호출 없음 (P6-4 invariant
  계승). PDF doc/chunk_id 모두 결정적.

통합 테스트 (`tests/pdf_pipeline.rs`, 8 passed + 1 ignored):
- 3-page text PDF → 1 doc + 3 chunk + Page span 검증
- identical re-ingest → Updated, doc_id 동일
- encrypted PDF → Error + `qpdf` hint 보존
- corrupt header PDF → Error + 미저장
- mixed page (page 2 빈) → 2 chunk + Warning 1개
- IngestReport 산술 invariant
- 50-page 긴 PDF → ≥50 chunk
- inspect doc → SourceSpan::Page round-trip
- (ignored) edited bytes re-ingest → storage UNIQUE bug 노출, P+ fix 대기

추가 발견 (HOTFIXES `2026-05-02 P7-3`): `assets.workspace_path` 의 UNIQUE
제약과 `upsert_asset_row` 의 `ON CONFLICT(asset_id)` 만 처리하는 부분 사이에
gap 존재. byte 변경 시 새 asset_id → 같은 workspace_path 충돌. md / image / pdf
모두 영향. P7-3 통합 테스트가 처음 노출. 본 PR 은 fix 안 함 — P+ storage task.

`docs/SMOKE.md` 에 PDF 섹션 + 검증 체크리스트 + 알려진 동작 4건 추가.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 09:28:06 +00:00
8de08cf38c review(p7-1): 회차 1 지적 반영
- Cargo.toml: 사용하지 않는 deps 제거 (`kebab-config`, `thiserror`,
  `pdf-extract`, dev `tempfile` / `serde_json` / `serde`). 특히
  `pdf-extract` 가 끌어오던 transitive ~150 crate (pom, postscript,
  type1-encoding-parser, adobe-cmap-parser, euclid, chrono, md5,
  linked-hash-map …) 가 모두 사라짐. lopdf 만 남음.
- info.rs: BOM 없는 PDFDocEncoded Title 디코드 버그 수정. `from_utf8_lossy`
  는 0x80–0xFF 를 U+FFFD 로 치환해 "Café" 같은 레거시 타이틀을 망가뜨림.
  byte → `char` 직접 캐스팅 (Latin-1 디코더) 로 교체. 회귀 테스트
  `info_dict_title_pdfdocencoding_latin1_high_bytes_decoded` 추가.
- info.rs: 모듈 doc 의 "Latin-1 superset" 부정확 표현 정정 — PDFDocEncoding
  은 0x18–0x1F / 0x80–0x9F 영역에서 Latin-1 과 다름.
- lib.rs: `saturating_sub(1)` 가 page=0 케이스를 silent 흡수하던 부분에
  `debug_assert!` 추가. release 는 saturating fallback 유지 (panic 보다
  garbled order 가 운영에 유리).
- tests: UTF-16 surrogate pair 커버리지 갭 보완 — 🥙 (U+1F959) 가 포함된
  타이틀로 `String::from_utf16_lossy` 의 페어-결합 경로 검증.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 08:40:40 +00:00
5a158d7343 feat(kebab-parse-pdf): P7-1 text PDF extractor — per-page CanonicalDocument
`PdfTextExtractor`(MediaType::Pdf) lopdf 기반 per-page 텍스트 추출.
페이지마다 `Block::Paragraph` + `SourceSpan::Page { page, char_start, char_end }`
emit. 본문이 비거나 추출 panic 인 페이지는 빈 paragraph + `Provenance::Warning`
("scanned candidate") 로 표시 — 이후 OCR fallback (별도 task) 의 입력.

핵심 동작:
- `lopdf::Document::load_mem` + `is_encrypted()` → 암호화 PDF 는 명시 에러
  (`qpdf --decrypt` 안내).
- 페이지 단위 `extract_text(&[page])` 를 `catch_unwind` 로 감싸 malformed
  page panic 을 recoverable warning 으로 변환.
- `/Info` dict 에서 Title/Producer/Creator best-effort 추출. UTF-16BE BOM
  prefixed 문자열도 디코드 (한국어 등 non-ASCII Title 정상 처리).
- 9개 통합 테스트: 3-page emit, scanned-mixed warning, encrypted refuse,
  corrupt header error, page_count 메타, UTF-16BE Title, filename
  fallback, determinism, snapshot.

`parser_version = "pdf-text-v1"`. Allowed deps: `lopdf 0.32` + `pdf-extract 0.7`
(원본 spec 그대로). 본문 다국어 OCR fallback 은 §9.2 후속 task (out of scope).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 08:34:55 +00:00
ca0567c72b feat(kebab-app): P6-4 image ingest wiring — kebab ingest 가 PNG/JPEG 자산도 처리
P6-1/P6-2/P6-3 의 라이브러리 (`ImageExtractor`, `OllamaVisionOcr`,
`apply_caption`) 가 그동안 CLI 에서 보이지 않던 미완 구간을 완성.
이제 `kebab ingest` 가 markdown 외에 이미지 자산을 end-to-end 로
색인하고, `kebab search` / `kebab ask` 가 OCR 텍스트 + caption 으로
이미지를 매칭/인용한다.

## kebab-app

- `[dependencies]` 에 `kebab-parse-image` 추가.
- `ingest_with_config` 진입 시 `image.ocr.enabled` / `image.caption.enabled`
  플래그에 따라 `OllamaVisionOcr` / `OllamaLanguageModel` 을 **ingest
  세션당 1회** 빌드. 자산 루프에서 trait object 로 공유.
  reqwest::blocking::Client 의 내부 Arc 덕분에 알로케이션 비용은
  자산 수와 무관.
- 두 어댑터 + ImageExtractor 를 한 묶음으로 `ImagePipeline` 구조체에
  담아 `ingest_one_asset` 매개변수 폭증 차단 (clippy::too_many_arguments
  대응).
- `ingest_one_asset` 의 markdown-only 가드를 `match media_type` 으로
  교체 — Markdown 은 기존 경로, Image(_) 는 새 `ingest_one_image_asset`
  로 분기, PDF/Audio/Other 는 종전대로 skipped.
- 신규 `ingest_one_image_asset`:
  - bytes 읽기 → `ImageExtractor::extract` (실패 시 caller 가 errors+=1)
  - `apply_ocr` (Lenient — 실패 시 ProvenanceKind::Warning 이벤트 +
    `IngestItem.warnings` 에 \"ocr_failed: ...\", `block.ocr` 는 None
    유지)
  - `apply_caption` (동일 Lenient 정책)
  - 기존 `MdHeadingV1Chunker` 호출 — 청커는 이미 `Block::ImageRef` 를
    단일 청크로 emit
  - 기존 persist + embed 시퀀스 그대로 (markdown 과 byte-identical)
- `lang_hint_from_doc` — `Lang(\"und\")` 또는 빈 문자열을 None 으로
  매핑 (image-pipeline 어댑터의 build_prompt 가 \"und\" 를 silent drop
  하지 않도록 caller 측에서 미리).

## kebab-chunk

- `render_block_text` 의 `Block::ImageRef` 분기를 P6-4 (β) plain
  concat 정책으로 교체 — `[alt, ocr.joined, caption.text]` 를 `\\n\\n`
  로 join, 빈 부분은 drop. alt 가 비면 `src` 의 basename 으로 fallback
  (P6-1 contract 의 defensive guard).
- 신규 unit 테스트 `image_ref_p6_4_plain_concat_drops_empty_parts` —
  alt-only / alt+ocr / alt+caption / alt+ocr+caption / 빈 alt → src
  fallback 다섯 케이스 모두 검증.
- 기존 `image_ref_emits_own_chunk_zero_tokens` 그대로 통과 — 청커의
  per-block dispatch 는 변경 없음, text 렌더링만 갱신.

## 통합 테스트 (kebab-app/tests/image_pipeline.rs)

wiremock 으로 Ollama 를 stub. 5건:

1. OCR-only happy path — 1 PNG + ocr.enabled → 1 doc + 1 chunk emit,
   `block.ocr.joined` 가 mock 의 \"Hello World 2026\".
2. OCR + caption 동시 활성 — 두 필드 모두 채워지고 chunk text 에
   alt + ocr + caption 세 부분 모두 포함.
3. Lenient 실패 검증 — OCR 503 시 자산은 indexed (kind=New),
   `errors=0`, ProvenanceKind::Warning attributed to \"kb-app\",
   `IngestItem.warnings` 에 \"ocr_failed:\" 노트.
4. 양쪽 비활성 — `image.ocr.enabled=false && image.caption.enabled=false`
   여도 자산은 chunk 1개로 indexed (chunk text=filename), EXIF +
   dimensions 그대로 채워짐.
5. 결정성 (re-ingest) — 동일 PNG 두 번 ingest 시 두 번째는
   `Updated` + 동일 `doc_id`.

## SMOKE.md

`kebab search --mode lexical \"Hello World\"` 단계를 명령 시퀀스에
추가. `[image.ocr]` / `[image.caption]` config 절 예시 + ingest 시간
추정 (자산당 ~5-10초) 추가. \"책은 P7 PDF 라인으로\" 가이드를 검증
체크리스트 와 \"알려진 동작\" 양쪽에 박음.

## 실 Ollama 통합 검증

192.168.0.47 + gemma4:e4b 기준:

```
$ kebab --config /tmp/kebab-smoke/config.toml ingest
scanned 2  new 2  updated 0  skipped 0  errors 0  (18395 ms)

$ kebab inspect doc <image_doc_id>
parser_version: image-meta-v1
blocks: [{
  alt: \"hello.png\",
  ocr: \"Hello World 2026\",
  caption: \"The image displays the text \\\"Hello World 2026\\\" in a large, black, sans-serif font.\"
}]

$ kebab --json ask \"Hello World 텍스트가 어디에 있나?\" --mode hybrid
grounded: true
citations: [{marker: \"[1]\", doc_path: \"hello.png\"}]
```

## 검증

- `cargo test --workspace --no-fail-fast -j 1` — 전부 pass
- `cargo clippy --workspace --all-targets -- -D warnings` — pass
- `cargo test -p kebab-chunk image_ref` — 2 pass (P1-5 회귀 + P6-4
  신규 unit)
- `cargo test -p kebab-app --test image_pipeline` — 5 pass

## 의존성 경계

- `kebab-app` 이 `kebab-parse-image` 추가 — spec Allowed dep 그대로.
- 새 forbidden 침범 없음 (기존 `kebab-tui` / `kebab-desktop` /
  `kebab-eval` 미참조 유지).
- 본 task 가 신설하는 image-specific 비즈니스 로직 0줄 — 모두
  `kebab-parse-image` 에 위임.

`tasks/p6/p6-4-image-ingest-wiring.md` status: planned → completed.

contract: docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
sections: §3.4 ImageRefBlock, §6.1 ingest pipeline, §7.2
Extractor/Chunker traits, §9.1 image extraction policy.
2026-05-02 07:37:56 +00:00
cd2213e48d feat(kebab-parse-image): P6-3 caption adapter — vision LM via trait
- 신규 모듈 `crates/kebab-parse-image/src/caption.rs` 추가:
  • `caption_image(llm, bytes, lang_hint, cfg)` — `&dyn LanguageModel`
    위에서 동작. 비전 LM (예: gemma4:e4b) 이 한 문장 객관 설명
    출력. temperature=0 / seed=0 결정성.
  • `apply_caption(llm, bytes, block, lang_hint, cfg, events)` —
    `block.caption = Some(...)` 으로 채우고 ProvenanceKind::CaptionApplied
    이벤트 1건 추가. `image.caption.enabled = false` 면 클린 no-op
    (Ok(())). LM 실패 시 block.caption None 그대로 + events 미기록.
  • 다운스케일 long-edge `[128, 1536]` 클램프. PNG passthrough hot
    path 보존, 그 외는 단일 디코드 + PNG 재인코딩.
  • 한국어 / 영어 프롬프트 분기 (lang_hint=\"ko\"/\"kor\" → 한국어).
  • `ModelCaption.model_version = \"<provider>/<prompt_template_version>\"`
    (예: \"ollama/caption-v1\") — prompt 또는 모델 회귀 감사 가능.

## kebab-core / kebab-llm-local 변경

- `kebab_core::GenerateRequest` 에 `images: Vec<String>` 필드 추가.
  `#[serde(default)]` 으로 기존 wire 페이로드 / snapshot 호환.
- `kebab-llm-local::OllamaLanguageModel` 가 req.images 를 Ollama
  `images: [base64, ...]` 와이어 필드로 라우팅.
  `#[serde(skip_serializing_if = is_empty)]` 로 비어 있을 때 wire
  shape 가 pre-P6-3 와 byte-identical.

## kebab-config

- 신규 `ImageCfg.caption: CaptionCfg`:
  - `enabled: bool` (default false)
  - `max_pixels: u32` (default 768, 클램프 [128, 1536])
  - `prompt_template_version: String` (default \"caption-v1\")
- `KEBAB_IMAGE_CAPTION_{ENABLED,MAX_PIXELS,PROMPT_TEMPLATE_VERSION}`
  3종 환경변수 추가.

## Spec deviations

`tasks/HOTFIXES.md` 2026-05-02 항목 추가:
- Symptom 1: spec p6-3 시그니처가 `&dyn LanguageModel` 인데 frozen
  trait + GenerateRequest 가 vision 미지원. → trait 확장.
- Symptom 2: spec 의 cargo feature `caption` (default OFF at compile
  time) → runtime gate 1개로 통합. base64/image/kebab-llm 외 추가
  deps 없어 cargo feature 의 binary 절감 가치 미미.

p4-1 / p4-2 / p6-3 spec 의 amends 명시.

## 테스트

`cargo test -p kebab-parse-image --test caption` — 9건 + 1 ignored:
- feature gate (disabled → no-op / Err on direct call)
- happy path (block.caption Some + Provenance CaptionApplied)
- 빈 토큰 stream → empty text + caption.is_some()
- CapturingMock 으로 req.images 라우팅 검증 (base64 1개, decode 가능)
- 한국어 / 영어 프롬프트 분기 (CapturingMock 의 system 캡처)
- LM Err → block.caption None 유지 + events 미기록
- 결정성 (동일 mock 입력 → 동일 caption)
- max_pixels 클램프 (99999 → 1536, 4000×3000 PNG 다운스케일 검증)
- opt-in 통합 (실 192.168.0.47 Ollama / gemma4:e4b → \"The image is
  a solid red color.\" 검증 완료, 4.3초)

`cargo test --workspace --no-fail-fast -j 1` 전체 pass.
`cargo clippy --workspace --all-targets -- -D warnings` pass.

## 의존성 경계

- 추가 deps: `kebab-llm` (trait 만), `base64` (이미 P6-2 에서 추가).
- dev-deps: `kebab-llm/mock` 으로 `MockLanguageModel`,
  `kebab-llm-local` (통합 테스트 전용 — 런타임 deps 에는 없음).
- forbidden 침범 없음: `kebab-source-fs / parse-md / normalize /
  chunk / store-* / embed* / search / rag / UI` 미참조.

contract: docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
sections: §3.4 ImageRefBlock.caption, §3.7a ModelCaption, §9.1
caption (model-generated, low trust).
2026-05-02 06:05:39 +00:00
4ed5536c92 feat(kebab-parse-image): P6-2 OCR adapter — Ollama-vision default
- 새 모듈 `crates/kebab-parse-image/src/ocr.rs` 추가. spec 의 `OcrEngine`
  trait 그대로 + `OllamaVisionOcr` default 구현 + `apply_ocr` 헬퍼.
- `OllamaVisionOcr`: `<endpoint>/api/generate` 비스트리밍 호출,
  `images: [base64]` 필드로 이미지 전달, 프롬프트는 언어 힌트
  + 화이트리스트 언어 목록 포함. 응답 prose 를 `OcrText.joined` 로,
  prepared image 전체 영역 단일 region (confidence 1.0) 으로 wrap.
  기본 모델 `gemma4:e4b`. endpoint 비어 있으면 `models.llm.endpoint`
  로 fallback.
- 이미지 전처리: long-edge `config.image.ocr.max_pixels` (기본 1600,
  256~4096 클램프) 초과 시 PNG 로 재인코딩 (image::imageops::resize,
  Triangle filter). PNG 입력이 max 이내면 zero-copy passthrough.
- `apply_ocr` 는 OCR 성공 시 block.ocr 를 Some 으로 채우고
  ProvenanceKind::OcrApplied 이벤트 추가. 실패 시 block.ocr 는
  None 그대로 + provenance 미기록 (부분 상태 누출 금지).
- `kebab-config`: 새 `ImageCfg.ocr: OcrCfg` 블록 (enabled/engine/model
  /endpoint/languages/max_pixels). `#[serde(default)]` 로 pre-P6
  TOML 호환. `KEBAB_IMAGE_OCR_*` 환경변수 5종 추가.

## Spec deviation

원래 P6-2 spec 은 Tesseract 를 default OCR 엔진으로 지정했으나, dev /
CI 호스트에서 `libtesseract-dev` 시스템 패키지 설치를 피하려고
Ollama-vision 으로 default 를 교체. `OcrEngine` trait 추상화는 spec
그대로 보존 — Tesseract / Apple Vision / PaddleOCR 어댑터는 같은
trait 으로 추후 feature-gate 추가 가능. 자세한 내역은
`tasks/HOTFIXES.md` 2026-05-02 항목 참조.

Trust 측면: vision LM 은 hallucinate 가능. `OcrText.engine = "ollama-vision"`
필드로 consumer 가 엔진 별 신뢰 분기 가능.

## 테스트

- 신규 (`tests/ocr.rs`, 8 + 1 ignored):
  - 200 happy → OcrText 디코딩 (joined / engine / engine_version /
    region count / bbox / confidence)
  - 빈 응답 → 빈 regions
  - 5xx → Err with status + body 포함
  - 200 error envelope → Err
  - apply_ocr → block.ocr Some + Provenance OcrApplied 1건
  - apply_ocr error → block.ocr None 유지 + events 미기록
  - 4000×3000 PNG → max_pixels=1024 까지 다운스케일, aspect ratio 보존
  - from_parts max_pixels 클램프
  - opt-in `KEBAB_OCR_INTEGRATION=1` 통합 (실제 192.168.0.47 Ollama
    `gemma4:e4b` 로 \"Hello World 2026\" 전사 검증 완료)
- 신규 (`src/ocr.rs` unit): truncate, build_prompt 언어/힌트 처리
- `kebab-config` 테스트 +3: defaults, env override, pre-P6 TOML 호환

전체: `cargo test -p kebab-parse-image` 28 pass + 1 ignored,
`cargo test -p kebab-config` 20 pass,
`cargo clippy --workspace --all-targets -- -D warnings` pass.

contract: docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
sections: §3.4 ImageRefBlock.ocr, §3.7a OcrText / OcrRegion, §9.1 OCR
vs caption provenance.
2026-05-02 05:38:24 +00:00
194dd34668 review(p6-1): 회차 1 지적 반영
- Cargo.toml: 미사용 deps 제거 (`serde`, `thiserror`) + dev-deps 의
  `serde_json` 중복 선언 제거.
- src/lib.rs: 변수명 `decode_warning` → `dim_warning` (16k cap 초과
  분기까지 포괄하므로 더 정확).
- src/exif_extract.rs: `ascii_field` / `u32_field` 의 dead-flexibility
  `In` 인자 제거 (모든 호출이 `In::PRIMARY` 였음). 두 단 `if let` 을
  Rust 2024 let-chain 으로 정리. EXIF 화이트리스트 출력 키를
  workspace wire-schema 컨벤션에 맞춰 snake_case 로 통일
  (`Make` → `make`, `DateTimeOriginal` → `date_time_original` 등).
- tests/common/mod.rs: 호출되지 않는 `fake_path` 헬퍼 + `Path` import
  제거.
- tests/extractor.rs: snake_case 키로 assertion 갱신.

cargo test -p kebab-parse-image — 14건 모두 pass.
cargo clippy -p kebab-parse-image --all-targets -- -D warnings — pass.
2026-05-02 05:11:40 +00:00
d11a810119 feat(kebab-parse-image): P6-1 image extractor + EXIF whitelist
- 새 crate kebab-parse-image 추가 (workspace 19개째). MediaType::Image(_)
  자산을 단일-블록 CanonicalDocument 로 변환하는 ImageExtractor 구현.
- parser_version "image-meta-v1" (§9 versioning).
- 본문은 Block::ImageRef 1건만 포함 — OCR / caption 필드는 None 으로
  남겨 두고 P6-2 / P6-3 에서 채운다.
- EXIF 화이트리스트 (§9.1, PII 표면 최소화):
  Make / Model / Software / DateTimeOriginal / Orientation /
  GPSLatitude(+Ref) / GPSLongitude(+Ref). MakerNote / Thumbnail / 기타
  태그는 폐기. DateTime 은 EXIF "YYYY:MM:DD HH:MM:SS" → ISO-8601 변환.
  GPS DMS triple + N/S/E/W ref → signed decimal degree.
- 차원: image::ImageReader 헤더만 읽어 (w, h, format) 획득. 16k×16k cap
  초과 또는 디코드 실패 → metadata.user.dimensions = null + Provenance
  Warning 이벤트 (Err 아님). 포맷 자체 인식 실패 → anyhow::Error
  (caller skip).
- SourceSpan::Region { 0, 0, w, h } 으로 전체 이미지 영역 표기. 결정성:
  동일 bytes + 동일 parser_version → 동일 doc_id + block_id (§4.2 ID
  recipe 그대로 사용).
- metadata.source_type = Reference, trust_level = Primary, lang = "und".
  title = 확장자 제외 파일명, alt = 파일명.
- 의존성 경계 (§8): kebab-core 만 + image 0.25 (default features off,
  png/jpeg/webp/gif/tiff 만), kamadak-exif 0.6, anyhow / serde /
  serde_json / time / tracing / thiserror. kebab-source-fs · parse-md ·
  store-* · embed* · llm* · rag · UI crate 미참조.
- 테스트 14개 (4 unit + 10 integration):
  • PNG 차원 추출, JPEG EXIF GPS 추출 (DMS → decimal 변환 정확도 1e-6),
    EXIF 없는 PNG → 빈 map, 손상 PNG → warning + null dims (panic 없음),
    인식 불가 bytes → Err, 결정성, 스냅샷, supports() 매칭, media_type
    불일치 거부.
  • 픽스처는 in-memory 생성 (PNG 는 image crate, EXIF JPEG 는 kamadak
    Writer 로 EXIF blob 만든 뒤 SOI 직후 APP1 splice) — 바이너리
    fixture 커밋 없음.
- HEIC / RAW 는 spec 상 v1 out of scope (image crate 미지원, Apple
  Vision sidecar 가 추후 P+ 에서 채움).
- tasks/p6/p6-1-image-extractor-exif.md status: planned → completed.

contract: docs/superpowers/specs/2026-04-27-kebab-final-form-design.md
sections: §3.4 Block::ImageRef + ImageRefBlock, §3.7a OcrText /
ModelCaption stubs, §9.1 image extraction policy, §9 versioning.
2026-05-02 05:05:47 +00:00
911fb49550 refactor(rename): kb crates → kebab — Cargo packages, folders, Rust modules
프로젝트 이름 `kb` → `kebab` rename 의 첫 단계.

- workspace `Cargo.toml`: members `crates/kb-*` → `crates/kebab-*`,
  repository URL `altair823/kb` → `altair823/kebab`.
- 18 crate 폴더 rename via `git mv` (history 보존).
- 각 crate `Cargo.toml`: `name = "kb-*"` → `"kebab-*"`, path deps
  `../kb-*` → `../kebab-*`.
- 모든 `.rs`: `kb_<id>` snake-case 모듈 path 18 개 (`kb_core`,
  `kb_config`, `kb_app`, `kb_cli`, `kb_eval`, `kb_search`, `kb_chunk`,
  `kb_normalize`, `kb_source_fs`, `kb_parse_md`, `kb_parse_types`,
  `kb_store_sqlite`, `kb_store_vector`, `kb_embed`, `kb_embed_local`,
  `kb_llm`, `kb_llm_local`, `kb_rag`) → `kebab_<id>` 일괄 sed (단어
  경계 \\b 사용해 영어 문장 안의 "kb" 약어 미오염).

CLI binary 이름 (`[[bin]] name = "kb"`), 환경변수 `KB_*`, XDG paths,
tracing target, 그리고 docs sweep 은 다음 commit 에서.

## 검증

- `cargo check --workspace` clean — 모든 crate 빌드 통과 후 commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:28:08 +00:00
d9a5b88d27 feat(p5-2): kb-eval metrics + compare — AggregateMetrics, CompareReport, kb eval CLI
P5-2 구현. 저장된 eval_runs / eval_query_results 위에서:

- `kb_eval::metrics`: hit@k / MRR / recall@k_doc / citation_coverage /
  groundedness / empty_result_rate / refusal_correctness 계산. NaN
  metrics (분모 0)는 JSON null. 4-decimal round + Deserialize 추가로
  aggregate_json 라운드트립.
- `kb_eval::compare`: 두 run 비교 → CompareReport (per-metric Δ + per-
  query Win/Loss/Draw/Regression). chunker_version drift 시 graceful
  doc-id fallback (chunker_version_match: "fallback_doc"), `strict`
  옵션이면 refuse.
- `render_report_md`: 인간용 Markdown (집계 + Wins/Losses/Regressions
  표).
- `SqliteStore::{load_eval_run, load_eval_query_results,
  update_eval_run_aggregate}` + owned `EvalRunRecord` /
  `EvalQueryResultRecord` 추가 — write 측 borrow-shape는 그대로.
- `kb eval` CLI: `run` (P5-1 위임), `aggregate <id>`, `compare <a> <b>
  [--strict-chunker-version] [--write-report]`. `--json` 으로 raw
  CompareReport, 기본은 Markdown 출력.

## Spec deviations (intentional, doc 명시)

- Graceful 매칭은 doc-id-only (chunker_version_match: "fallback_doc")
  — 50% span overlap은 chunker re-index 후 양쪽 chunks 동시 보존이
  현실적으로 안 돼서 P6+ 로 deferred.
- `*_with_config` 헬퍼 추가: 통합 테스트가 TempDir Config 로 드라이브.
  no-arg 형태는 Config::load(None) 로 위임.
- CLI 는 kb-cli → kb-eval 직접 wire (kb-app cycle 회피). DoD 의 "via
  kb-app" 의도는 facade 단일화였지만 cycle 발생.
- `AggregateMetrics: Deserialize` 추가 — aggregate_json 라운드트립.

## 검증

- `cargo test -p kb-eval` 30/30 (15 unit + 2 loader + 8 metrics+compare
  통합 + 7 runner). 8 통합 중 snapshot 1 건 (`compare-1.json`).
- `cargo test -p kb-store-sqlite` 33/33.
- `cargo clippy --workspace --all-targets -- -D warnings` clean.
- forbidden imports 부재 (`kb-source-fs|kb-parse|kb-normalize|kb-chunk|
  kb-store-vector|kb-embed|kb-search|kb-llm|kb-rag|kb-tui|kb-desktop|
  kb-app` — kb-app 는 metrics/compare 모듈에 부재; runner 만 사용).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:05:13 +00:00
58a11cc2b8 feat(p5-1): kb-eval crate — golden-fixture runner + eval persistence
- new kb-eval crate: load_golden_set (YAML) + run_eval (per-query search/ask + persistence)
- new kb-store-sqlite::eval module: record_eval_run_with_results (transactional), document_exists / chunk_exists probes
- fixtures/golden_queries.yaml: 5-entry KO+EN template
- tests: 13 pass (loader: parse, dup-id, missing chunk_id; runner: elapsed, snapshot, error capture, JSONL, determinism, persistence, config_snapshot)
- per_query.jsonl mirror written to runs_dir/<run_id>/
- temperature=0 + fixed seed → byte-identical per_query.jsonl (lexical)

deviations from spec (documented in code):
- run_id uses uuid::Uuid::now_v7().simple() (timestamp-ordered hex) instead of ULID — uuid already in workspace deps
- load_golden_set_validated kept #[cfg(test)] pub(crate) — production inlines validate_against_db
- snapshot fixture uses normalized projection (id/query/mode/first_hit) — full byte-determinism covered by separate test
- index_version in config_snapshot left null (composed per call by kb-app, not config-level)

deferred to follow-up:
- App reuse across queries (currently rebuilds App per query)
- expand_path hoist to kb-config (3 crate clones now)
- --max-queries flag (deferred to P5-2 per updated spec)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 18:01:09 +00:00
e35b06d0d0 feat(p4-3): kb-rag crate — full RAG pipeline + kb-app::ask wired
P4 terminal task. Implements the user-facing payoff: retrieve →
score gate → pack → render → generate → cite-validate → persist.
After this commit, `kb ask` actually works against an Ollama
backend; the pipeline grounds the answer in retrieved chunks and
refuses cleanly when the gate trips or the model self-judges.

New crate kb-rag:
- pub struct RagPipeline { retriever, llm, docs, config } — all
  Arc<dyn Trait + Send + Sync> so the pipeline shares + Sync.
- pub fn ask(query, opts) -> Result<Answer> drives the nine-stage
  flow per spec §1.
- pub struct AskOpts { k, explain, mode, temperature, seed,
  stream_sink: Option<mpsc::Sender<String>> }. k acts as a floor
  over config.search.default_k so a low-k caller can't starve
  retrieval (documented in field doc).

Pipeline stages:
1. Retrieve via the injected dyn Retriever.
2. Score gate: empty hits → NoChunks refusal (no LLM call); top-1 <
   config.rag.score_gate → ScoreGate refusal (no LLM call) with
   top-3 candidates listed in the synthesized answer text.
3. Pack: budget = config.rag.max_context_tokens.saturating_sub
   (prompt overhead). Per-hit `[#n] doc=… heading=… span=…\n<text>`
   with deterministic enumeration. If every hit's chunk is
   unfetchable from the store (deleted between search and pack),
   fall back to NoChunks refusal with a tracing::warn rather than
   feeding an empty [근거] to the LLM.
4. Render rag-v1 prompt with the spec's verbatim Korean system
   string + `[질문]/[근거]` user template.
5. Generate via dyn LanguageModel. Single-thread token loop owns
   the iterator; tokens optionally forward to opts.stream_sink (a
   `mpsc::Sender<String>`). SendError silently dropped — caller
   cancellation never panics the pipeline. After Done the loop
   reads (acc, finish_reason, usage) in lockstep with no race.
   max_completion = llm.context_tokens().saturating_sub
   (used_for_input).max(64) — explicitly NOT capped by
   config.rag.max_context_tokens (that's the packing budget for
   [근거], not the LM completion ceiling).
6. Citation extract via STRICT regex `\[#(\d{1,3})\]` (compiled
   once via OnceLock). Loose forms `[1]`, `[ #1 ]`, `[#foo]`,
   `[#1234]`, `vec![1]` are all rejected to prevent prose
   false-positives.
7. Citation validate covers four cases:
   - unknown marker (e.g. `[#7]` when only 3 packed) →
     LlmSelfJudge refusal.
   - empty answer with hits → LlmSelfJudge.
   - non-empty + no marker + matches `근거 (가|이) 부족` regex →
     LlmSelfJudge (model self-refused with the canonical phrase;
     phrase match logged via tracing::debug for observability).
   - non-empty + no marker + no refusal phrase → LlmSelfJudge
     (silent ungrounded answers are still refusals).
   - non-empty + ≥1 valid marker → grounded = true.
8. Build Answer per kb_core::Answer shape:
   - citations: filter packed list to exactly the markers cited.
     Wire format `marker: Some("[1]")` (square-bracketed bare
     index) per design §2.3, distinct from the prompt-side
     `[#n]` grammar.
   - embedding ModelRef: read from config.models.embedding for
     Vector/Hybrid; None for Lexical. Documented deviation since
     the Retriever trait doesn't expose the embedder. For
     ScoreGate/NoChunks refusals on Vector/Hybrid the embedding
     model is still recorded — the vector retriever WAS consulted
     even when the gate tripped.
   - TraceId minted as `ret_<8-hex>` from blake3(query, top_score,
     model_id, ns).
   - retrieval AnswerRetrievalSummary populated.
   - usage from the final Done chunk; latency_ms wall-clock
     fallback when the LLM reports zero.
   - created_at OffsetDateTime::now_utc().
9. Persist via SqliteStore::put_answer (new inherent method on
   SqliteStore, not on the DocumentStore trait — answers aren't
   documents and adding to kb-core was forbidden). Always inserts,
   refusals included. packed_chunks_json is null unless
   opts.explain == true.

kb-store-sqlite extension:
- pub fn put_answer(&Answer, query, packed_chunks_json) ->
  Result<AnswerId>. Maps all 22 fields of the answers table per
  V001 schema in a single INSERT under a transaction.

kb-app::ask wired:
- bail!("not yet wired (P4-3)") replaced with a real body that
  builds the retriever per opts.mode (Lexical | Vector | Hybrid),
  instantiates OllamaLanguageModel from config, constructs
  RagPipeline, calls pipeline.ask. AskOpts moves to kb-rag and is
  re-exported via `pub use kb_rag::AskOpts` so kb-cli's
  `use kb_app::AskOpts` keeps working.
- kb-app/Cargo.toml gains kb-rag, kb-llm, kb-llm-local. P3-5's
  forbids on these are lifted by P4-3 spec — kb-app is the
  orchestrator and ask requires both the trait crate and the
  Ollama adapter.
- kb-cli/main.rs's AskOpts literal updated with stream_sink: None
  for the CLI path (TUI in P9 will plumb a real sink).

Tests (kb-rag: 18; kb-app: 1 ignored):
- 3 unit in src/pipeline.rs: marker regex strictness (rejects all
  loose forms with byte-equal expectations), Send+Sync compile
  check, embedding_ref_for behavior across modes.
- 15 integration in tests/pipeline.rs covering every spec test row
  + the new "all chunks unfetchable falls back to NoChunks" guard:
  empty-hits, score-gate, grounded happy path, unknown-marker,
  prose-`[1]` rejection, `vec![1]` rejection, refusal-phrase,
  packing-budget overflow, streaming-forwards-to-mpsc, dropped-
  receiver-no-panic, usage-from-final-Done, answers-row-inserted-
  for-each-refusal-kind, determinism temp=0 seed=0, Answer JSON
  shape, unfetchable-chunks-fall-back-to-no-chunks (the new
  M3 test).
- kb-app/tests/ask_smoke.rs: 1 #[ignore]'d real-Ollama smoke that
  drives the wired ask end-to-end against `localhost:11434`.

Workspace: 319 passed / 26 ignored / 0 failed. cargo clippy
--workspace --all-targets -- -D warnings clean.

Allowed deps respected (kb-core, kb-config, kb-search, kb-llm,
kb-store-sqlite, serde, serde_json, regex, time, tracing,
thiserror) plus forced waivers anyhow (Retriever / LanguageModel
trait return types) and blake3 (TraceId minting). Forbidden
(kb-source-fs, kb-parse-md, kb-normalize, kb-chunk, kb-store-
vector direct, kb-embed* direct, kb-llm-local direct, kb-tui,
kb-desktop) all absent from `cargo tree -p kb-rag` — concrete
adapters reach the pipeline only through trait objects.

Out of scope: reranker between retrieve and pack (P+), multi-turn
chat memory (P+), LLM-as-judge eval (P5 uses rule-based
must_contain), --json streaming (buffers per §0 Q5 hybrid).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 15:06:10 +00:00
3e38a9bcb4 feat(p4-2): kb-llm-local crate — Ollama HTTP adapter (reqwest::blocking)
First real LanguageModel implementation. Wraps Ollama's local HTTP
API at POST {endpoint}/api/generate with stream:true, parses the
NDJSON streaming response into TokenChunk events, and maps Ollama
error states to a thiserror-derived LlmError with actionable hints.
Synchronous trait surface; reqwest::blocking handles the HTTP I/O.

Public surface:
- pub struct OllamaLanguageModel
- pub fn new(config: &Config) -> Result<Self> — lazy connect; never
  hits the network. Spec line 96.
- pub enum LlmError { Unreachable, ModelNotPulled, Timeout, Stream,
  Malformed }. Lives in this crate per spec — kb-core / kb-llm stay
  free of error taxonomy.
- impl kb_core::LanguageModel via re-export from kb-llm.

Streaming:
- POST body shape per spec §11.2: model, prompt = system + "\n\n" +
  user, stream: true, options { temperature, seed, num_ctx, stop }.
- OllamaStream owns BufReader<reqwest::blocking::Response>, reads
  NDJSON lines via read_until(b'\n'), parses each as
  {response, done, done_reason?, prompt_eval_count?, eval_count?,
  total_duration?}. Token frame → TokenChunk::Token; done frame →
  TokenChunk::Done { finish_reason, usage }.
- done_reason mapping: "length" → Length, "abort" → Aborted,
  "stop" / missing / unknown → Stop (forward-compat with future
  Ollama tags).
- Missing prompt_eval_count / eval_count default to 0 + tracing::warn
  (do NOT fail). Spec line 135.
- EOF without a done line synthesizes Done { Aborted, zeros } so
  downstream pipelines never deadlock waiting for a terminal frame.
- UTF-8: line-delimited framing means each JSON line is a complete
  UTF-8 sequence — no cross-HTTP-chunk codepoint splits to worry
  about. read_until accumulates whole lines regardless of how the
  underlying reqwest body chunks.

Error mapping (LlmError):
- reqwest::Error::is_connect() → Unreachable { endpoint, source }
  with hint "ensure `ollama serve` is running and reachable at
  <endpoint>".
- reqwest::Error::is_timeout() → Timeout.
- 200 with non-NDJSON first line (e.g., transparent-proxy HTML
  error page) → Stream(truncated body) — distinguished from
  Malformed by the iterator's has_emitted flag.
- 404 with body containing model_id (case-insensitive) OR English
  "model" + "not found" → ModelNotPulled(model_id) with hint
  "ollama pull <model_id>". Tightened beyond spec to survive
  Ollama localizing the error message (Korean / Japanese / etc.)
  while keeping the original English-substring fallback.
- Other 4xx/5xx → Stream(truncated body).
- Mid-stream JSON parse failure (after at least one valid line) →
  Malformed(line). Truncate all error bodies to 512 chars
  (chars-based, multibyte safe) so an nginx 500 page can't blow up
  the diagnostic.
- Trailing slash in endpoint stripped before formatting the URL —
  endpoint = "http://x:1234/" produces .../api/generate, not
  .../api//generate. Pinned by trailing-slash test.

Tokio note: reqwest 0.12's blocking feature internally wraps a
private current-thread tokio runtime, so cargo tree --edges normal
shows tokio. The auditable invariant is "no top-level tokio dep +
no async surface exposed to callers" — verified: src/ has zero
async/await/tokio::*. default-features = false drops default-tls
(rustls only) but does NOT drop tokio. Documented honestly in
Cargo.toml + lib.rs. Switching to ureq would remove tokio
entirely; deferred since reqwest is the spec's allowed dep.

Tests (24 total: 23 default + 1 ignored):
- 7 unit in src/ollama.rs: prompt-build, options-build, finish-
  reason mapping, truncate_body bounds (under_cap / over_cap_marker
  / multibyte_chars_not_bytes), 404+model-id heuristic.
- 3 in tests/construction.rs: ModelRef shape, context_tokens
  passthrough, lazy-connect proven via port-1 pointing.
- 13 in tests/streaming.rs: streamed tokens then Done, multibyte
  chars within a line round-trip (renamed from "split across
  chunks" to honestly reflect what's tested), Unreachable-with-
  hint, 4xx→Stream, 404→ModelNotPulled, concat-equals-canned,
  done_reason length / abort, missing eval counts default to zero,
  missing done_reason defaults to Stop, determinism-by-mock,
  trailing-slash endpoint, non-NDJSON 200 body → Stream not
  Malformed.
- 1 #[ignore] in tests/integration.rs: real Ollama on
  localhost:11434 with the configured model. Opt-in via
  cargo test -p kb-llm-local -- --ignored after `ollama serve`
  + `ollama pull`.

Workspace: 288 passed / 25 ignored / 0 failed. cargo clippy
--workspace --all-targets -- -D warnings clean. No native-tls,
no openssl in the dep graph.

Allowed deps respected: kb-core, kb-config, kb-llm, reqwest 0.12
(default-features=false; blocking, json, rustls-tls), serde,
serde_json, tracing, thiserror plus anyhow (forced by trait return
type). wiremock + tokio in [dev-dependencies] only.

Out of scope: llama.cpp / candle adapters (P+), Ollama embed
endpoint (separate adapter inside kb-embed-local if requested),
cancellation / abort tokens (P+), connection-pool tuning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:28:34 +00:00