feat(kebab-app + kebab-store-sqlite): p9-fb-19 search LRU cache + corpus_revision

도그푸딩 item 15 — TUI / 같은 process 안에서 동일 query 반복 시 SQLite
FTS + Lance + RRF 재계산이 매번 발생하던 비용 해소. in-process LRU
캐시 + 모노토닉 corpus_revision 카운터로 ingest commit 발생 시 모든
entry 자동 stale.

## 핵심 변경

- **SQLite V004 migration**: `kv (key TEXT PRIMARY KEY, value TEXT)
  STRICT` + `corpus_revision = '0'` seed. 미래의 다른 scalar 도 같은
  테이블에 들어갈 수 있는 generic shape.
- **`SqliteStore::corpus_revision()` / `bump_corpus_revision()`** —
  `UPDATE ... CAST AS INTEGER + 1` atomic. INSERT-OR-IGNORE 도 함께
  실행 (V004 seed 가 무슨 이유로 누락된 케이스 paranoid).
- **`kebab-app::ingest_with_config_cancellable`** — `new + updated > 0`
  시 bump, no-op (skipped-only) reingest 는 cache 보존.
- **`App.search_cache: Option<Mutex<LruCache<SearchCacheKey, Vec<
  SearchHit>>>>`** — `config.search.cache_capacity` (default 256, 0
  비활성). `lru = "0.12"` workspace dep 추가.
- **`SearchCacheKey`** = `query_norm` (NFKC + trim + lowercase) +
  `mode` + `k` + `snippet_chars` + `embedding_version` (vector/hybrid
  만, lexical 은 빈 문자열) + `chunker_version` + `corpus_revision`
  snapshot.
- **`App::search`** rewrite — cache 활성 시 lookup → miss 면 기존
  `search_uncached` 호출 후 put. cache 비활성이거나 lock 실패면
  straight-line.
- **`App::search_uncached`** (rename of pre-fb-19 `search` body) +
  `search_uncached_with_config` facade — CLI `kebab search --no-cache`
  로 진입.
- **`Config.search.cache_capacity: usize`** field, `#[serde(default)]`
  로 기존 config 호환.
- **CLI `--no-cache`** flag — 디버깅용 (CLI 는 매 호출이 새 process
  라 사실상 no-op 이지만 spec 명시 + 향후 long-lived process 호환).
- **frozen design §9 versioning** 표에 `corpus_revision` row 추가
  (기존 `index_version` 라벨과 다른 차원: 라벨은 retrieval 형상,
  corpus_revision 은 ingest commit ack).

## 테스트

- `kebab-store-sqlite` 신규 3 unit (fresh=0, monotonic bump, persist
  across reopen)
- `kebab-app` 신규 4 integration (cached repeat 같은 hits, NFKC 정규화
  로 case/whitespace collapse, --no-cache parity, first ingest bumps
  corpus_revision)
- 워크스페이스 전체 `cargo test --workspace --no-fail-fast -j 1` exit 0
- `cargo clippy --workspace --all-targets -- -D warnings` clean

## 문서

- README `kebab search` 행: 캐시 동작 + `--no-cache` 안내 + corpus_
  revision 무효화 메커니즘
- docs/SMOKE.md `[search]` 절에 `cache_capacity` 라인 추가
- HANDOFF: 2026-05-03 entry
- spec status planned → in_progress

## Out of scope

- patch-and-merge incremental (RRF 정규화 전체 hit set 기준이라 어려움)
- SQLite 영속 cache (P+)
- 다른 process 간 cache 공유 (in-process 만 — corpus_revision 이
  cross-process 무효화는 O(1))

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-03 05:01:31 +00:00
parent 07dc2346fe
commit 0e408fb1b5
16 changed files with 424 additions and 5 deletions

24
migrations/V004__kv.sql Normal file
View File

@@ -0,0 +1,24 @@
-- V004__kv.sql — single-row key/value table for monotonic counters.
--
-- p9-fb-19 introduces an in-process LRU search cache; cache keys carry
-- a `corpus_revision` snapshot so a successful `kebab ingest` (which
-- bumps the counter) automatically invalidates every prior entry.
-- Persisting the counter in SQLite (rather than holding it in memory)
-- means a fresh process picks up the latest value, so a CLI invocation
-- after an ingest in another process correctly skips the stale cache.
--
-- Schema is a generic `key/value` so future scalars (last_compaction,
-- last_doctor_run, ...) can land here without another migration. The
-- value column is TEXT because SQLite has no opinion on integer width
-- and downstream code can parse `u64` / `i64` / strings as needed.
--
-- Seed `corpus_revision = 0` so the first cache miss after a fresh
-- install gets a defined snapshot; ingest's `bump_corpus_revision`
-- moves it to 1 on the first successful commit.
CREATE TABLE kv (
key TEXT PRIMARY KEY NOT NULL,
value TEXT NOT NULL
) STRICT;
INSERT INTO kv (key, value) VALUES ('corpus_revision', '0');