feat(search): heading_path FTS5 text column filter (default text-only matching)

v0.17.0 trigram tokenizer entry 가 미수정으로 남겨둔
heading_path_json JSON 노이즈 (HOTFIXES 2026-05-24) closure.
trigram 이 chunks_fts.heading_path 컬럼 (V002/V007 트리거가
chunks.heading_path_json 그대로 INSERT) 의 JSON 표기 + 안의 path
세그먼트 (app, src) 까지 3-gram 색인해서 query 가 우연히 false
positive hit 하는 문제. column filter 채택 — heading 색인 유지
(V007 verbatim 불변), 매칭 대상만 text 컬럼 한정.

- build_match_string 가 non-raw 분기에서 combined expression 을
  `text : (<expr>)` 로 wrap. FTS5 column filter syntax 가 OR/AND
  sub-expression 허용.
- Raw mode (`'...'`) 는 그대로 — 사용자가 명시 의도로
  `'heading_path : agent'` 같은 explicit opt-in 가능 (escape hatch).
- 8 기존 build_match_string unit test expected string 갱신 +
  `build_match_string_raw_mode_preserves_heading_filter` 신규.
- `lexical_heading_only_token_does_not_hit_default_mode` 신규 회귀 핀
  (heading-only unique token 이 default mode 에서 0 hit).
- `lexical_raw_mode_can_opt_into_heading_path_filter` 신규 — 같은
  fixture 가 raw mode 로 hit 확인 (escape hatch 동작 핀).

사용자 영향: lexical / hybrid 검색의 본문 precision ↑. recall
변화 없음 (text 본문 token 매칭은 동일). re-ingest 불필요 (FTS
query 시점 매칭만 변경). lexical_snapshot_run_1 + hybrid_snapshot
도 fixture regenerate 불필요 (text 본문 매칭 query 라 BM25 동일).

HOTFIXES: 2026-05-24 v0.17.0 entry 의 `heading_path_json` 노이즈
항목 closure 표기 + 새 2026-05-25 post-v0.17.1 dogfood entry 추가.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-25 05:40:51 +00:00
parent f2867540d2
commit 271329efbd
3 changed files with 183 additions and 25 deletions

View File

@@ -1060,3 +1060,99 @@ fn lexical_snapshot_run_1() {
let expected: serde_json::Value = serde_json::from_str(&baseline_text).unwrap();
assert_eq!(actual, expected, "lexical run-1 snapshot drift");
}
// ── post-v0.17.1 dogfood — `text` column filter ──────────────────────────
/// Heading-only token (unique to `chunks.heading_path_json`, absent
/// from `chunks.text`) must NOT hit in default mode after the column
/// filter clamp. Pins HOTFIXES 2026-05-24 closure — the JSON
/// punctuation + path segments in `heading_path_json` are no longer
/// matchable from a plain query.
#[test]
fn lexical_heading_only_token_does_not_hit_default_mode() {
let env = Env::new();
let conn = env.raw_conn();
insert_document(
&conn,
&id32("d"),
"notes/heading-only.md",
"Heading-only fixture",
"en",
"primary",
&[],
);
insert_chunk(
&conn,
&id32("c1"),
&id32("d"),
"bravo charlie delta echo",
&["kubernetes-agent-controller"],
Some("Heading"),
r#"[{"kind":"line","start":1,"end":2}]"#,
"v1",
);
drop(conn);
let r = env.retriever();
let hits = r
.search(&SearchQuery {
// "kubernetes-agent-controller" is in heading_path only.
text: "kubernetes-agent-controller".to_string(),
mode: SearchMode::Lexical,
k: 10,
filters: SearchFilters::default(),
})
.unwrap();
assert!(
hits.is_empty(),
"heading-only token must not hit text column; got {} hits",
hits.len()
);
}
/// Raw mode (`'heading_path : <token>'`) is the opt-in escape hatch
/// for users who deliberately want heading-column matching after the
/// default text-only clamp. The same fixture that 0-hits in default
/// mode must hit when the user explicitly scopes to `heading_path`.
#[test]
fn lexical_raw_mode_can_opt_into_heading_path_filter() {
let env = Env::new();
let conn = env.raw_conn();
insert_document(
&conn,
&id32("d"),
"notes/heading-only.md",
"Heading-only fixture",
"en",
"primary",
&[],
);
insert_chunk(
&conn,
&id32("c1"),
&id32("d"),
"bravo charlie delta echo",
&["kubernetes-agent-controller"],
Some("Heading"),
r#"[{"kind":"line","start":1,"end":2}]"#,
"v1",
);
drop(conn);
let r = env.retriever();
let hits = r
.search(&SearchQuery {
// Raw mode: outer single quotes opt out of column-filter
// wrapping and pass the FTS5 expression through verbatim.
text: "'heading_path : \"kubernetes-agent-controller\"'".to_string(),
mode: SearchMode::Lexical,
k: 10,
filters: SearchFilters::default(),
})
.unwrap();
assert_eq!(
hits.len(),
1,
"raw-mode heading_path filter must hit the seeded chunk"
);
}