feat(search): provenance 출처 필터 — [[workspace.sources]] 멀티소스 + --source/--source-type

혼합 출처 KB(위키+jira 등)에서 색인은 전부 하되 질의 시 출처로 좁히는 provenance
레버. 전역 trust 곱셈가중(weighted-RRF)은 A/B 에서 반증(θ=0.85 만으로 incident MRR
0.918→0.340 절벽, 점수 압축) — 필터가 see-saw 없는 올바른 레버.

- config [[workspace.sources]] (각 id/root/exclude/trust_level/source_type);
  단일 root 는 implicit `default` source 로 정규화. validate: id 유일·비어있지 않음.
- config schema v3→v4 (step_3_to_4, root→[[workspace.sources]] id=default 미러, 멱등)
- V014 documents.source_id 컬럼+인덱스 (additive, DEFAULT 'default', 재색인 0)
- Metadata.source_id + BodyHints trust precedence(frontmatter > source 기본값 > Primary)
- ingest: --root 미지정 시 resolved_sources() 순회 + doc 마다 source_id/trust stamp
- 검색 SearchFilters.source_type/source_id → lexical + vector 두 site (IN, OR)
- CLI kebab search --source <id> / --source-type <type> (repeatable/comma-sep)

도그푸딩(620 doc, jira400+wiki220): --source wiki 로 개념 질의 MRR 0.780→0.810,
--source jira 로 incident 0.918→0.975. trust precedence 실측(jira=secondary 기본값).

version bump 0.28.0 → 0.29.0 (신규 CLI flag + config 키 + V014 migration → minor).
follow-up: MCP search 필터 미노출 · kebab list source_id 미표시 · RAG provenance 라벨.

자세한 내용: tasks/HOTFIXES.md (2026-06-21), docs/release-notes/v0.29.0-draft.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Mc6W1fgsrbFKTsqA6P8La
This commit is contained in:
2026-06-21 08:35:19 +00:00
parent 403e162ac0
commit 58ac62d53a
101 changed files with 1201 additions and 111 deletions

View File

@@ -36,6 +36,14 @@ pub struct Metadata {
/// for markdown / pdf / image. Set by the local-filesystem source connector during ingest.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub code_lang: Option<String>,
/// `[[workspace.sources]]`: id of the named source this document was
/// ingested from (the `id` of the matching `[[workspace.sources]]`
/// entry; `"default"` for single-root workspaces normalized to the
/// implicit `default` source). null on documents ingested before the
/// multi-source feature; the store column defaults to `"default"`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_id: Option<String>,
}
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Serialize, Deserialize)]
@@ -105,12 +113,14 @@ mod tests {
git_branch: None,
git_commit: None,
code_lang: None,
source_id: None,
};
let v = serde_json::to_value(&m).unwrap();
assert!(v.get("repo").is_none());
assert!(v.get("git_branch").is_none());
assert!(v.get("git_commit").is_none());
assert!(v.get("code_lang").is_none());
assert!(v.get("source_id").is_none());
}
#[test]
@@ -128,8 +138,10 @@ mod tests {
git_branch: Some("main".into()),
git_commit: Some("a".repeat(40)),
code_lang: Some("rust".into()),
source_id: Some("notes".into()),
};
let v = serde_json::to_value(&m).unwrap();
assert_eq!(v["source_id"], "notes");
assert_eq!(v["repo"], "kebab");
assert_eq!(v["git_branch"], "main");
assert_eq!(v["git_commit"].as_str().unwrap().len(), 40);

View File

@@ -69,6 +69,20 @@ pub struct SearchFilters {
/// Unknown values produce empty hits (consistent with `media` policy).
#[serde(default)]
pub code_lang: Vec<String>,
/// Phase-2 (jira-contamination experiment): filter by `documents.source_type`
/// (`markdown` | `note` | `paper` | `reference` | `inbox`). Empty = no filter;
/// multi-value = OR. Direct indexed column — the clean provenance/source lever:
/// filtering recovers concept-query precision without the see-saw of global
/// trust-weighting (see tasks/HOTFIXES.md A/B evidence).
#[serde(default)]
pub source_type: Vec<String>,
/// `[[workspace.sources]]`: filter by `documents.source_id` (the `id` of
/// the `[[workspace.sources]]` entry a document was ingested from; e.g.
/// `default`, `notes`, `code`). Empty = no filter; multi-value = OR.
/// Direct indexed column (idx_docs_source_id) — the named-source
/// provenance lever for multi-source KBs.
#[serde(default)]
pub source_id: Vec<String>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]