feat(rag): fb-41 PR-9c-1 — core types + wire scaffolding (NLI verification)

Surface-only PR (no behavior wiring — that's PR-9c-2):
- kebab-core: RefusalReason::NliVerificationFailed + NliModelUnavailable (serde rename_all="snake_case", wire = identical strings).
- kebab-core: Answer.verification: Option<VerificationSummary> field (additive minor wire — pre-v0.18 reader 무영향).
- kebab-core: VerificationSummary { nli_score: f32, nli_threshold: f32, nli_passed: bool } struct + lib.rs 재-export.
- kebab-config: NliCfg { model, provider } + ModelsCfg.nli (default Xenova/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7).
- kebab-config: RagCfg.nli_threshold: f32 (default 0.0 = disabled, spec §2.6 single gate).
- kebab-config: env override KEBAB_MODELS_NLI_MODEL/PROVIDER + KEBAB_RAG_NLI_THRESHOLD (parse 실패 시 tracing::warn + default 유지).
- kebab-rag: RagPipeline.verifier: Option<Arc<dyn NliVerifier>> field + with_verifier builder (모두 #[allow(dead_code)] — PR-9c-2 의 step 8.5 hook 가 활성화 시 제거). RagPipeline::new signature 유지 (round-2 NEW-M1 Option B).
- kebab-rag: Cargo.toml 에 kebab-nli path 의존 추가.
- kebab-store-sqlite + kebab-tui: 두 신규 RefusalReason variant 에 대한 exhaustive match arm 추가 (snake_case label / 표시 문구).
- 모든 Answer 구축 site (rag 6 + cli/tui/eval 3 fixture) 에 verification: None 추가.
- wire schemas: answer.schema.json verification field + \$defs.VerificationSummary + refusal_reason.enum 2 추가. error.schema.json code.enum + details.description 2 추가 (forward-looking reserved).
- docs/ARCHITECTURE.md: Mermaid Adapters subgraph 의 nli 노드 + rag→nli + app→nli (forward-looking) + nli→config edges. nli→core edge 는 skip (kebab-nli/Cargo.toml direct dep 가 config 만, ARCHITECTURE 컨벤션 = direct deps only). 디렉토리 트리에 crates/kebab-nli/ 추가.

Tests: kebab-core 3 (serde rename + verification skip + struct shape) + kebab-config 6 (defaults + legacy + env + malformed env) + kebab-cli wire 5 (schema verification + enum 검증).
검증: cargo test --workspace -j 1 회귀 0 (pre-existing kebab-mcp::tools_call_ask_multi_hop flaky 1개 동일 — spec 에 명시된 known-flaky). cargo clippy --workspace --all-targets -D warnings clean.
Wire 영향: additive minor — answer.v1 의 verification optional + refusal_reason.enum 확장 + error.v1.code 확장.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-25 23:27:36 +00:00
parent 79ad6e376f
commit 546c1564b0
15 changed files with 479 additions and 5 deletions

View File

@@ -37,6 +37,29 @@ pub struct Answer {
/// reading older single-pass answers see `hops: None` (or absent).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub hops: Option<Vec<HopRecord>>,
/// p9-fb-41 PR-9c-1: NLI-based post-synthesis verification summary.
/// `None` for single-pass asks and for multi-hop runs with
/// `[rag].nli_threshold == 0` (verification disabled — the default).
/// Present only when the multi-hop pipeline reached the post-
/// synthesize verification step (PR-9c-2 wires step 8.5). Wire-
/// additive: `answer.v1` schema_version unchanged; consumers
/// reading pre-v0.18 answers see `verification: None` (or absent).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub verification: Option<VerificationSummary>,
}
/// p9-fb-41 PR-9c-1: post-synthesize NLI verification summary stamped
/// onto [`Answer::verification`] when multi-hop runs reach step 8.5
/// (NLI gate). Three required fields ride together on every wire emit:
/// `nli_score` is the entailment channel of the XNLI verifier,
/// `nli_threshold` mirrors `[rag].nli_threshold` for audit, and
/// `nli_passed` is `nli_score >= nli_threshold`. The whole struct is
/// omitted (serde skip) when no verification ran.
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct VerificationSummary {
pub nli_score: f32,
pub nli_threshold: f32,
pub nli_passed: bool,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
@@ -153,6 +176,18 @@ pub enum RefusalReason {
/// 단계 진입 못 함. CLI / MCP / TUI 가 받는 wire error code
/// = `"multi_hop_decompose_failed"` (PR-4 의 error_wire 매핑).
MultiHopDecomposeFailed,
/// p9-fb-41 PR-9c-1: post-synthesize NLI verification gate fired —
/// `NliScores::faithfulness()` (entailment channel) fell below
/// `[rag].nli_threshold`. Wire string = `"nli_verification_failed"`
/// (single source of truth: also the matching `error.v1.code`).
/// Multi-hop only; behavior wiring lands in PR-9c-2.
NliVerificationFailed,
/// p9-fb-41 PR-9c-1: NLI verifier was configured (threshold > 0)
/// but the model / runtime is unavailable (download failure,
/// missing tokenizer, ONNX session init error). Treated as a soft
/// refusal — the user sees an unverified-answer outcome rather
/// than crashing the ask. Wire string = `"nli_model_unavailable"`.
NliModelUnavailable,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
@@ -190,6 +225,81 @@ mod tests {
use crate::citation::Citation;
use time::macros::datetime;
/// p9-fb-41 PR-9c-1: pin the wire-side spelling of the new
/// `RefusalReason` variants. The strings here must match
/// `answer.schema.json::refusal_reason.enum` AND
/// `error.schema.json::code.enum` byte-for-byte (single source of
/// truth per spec §2.4).
#[test]
fn refusal_reason_nli_variants_serialize_to_snake_case() {
assert_eq!(
serde_json::to_string(&RefusalReason::NliVerificationFailed).unwrap(),
"\"nli_verification_failed\""
);
assert_eq!(
serde_json::to_string(&RefusalReason::NliModelUnavailable).unwrap(),
"\"nli_model_unavailable\""
);
}
/// p9-fb-41 PR-9c-1: `Answer.verification` is `Option<...>` with
/// `skip_serializing_if = None`. A `verification: None` answer
/// must NOT emit a `"verification"` key on the wire — the field
/// is additive and pre-v0.18 readers see no new key.
#[test]
fn answer_omits_verification_field_when_none() {
let ans = Answer {
answer: "x".into(),
citations: vec![],
grounded: true,
refusal_reason: None,
model: ModelRef {
id: "m".into(),
provider: "p".into(),
dimensions: None,
},
embedding: None,
prompt_template_version: PromptTemplateVersion("rag-v2".into()),
retrieval: AnswerRetrievalSummary {
trace_id: TraceId("t".into()),
mode: crate::SearchMode::Lexical,
k: 1,
score_gate: 0.0,
top_score: 0.0,
chunks_returned: 0,
chunks_used: 0,
},
usage: TokenUsage {
prompt_tokens: 0,
completion_tokens: 0,
latency_ms: 0,
},
created_at: datetime!(2026-05-09 12:00:00 UTC),
conversation_id: None,
turn_index: None,
hops: None,
verification: None,
};
let v = serde_json::to_value(&ans).unwrap();
assert!(
v.get("verification").is_none(),
"verification: None must be omitted from wire output, got: {v}"
);
}
#[test]
fn verification_summary_serializes_all_three_required_fields() {
let vs = VerificationSummary {
nli_score: 0.87,
nli_threshold: 0.5,
nli_passed: true,
};
let v = serde_json::to_value(vs).unwrap();
assert!((v["nli_score"].as_f64().unwrap() - 0.87).abs() < 1e-5);
assert!((v["nli_threshold"].as_f64().unwrap() - 0.5).abs() < 1e-5);
assert_eq!(v["nli_passed"], true);
}
#[test]
fn answer_citation_serializes_indexed_at_and_stale() {
let ac = AnswerCitation {