fb-41 multi-hop RAG 의 **PR-4** (PR-3b-ii 의 ScriptedLm + tests 위에서
user-facing CLI surface + JSON Schema 확장). PR-3b-i / PR-3b-ii 의 multi-hop
pipeline 을 `kebab ask --multi-hop` 으로 사용자에게 노출.
설계: docs/superpowers/specs/2026-05-25-p9-fb-41-multi-hop-rag-design.md
계획: docs/superpowers/plans/2026-05-25-p9-fb-41-multi-hop-rag.md (PR-4 단락)
## CLI surface
- `kebab ask --multi-hop <query>` — 새 flag (default false). `AskOpts.multi_hop`
로 전달, stream + non-stream 두 callsite 모두 갱신.
- `--show-citations` / `--hide-citations` / `--stream` / `--session` 등 기존
flag 와 orthogonal.
- `--json` 모드에서 `Answer.hops` 배열이 multi-hop happy path / refusal-with-
partial-trace 양쪽 경로에서 노출됨 (PR-3b-i + PR-3b-ii 의 wiring).
## Wire schema 확장
- `docs/wire-schema/v1/answer.schema.json`:
- 신규 `hops: array | null` 필드 (optional, additive). `HopRecord` 의
`$defs` 추가 — `iter` / `kind` (decompose|decide|synthesize) /
`sub_queries` / `context_chunks_added` / `forced_stop` / `llm_call_ms`
6 필드 + per-field doc.
- `refusal_reason` 필드를 `anyOf [enum, null]` 로 명시 — 6 variant
(`score_gate`, `llm_self_judge`, `no_index`, `no_chunks`,
`llm_stream_aborted`, `multi_hop_decompose_failed`). 이전 schema 는
`type: string|null` 만 명시 → enum 명시는 agent / consumer 의 strict
validate 강화 (additive — 기존 producer 값 모두 enum 안).
- `$id` / `schema_version` 변경 없음 — additive minor.
- `docs/wire-schema/v1/error.schema.json`:
- `code` enum 에 `multi_hop_decompose_failed` 추가. **이는 forward-looking
enum extension** — 현재 RefusalReason 은 `Answer.refusal_reason` (stdout)
으로만 노출되고 `error.v1` (stderr) 경로 안 거침. 미래 PR 에서 fatal
promotion 정책 결정 시 trigger 가능하도록 enum 만 미리 reserve.
- details.description 의 per-code 안내에 `multi_hop_decompose_failed: {}`
note 추가 — reserved 상태 명시.
## Tests
- `crates/kebab-cli/tests/wire_ask_multi_hop.rs` 신규 (4 Ollama-free pins):
- `cli_ask_help_advertises_multi_hop_flag`: clap-level smoke, `kebab ask
--help` 출력에 `--multi-hop` 등장 확인.
- `answer_schema_declares_hops_property_with_hop_record_defs`: `hops`
property 존재 + `$defs.HopRecord` 의 `kind` enum 3 variant
(decompose/decide/synthesize) 회귀 핀.
- `answer_schema_refusal_reason_enum_includes_multi_hop_decompose_failed`:
6 variant 모두 enum 에 존재 — 기존 5 도 함께 핀 (회귀 방지).
- `error_schema_code_enum_includes_multi_hop_decompose_failed`: 신규
code enum 확장 + 기존 code (config_invalid / not_indexed / ...) 보존 핀.
End-to-end multi-hop ask 의 live Ollama 검증은 후속 `#[ignore]` test 로
(같은 `wire_ask_stale.rs` 패턴). PR-4 의 범위 = clap + schema 정합성 만.
## 변경 없음
- `crates/kebab-app/src/error_wire.rs` — plan 의 "error_wire 매핑" 항목은
현재 RefusalReason 가 `Answer.refusal_reason` 로만 노출 (anyhow chain 안
거침) 라 trigger 가 없음. enum reservation 만으로 충분, 매핑 코드는 dead
code 회피. 향후 fatal-promotion 정책 (refusal → error.v1) 결정 시 PR-4b
로 split.
- `prompt_template_version` — `rag-multi-hop-v1` 그대로.
- TUI / MCP surface — PR-5 / PR-6 에서.
## 검증
- `cargo test -p kebab-cli -j 1` — 모든 test 통과 (신규 wire_ask_multi_hop 4 +
기존 ask / search / schema / ingest / mcp / reset 등 모두).
- `cargo clippy -p kebab-cli --all-targets -j 1 -- -D warnings` clean.
- 단일 crate 직렬 build (16 GB RAM 제약).
## 다음 PR
- PR-5: MCP `ask` tool 의 `multi_hop: bool` argument + `integrations/claude-
code/kebab/SKILL.md` 의 ask 절 갱신.
- PR-6: TUI Ask 패널 multi-hop toggle (F2 / Ctrl-T) + hop trace render.
- v0.18.0 cut (PR-6 머지 후): `Cargo.toml` 0.17.2 → 0.18.0 + HANDOFF /
HOTFIXES / INDEX 갱신 + gitea-release.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
103 lines
4.3 KiB
JSON
103 lines
4.3 KiB
JSON
{
|
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
"$id": "https://kb.local/wire/v1/answer.schema.json",
|
|
"title": "Answer v1",
|
|
"description": "Stub schema — declares the schema_version label and the required fields per design §2.3.",
|
|
"type": "object",
|
|
"required": [
|
|
"schema_version",
|
|
"answer",
|
|
"citations",
|
|
"grounded",
|
|
"model",
|
|
"prompt_template_version",
|
|
"retrieval",
|
|
"usage",
|
|
"created_at"
|
|
],
|
|
"properties": {
|
|
"schema_version": { "const": "answer.v1" },
|
|
"answer": { "type": "string" },
|
|
"citations": { "type": "array" },
|
|
"grounded": { "type": "boolean" },
|
|
"refusal_reason": {
|
|
"anyOf": [
|
|
{
|
|
"type": "string",
|
|
"enum": [
|
|
"score_gate",
|
|
"llm_self_judge",
|
|
"no_index",
|
|
"no_chunks",
|
|
"llm_stream_aborted",
|
|
"multi_hop_decompose_failed"
|
|
]
|
|
},
|
|
{ "type": "null" }
|
|
],
|
|
"description": "p9-fb-41: `multi_hop_decompose_failed` added in PR-3a for the multi-hop pipeline (only emitted when AskOpts.multi_hop = true and the decompose LLM call fails to parse). Other variants are unchanged from earlier phases."
|
|
},
|
|
"model": { "type": "object" },
|
|
"embedding": { "type": ["object", "null"] },
|
|
"prompt_template_version": { "type": "string" },
|
|
"retrieval": { "type": "object" },
|
|
"usage": { "type": "object" },
|
|
"created_at": { "type": "string", "format": "date-time" },
|
|
"conversation_id": {
|
|
"type": ["string", "null"],
|
|
"description": "p9-fb-15: same conversation 의 turn 들이 공유. CLI single-shot / TUI 첫 turn 은 null."
|
|
},
|
|
"turn_index": {
|
|
"type": ["integer", "null"],
|
|
"minimum": 0,
|
|
"description": "p9-fb-15: 같은 conversation 안 0-based 순서. null 이면 single-shot."
|
|
},
|
|
"hops": {
|
|
"anyOf": [
|
|
{
|
|
"type": "array",
|
|
"items": { "$ref": "#/$defs/HopRecord" }
|
|
},
|
|
{ "type": "null" }
|
|
],
|
|
"description": "p9-fb-41 multi-hop trace. Present (non-null array) only when the ask routed through the multi-hop pipeline (`AskOpts.multi_hop = true`); single-pass answers omit the field entirely (serde `skip_serializing_if = None`). Each entry records one LLM hop — decompose / decide / synthesize — with sub-queries, retrieval count, and per-hop latency. Wire-additive: pre-fb-41 readers tolerate the missing field; new readers branch on its presence to render the per-hop trace."
|
|
}
|
|
},
|
|
"$defs": {
|
|
"HopRecord": {
|
|
"type": "object",
|
|
"required": ["iter", "kind", "context_chunks_added", "forced_stop", "llm_call_ms"],
|
|
"properties": {
|
|
"iter": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "0-based hop index. iter=0 is always the initial decompose; subsequent iters are decide calls; the final iter is the synthesize call."
|
|
},
|
|
"kind": {
|
|
"type": "string",
|
|
"enum": ["decompose", "decide", "synthesize"]
|
|
},
|
|
"sub_queries": {
|
|
"type": "array",
|
|
"items": { "type": "string" },
|
|
"description": "Per-kind semantics. Decompose: the initial sub-queries the LLM produced. Decide: the *new* sub-queries to retrieve next iter (empty when the LLM signalled stop or when forced_stop=true). Synthesize: always empty."
|
|
},
|
|
"context_chunks_added": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Number of *new* chunks the retrieval round contributed to the pool (deduped by chunk_id). 0 for decompose / synthesize hops."
|
|
},
|
|
"forced_stop": {
|
|
"type": "boolean",
|
|
"description": "True when the pipeline cut the loop short due to a safety cap (max_depth / max_pool_chunks) rather than the LLM's own stop signal. Tracing signal, not a refusal."
|
|
},
|
|
"llm_call_ms": {
|
|
"type": "integer",
|
|
"minimum": 0,
|
|
"description": "Wall-clock latency of the LLM call for this hop. `0` is overloaded — means 'no LLM call happened' when (a) the Decide hop was skipped due to forced_stop or (b) the pool was empty before any decide could run. Treat 0 as absent or instantaneous."
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|