I1: warning_agent maps ExtractFailed → "kb-parse-md" (the panic-recovery
emitter in kb-parse-md/src/blocks.rs). Lift-stage warnings from
build_canonical_document are tracked separately and attributed to
"kb-normalize", so the I1 mapping change does not lie about
kb-normalize-originated drops.
I2: ParsedPayload::AudioRef no longer synthesizes Block::AudioRef with
an invalid empty AssetId (would violate AssetId::from_str's 32-hex
invariant). Block is dropped, Warning surfaces in Provenance with src
mention, attributed to kb-normalize (lift-stage decision). TODO(P8)
comment marks this as a placeholder until the audio extractor lands.
I3: NFC-normalize each heading_path string in lift_block before feeding
into id_for_block AND into CommonBlock.heading_path. pulldown-cmark does
not NFC heading text and serde_json_canonicalizer v0.3 does not either,
so canonically-equivalent NFD/NFC inputs would produce different
block_ids without this normalization. Mirrors the existing doc_id NFC
handling via to_posix.
Minors:
- M4: trim Cargo.toml — drop kb-config, serde_json_canonicalizer,
blake3 (unused); keep tracing (now wired) + unicode-normalization
(now used by I3).
- M5: determinism_1000_iterations_under_1s now uses the same 5-block
fixture as block_ordinals_scoped_per_heading_and_kind (extracted into
fixture_blocks_five helper) so the determinism property is exercised
on a real lift_block path, not just an empty Vec. Still < 1s.
- M6: snapshot integration test now passes BodyHints { first_h1:
Some("Code And Table"), .. } and asserts doc.title == "Code And Table"
end-to-end. Baseline JSON updated.
- M7: title/lang edge-case unit tests pin policy: empty string lifts to
empty string; non-stringy values silently drop. Rustdoc updated.
- M10: provenance_contains_stage_events_in_order asserts events[1].at
== events[2].at to pin the shared-now_utc invariant.
New tests (unit, kb-normalize):
- provenance_with_extract_failed_warning_attributes_to_kb_parse_md (I1)
- audio_ref_block_skipped_with_warning (I2)
- nfc_nfd_korean_heading_path_same_block_id (I3)
- title_empty_string_in_user_map_falls_back_to_default (M7)
- title_non_string_in_user_map_silently_drops (M7)
- lang_invalid_shape_silently_drops (M7)
kb-normalize unit tests: 9 → 14. Integration snapshot: 1 (unchanged).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the integration snapshot test pinning the full `CanonicalDocument`
JSON for `fixtures/markdown/code-and-table.md` (run through the real
`kb-parse-md::parse_frontmatter` + `parse_blocks`, dev-dep only).
Non-deterministic `provenance.events[*].at` for the Parsed and
Normalized events is stripped before comparison; the Discovered
event's `at` is pinned by constructing the test `RawAsset` with a
fixed `discovered_at`. Run with `UPDATE_SNAPSHOTS=1` to regenerate.
Add the 1000-iteration determinism property: same inputs ⇒ byte-
identical JSON (modulo the same stripped timestamps), in under one
second of wall-clock time. A regression in canonical JSON, BLAKE3
hashing, ordinal counting, or any other deterministic field would
surface here immediately.
The integration test depends on `kb-parse-md` only as a dev-dep, so
`cargo tree -p kb-normalize --depth 1 --edges normal` confirms no
parser implementation appears in the production dep tree per design
§8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Build a `Provenance` with one event per pipeline stage (`Discovered`
sourced from `RawAsset.discovered_at`, then `Parsed` and `Normalized`
stamped with one shared `now_utc()` reading), plus one `Warning` event
per upstream warning. Sharing `now` between Parsed and Normalized
bounds intra-call timestamp jitter — event ordering is preserved by
`Vec` position regardless. Warning agents are routed back to the
upstream component (`kb-parse-md` for parse warnings,
`kb-normalize` for `ExtractFailed`).
Lift `metadata.user["title"]` and `metadata.user["lang"]` (where P1-2
stashes them since the `Metadata` struct itself does not carry those
fields) into `CanonicalDocument.title` / `CanonicalDocument.lang`.
Both keys are removed from the user map after lifting so the wire
form does not duplicate the data; missing keys default to empty
string / empty `Lang`. Other user-map keys survive.
Tests pin the event ordering, the warning routing, and the lift
behavior (including non-duplication in `metadata.user`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implement the §4.3 ordinal rule and §3.4 block lift. Each `ParsedBlock`
maps to a `kb_core::Block` variant carrying a `CommonBlock` whose
`block_id = id_for_block(doc_id, payload_kind, heading_path, ordinal,
source_span)`. Ordinals are scoped to `(heading_path, payload_kind)`,
0-based, in document order — three paragraphs under one H1 get 0/1/2,
a code block under the same H1 starts fresh at 0, a paragraph under a
different H1 also starts at 0.
`payload_kind` is the lowercase-no-spaces convention from §4.2:
"heading", "paragraph", "list", "code", "table", "quote", "imageref",
"audioref".
`ListBlock.items` re-uses the parent list's `CommonBlock` per §3.4 (no
per-item BlockId is allocated). `AudioRefBlock` placeholder fields
(`asset_id`, `duration_ms`) are filled in by P8 — for now we synthesize
the minimal record so the document is well-typed.
Tests pin the four §4.4 ID properties (1000-iteration determinism, NFC
≡ NFD Korean path, `./a/b.md` ≡ `a/b.md`, ordinal grouping). Provenance
and title/lang lift land in the next commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the workspace member, `Cargo.toml` with the §8-allowed dep set
(kb-core, kb-parse-types, kb-config, serde, serde_json_canonicalizer,
blake3, unicode-normalization, time, anyhow, tracing) and a stubbed
`build_canonical_document` that pins the public signature plus
`doc_id` derivation. `kb-parse-md` is permitted only as a *dev*-dep so
the integration snapshot test (added later in this series) can drive
a fixture through the real parser without violating the production
boundary — `cargo tree -p kb-normalize --depth 1 --edges normal`
confirms no parser implementation appears in the regular dep tree.
`id_for_doc` and `id_for_block` are re-exported from kb-core (which
holds the canonical recipe per §4.2); kb-normalize is the canonical
*entry point* per design §8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
