Session Log: Sprint 01 1a.4 — KMaps search-root productionization + ADR 011

Date: 2026-06-18 Participants: Than Grove, Yuji Shinozaki, Xiaoming Wang, Claude Opus 4.7 Outcome: - Decision: ADR 011 — Group collections inheritance (Option D) drafted, accepted, and committed to main; deferred notes superseded; Spike 3 closed Proven; sprint task 1b.2 scope expanded; decision brief closed. - Implementation (on branch sprint-01/kmaps-search-root): Sprint 1 task 1a.4 — search_root_kmapid field setting + config schema + widget/controller plumbing; KmapsPathResolver service; live-Solr verification of the search-root fq (turned up a real domain-mismatch fix: field_kmap_collections + field_language belong to the subjects domain, not terms); KMaps D7→D11 migration mapping doc.

This is the raw conversation transcript. Tool calls and code output are omitted; only the text exchanges are recorded.

User — 2026-06-18 18:27

Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.

User — 2026-06-18 18:27

/model model

User — 2026-06-18 18:27

Set model to [1mOpus 4.7[22m and saved as your default for new sessions

User — 2026-06-18 18:27

review where we are at

Claude — 2026-06-18 18:28

Where we're at — 2026-06-18

Branch: main, clean, up to date with origin. The sprint-01 branch was merged in PR #6 yesterday (commit 7a6c0ca), with a follow-up D11-compatibility fix (9a7f3db) on top.

What just landed (last session, 2026-06-17)

Sprint 1 Step 1a, tasks 1a.1–1a.3 — Images content model on D11: - D10.6 → D11.3.11 bump (composer, DDEV PHP 8.3, Drush 13) - 382-file CMI baseline in drupal/config/sync (shanti_image + 3 paragraph types + external_classification_scheme vocab), round-trips clean - DSF settings convention adopted (committed env-driven settings.php, config_sync_directory = '../config/sync') - Dockerfile → D11 with monorepo paths fixed - ADR 010 accepted (refines "migrate, not improve" — internal modernization is fair game if user-invisible) - Yesterday's hotfix: custom modules' core_version_requirement widened to ^10 || ^11, rebuild.sh switched to site:install --existing-config

Top of today's agenda — Collections Inheritance Decision

The decision brief is teed up as the first item. Three questions, decision-owner is you (Than) as Spike 3 lead:

Q1 — Inheritance implementation. Standing recommendation Option D (entity-ref + custom hooks); no Group contrib gives visibility inheritance for free. Job is ratify / amend / reject.
Q2 — Sequencing: model structural Group types into the CMI baseline now (deferring inheritance), or hold all collections work for Step 1b?
Q3 — Phasing: reconcile deferred-notes "Phase 3" vs Sprint 1 task 1b.2 "Step 1b".

5-min prep suggested: re-check ggroup/subgroup D11 release status (findings are from 2026-06-12).

Still open in Sprint 1 Step 1a

1a.4 (KMaps search-root productionization — placeholders flagged in build script), 1a.5 (IIIF wiring), 1a.6/1a.7 (migrations from the D7 dump), 1a.8 (Solr sync), 1a.9 (rollback). Acceptance criterion #1 ("installs via CMI") would be most rigorously proven by a fresh site:install --existing-config.

Spike status (unchanged)

1 ✓, 2 ✓, 3 ◐ (you — gating today's decision), 4 pending (you), 5 pending (Xiaoming), 6 pending (you — Phase 3 gate), 7 pending, 8 ◐ (Yuji).

Ready to start on Q1 whenever you are.

User — 2026-06-18 18:29

For spike 3 you say "gating today's decision" what is that decision?

Claude — 2026-06-18 18:29

The decision gating on Spike 3 is Q1 in today's brief: which inheritance approach do we use for Group collections → subcollections?

Spike 3 proved the structural half (Group collection/subcollection types, one-level nesting via a parent_collection entity-ref, basic access control) but explicitly did not resolve the inheritance model:

Visibility inheritance — private parent ⇒ subcollections private by default (overridable to public).
Membership inheritance — subcollection inherits all parent members (additive locally).

Both must match D7 og_subgroups behavior. The brief's table:

Option	Membership	Visibility	D11 status	Verdict
A — entity-ref only	✗	✗	proven	insufficient
B — `ggroup`	✓	✗ (custom)	not D11-ready	blocked
C — `subgroup`	✓	✗ (custom)	unconfirmed	risky
D — entity-ref + custom hooks	✓	✓	no dep risk	recommended (~2–3 d)
E — ggroup/subgroup + custom visibility hook	✓	✓	dep-blocked	revisit later

Key fact driving the recommendation: no Group contrib option provides visibility inheritance out of the box — visibility always needs custom logic, which weakens the case for taking on ggroup/subgroup as a dependency.

Why it gates everything else: the Images model (just built) chose Paragraphs for access inheritance, but paragraph access only holds if the parent image's access is correct — and that comes from Group collection visibility. So the 1a.7 image migration's access story depends on resolving this. That's why it's first on the agenda before any more collections work happens.

User — 2026-06-18 18:35

let's go with Option D

Claude — 2026-06-18 18:35

Option D locked in — entity-reference nesting + custom hooks for both visibility and membership inheritance. That becomes ADR 011 when written up.

Two follow-ups from the brief still need answers before I touch anything:

Claude — 2026-06-18 18:37

What would you like to clarify? Happy to unpack any of Q2, Q3, or the ADR-drafting question — or reframe them entirely if I'm asking the wrong thing.

User — 2026-06-18 18:37

Please unpack Q2

Claude — 2026-06-18 18:38

Q2 unpacked — what "structural Group types now" actually means

The question is whether to extend the CMI baseline we just committed with a few more config files before we know how inheritance will be implemented, or to leave the entire Group/collections layer empty until Step 1b.

What "model structural Group types" includes (and doesn't)

If we go now, the additions to drupal/config/sync would be roughly:

Config	What it does	Risk
`group.type.collection`	Group bundle for top-level collections	None — bundle definition
`group.type.subcollection`	Group bundle for child collections	None
`field.field.group.collection.parent_collection` (or only on `subcollection`)	Entity-reference field pointing a subcollection at its parent	None — pure field config
`group_content_type.group_content_type.collection-group_node-shanti_image`	Plugin enabling `shanti_image` nodes to be members of `collection` groups (and same for `subcollection`)	None — relation declaration
Group permissions / role configs per type	Member/anonymous/outsider role permission grids	Low — declarative

What it excludes (deliberately): - Custom hooks for visibility inheritance (the Option D logic) - Custom membership-sync code - The visibility_overridden flag mechanism - Any migration code from D7 og_collection_ref → Group membership

So "structural" = the schema (group bundles, fields, relations, permissions). "Inheritance" = the behavior (hooks + custom logic). Q2 is asking whether to land the schema now and the behavior later.

Case for (a) "model now"

It's the same kind of low-risk CMI work we just finished — declarative bundle/field/plugin config, no contrib dependencies, no custom code. Nothing in this list pre-commits us to anything Option D doesn't already need.
Closes a real gap in the Images baseline. Right now shanti_image has no group-relation config, so even a manual test of "put an image in a collection" isn't possible against the committed baseline. The skeleton lets the team start exercising the model.
Surface for 1a.7 migration planning. The image migration script needs target Group bundles to map D7 field_og_collection_ref against. Having the bundles in config makes 1a.7 design concrete instead of hypothetical.
Reversible. It's CMI config — if the inheritance design forces a structural change later, we edit the config and re-export. The cost of "wrong now" is small.

Case for (b) "hold for 1b"

ADR 009 deliberately decoupled Step 1a (public win) from Step 1b (auth/groups). The whole point was to ship Images-as-a-readable-thing without entangling auth complexity. Adding Group config to 1a partially blurs that line — even if the behavior stays out, the config is now Group-aware.
Permissions matrices are easy to get wrong. Group's per-bundle permission grids are detailed, and getting them right depends on the inheritance approach (Option D's hooks need certain hooks/permissions to exist). Landing them now means a permissions revision when 1b actually implements inheritance.
The 1a.7 migration argument cuts both ways. Images can be migrated in 1a without group membership and have it backfilled in 1b — that's exactly the OG→Group sequencing 1b.2 already implies.

Hidden third option worth naming

(c) Half-step: bundles + relation, no permissions yet. Land collection / subcollection group types and the group_node:shanti_image relation plugin, but defer the permission-grid config to 1b.2 when inheritance behavior lands. This keeps the schema visible (helps 1a.7 migration design and team familiarity) without committing to a permission model that Option D's hooks may need to revise.

What I'd point at

The strongest argument either way: how soon will the 1a.7 migration script be written? If 1a.7 is near-term (this sprint), having the Group bundles in config now makes the migration target real instead of a placeholder — argues for (a) or (c). If 1a.7 will wait for 1b anyway, the case for landing config now weakens — argues for (b).

What's your read on the 1a.7 timeline? That probably decides it.

User — 2026-06-18 18:41

What is 1a.7?

Claude — 2026-06-18 18:42

1a.7 — the Images data migration (the pattern-setting one)

From the Sprint 1 backlog (line 82):

1a.7 — Migrate shanti_image with node→paragraph transform for satellites; source satellites via the image reference field, not the raw node table (skips ~12k orphan agents, ~17k orphan descriptions); expect shared-agent fan-out (~111k agent paragraphs)

In plain terms, it's the Drupal Migrate API script that takes the D7 production Images dump (the 111,340-image mandala-prod-images-db_2026-06-11.sql.gz sitting at /Users/ys2n/Code/mandala-legacy/) and writes it into the D11 model we just built.

Why it's a big deal in the sprint

The backlog calls it out as "the pattern-setting migration" and the only mob task explicitly marked that way. The plan (lines 86–87):

After 1a.7 establishes the pattern + test + rollback story, individuals replicate it for the remaining field/paragraph types.

So 1a.7 is where the team works through the shape of: - node → entity transform (image D7 nodes → shanti_image D11 nodes) - node → paragraph transforms for the three satellite types (agents, descriptions, classifications) — sourced via the parent image's reference field, not the raw D7 node table (that's how the orphan-skipping works) - handling shared-agent fan-out (~111k paragraph rows from a smaller agent-node population, because a single D7 agent node is reused across many images) - transliteration NFC/NFD preservation through Migrate API → MySQL → Solr - the test-run → validate → rollback cycle that 1a.9 then documents

Every subsequent migration (Texts, Sources, AV, etc. in Phase 2/3) is supposed to be a variation on 1a.7's pattern, so getting it right matters disproportionately.

Why it connects to today's Q2

1a.7's acceptance includes "Security: a restricted Images item is non-retrievable by an unauthorized user" (criterion 7). That security check requires Group collection membership to be present and access to work correctly. If Group config doesn't exist when 1a.7 runs, either (a) we can't validate that criterion in Step 1a — it has to wait for 1b, or (b) we'd land Group structural config for 1a.7's validation.

So Q2 is really asking: does 1a.7 need to validate the security path against real Group config, or is that explicitly punted to 1b.3?

Reading the sprint doc carefully, the security acceptance is listed under Sprint 1 as a whole, not Step 1a — and Step 1b's tasks (1b.1, 1b.3, 1b.4) are exactly the ones that exercise it. So the current sprint plan does intend for Step 1a's 1a.7 to migrate the data (including any group-membership fields, presumably to placeholder/empty Group bundles) without yet proving the security path.

That's a softer answer than I initially implied: 1a.7 doesn't strictly require the Group config to land before it runs, but it does need somewhere to write the OG membership data. Which means Q2 has a sharper version:

Does 1a.7 migrate OG membership at all, or is that 1b.2's job entirely?

If 1b.2 owns the membership migration (which the task title "Migrate OG → Group: field_og_collection_ref → Group membership" strongly suggests), then 1a.7 can run with no Group config at all — and (b) "hold for 1b" becomes much more defensible than I gave it credit for.

Worth pulling up the audit's view on this to confirm — want me to check, or does that match your reading?

User — 2026-06-18 18:43

yes, check the audit

Claude — 2026-06-18 18:43

The audit is unambiguous — OG/Group is Step 1b's job, not 1a.7's

Three direct citations:

Line 232–233 — the OG fields are explicitly tagged as Phase 1b:

group_content_access — "OG access — Phase 1b auth concern" field_og_collection_ref — "OG membership → Group relationship"

Line 324–326 — the audit's own "open items" list:

OG → Group migration mapping. group_content_access (Visibility) and field_og_collection_ref depend on the collections architecture (Spike 3, still partial) and the proxy-auth contract (ADR 009 Step 1b). Out of scope for Step 1a public plumbing.

Line 346–348 — the audit's recommendation for the migration script:

Write the D7 → D11 migration as a node→paragraph transform for the satellites … defer OG/auth to Step 1b.

And on the structural side, the audit confirms (lines 270–274) that production has only 55 collections + 116 subcollections as their own node bundles in the D7 dump — that's a small, separate migration, and they aren't being modeled in the Sprint 1 baseline at all yet.

What this means for Q2

The audit closes the door I left open in my previous answer:

1a.7 does not touch OG/Group fields. Both group_content_access and field_og_collection_ref are explicitly punted to 1b.
The Images CMI baseline doesn't even include those two fields yet. I authored the model with the 48 fields the audit specified and dropped the 2 OG fields, exactly as the audit directs. (Confirmed in last session's wrap: "48 fields (the 50-field D7 inventory minus the 2 OG fields, deferred to Step 1b)".)
So 1a.7 can run start-to-finish with no Group config in the system at all. The image bodies, paragraphs, KMaps fields, IIIF linkage, transliteration fidelity — all of that is Step 1a's concern. Group bundles, membership, visibility, and the security-path acceptance criterion belong to 1b.

My read on Q2 now

The audit's "out of scope for Step 1a" is pretty pointed. Landing Group structural config in the 1a baseline would push against the audit's stated scope, not align with it. The case for (b) "hold for 1b" — which I underweighted earlier — is now the stronger one:

It respects ADR 009's decoupling and the audit's explicit scope boundary.
It avoids landing permission grids that Option D's hooks may need to revise.
Nothing in 1a.7 needs it.
The collections architecture itself (one-level nesting, Option D hooks, edge cases) is still being designed and will land as a coherent unit in 1b.

The case for (a) "model now" really only stood on "1a.7 needs a target" — which the audit just dissolved.

I'd flip my earlier framing: recommend (b) — hold all collections config for Step 1b, where it lands together with the inheritance hooks (1b.2) and the security validation (1b.3/1b.4). Q3 then becomes: does the inheritance behavior land with 1b.2 in this sprint, or slip to Phase 3?

Does (b) match your read now too?

User — 2026-06-18 18:44

yes, hold for 1b

Claude — 2026-06-18 18:45

Q2 resolved: hold all collections work for Step 1b.

Q3 — which phase implements the inheritance hooks?

The deferred notes (written 2026-06-12, when Option D was still tentative) said "before Phase 3." Sprint 1 task 1b.2 says Step 1b. With Option D chosen, the reason for that gap has changed.

What 1b.2/1b.3/1b.4 actually need

Looking at Step 1b's tasks together:

Task	What it does
1b.2	Migrate OG → Group: `group_content_access` (Visibility) + `field_og_collection_ref` → Group membership
1b.3	Solr-proxy visibility filtering; prove access-control coherence (search results agree with node/Group access)
1b.4	Confirm paragraph access inheritance: a private image's satellite paragraphs are not independently retrievable

And Sprint 1's security acceptance criterion (line 111):

A restricted Images item is non-retrievable by an unauthorized user via the D11 search path, and retrievable by an authorized one.

The crunch point: for a private collection containing subcollections (which is the production shape — 55 collections / 116 subcollections, one-level nested), the security test only succeeds if subcollection visibility actually inherits. Without the Option D hooks, an image in a subcollection of a private collection would leak — because the subcollection's visibility wouldn't track the parent's.

So 1b.3 (access-control coherence) and the Sprint 1 security acceptance gate can't truthfully pass with inheritance hooks deferred to Phase 3. We'd be shipping a Step 1b that proves the easy half of the security model.

What changed since the deferred notes were written

The Phase 3 deferral was defensive — written when the inheritance approach was unsettled (could it be ggroup? was that D11-ready? would it require contrib bets?). Option D is ~2–3 days of custom hook work with no contrib dependency. The original reason to push it back doesn't apply anymore.

Recommendation

Inheritance hooks land with 1b.2 in Step 1b (this sprint), not Phase 3. Specifically:

1b.2 owns both the membership migration and the Option D hooks (visibility inheritance, membership cascade, visibility_overridden flag, sub-only member retention edge case).
1b.3 validates that 1b.2's hooks produce coherent search-path access.
The two High-priority deferred notes (group-subgroup-nesting-approach.md, group-access-inheritance-subcollections.md) get superseded by ADR 011 rather than carrying a "Phase 3" tag forward.

The cost is Step 1b grows by 2–3 days of hook work. The benefit is Sprint 1's security acceptance criterion is honestly meetable.

Sound right, or do you see a reason to keep it in Phase 3?

User — 2026-06-18 18:48

What is ADR 011?

Claude — 2026-06-18 18:49

Fair catch — I've been name-dropping it as if it exists. It doesn't yet.

ADR 011 is the ADR you would write to record today's decision. The number is just the next available slot (001–010 are the accepted ones; 011 is the next number). The decision brief itself anticipates this:

Outcome: becomes an ADR once resolved (supersedes the "recommended" status of the deferred notes)

So once Q1/Q2/Q3 are settled — which after this conversation they essentially are — someone needs to write docs/adr/011-group-collections-inheritance-option-d.md (or similar slug) that captures:

Decision: Option D — entity-reference nesting + custom hooks for visibility and membership inheritance.
Context: Group 3.x on D11 has no built-in subgroup plugin; ggroup not D11-ready, subgroup D11-unconfirmed; none of the contrib options give visibility inheritance out of the box. Production is one-level deep (55 / 116). Spike 3 proved the entity-ref structural half.
Consequences: ~2–3 days custom hook work in Sprint 1 Step 1b; lands in 1b.2 alongside the OG→Group membership migration. Edge cases scoped in: visibility_overridden flag (so parent changes don't stomp independent settings), sub-only member retention (a member removed from parent but added directly to subcollection must not be cascade-removed).
Sequencing: all collections work held for Step 1b (Q2). Inheritance hooks land with 1b.2 in Sprint 1, not deferred to Phase 3 (Q3).
Supersedes: the "Option D recommended" standing in docs/deferred/group-subgroup-nesting-approach.md and docs/deferred/group-access-inheritance-subcollections.md — both notes get closed out (or marked Superseded by ADR 011).

Per the workflow rules in CLAUDE.md, once an ADR is accepted it's immutable — so the wording matters. Writing it now (rather than later) is mostly about not letting the decision drift before it's recorded.

Want me to draft ADR 011 now so it's reviewable before the next session?

User — 2026-06-18 18:51

yes, draft ADR 011

Claude — 2026-06-18 18:59

Now the index updates so the ADR is visible.

Claude — 2026-06-18 19:00

ADR 011 drafted and indexed.

What landed

docs/adr/011-group-collections-inheritance.md — full ADR
docs/adr/README.md — table updated
docs/adr/.pages — mkdocs nav updated

Structure

Status: Accepted, dated today (2026-06-18), with Than as decision owner + Yuji
Context — Spike 3 baseline, settled requirements, production scale (55/116, one level), the contrib comparison table, the decisive observation that no contrib gives visibility inheritance
Decision — Option D mechanics: structural shape, the five hooks, the visibility_overridden flag mechanics, sub-only member retention behavior, plus Q2 (hold for 1b) and Q3 (hooks land with 1b.2, not Phase 3) inline
Consequences — concrete work added to 1b.2, Sprint 1 security gates that now have a basis, architectural consequences, explicit non-decisions (module name, UI affordance, performance), documentation effects
Supersedes — the "Recommended" standing of both deferred notes; closes the decision brief; promotes Spike 3 from ◐ Partial → ● Proven

What it does NOT do yet (your call)

Not committed — sitting in the working tree, nothing staged
Deferred notes still carry their old "Recommended" language. They need a "Superseded by ADR 011" header per the workflow ritual
Spike 3 doc still reads as Partial — needs a closing note pointing at ADR 011
Decision brief still says OPEN — needs a closing note
Sprint 1 doc task 1b.2 still reads as just "OG → Group migration" — should reflect the expanded scope (membership migration + inheritance module)
project-mandala-state.md memory lists ADRs 001–010 and Spike 3 as Partial — needs an update

Want me to do the closing-out documentation pass and then commit the whole set as a single "ADR 011 + closeouts" commit? Or pause here for review of the ADR text first?

User — 2026-06-18 19:01

Xiaoming is here too.

Claude — 2026-06-18 19:01

Hi Xiaoming. Glad to have you in the room.

Want me to add you as a decider on ADR 011? You'd be a natural fit given you'll likely be touching the inheritance module work on the engineering side, and the brief left room for "with Yuji Shinozaki" alongside Than as the decision owner — adding you reflects the actual room.

If yes, I'll update line 5 of the ADR:

**Deciders:** Than Grove (Spike 3 lead, decision owner), Yuji Shinozaki (Lead Architect), Xiaoming Wang

Confirm and I'll make that one edit before the closeout pass.

User — 2026-06-18 19:01

yes add Xiaoming

Claude — 2026-06-18 19:01

Done. Xiaoming added to the deciders line.

Ready for the closeout documentation pass when you are — the list from before:

Stamp "Superseded by ADR 011" on the two deferred notes
Close the Spike 3 doc with a pointer to ADR 011 (◐ Partial → ● Proven)
Close the decision brief (OPEN → resolved by ADR 011)
Update Sprint 1's 1b.2 task to reflect the expanded scope
Update my project-mandala-state.md memory with ADRs through 011 and Spike 3 status

Want me to do all five and commit as one "ADR 011 + closeouts" commit?

User — 2026-06-18 19:01

yes do all five and commit

Claude — 2026-06-18 19:02

Doing the five closeouts now.

1. Stamp the first deferred note:

Claude — 2026-06-18 19:02

2. Stamp the second deferred note:

Claude — 2026-06-18 19:02

3. Move the two items off the deferred open-items table:

Claude — 2026-06-18 19:02

4. Close the Spike 3 doc:

Claude — 2026-06-18 19:03

5. Close the decision brief:

Claude — 2026-06-18 19:03

Also update the planning .pages to reflect the brief's resolved status:

Claude — 2026-06-18 19:04

6. Update Sprint 1 to reflect 1b.2's expanded scope:

Claude — 2026-06-18 19:04

7. Update memory:

Claude — 2026-06-18 19:04

Now let me check the working tree and commit.

Claude — 2026-06-18 19:05

Committed e7a3f5f on main — 10 files, 254 insertions, working tree clean.

What landed

File	Change
`docs/adr/011-group-collections-inheritance.md`	New — Option D ADR (deciders: Than, Yuji, Xiaoming)
`docs/adr/README.md` + `.pages`	ADR 011 indexed
`docs/deferred/group-subgroup-nesting-approach.md`	SUPERSEDED stamp
`docs/deferred/group-access-inheritance-subcollections.md`	SUPERSEDED stamp
`docs/deferred/README.md`	Both items moved to new "Resolved / superseded" section
`docs/spikes/spike-03-group-collections.md`	Status → ● Proven, closeout note added
`docs/planning/collections-inheritance-decision-brief.md`	Status → RESOLVED, full Resolution section appended (Q1/Q2/Q3 outcomes)
`docs/planning/.pages`	Brief renamed to "(RESOLVED → ADR 011)"
`docs/sprints/sprint-01-images-implementation.md`	Next-session-blocker note resolved; task 1b.2 rewritten to include bundles + relation + inheritance module + OG migration
(memory) `project-mandala-state.md`	ADRs through 011; Spike 3 → Proven; both Group deferred notes struck through

Where the work stands now

Sprint 1 Step 1a tasks 1a.1–1a.3 done (last session). Today resolved the inheritance question that was gating Step 1b. Remaining in Step 1a: 1a.4 (KMaps search-roots), 1a.5 (IIIF wiring), 1a.6–1a.7 (migrations), 1a.8 (Solr sync), 1a.9 (rollback). Step 1b's 1b.2 now has a definitive scope description backed by ADR 011.

Nothing pushed to origin. Want me to push, or hold for review?

User — 2026-06-18 19:06

I pushed it. Let's continue with 1a.4.

Claude — 2026-06-18 19:06

Let me orient on 1a.4 before diving in — Spike 1, the current state of the KMaps field module, and what's already in the baseline.

Claude — 2026-06-18 19:08

I've oriented on 1a.4. Here's what I found and the proposed shape — couple of things to confirm before I touch anything.

What 1a.4 actually entails

The sprint backlog calls for: "Productionize the KMaps field (Spike 1 tail): write path, ancestor-path resolution, migration mapping; wire field_subjects / field_places / field_kmap_terms / field_kmap_collections to kmassets."

Mapping that against the code as it stands:

Capability	State today
Field type schema (raw/id/header/domain/path/defids)	✅ Working (Spike 1)
`kmap_domain` field setting	✅ Working
`search_root_kmapid` field setting	❌ Missing — flagged as placeholder last session
Autocomplete picker	✅ Working (UX deferred separately)
Search-root filtering in autocomplete	❌ Controller has no concept of a search root
`field_kmap_collections` wired to root 2823	❌ Generic `kmap_domain: terms` only
`field_language` wired to root 301	❌ Generic `kmap_domain: terms` only
Path column population via widget	✅ From Solr `ancestor_id_path` in autocomplete payload
Ancestor-path resolution outside the widget (for migrations/programmatic save)	❌ No service; only the widget path works
D7 → D11 migration mapping	❌ Not specified
Write to kmassets (Solr indexing on save)	Out of scope — 1a.8 / reindeer_x territory
`field_kmaps_subject` on `article` bundle	⚠️ Spike 1 demo leftover; doesn't belong in prod

The 4 KMaps fields on shanti_image + field_agent_place (on image_agent) + field_language (on image_descriptions) are all already in the baseline structurally — what's missing is making them per-instance-tunable to their actual KMaps roots and giving non-widget code (migrations) a way to populate the path column.

Two things to confirm before I start

"Wire to kmassets" — read direction, right? My reading: the autocomplete picker queries the kmterms/kmassets Solr (read). Writing kmasset docs back on node save is the D7 pattern that's being replaced by 1a.8 / reindeer_x. So 1a.4 doesn't include any save-time Solr writes. Confirm?
The stray field_kmaps_subject on article — Spike 1's demo field, committed to the baseline. Three options: remove from CMI, leave as a dev fixture, or move to a separate test config directory. My pick: remove, since the baseline should be production-shaped and Spike 1's demo can be re-created via Drush eval if needed.

Proposed work order (5 chunks)

#	Work	Size
A	Add `search_root_kmapid` setting to `KmapsItem.php` (defaultFieldSettings + fieldSettingsForm)	Small
B	Plumb the search root through to the autocomplete: widget passes it on the URL, controller adds `fq=ancestor_id_path:<root>` to the Solr query	Small/medium
C	Update `field_kmap_collections` (→ root 2823) and `field_language` (→ root 301) field configs; re-export	Small
D	New `KmapsPathResolver` service — given `(domain, id)`, returns the ancestor path by querying Solr. The widget and the migration both use it. (D7 called the KMaps API; we can do the same against the kmterms index.)	Medium
E	Migration-mapping spec doc — schema is unchanged, so it's largely a column-for-column copy from D7 `field_data_field_*`; path-refresh policy needs a decision (preserve D7 paths vs. re-resolve at migration time)	Small (doc)
F	Remove `field_kmaps_subject` on `article` from the baseline	Trivial

A/B/C are the natural first batch — they get the field type to where the audit specifies and unblock the rest. D is the heaviest piece (needs Solr query design + tests). E is a planning doc. F is cleanup.

Sound right? And confirm read-direction-only on item 1?

User — 2026-06-18 19:14

How do we verify the solr documents written by the D11 site against our current production index?

Claude — 2026-06-18 19:15

Good question — and worth pausing on, because it's bigger than 1a.4. The Solr docs are written in 1a.8 (the reindeer_x / replacement pipeline), but the contract needs to be defined now, because 1a.4 produces the field data that feeds that pipeline.

What's actually in scope

The kmassets Solr index in prod has two populations (per ADR 006/007): 1. Shadow entries for kmterms — written by reindeer_x. Already running. Not what this conversation is about. 2. Drupal content asset docs (shanti_image and friends) — written by the D7 pipeline: shanti_kmaps_fields → private://solrdocs/ JSON → clsync → synchandler → S3 → SQS → ECS → Solr. This is what we need to reproduce on D11.

ADR 004 says the Solr schema stays as-is. ADR 008 says faithful migration; the explicit Sprint 1 acceptance criterion is "retrievable via existing query patterns" — which is a weaker bar than per-doc bit-parity. That matters because it changes the verification strategy.

Three tiers of verification, ordered by rigor

Tier	What it checks	When useful	Cost
1. Query-result parity	A defined query suite returns the same docs / same result sets against prod vs. D11-shadow. This is what ADR 008 actually demands.	Acceptance gate	Medium — need a representative query corpus
2. Per-doc field diff	Pick N representative nodes; capture their prod kmassets docs; run D11 write path on the same source; field-by-field diff.	Catches per-field regressions before query parity breaks	Low for N=20–50; just needs sample + diff
3. Distribution checks at scale	After full migration: doc counts by type, field cardinality, field length percentiles, KMaps ID coverage.	Catches systematic divergence that small samples miss	Medium — needs reports against the full set

You probably want all three: per-doc diff for confidence on the write path itself, query parity for the acceptance gate, distributions to catch what samples miss.

The harder, unresolved layer

Two infrastructure questions block the execution of any of this:

Can we run a parallel D11 kmassets index (separate core or separate Solr) without disrupting prod, so D11 writes go to a shadow index for comparison? That's part of the Dave Goldstein cost/architecture conversation flagged in solr-pipeline-cost-discussion.md and solr-sync-architecture-d11.md (both High priority deferred). Without a shadow target, verification has to happen offline against captured fixtures.
What is D11's write path? 1a.8 + Spike 8 — still partial. The D7 pipeline (clsync → synchandler → S3 → SQS → ECS) is being replaced; whether by reindeer_x extension, Lambda, or direct Solr write is unsettled. Until that's chosen, "what D11 writes" isn't testable end-to-end.

What we can actually lock down during 1a.4

The contract-definition work is independent of the infrastructure decision and worth doing now while we're in the KMaps field:

Inspect the D7 write path in mandala-drupal — shanti_kmaps_fields module's hook_node_insert/update logic that produced the private://solrdocs/ JSON. That's the authoritative spec of what the doc looks like.
Capture a fixture set — dump the prod kmassets docs for ~50 representative shanti_image nodes (varied: with and without KMaps fields, multi-domain, Tibetan content, restricted vs public). These become the golden reference any D11 implementation has to match. Can be stored as test fixtures in this repo.
Document the doc-shape contract — field-by-field expected shape, sourced from the D7 code + the fixture set. Becomes the acceptance spec for 1a.8.
Sketch the query corpus — what existing queries the React app + any consumers actually run. (Likely a separate small audit; probably owned by you given your front-end / KMaps React app history.)

Items 1–3 are roughly a half-day to a day. They don't depend on Spike 8 resolving or Dave's conversation happening, and they tee both up by giving them a concrete acceptance target.

Two suggestions

Make verification an explicit work item. Right now it's implicit in 1a.8 and the acceptance criteria. I'd lift it out as a documented contract — either a new section in the sprint doc under 1a.8 or a docs/planning/solr-doc-contract.md.
Get the fixture capture out of the way now while we have prod-dump access and we're already in the KMaps field code. Doing it as part of 1a.4 means 1a.8 has a target to hit instead of having to design verification from scratch.

Want me to: - (a) Do the D7-code inspection + fixture capture + contract doc as a 1a.4 sub-task now, or - (b) Punt to 1a.8 and just note it as a pending sub-item there, or - (c) Different scope entirely (e.g., bring Dave into this first)?

User — 2026-06-18 19:22

Are you saying that we don't need a shadow Solr index just because we can compare the documents written to private://solrdocs/?

Claude — 2026-06-18 19:23

No — I was sloppy if it came across that way. Comparing private://solrdocs/ JSON is necessary but not sufficient, and possibly not even applicable on D11. Let me untangle it.

What `private://solrdocs/` actually represents

It's the D7 Drupal module's write output, before the rest of the pipeline runs:

shanti_kmaps_fields (Drupal) → private://solrdocs/*.json
                                ↓
                         clsync → synchandler → S3 → SQS → ECS → Solr

Comparing those JSON files tells you whether D11's field-extraction and Drupal-side write logic matches D7's. That's a useful test, but it stops well before the doc is actually in Solr.

What it doesn't catch

Post-JSON pipeline behavior — clsync/synchandler/the ECS step may transform, validate, or reject docs in ways tied to assumptions about the JSON shape. A D11 write that looks identical at the JSON layer could behave differently downstream.
Solr ingestion itself — schema validation, analyzer chain, type coercion, NFC/NFD normalization. The "indexed" doc isn't the same as the "written" doc; Solr transforms on the way in.
Query-time behavior — two docs with different shapes can satisfy the same query, and two docs with identical shapes can return different results if any of the analysis chain shifted.
D11 may not produce private://solrdocs/ at all. That's a D7 implementation detail. If 1a.8 / reindeer_x writes directly to Solr (or via SQS in a different format, or via Lambda), there's no JSON intermediary to compare. The format is an artifact of the pipeline being replaced.

What the contract actually is

The only layer D7 and D11 are obliged to produce identically is the Solr index — that's the surface ADR 004 declared the source of truth. Everything before that is implementation detail and is allowed to differ. So the verification has to ultimately land at the Solr layer.

"Shadow index" can mean several things

I conflated them in my previous answer. The real options:

Form	Cost	What it verifies
Long-running parallel prod index (e.g., `kmassets_d11_shadow` writing alongside prod)	High (infra, ops)	Full real-traffic divergence over time
Short-lived staging index that D11 writes into during test cycles	Low/medium	Per-batch verification against fixtures
Ephemeral CI Solr spun up per test run, fed a fixture set	Low	Schema + ingestion contract; no live drift detection
Offline JSON comparison only (what I implied)	Trivial	Drupal write logic — but blind to ingestion + queries

The last one alone isn't enough. The middle two — staging or CI Solr — are usually sufficient and don't require a production-parallel shadow. So the cheap answer is: a staging Solr core that D11 writes into during verification runs, fed the same source content. That's a more limited "shadow" than what I implied at first, but it's still genuinely indexing the docs, which JSON comparison can't substitute for.

The practical recommendation, corrected

A two-layer verification, both needed:

JSON-layer diff (if D11 retains a serialized intermediary): cheap regression check on Drupal-side write logic. Fast feedback for 1a.8 development.
Solr-layer fixture verification in a staging/CI Solr: capture prod docs for ~50 representative nodes as the golden reference; D11 writes its output to a staging core; per-doc field diff + query-result parity against the captured fixtures.

A long-running production-parallel shadow is an optional third layer (catches real-traffic drift) but not the minimum bar. The Dave Goldstein conversation is mostly about whether (1) the prod Solr can host an extra core for staging verification cheaply and (2) the deployed pipeline pattern, not about needing a permanent shadow.

Does that land better? And: do you want me to make this verification scoping a docs/planning/solr-doc-contract.md (so it's referenceable from 1a.4 and 1a.8) or keep it conversational for now?

User — 2026-06-18 19:29

defer to 1a.8 and proceed with the field-type changes

Claude — 2026-06-18 19:31

Setting up task tracking, then starting on A.

Claude — 2026-06-18 19:31

Starting with the field-type setting and config schema.

Claude — 2026-06-18 19:32

Now the config schema (the module has none today — current kmap_domain works on Drupal's permissive fallback, but explicit schema is correct):

Claude — 2026-06-18 19:32

Now plumbing the search root through the widget and controller.

Claude — 2026-06-18 19:32

Now the autocomplete controller — read search_root from the query string and apply it as an ancestor_id_path filter:

Claude — 2026-06-18 19:32

Quick PHP-lint pass before moving on, to catch syntax issues early:

Claude — 2026-06-18 19:33

Lint clean. DDEV came up. Moving to task 3.

Claude — 2026-06-18 19:33

Now removing the stray article KMaps demo field. I need to find every config file that references it.

Claude — 2026-06-18 19:34

All field_kmaps_subject references gone. Now the round-trip verification.

Claude — 2026-06-18 19:35

All targeted changes imported. Now check round-trip:

Claude — 2026-06-18 19:35

Clean round-trip. Quick spot-check that the new settings survived:

Claude — 2026-06-18 19:36

All five tasks done. Quick commit then status report.

Claude — 2026-06-18 19:36

The four search_api_solr files are pre-existing key-order drift (module's bundled config got reordered between runs) — same logical content. I'll commit them separately so the 1a.4 commit stays focused.

Claude — 2026-06-18 19:37

Committed b84b3b5 on main.

What landed (commit b84b3b5)

Layer	Change
Field type	New `search_root_kmapid` setting (nullable int) on `shanti_kmaps_fields_default`
Config schema	New `config/schema/shanti_kmaps_fields.schema.yml` — declares both settings, constrains `kmap_domain` to `subjects\\|places\\|terms`
Widget	Passes `search_root` through `#autocomplete_route_parameters` when set
Controller	Reads `search_root` from query; adds `ancestor_id_path:<root>` to Solr `fq` alongside the domain filter
`field_kmap_collections`	`search_root_kmapid: 2823`
`field_language` (on `image_descriptions`)	`search_root_kmapid: 301`
Removed	`field_kmaps_subject` on `article` + storage + 4 display refs (Spike 1 demo leftover)

Verification done: - PHP lint clean (all 3 files) - drush cim/cex round-trip → "No differences between DB and sync directory"

Still untracked in working tree

Four search_api_solr.solr_field_type.text_* files — pure key-order drift from the search_api_solr module (same logical content, reordered between runs). I left them out of the 1a.4 commit. Want me to commit them separately as "incidental drift cleanup" or leave them for someone else / next session?

1a.4 scope still open

The user-said scope was "field-type changes" — done. But two items from the original 1a.4 sprint task aren't done yet:

KmapsPathResolver service — ancestor-path resolution outside the widget, for migrations and programmatic save. Today the path column gets populated from the autocomplete payload; non-widget code can't reliably set it.
D7 → D11 migration mapping doc — schema is unchanged so the mapping is largely column-for-column; key decision is whether to preserve D7-stored paths or re-resolve at migration time.

And one verification gap:

Live-Solr exercise of the search-root filter — requires UVA VPN. The change is logic-only and lint-clean, but I haven't actually proved the new fq returns the expected subset against the live kmterms index.

Want me to continue with (1) and (2), or pause here and pick those up next session? And — do you have VPN to do (3) before we move on?

User — 2026-06-18 19:39

Make a separate branch for this code for now.

Claude — 2026-06-18 19:40

Branch created, main reset. The working tree and Drupal DB are now in slightly inconsistent states though — the active site DB still has the 1a.4 changes applied (from the cim earlier), but sync on main is back to the pre-1a.4 state. Re-syncing the DB to match committed main so future sessions on main are consistent:

Claude — 2026-06-18 19:41

DB is back in sync with main. The four search_api_solr files still show as Different — that's the preexisting key-order drift from before this session; not from my work.

State of things


`main`	`e7a3f5f` (ADR 011 + closeouts) — matches `origin/main`
`sprint-01/kmaps-search-root` (local only)	`b84b3b5` — the 1a.4 KMaps work
Working tree	Clean against `main` (modulo the preexisting `search_api_solr` drift)
Drupal active DB	Matches `main`'s committed config

To pick up the 1a.4 work later

git checkout sprint-01/kmaps-search-root
ddev drush cim -y

That brings the active DB forward to the 1a.4 state. To come back, git checkout main && ddev drush cim -y.

Where 1a.4 stands

Done on the branch: search-root setting + plumbing, two field configs wired to roots 2823 and 301, config schema, baseline cleanup. Round-trip clean.

Still open (would be follow-ups to that branch, or new commits): 1. KmapsPathResolver service — ancestor-path resolution outside the widget 2. D7 → D11 migration mapping doc 3. Live-Solr verification of the search-root fq (VPN-gated)

What's next — pause here, continue on the branch with (1)/(2), or shift to something else?

User — 2026-06-18 19:42

We'll try to do all 3 but let's start with 1

Claude — 2026-06-18 19:43

Cleaning up the completed task list and creating fresh tasks for the three follow-ups.

Claude — 2026-06-18 19:43

Switching to the branch to continue.

Claude — 2026-06-18 19:44

Lint and verify the service registers in the container:

Claude — 2026-06-18 19:45

Service registers cleanly. Now a couple of functional smoke tests — cache hit path (doesn't need Solr) and the live lookup (will reveal whether VPN is on):

Claude — 2026-06-18 19:45

All three smoke tests pass. The live lookup means VPN is actually reachable — so I can probably verify the search-root filter (task 8) too once we get there. Committing the resolver:

Claude — 2026-06-18 19:46

Resolver committed as 76a7cb1. Since live Solr is reachable right now, I'll opportunistically do task 8 (search-root verification) next — VPN could drop — then come back to task 7 (the doc).

Claude — 2026-06-18 19:46

Zero results when filtered. That could mean the filter is working but my probe queries are wrong, OR the fq shape is wrong. Let me check directly what's in Solr:

Claude — 2026-06-18 19:47

Important schema finding. Two things going on:

The terms-2823 doc exists but has empty ancestor_id_path — it's the Tibetan term "kuN+DI/", not "Collections." All 1298 docs with ancestor 2823 are subjects-*, not terms-*. So root 2823 lives in the subjects domain, not terms — but field_kmap_collections is configured as kmap_domain: terms. Either the audit conflated KMaps "subjects" with collections, or the domain is mis-configured.
The terms domain uses a completely different ancestor field: ancestor_id_new.alpha_path (or variants like tib.alpha_path / eng.alpha_path). The terms tree has multiple orderings, each with its own ancestor path. Subjects and places have plain ancestor_id_path; terms doesn't.

Let me probe further before any code conclusions:

Claude — 2026-06-18 19:47

Clear answer:

subjects-2823 = "Collections" ✅ — collections root is in the subjects tree, not terms
subjects-301 = "Language Tree" ✅ — language root also in subjects
The prior session's kmap_domain: terms was wrong; should be subjects for both

Fixing the configs:

Claude — 2026-06-18 19:48

Search-root filter verified working — and the domain correction was real and necessary. Committing both:

Claude — 2026-06-18 19:49

Now task 7 — the migration mapping doc.

Claude — 2026-06-18 19:50

All three follow-ups done. Summary of where the branch stands now:

`sprint-01/kmaps-search-root` — final state

Commit	What
`b84b3b5`	Field type: `search_root_kmapid` setting + schema + widget/controller plumbing + collections/language root configs + baseline cleanup
`76a7cb1`	`KmapsPathResolver` service (single + batched ID-to-path lookup, cached, Solr-fallback-safe)
`fe6117a`	Domain fix: `field_kmap_collections` and `field_language` were on `kmap_domain: terms`; Solr probes proved both roots live in subjects. Both fields corrected
`b72428b`	`docs/planning/kmaps-migration-mapping.md` — 1a.7 spec, three policy decisions pinned (preserve paths, preserve headers, don't validate orphans at migration time)

What was verified live

Resolver: cache-hit returns seeded value; live lookup against kmterms Solr returns the Spike 1 fixture path verbatim (subjects-2610 → 6403/272/282/2610); batched lookups mix cached and live correctly
Search-root filter: subjects + ?search_root=2823 + q=image → 3 results, all paths begin with 2823/.... subjects + ?search_root=301 + q=tib → 7 results, all paths begin with 301/.... Pre-filter (no root) returns the expected larger sets. Filter narrows correctly

What 1a.4 produced beyond the planned scope

The live-Solr probe was meant to be a verification step but turned up a real configuration error — both target fields had been built against the wrong KMaps tree. That correction is captured in fe6117a and also documented in the migration mapping doc so 1a.7 picks up the right domains.

Open questions to flag before next session

Branch lifecycle. Four commits on sprint-01/kmaps-search-root, all local. Ready for review / push / PR whenever you are — same path as the previous sprint-01 branch.
Sprint task 1a.4 status. Strictly speaking the sprint task says "Productionize the KMaps field … write path, ancestor-path resolution, migration mapping; wire field_subjects/places/kmap_terms/kmap_collections to kmassets." All items addressed except "wire to kmassets" in the write direction — that's 1a.8 / reindeer_x. So 1a.4 is done as scoped, modulo the explicit 1a.8 handoff.

Want me to push the branch / open a PR, or stop here?

User — 2026-06-18 19:56

I'll push the branch. Please write the transcript and update the documentation as needed on this branch, and remember where we are.