Skip to content

Session Log: Dev DB Decisions, User Migration Draft, Group-Delete Fix

Date: 2026-07-16
Participants: Than Grove (driving), Yuji Shinozaki, Xiaoming Wang, Claude Code
Outcome: Resolved the flagged dev-DB bootstrap discussion (d11-dev-database-bootstrap-and-migration-source.md, decisions A/B/C — PR #44); scaffolded the cross-cutting D7 user migration (draft PR #45, d7-shared-user-database.md); and fixed the High group-relationship-delete bug (group-relationship-delete-broken-no-data-field.md, PR #46). See [[project-dev-db-workflow]].

This is the raw conversation transcript. Tool calls and code output are omitted; only the text exchanges are recorded.

User — 2026-07-16 13:37

Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.

User — 2026-07-16 13:37

/model model

User — 2026-07-16 13:37

Set model to Sonnet 5 (default) and saved as your default for new sessions

User — 2026-07-16 13:38

ok so we are getting ready for another session with yuji, xiaoming, and myself. They have done a couple of sessions without me. I have pulled down the latest changes on the main branch and fetched the other ones. Please asses and let me know where we are at, right now.

Claude — 2026-07-16 13:39

I'll get oriented by reading the shared docs (per session-startup convention) and checking git for what's landed since your last session.

Claude — 2026-07-16 13:39

Good — local main matches origin/main exactly (the earlier fetch error was transient). Let me check the memory state and the item flagged for discussion today.

Claude — 2026-07-16 13:39

Good — confirmed no open PRs; everything through PR #43 is merged to main. Local branches are all merged/stale copies. Now let's check for any local uncommitted work on the side branches, and glance at the dev-0 database discussion status (the flagged item) and the group-relationship-delete bug since it's marked High.

Claude — 2026-07-16 13:39

Both local branches are fully merged into main (they're stale local copies) and there's no uncommitted work. Here's where things stand.

Status as of now (2026-07-16)

Repo: main is fully synced with origin/main at cfbbad5, working tree clean. Your two local branches (feat/1b1-hybrid-solr-proxy, spike/4b-book-display-model) are stale — fully merged, safe to delete if you want. No open PRs (#19–#43 all merged).

Since your last session (2026-07-10), Yuji has driven 1b.1 Part 4 solo — SAML/NetBadge wiring and getting the CI/CD pipeline actually working:

  • Spike 10 (SAML+OAuth2 coexistence) proven, unblocked 1b.1.
  • 1b.1 Part 4 hit five stacked infra defects (build-tag chain, missing ECR pull permission, legacy Aegir stack squatting on port 8080, a double-sourced ENVIRONMENT var, missing .dockerignore) before reaching first fully green pipeline run on 2026-07-15: Source → Build → Deploy, D11 container now actually serving on dev-0.
  • Legacy Aegir stack on dev-0 is now stopped (not just idle) — both volumes snapshotted first.
  • Dave G pushed 6 commits directly to main on 2026-07-15 (build-tag/SSM convention changes) that briefly broke the deploy playbook before Yuji reconciled them.

Where it's stuck right now: dev-0 serves /core/install.php — the container runs, but mandala_drupal_0 database doesn't exist yet. This is flagged for team discussion today (2026-07-16) in docs/deferred/d11-dev-database-bootstrap-and-migration-source.md — Yuji raised it, nothing decided. Open questions: - Bootstrap dev via a proper deploy_install.yml playbook vs. a manual runbook - Whether dev's deploy should run updb/config:import automatically (it currently does neither) - Where dev's D7 migration source lives — load into RDS as mandala_d7_images (my prior recommendation), point at the live D7 dev DB (recouples to the Aegir stack you just decommissioned), or skip migration on dev entirely

Still-open High-priority items worth surfacing in the session: - group-relationship-delete-broken-no-data-field.md — breaks membership removal in the already-merged 1b.2 collections work - d7-shared-user-database.md — blocks user migration; dev has no real users until resolved - d11-app-has-no-cicd-pipeline.md / saml-alb-routing-assumes-mod-shib.md — largely addressed by Part 4 work but worth confirming closed - ⚠️ Not yet mitigated: kmassets-sync-hook-fires-during-migration.md — a real migration on dev would fire ~111k live Solr writes unless disabled first

Spikes still pending: 4a (Tibetan Unicode, yours), 4b (footnotes, yours — team sign-off still needed on 3 response options), 5 (bibcite, Xiaoming), 6 (API compat, yours), 7 (Kaltura), 9 (docs hosting, low priority).

Want me to pull up the full text of the dev-database discussion doc or the group-relationship-delete bug before the session starts?

User — 2026-07-16 13:53

Can you get rid of any local stale branches?

Claude — 2026-07-16 13:53

Both are fully merged, so -d (safe delete) will work.

Claude — 2026-07-16 13:53

Deleted both. main is now the only local branch — clean slate for the session.

User — 2026-07-16 13:55

what does missing ECR pull permission mean?

Claude — 2026-07-16 13:55

ECR is AWS's Docker registry (Elastic Container Registry) — that's where the pipeline pushes the built D11 image. For the EC2 instance on dev-0 to docker pull that image, the instance needs AWS IAM permission to authenticate to and read from ECR.

The mandala-drupal-staging-instance-role only had CloudWatchAgentServerPolicy attached — a policy for shipping logs/metrics, nothing about container registries. dsf's equivalent role also has AmazonEC2ContainerRegistryReadOnly, but Mandala's didn't, because the legacy Aegir/docker-compose setup never pulled images from ECR (it built/ran containers locally), so nobody had ever needed to grant it.

The reason this took a while to surface: global/playbooks/ecr_authenticate.yml runs docker login to ECR but pipes the result through || /bin/true — so even when the login silently fails due to missing permissions, the playbook reports success and moves on. The actual failure only showed up downstream when the docker pull/docker run step couldn't get the image. Yuji fixed it in terraform commit bf815436b by adding the read-only ECR policy to the instance role, and flagged the || /bin/true pattern as a fleet-wide trap worth raising with Dave (it'll mask this same class of failure on any other service that copies that playbook).

User — 2026-07-16 14:01

yes pull up the full text of the dev-database discussion doc or the group-relationship-delete bug before the session starts

Claude — 2026-07-16 14:01

Both docs, in full.

1. d11-dev-database-bootstrap-and-migration-source.md

(flagged for today's discussion, raised by Yuji)

Priority: High — dev serves /core/install.php until decided; blocks 1a.9 staging execution and part-4 item 7 validation

Status: FLAGGED FOR TEAM DISCUSSION — 2026-07-16. Nothing decided yet; findings are verified, options/recommendation are not.

Why now: Pipeline went green 2026-07-15, D11 container is up on dev-0 — but mandala_drupal_0 doesn't exist, so it redirects to install.php. Yuji's framing: he and Than each have a working local DB, "probably in slightly different states." Question is whether either seeds dev, and whether migrations run there at all.

Verified findings: 1. Nothing automates bootstrap. deploy_backend.yml only runs drush cr and a partial drush cim for SimpleSAMLphp settings — no DB creation, no site:install, no full config:import, no updb, no migrate. dsf's deploy_backend_0.yml is identical, so this is the house pattern. Consequence: config shipped in a commit doesn't reach the site on deploy — someone must run drush updb && drush cim by hand. 2. Drush execution path is solved — the green deploy proved docker exec mandala-drupal-0 drush cr works. staging-migration-execution-prerequisites.md is half stale now; only the source DB blocker remains. 3. D7 source is DDEV-only today. scripts/load-d7-source.sh uses ddev mysql into a d7_images DB; the migration group config literally documents the source as "the secondary d7_images DDEV database." Neither survives contact with RDS. 4. The local source DB name isn't grantable on RDS. mandala_drupal account holds ALL on mandala% only — d7_images doesn't match that pattern, so an RDS source would need to be named e.g. mandala_d7_images, and the source DB name needs to become a parameter, not a hardcoded DDEV name. 5. Dev's DB config (container_0.env.generated): host rds-mysql8-staging.internal.lib.virginia.edu, db mandala_drupal_0, user mandala_drupal. Self-serve — existing grant covers it, no DBA, no new secret needed.

The principle to argue about first: config is code (committed config/sync), content comes from migration (from D7 source via migration-cycle.sh) — both reproducible from the repo. A dump of anyone's laptop is neither — it promotes one person's local state, creates a third unreproducible state, and bakes in local hand-fiddling (manual UUID set, shortcut deletions, test content). So the proposed bootstrap mirrors what a laptop does: 1. create mandala_drupal_0 2. drush site:install 3. drush config:set system.site uuid dfc3f060-3fa3-4a1e-b081-dbc07bdc4323 4. drush entity:delete shortcut + shortcut_set 5. drush config:import

(Steps 3–4 exist only because site:install --existing-config is broken on the standard profile — an argument for fixing rebuild.sh rather than hand-running the workaround on a server.)

Open decisions: - A. Bootstrap: runbook or playbook? A deploy_install.yml makes dev rebuildable on demand and keeps the sequence honest; a runbook is faster today. Fixing rebuild.sh could let both share one path. - B. Should dev's deploy run updb + full cim? Doesn't today. Reasonable for dev; dangerous as a production default (probably why dsf abstains) — if dev diverges from dsf here, do it deliberately. - C. Where does dev's D7 source live? - (a) Load the D7 dump into mandala_d7_images on staging RDS — mirrors local, repeatable, needs a non-DDEV loader + parameterization. - (b) Point at the live D7 dev database — moving target, recouples to the Aegir stack just decommissioned. - (c) Don't migrate on dev at all — config-only. - Recommendation (Claude, not agreed): (a).

⚠ Decide before the first dev migration: kmassets-sync-hook-fires-during-migration.md is only Medium on DDEV evidence, but on dev it's sharp — the 1a.8 direct-to-master sink means a 111k-node migration would fire ~111k live writes into the real kmassets index unless disabled or redirected first.

Also constrains Part 4: d7-shared-user-database.md still blocks user migration, so dev has no real users — Part 4's 5-row provisioning matrix can only test the auto-provision path until that's unblocked.

2. group-relationship-delete-broken-no-data-field.md

Priority: High — affects already-merged 1b.2 functionality (PR #27), not just 1b.1

What we found: Group::removeMember($account) — any path that removes a user from a collection/subcollection group — throws on any collection whose membership-cascade hook runs:

InvalidArgumentException: Field data is unknown.
  in ContentEntityBase->getTranslatedField() (ContentEntityBase.php:630)
  in SqlContentEntityStorage->delete()

Confirmed by disabling mandala_group_inheritance — delete succeeds cleanly with the module off; re-enabling reproduces the failure every time.

Root cause: mandala_group_inheritance_group_relationship_delete() → _mandala_group_inheritance_cascade_member_remove() reads $relationship->get('data') to check an _inherited flag (meant to retain sub-only members when cascading a removal from the parent collection). But group_relationship has no data field at all in this codebase's Group module version — confirmed at both entity-storage and bundle level (collection-group_membership).

This means two things: 1. The _inherited flag has never actually been stored anywhere. The write path (addMember($account, ['_inherited' => TRUE])) silently drops the unknown key on create() rather than erroring — so the "sub-only retention" feature has never functioned, in either direction, since it was written. 2. Any removal that reaches this code throws, because ->get('data') on the delete path is a strict field lookup that errors on an undefined field — unlike create(), which tolerates unknown keys. Group::removeMember() on a collection is currently unusable for any collection with subcollections.

Why 1b.2 didn't catch it: verification exercised counts, private-collection access denial, and the cascade-add path — never a real removal. Add silently masks the bug; only remove throws.

What needs to happen: either - add a real data field to group_relationship (base field or per-bundle config field) so _inherited can actually be stored/read, or - replace the flag mechanism — e.g. a dedicated boolean field for direct-vs-cascaded membership, or diff group rosters instead of a stored flag.

Either way, _mandala_group_inheritance_cascade_member_remove() needs a real fix before membership removal is safe, and "sub-only retention" needs test coverage that actually exercises removal, not just addition.

Cross-references: ADR 011 (Option D, the visibility + membership inheritance feature this breaks); mandala_group_inheritance.module → _mandala_group_inheritance_cascade_member_remove().

User — 2026-07-16 14:30

Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.

User — 2026-07-16 14:30

/model model

User — 2026-07-16 14:30

Set model to Opus 4.8 and saved as your default for new sessions

User — 2026-07-16 14:30

Could you reassess what our next steps are?

Claude — 2026-07-16 14:31

Here's my read on what comes next, in dependency order.

The one decision that gates everything today

Dev database bootstrap (d11-dev-database-bootstrap-and-migration-source.md). The pipeline is green and the container runs, but dev is a dead site until mandala_drupal_0 exists. Every downstream thing — validating Part 4's NetBadge provisioning, closing the 1a.9 staging-execution criterion, testing anything on real infrastructure — is blocked behind this. It's flagged for today, so it's the natural top of the session agenda. Three concrete decisions to close:

  • A — playbook vs. runbook. My lean: a deploy_install.yml (or fix rebuild.sh so laptop and server share one path). A runbook is faster today but the bootstrap sequence has enough sharp edges (the UUID set, shortcut deletions from the broken --existing-config) that a one-off runbook will rot.
  • B — should dev's deploy run updb + full cim? dsf deliberately doesn't. If dev diverges, do it on purpose and write down why — otherwise config in a commit silently never reaches the site.
  • C — where dev's D7 source lives. My lean is (a): load the dump into mandala_d7_images on staging RDS. It's the only option that's both reproducible and doesn't recouple you to the Aegir stack you just stopped. Costs a non-DDEV loader + parameterizing the hardcoded d7_images name.

I'd frame this to Yuji as: the doc already argues the principle well (config is code, content is migration, a laptop dump is neither) — the session is really just ratifying A/B/C and turning them into work.

The landmine to clear before acting on C

Before the first dev migration runs, the 1a.8 direct-to-master kmassets sink has to be disabled or pointed somewhere safe. On DDEV it's harmless (solr_master_url unset), but on dev a 111k-node migration fires ~111k writes into the real kmassets index. This is easy to forget in the moment and expensive to discover after. It should be a checklist item on whatever bootstrap decision you make, not a separate ticket you get to later.

Real code work that does not wait on the above

The group-relationship-delete bug is independent of the dev-DB question and is a genuine defect in already-merged 1b.2 code: Group::removeMember() throws on any collection with subcollections, and the _inherited retention feature has never actually worked. This is the best candidate for hands-on coding this session if the dev-DB question turns into a devops/decision discussion that doesn't need three people. It needs a design call first though — add a real data/boolean field to group_relationship, or drop the stored-flag mechanism and diff rosters. That's an ADR-sized-ish decision worth 10 minutes with the team before anyone codes it.

The bigger blocker on the horizon

User migration is stuck behind d7-shared-user-database.md (the mandala_shared cross-database prefix — per-site dumps have no real user data). Until that's unblocked, dev has no real users, so Part 4's provisioning matrix can only exercise the auto-provision path, not match-existing. Worth naming today so it's scheduled, but it's a thread of its own, not something to solve in this session.

Your own lane, Than

Distinct from the team infra track: Spike 4b still needs team sign-off on the 3 footnote response options before any implementation branch opens, and Spike 4a (Tibetan Unicode round-trip) and Spike 6 (API compat, the Phase 3 cutover gate) are both still pending. If today skews heavily toward Yuji's dev-DB/pipeline work, 4b sign-off is a cheap thing to extract from the group while everyone's together.

My one-line recommendation for the session: ratify A/B/C on the dev database (with the kmassets-sink safety step baked in), and if that frees people up, spend the rest on the group-relationship-delete design call since it's a High bug sitting in merged code. Want me to draft the deploy_install.yml sequence or dig into the group_relationship field options so you walk in with something concrete rather than a blank discussion?

User — 2026-07-16 14:37

We should start moving our development for the infrastructural work to the mandala-dev server. So we need to resolve how database is loaded and when anybody works on a feature on their laptop, they will sync to that central server before starting any branch work. So for example we need to resolve the users on Mandala-dev.

Claude — 2026-07-16 14:38

This is a good shift, and it actually resolves the drift problem the dev-database doc was worried about. Let me ground the users part in the actual migration code before I advise, since that's the concrete ask.

Claude — 2026-07-16 14:38

Interesting — there's already an update-db-from-remote.sh. Let me look at it and how memberships reference users.

Claude — 2026-07-16 14:39

Grounded now, and two findings sharpen the plan considerably.

The sync mechanism you're describing already exists as a stub. scripts/update-db-from-remote.sh is the dsf-pattern "pull from central before you branch" script — it dumps a remote env, imports into local DDEV, runs config:import + cr. But the actual remote dump line is a commented TODO (needs the SSH/bastion access wired). So "laptops sync from the central server before branch work" isn't new architecture to design — it's finishing this script and establishing the discipline. And note the direction: it pulls server → laptop. That's the coherent reading of your proposal, and it's what resolves the Yuji/Than drift — dev becomes canonical, laptops rebase their DB onto it, nobody's local hand-fiddling propagates.

That also reconciles cleanly with the dev-database doc's principle. The doc argued "a dump of a laptop is neither config nor migration, so don't seed dev from one." Your model doesn't violate that — it inverts it: dev is bootstrapped reproducibly (config:import + migration), and then laptops dump from dev. A dump of dev is legitimate precisely because dev is reproducible from the repo; a dump of a laptop wasn't. So the rule becomes: dev is the only thing that runs migrations; laptops only ever pull dev's snapshot.

The users problem is bigger than "run a migration" — there is no user migration in this repo at all. Only the unused core d7_user templates exist; nothing custom in mandala_migrations. And the membership migration exposes a hard constraint:

# d7_images_collection_memberships
entity_id: etid          # raw D7 uid, no migration_lookup, no stub

Memberships reference users by raw D7 uid with no remap. That means preserving D7 uids is mandatory, not optional — if a user migration reassigns uids, all 111k membership references point at the wrong accounts (or nonexistent ones). This is also why 1b.2 landed "38 admin-only": with no real users present, only the admin uid resolved.

What resolving users on mandala-dev concretely requires

  1. Load the shared user DB as a migration source on RDS. Per d7-shared-user-database.md, real users live in mandala_shared / mandala_shared_dev (tables users, users_roles, role, authmap, sessions), not in the per-site d7_images dump. Good news: mandala_shared_dev matches the mandala% grant, so it's self-serve on RDS — same loader parameterization the d7_images source needs.
  2. Build a d7_users migration in mandala_migrations (from the core d7_user template) that:
  3. preserves uid (forced by the constraint above),
  4. migrates roles (map D7 role ids → D11 roles),
  5. does not migrate password hashes — auth is external via SAML/NetBadge, so this simplifies,
  6. migrates authmap — the externalauth d7_authmap.yml and simplesamlphp_auth d7_simplesamlphp_auth.yml templates already ship in the tree. authmap is what links a NetBadge computing-id to a local uid, i.e. it's exactly what makes Part 4's "match existing account" path work instead of always auto-provisioning.
  7. Re-run the membership migration after users exist, to replace the 38-admin stub with real memberships — this is what finally makes private-collection access mean something.
  8. Verify the authname format matches. The one genuine unknown: does the D7 authmap's stored external name match what SimpleSAMLphp will present on D11 (eppn vs. bare computing-id)? If it doesn't line up, match-existing silently fails and everyone gets auto-provisioned as a duplicate. This may be a small spike rather than a given.

Sequencing for the session

  1. Ratify the dev-DB bootstrap (A/B/C from the flagged doc) — this now has a corollary: only dev migrates; laptops pull.
  2. Load two sources on RDS: mandala_d7_images (content) and mandala_shared_dev (users). The users source was the missing half.
  3. Clear the kmassets-sink landmine before any dev migration run — still the sharpest prerequisite.
  4. Order the migration run: users (uid-preserving) → content → memberships (re-run against real users).
  5. Finish update-db-from-remote.sh (wire the bastion dump) so laptops can pull dev going forward.

Decisions the team owns

  • Migrate all users, or only collection owners/members + auto-provision the rest? My lean: migrate the full set — uid preservation is required anyway, and memberships need them. Auto-provision then only handles genuinely new people post-migration.
  • authname reconciliation — decide whether that's a 30-minute verification or its own small spike.
  • Where the shared user dump comes from — same (a)/(b)/(c) shape as the content source: dump into mandala_shared_dev on RDS (repeatable) vs. point at the live D7 dev shared DB (recouples to the Aegir stack you just stopped).

Want me to draft the d7_users migration YAML (uid-preserving, roles + authmap, no passwords) and finish the bastion dump line in update-db-from-remote.sh so you walk into the session with something runnable rather than a whiteboard?

User — 2026-07-16 14:43

When we start do development separately on our individual laptops, we will be doing migrations on our laptops to develop the migration code. So strictly speaking it won't be true that migrations are only done on dev. It's just the the FINAL migration is done on dev.

Claude — 2026-07-16 14:43

Right — that's the correct refinement, and the distinction matters for the workflow.

The precise principle is: the migration of record runs on dev; laptops run migrations freely, but only to develop the migration code. Both are "doing migrations" — they're just different modes with different products.

What makes that consistent with the anti-drift goal is what each mode commits:

  • Migration-development mode (laptop): load a D7 source, run migrate, inspect results, iterate the YAML, throw the resulting DB away, repeat. The product that leaves the laptop is the committed migration code, not the database it produced. The laptop DB is scratch.
  • Authoritative run (dev): dev re-runs that committed migration code against the canonical source to produce the canonical DB. That DB is what everyone else pulls.
  • Feature-development mode (laptop, non-migration work): pull dev's snapshot, branch, build the feature. Here you specifically don't re-run migrations — you start from canonical state.

So "content comes from migration" still holds strictly — it comes from the authoritative migration run on dev, using code that was developed (and its scratch DB discarded) on laptops. The thing the doc warned against was seeding dev or a teammate from a laptop DB dump; a laptop's migration run never does that — its output is reviewable YAML.

Practical implication: laptops keep both loader paths, and a developer picks the mode: - migration work → load-d7-source.sh (D7 source) + run migrate, - feature work → update-db-from-remote.sh (pull dev snapshot).

For users concretely, this means the d7_users migration gets developed on a laptop against a loaded mandala_shared_dev copy — iterate until uid-preservation, roles, and the authmap/eppn matching all check out — then the authoritative user run happens on dev and becomes the canonical accounts everyone pulls. The authname-format unknown is exactly the kind of thing you want to shake out in laptop migration-dev before the dev run, not discover on dev.

Want me to still draft that d7_users migration so there's something to iterate on, and wire the bastion dump line so the pull path is real?

User — 2026-07-16 14:50

We don't need to use the bastion, because the only reason for the bastion is that the staging and production environments can't communicate to each other directly, not for VPN to Staging work. Please remember this.

Claude — 2026-07-16 14:50

Got it — recording that so it sticks across sessions.

Claude — 2026-07-16 14:50

Recorded. So the correction to the sync plan: update-db-from-remote.sh should reach mandala-drupal-staging-0/dev directly over the VPN to run drush sql:dump — no -J bastion jump. That commented ssh -J bastion ... TODO in the script is the wrong template; the bastion is only for staging↔production paths, which this isn't.

That actually makes finishing the pull script simpler than I framed earlier — a developer on the VPN just needs direct SSH + drush dump, no bastion config.

User — 2026-07-16 14:52

Draft the d7_users migration and finish the pull script

Claude — 2026-07-16 14:52

Let me examine the existing migration conventions before drafting, so the new files match — including the config/install vs config/sync lesson from PR #28.

Claude — 2026-07-16 14:52

Now let me look at a complete existing migration YAML and how the source DB connection key is configured, so the new files match exactly.

Claude — 2026-07-16 14:52

Let me read the settings.php migrate connection block to understand how to add the shared-user source DB.

Claude — 2026-07-16 14:55

I have everything I need. Creating the user migration set (in config/install/, the module source of truth), the source DB connection, a loader for the shared user DB, and finishing the pull script.

Claude — 2026-07-16 14:56

Now the source DB connection in settings.php, mirroring the existing migrate block:

Claude — 2026-07-16 14:56

Now the loader for the shared user DB, mirroring load-d7-source.sh:

Claude — 2026-07-16 14:56

Now the pull script — direct SSH over VPN, no bastion:

Claude — 2026-07-16 14:57

Let me make the loader executable (matching the other scripts) and sanity-check the YAML parses.

Claude — 2026-07-16 14:58

All four YAMLs parse and both scripts pass syntax check. Here's what I built.

The d7_users migration set

Cross-cutting (not under mandala_images), in a new mandala_users group with its own source connection, since the real users live in the shared DB — four files in mandala_migrations/config/install/:

File Role
migrate_plus.migration_group.mandala_users.yml New group, reads from a separate migrate_shared connection
d7_user_role.yml Remaps D7 role names → existing D11 role ids (does not migrate permissions — those stay in committed config)
d7_users.yml The user migration — uid-preserving, roles via lookup, no picture
d7_user_authmap.yml Links accounts to NetBadge identity via externalauth — the "match existing account" enabler

The load-bearing decision is uid: uid in d7_users, heavily commented: the membership migration maps entity_id: etid (raw D7 uid, no lookup, no stub), so if uids aren't preserved, all ~111k membership references break. This is why it's mandatory, not stylistic.

Supporting pieces: added a migrate_shared DB connection to settings.php (pointed at mandala_shared_dev, mirroring the existing migrate block), and scripts/load-d7-users-source.sh which loads the shared dump and then prints the two facts the migrations still need — the actual D7 role list and a sample authname.

Two things flagged in-file to resolve during laptop migration-dev (can't be known without the real dump, exactly the mode you described): 1. d7_user_role's static_map only covers the three default roles — custom roles need filling from SELECT rid, name FROM role. 2. d7_user_authmap's provider filter and the authname format (bare computing-id vs. scoped eppn) must be verified against what SimpleSAMLphp presents on D11, or match-existing silently fails. This is the crux unknown from the deferred doc.

The pull script

scripts/update-db-from-remote.sh is finished: direct ssh $REMOTE_HOST over the VPN, no bastion (with a comment explaining why, per your correction). It does docker exec <container> .../drush sql:dump --gzip on the remote, imports into DDEV, reasserts committed config, and fails loudly on an empty dump. It also documents the two-mode workflow at the top (pull-for-feature-work vs. load-source-for-migration-dev).

Two values to confirm before it runs clean — I used placeholders I couldn't verify: the SSH host aliases (mandala-dev etc. — must resolve on your VPN / ~/.ssh/config) and DRUPAL_HOME=/opt/drupal/app (inferred from the deploy's docker exec path; overridable via env var).

Not yet done (deliberately)

  • These are in config/install/ (module source of truth). To activate they need ddev drush cim to enter active config, then drush cex to write the config/sync/ copies — the config/install-vs-sync discipline from PR #28. I didn't run it since it depends on DDEV state and the actual dumps.
  • Nothing committed — all changes are in the working tree for you to review.

Want me to stage these into a feat/user-migration branch, or leave them loose for you to run through DDEV first?

User — 2026-07-16 15:05

Why would we need to run through DDEV first?

Claude — 2026-07-16 15:06

Fair challenge — I overstated it. "Verify it works" isn't actually available to us: you can't run these migrations to completion until the shared user dump is in hand (it's PII, out-of-band), and the two flagged unknowns (custom roles, authname format) can only be resolved against that real data. So DDEV can't prove correctness right now regardless.

The one genuine reason I had in mind is narrower and it's the PR #28 convention: these live in config/install/, but the committed source of truth is config/sync/, and those sync files are supposed to be the drush cex output — with the uuid:, langcode:, dependencies: {}, sorted keys, and class: null-style fields you saw in the existing d7_images_collection_memberships sync file. If I hand-author the config/sync/ copies I'll produce something subtly different from what cex generates, which is exactly the config/install-vs-sync drift that PR #28 was cleaning up. Running ddev drush cim && drush cex is the only reliable way to generate that canonical form.

But even that is deferrable, not a gate. Two honest options:

  • Commit the working tree as-is — the config/install/ YAMLs, settings block, and scripts. They're correct drafts; the config/sync/ export happens later on whoever's laptop first loads the real shared dump, as a natural part of migration-dev. Cleanest if the point of this branch is "here's the scaffolding to iterate on."
  • Run it through DDEV first only to produce the matching config/sync/ exports so the branch lands convention-complete — worth it only if you want this mergeable rather than a work-in-progress.

Given the migrations are unrunnable until the dump exists, I'd lean toward the first: commit the drafts now, and fold the cim/cex export into the same session where someone loads mandala_shared_dev and starts resolving the roles/authname unknowns. Want me to stage them onto a feat/user-migration branch that way?

User — 2026-07-16 15:07

go ahead and make the branch

Claude — 2026-07-16 15:08

Done. Branch feat/user-migration created and committed (970502c), 7 files:

  • 4 migration YAMLs in mandala_migrations/config/install/
  • settings.php — migrate_shared connection
  • scripts/load-d7-users-source.sh (new)
  • scripts/update-db-from-remote.sh (finished)

Nothing pushed — it's local only, as expected. When you're ready to pick this up on a laptop with the shared dump, the sequence is: load-d7-users-source.sh <dump> → fill in the role map + verify authname → ddev drush cim && drush cex to produce the config/sync/ copies → run migrate:import --group=mandala_users.

Want me to push it to origin, or leave it local until the team's seen the dev-DB decisions land?

User — 2026-07-16 15:08

what are the dev-DB decisions?

Claude — 2026-07-16 15:09

The three flagged for today's discussion, from d11-dev-database-bootstrap-and-migration-source.md (Yuji raised them; nothing decided yet). Dev serves /core/install.php because mandala_drupal_0 doesn't exist, and these decide how it gets a database:

  • A — Bootstrap: playbook or runbook? Codify the create-DB → site:install → set uuid → delete shortcuts → config:import sequence as a deploy_install.yml (rebuildable on demand, stays honest), or do it once by hand (faster today). Related: fixing rebuild.sh could let laptop and server share one path.
  • B — Should dev's deploy run updb + full cim? It doesn't today — so config shipped in a commit never reaches the site until someone runs it by hand. Reasonable for dev, dangerous as a prod default (why dsf abstains). If dev diverges, do it deliberately.
  • C — Where does dev's D7 source live? (a) load the dump into mandala_d7_images on staging RDS (mirrors local, repeatable), (b) point at the live D7 dev DB (moving target, recouples to the Aegir stack you just stopped), or (c) don't migrate on dev at all (config-only). Recommendation in the doc is (a).

Plus the non-negotiable prerequisite that rides along: before any dev migration runs, the 1a.8 direct-to-master kmassets sink must be disabled or redirected — otherwise a 111k-node run fires ~111k writes into the real kmassets index.

How this branch relates: the user migration is a fourth source in the same shape as decision C — it needs mandala_shared_dev loaded somewhere. So resolving C (a) really means "load two sources on RDS: mandala_d7_images and mandala_shared_dev." The feat/user-migration work is the code half; C is the where-does-the-data-live half. That's why I'd hold the push until C lands — if the team picks (a), the branch's migrate_shared connection and loader slot straight in; if they somehow pick (c), the whole thing waits.

User — 2026-07-16 15:13

We are all here and will answer C first. Since the dev database is up but nothing is connected to it, it can serve as the D7 source for both image and mandala_shared users.

Claude — 2026-07-16 15:14

Good — that's C(a), and doing it on the already-up RDS instance is clean. Let me play back what I'm hearing so we're aligned, because there's one naming gotcha baked into it:

Decision (C): The dev/staging RDS instance (rds-mysql8-staging, up, nothing using it yet since mandala_drupal_0 doesn't exist) hosts the D7 source databases — as separate databases alongside the eventual D11 site DB. Two of them: the image content source and the shared-user source. The authoritative migration then runs on dev reading from those.

The gotcha that decision forces — DB naming. The mandala_drupal account holds ALL on mandala% (the Aegir-era grant). So: - mandala_shared_dev → matches mandala% ✅ (already what I named it, no change) - d7_images → does not match mandala% ❌ — on RDS the image source must be named mandala_d7_images to be grantable without a DBA.

That means the source DB name becomes environment-specific: d7_images in DDEV (my committed $databases['migrate'] block), mandala_d7_images on the dev server. migrate_shared → mandala_shared_dev is the same in both, so that half's already right. The server's migrate/migrate_shared connections live in server settings (not the DDEV if-block), so this is a server-config line, not a change to what I committed — but worth writing down now so nobody loads it as d7_images on RDS and then can't grant it.

One follow-on this creates: both loaders (load-d7-source.sh, load-d7-users-source.sh) use ddev mysql. Loading onto RDS needs a non-DDEV path — either run them from the dev-0 box with a plain mysql client, or parameterize host/DB. Small, but it's the "non-DDEV loader" the deferred doc's finding 3 called out.

And the standing prerequisite is now sharper, not hypothetical: the authoritative image run on dev will hit the real kmassets index through the 1a.8 direct sink — it has to be disabled/redirected before that run.

Have I got C right (RDS instance hosts separate source DBs, not mandala_drupal_0 itself serving as source)? If so, I'm ready for A and B.

User — 2026-07-16 15:16

First off, since mandala-dev will become the de facto source of truth for the DB we don't need the local DDEv D7 images DB.

Claude — 2026-07-16 15:17

Agreed, and that's the right consequence of the workflow — with one refinement worth stating so the rule is clean:

The rule: a laptop keeps a local D7 source only for a migration it's actively developing. Images migration is done (1a complete, 111,343 nodes, merged) — so the local d7_images DB is now vestigial. Laptops get image content by pulling dev via update-db-from-remote.sh, not by re-running the migration. So yes, drop it.

Concrete cleanup that follows (dead once nobody develops the images migration locally): - the $databases['migrate'] → d7_images block in settings.php - scripts/load-d7-source.sh

I'd do that as a small separate cleanup rather than fold it into this user-migration branch — it's touching 1a's committed setup. Want me to, or leave those as reference?

But this raises the real question for my branch — the user migration is not done, it's the one thing still in development. So: where does user-migration dev happen, and where does the shared source live? Two options, and I've now committed the first one:

  • (a) Keep it local — the migrate_shared DDEV connection + load-d7-users-source.sh I just committed. Faster iteration.
  • (b) Develop against dev's RDS source directly — no local shared DB at all; consistent with "dev is source of truth," and it never puts the source on a laptop.

And here's the reason I'd lean (b): the shared-user DB is real PII (load-d7-users-source.sh even warns on it). Copying it to every developer's laptop is a data-governance problem the images content never was. Centralizing it on dev, behind the VPN, and developing the user migration there sidesteps that entirely — and it's exactly the "dev is source of truth" instinct you just stated, applied to the source side too.

If the team agrees, then the migrate_shared DDEV block and load-d7-users-source.sh on my branch are the wrong shape — they'd become a dev-server connection + a non-DDEV loader instead. That's a small rework, but better to know before this branch goes anywhere. What's the call — local or on-dev for the user source?

User — 2026-07-16 15:18

Since user migration is core to everything, we are goint to do that on dev. So we want b.

Claude — 2026-07-16 15:19

The team's on (b) — got it. That reshapes the branch: no local shared DB, no local loader; the migrate_shared connection becomes a dev-server connection pointed at RDS. Let me look at how the server gets its DB config so I put it in the right place rather than the DDEV block.

Claude — 2026-07-16 15:20

Now I understand the server path: the D11 connection is built from MYSQL_* env vars, and the source DBs sit on the same RDS instance reached by the same mandala_drupal account (the mandala% grant covers all of them). So the migrate_shared connection can derive from the default one and just swap the database name — no new secret. Reworking to (b):

Claude — 2026-07-16 15:20

Now remove the DDEV-local migrate_shared block I'd added, and delete the local loader:

Claude — 2026-07-16 15:20

All correct. Amending the commit to reflect the (b) decision (it's local and unpushed, so a clean amend):

Claude — 2026-07-16 15:21

Reworked to match decision (b). Branch feat/user-migration (024733b) now has 6 files:

  • 4 migration YAMLs — unchanged; environment-agnostic (they reference the migrate_shared key, wherever it's defined)
  • settings.php — migrate_shared is now a dev-server connection derived from the MYSQL_* env (same RDS instance, same mandala% grant, DB name env-overridable), with an explicit note that there's deliberately no DDEV/local shared connection because the data is PII
  • update-db-from-remote.sh — the direct-over-VPN pull, unchanged
  • removed load-d7-users-source.sh — no local load

So the branch now encodes the workflow you chose: user-migration dev happens on dev against the RDS shared source; laptops never hold the PII, they pull the migrated result via update-db-from-remote.sh.

One open ops gap this leaves (out of scope for a config branch): loading mandala_shared_dev onto RDS needs a non-DDEV loader — a mysql client run from the dev-0 box, not the ddev mysql script I deleted. I can write that once you tell me how you'll run it (from the dev-0 host against the RDS endpoint, presumably using the container's MYSQL_* creds).

Want me to push the branch, or hold it while you finish A and B?

User — 2026-07-16 15:22

Hold it while we finish A and B

Claude — 2026-07-16 15:22

Holding. Branch feat/user-migration (024733b) stays local and unpushed.

When you're ready, the two remaining decisions: - A — bootstrap dev via deploy_install.yml playbook vs. one-time runbook (and whether to fix rebuild.sh so laptop + server share one path) - B — whether dev's deploy runs updb + full cim (or config keeps arriving by hand)

I'll pick back up when you land those — and A/B may reshape whether this branch pushes as-is or needs the config/sync export folded in first.

User — 2026-07-16 15:23

For A, let's do the deploy_install.yml playbook

Claude — 2026-07-16 15:23

The team's on A. Let me find where the Ansible playbooks actually live before I draft deploy_install.yml — they may be in the terraform-infrastructure repo, not this monorepo.

Claude — 2026-07-16 15:24

The playbooks live in the terraform-infrastructure repo on GitLab (mandala/drupal/${ENVIRONMENT}/ansible/), not this monorepo — so deploy_install.yml gets created there, modeled on deploy_backend.yml. Let me check whether that repo is cloned locally so I can match the existing playbook's conventions rather than guess.

Claude — 2026-07-16 15:25

Two things I need to flag before I write it.

Where it lives: deploy_install.yml belongs in the terraform-infrastructure repo (GitLab) under mandala/drupal/staging/ansible/, next to deploy_backend.yml — not in this monorepo. And that repo isn't cloned anywhere on this machine (I checked ~/Sandbox and the whole tree). So to write it matching the real conventions — the host group name, become semantics, how container/drupal_home are templated, the include_vars for container_0.env — I need to see deploy_backend.yml. Can one of you clone terraform-infrastructure into ~/Sandbox/Mandala/, or paste that playbook? Otherwise I'm guessing at var names for infra code, which I'd rather not.

The design, which I can nail now regardless — and the one part that makes an install playbook genuinely different from deploy_backend.yml:

The sequence itself is the doc's five steps, run via the proven docker exec mandala-drupal-0 {{ drupal_home }}/vendor/bin/drush … pattern: 1. create mandala_drupal_0 (self-serve — mandala% grant covers it) 2. site:install (standard profile) 3. config:set system.site uuid dfc3f060-3fa3-4a1e-b081-dbc07bdc4323 4. entity:delete shortcut + shortcut_set 5. config:import

But steps 2–5 are destructive — a site:install against a DB that already has tables silently wipes dev. So the thing this playbook must have that deploy_backend.yml doesn't:

  • A hard idempotency guard up front — check whether mandala_drupal_0 already bootstraps (has tables / a valid Drupal schema), and refuse/skip if so. Without it, re-running the playbook — or worse, someone wiring it into the pipeline — destroys the canonical dev DB. This is the whole reason it's a separate, deliberately-invoked playbook.
  • Kept out of the deployspec build phase. It's run by hand (ansible-playbook deploy_install.yml) exactly once at bootstrap, never on every deploy. deploy_backend.yml stays the per-deploy playbook.

Steps 3–4 are only there because site:install --existing-config is broken on the standard profile (its hook_install) — so this is also the concrete argument for the rebuild.sh fix you flagged under A, since a fixed rebuild.sh and this playbook could share one honest path instead of both hand-coding the workaround.

So: get me deploy_backend.yml and I'll write deploy_install.yml to match, guard included — or say the word and I'll draft it blind for you to reconcile against the real conventions. Which do you want, and shall we move to B?

User — 2026-07-16 15:27

Could we postpone doing that until Yuji or Xiaoming are driving because they have access to that repo?

Claude — 2026-07-16 15:27

Makes sense — it's their repo and their turf. Parking the deploy_install.yml implementation for a Yuji- or Xiaoming-driven session.

The design is captured above, so whoever picks it up has the spec: clone terraform-infrastructure, model on deploy_backend.yml, the five-step sequence via docker exec … drush, and the two things that make it different — the hard idempotency guard against re-wiping mandala_drupal_0, and keeping it out of the deployspec build phase (hand-invoked once, not per-deploy).

So decision A is: deploy_install.yml playbook, implementation deferred to a driver with terraform-infrastructure access. The feat/user-migration branch stays held regardless.

One offer before we move on: the doc this whole discussion resolves — d11-dev-database-bootstrap-and-migration-source.md — is the one flagged "FOR TEAM DISCUSSION 2026-07-16." I can record today's decisions into it (C=b, A=deploy_install.yml deferred) as they land, so the handoff to Yuji/Xiaoming is clean and nothing's lost when the driver changes. Want me to do that now or at session end?

Ready for B — whether dev's deploy runs updb + full cim, or config keeps arriving by hand.

User — 2026-07-16 15:27

why wouldn't we want dev's deploy to run updb + full cim?

Claude — 2026-07-16 15:28

Good question — and mostly the reasons don't apply to dev. Let me separate the generic caution from the one that actually bites here.

Why the house pattern (dsf) abstains — and why it's mostly a prod concern: - config:import isn't transactional. If an export is inconsistent — references a module that isn't enabled, dependency ordering is off — it throws partway and leaves the site in a half-applied, possibly unbootable state. On prod that's an availability incident triggered by a deploy. On dev, the site going down briefly is fine and recoverable. - updb runs hook_update_N + entity-schema updates, which can be long-running and irreversible. On prod you want that done deliberately, in a window, with a backup taken first — not automatically mid-deploy where a failure leaves the schema half-migrated.

Those are real, but they're prod-availability arguments. Dev tolerates all of them.

The one reason that genuinely applies to dev: dev is no longer just config — as of today's decisions it holds the canonical migrated content (the 111k-node dataset, and soon the migrated users). Both updb and a cim that drops or changes a field will mutate or destroy that content — automatically, on a bad commit, with no human checkpoint and no backup taken first. The risk on dev isn't downtime; it's silently damaging the expensive dataset everyone pulls from.

So my recommendation is: yes, run updb + full cim on dev's deploy — but take a DB snapshot immediately before the mutating steps. Reasoning:

  • Auto-cim is actually aligned with everything you've decided today: it enforces config-is-code, kills drift, and means merging a branch like feat/user-migration and deploying makes its new migration configs live automatically — no hand-running. That's the anti-drift discipline you've been driving.
  • The snapshot (RDS snapshot or a drush sql:dump guard) is the cheap insurance that makes it safe on a content-bearing dev — a broken config commit costs you a re-run, not the migrated content.
  • Mark it explicitly as a deliberate divergence from dsf (the doc asks for exactly that), because prod will not do this — prod keeps updb/cim as deliberate, backed-up, human steps.

It also makes decision A's split clean: deploy_install.yml = one-time create + site:install + first import; deploy_backend.yml = updb + cim (snapshot-guarded) on every subsequent deploy.

The snapshot-before-mutate detail is really the whole answer to "why wouldn't we" — without it, auto-updb/cim on a dataset-bearing dev is the one genuinely reckless version. Want me to fold that (B = auto updb+cim on dev, snapshot-guarded, dsf divergence) into the decision record alongside C and A?

User — 2026-07-16 15:30

Yes, fold B into the decision record

Claude — 2026-07-16 15:31

Recording all three into the doc that flagged the discussion. Updating d11-dev-database-bootstrap-and-migration-source.md and the deferred README row.

Claude — 2026-07-16 15:31

Done — recorded in both the doc and the README index. One thing to sort, though: I'm currently on feat/user-migration, so these doc edits are sitting in that branch's working tree. Let me check, because the decision record shouldn't get buried inside the held feature branch.

Claude — 2026-07-16 15:32

As I suspected — the two doc edits are uncommitted on feat/user-migration. They shouldn't ride in the held feature branch; the decision record belongs on its own docs branch off main (matching how the other docs PRs — #37–#43 — landed). The feat/user-migration commit itself is untouched and clean.

My recommendation: stash these, branch docs/dev-db-decisions off main, and commit them there so the decision record can merge independently of the held migration code.

Claude — 2026-07-16 15:33

I'm now on docs/dev-db-decisions (off main), so settings.php and the pull script correctly show main's original versions here — my code changes are safe on the feat/user-migration commit (024733b), not lost. The two doc files are staged. Committing:

Claude — 2026-07-16 15:33

Both branches are intact and clean:

  • docs/dev-db-decisions (34be6b5) — decision record for A/B/C, off main, ready to push/PR independently
  • feat/user-migration (024733b) — verified the code changes survived the branch dance (migrate_shared still present in its settings.php), still held/unpushed
  • main — untouched

Where the session landed

The d11-dev-database-bootstrap-and-migration-source.md discussion — the item flagged for today — is now fully resolved:

Decision
C D7 sources (mandala_d7_images, mandala_shared_dev) live on dev RDS; user-migration dev happens on dev (PII never on laptops)
A deploy_install.yml playbook (idempotency-guarded, out of the build phase) — implementation deferred to a Yuji/Xiaoming-driven session
B dev deploy runs updb + full cim, snapshot-guarded — deliberate dsf divergence

Two things carried forward for the next (DevOps-access) driver: 1. Write deploy_install.yml in terraform-infrastructure per the captured spec 2. Load mandala_shared_dev + mandala_d7_images onto RDS (needs a non-DDEV loader), then develop the feat/user-migration branch on dev — resolving the two in-file unknowns (custom role map, authname format), and disabling the kmassets sink before the first run

Want me to push either branch, or leave both local until the next session? And should I update your project memory with today's decisions so the next session doesn't start stale?

User — 2026-07-16 15:34

is there anything else that can be taken care of now?

Claude — 2026-07-16 15:35

A few things — but let me separate genuinely-useful-now from busywork, and flag the one that's actually well-timed while all three of you are in the room.

Best real-work candidate: the group-relationship-delete bug. It's the one open High item that's (a) real code, (b) fully doable on your laptop in DDEV, and (c) independent of everything you just deferred — it's a landmine sitting in already-merged 1b.2 code (Group::removeMember() throws on any collection with subcollections; the _inherited retention feature has never actually worked). It needs one design call before coding, and that call is exactly the kind of thing to make now while the team's present: - add a real data/boolean field to group_relationship so the _inherited flag can be stored, or - drop the stored-flag mechanism and derive direct-vs-cascaded membership by diffing rosters.

If the room picks a direction, I can implement it this session — it's self-contained and comes with a clear test gap (exercise removal, not just add).

Housekeeping I can do right now: - Refresh your project memory — project-mandala-state is 2 days stale and predates today's A/B/C decisions and the user-migration branch. Worth doing so the next session doesn't start wrong. - Save a session log — this was a substantial planning session (dev-DB decisions + user-migration draft); scripts/save-session-log.py is the ritual for exactly this. - Generate the config/sync exports for feat/user-migration — this is doable locally without the real dump (it's just config entities), and it's the one step that makes that branch convention-complete/mergeable per the PR #28 lesson. Fiddly (needs the module reinstall dance in DDEV) but real.

What I'd not do yet: the local d7_images source cleanup (remove the DDEV migrate connection + load-d7-source.sh). It's implied by today's decisions, but I'd hold it until the images authoritative run exists on the dev server — removing the laptop path before the server path is proven is premature.

My pick: make the group-bug design call now and let me implement it, then I'll do the memory refresh + session log on the way out. Want to go that way?

User — 2026-07-16 15:35

I have pushed both branches

Claude — 2026-07-16 15:38

Both are on origin at the right commits (docs/dev-db-decisions 34be6b5, feat/user-migration 024733b), no PRs open yet. (The fetch spat an SSH warning again but the ref check confirms they're there.)

Two are ready for different treatment:

  • docs/dev-db-decisions — ready for a normal PR; it's self-contained and matches how #37–#43 landed. This one just records today's decisions.
  • feat/user-migration — this is a held draft; the migrations are unrunnable until the on-dev work happens, and it still needs the config/sync export. If you PR it, I'd make it a draft PR so it's visible to Yuji/Xiaoming as the starting point without signaling "ready to merge."

Want me to open PRs? I'd suggest: 1. A regular PR for docs/dev-db-decisions 2. A draft PR for feat/user-migration whose body captures the two unresolved in-file unknowns (custom role map, authname format), the "develop on dev / PII" constraint, and the "needs config/sync export" note — so whoever picks it up on dev has the full context

Say the word and I'll open one or both (I'll need to confirm the PR titles/bodies with you first if you'd like to tweak them). And I can still do the memory refresh + session log whenever you're ready to wrap.

User — 2026-07-16 15:38

Open both PRs, feat one as draft

Claude — 2026-07-16 15:39

Both open:

  • PR #44 — docs/dev-db-decisions → the A/B/C decision record (regular, ready for review/merge): https://github.com/uvalib/mandala-navina/pull/44
  • PR #45 — feat/user-migration → draft, with a checklist body capturing the on-dev/PII constraint, the two unresolved unknowns, and everything that has to happen before it can run or merge: https://github.com/uvalib/mandala-navina/pull/45

PR #45's draft status + checklist means whoever drives the on-dev session (Yuji/Xiaoming) has the full picture as their starting point.

That's the whole session's work landed. Want me to do the memory refresh (project-mandala-state is stale — predates today's decisions and both PRs) and save the session log to close out the ritual?

User — 2026-07-16 15:40

Let's work on the the group-relationship-delete bug

Claude — 2026-07-16 15:40

I'll start by reading the actual module code so the design call is grounded in what's really there, not just the deferred doc's summary.

Claude — 2026-07-16 15:41

Read the whole module. The bug is confirmed and slightly worse than "delete throws" — the retention feature is broken in both directions. Here's what's actually there:

The write side (silently broken): three places call addMember($account, ['_inherited' => TRUE]) — cascade-add (line 242) and subcollection-backfill (line 313). Since group_relationship has no data field (and no _inherited field), that key is silently dropped on save. The flag has never been persisted, ever.

The read side (throws): _mandala_group_inheritance_cascade_member_remove() at line 270 calls $relationship->get('data'). On an entity with no data field, ->get('data') throws InvalidArgumentException: Field data is unknown immediately — the && !->isEmpty() guard is useless because the left operand itself throws. So every collection-member removal that reaches a subcollection blows up.

A useful consequence for the fix: the existing migrated subcollection memberships (from 1b.2) were all created without the flag — and that's actually correct, because in D7 every og_membership was an explicit membership. So "no flag = direct member = retain on parent removal" is the right default for migrated data. No data backfill needed.

Recommendation

Of the doc's two options, I'd go with a sharpened version of "dedicated boolean field": add a base field mandala_inherited to group_relationship via hook_entity_base_field_info() in the module itself. Why this over the alternatives:

  • vs. a generic data field — typed, queryable, self-documenting; we don't need arbitrary JSON blob storage, just one boolean.
  • vs. roster-diffing (no stored flag) — diffing can't distinguish "direct sub member who also happens to be in the parent" from "inherited member," so it silently loses information. A stored flag is unambiguous.
  • base field vs. config field — this is an internal, non-admin-configurable flag; a base field is one hook in the .module, no per-bundle config entities, no config/sync files. Consistent with how the module already works.

The change is small: add the base-field hook, swap the three ['_inherited' => TRUE] writes to ['mandala_inherited' => TRUE], replace the throwing get('data') read with ->get('mandala_inherited')->value, then drush updb to install the field. Plus — per the doc — a test that actually exercises removal, which the module has zero of today.

Before I write it: the team wanted this design call. Confirm base field mandala_inherited is the direction, and whether you want me to include a kernel test now or defer that as a follow-up (it's the larger part of the work)?

User — 2026-07-16 15:43

Sounds good

Claude — 2026-07-16 15:44

Team's confirmed — base field mandala_inherited, with a test. Let me implement the fix, then verify it end-to-end in DDEV, then add the test. First the module changes.

Claude — 2026-07-16 15:44

Got it — deferring the test. I'll implement the fix and verify it in DDEV, and note the test as a follow-up. Continuing with the write and read sites.

Claude — 2026-07-16 15:45

Now the read site (the throwing code) and its docblock:

Claude — 2026-07-16 15:45

Now the .install file to install the field storage on existing sites via drush updb:

Claude — 2026-07-16 15:45

PHP changes done. But I need to check which branch I'm on — these edits shouldn't land on the docs branch. Let me sort that and lint the PHP.

Claude — 2026-07-16 15:45

PHP lints clean. I'm on docs/dev-db-decisions — moving these to a dedicated branch off main:

Claude — 2026-07-16 15:45

Now on fix/group-relationship-delete-inherited-field. Let me verify end-to-end in DDEV: install the field, then run a real scenario exercising the exact path that was broken (cascade-add, then parent-removal with both an inherited and a direct sub-member).

Claude — 2026-07-16 15:46

DDEV up, Group 3.3.5. Applying the field install and rebuilding:

Claude — 2026-07-16 15:47

Update hook ran. Now the real test — a scenario that exercises the exact path that used to throw, plus the retention branch. Writing it as a drush script:

Claude — 2026-07-16 15:48

Path resolution — DDEV's drush root differs from repo root. Let me pass the absolute container path:

Claude — 2026-07-16 15:48

All nine checks pass — including the two that matter most: the previously-throwing removeMember path no longer throws, and direct sub-members are retained while inherited ones cascade out. Let me confirm no stale _inherited/data references remain, then update the deferred doc and commit.

Claude — 2026-07-16 15:48

All matches are the new mandala_inherited field — no stale _inherited or get('data') left. Now updating the deferred doc to mark it resolved (goes on this branch so it's true when the fix merges):

Claude — 2026-07-16 15:49

Now move its README row from Open to Resolved:

Claude — 2026-07-16 15:49

Done. The fix is committed on fix/group-relationship-delete-inherited-field (88e6519).

What the fix does

The bug was worse than "delete throws" — the whole _inherited mechanism was reading and writing a data field that doesn't exist on Group 3.x's group_relationship. Write silently dropped the flag; read threw. Fixed with the design the team picked:

  • hook_entity_base_field_info() adds a boolean mandala_inherited base field (internal, default FALSE)
  • update_9001() installs the field storage on deploy (drush updb)
  • The three flag sites now use the real field — cascade-add/backfill write it, the removal cascade reads it

No data backfill needed — existing migrated memberships default FALSE (direct), which is correct since every D7 og_membership was explicit.

Verified end-to-end in DDEV (Group 3.3.5)

A throwaway scenario (collection + subcollection, an inherited member A and a direct member B) — all 9 assertions passed, including the two that are the whole point: - removeMember(A) on the collection no longer throws (the reported bug) - inherited member A cascade-removed from the subcollection; direct member B retained

Also updated the deferred doc + README to mark it RESOLVED, on this same branch so it's accurate when the fix merges.

Deferred, per your call: the automated removal-path kernel test — the doc's noted gap that let this ship. The .install cleanly installed on the local DB.

Branches now local/unpushed: this fix, plus feat/user-migration (PR #45 draft) and docs/dev-db-decisions (PR #44) already pushed. Want me to push this fix and open a PR? And should I still do the memory refresh + session log to close out?

User — 2026-07-16 15:51

I pushed it, create the PR

Claude — 2026-07-16 15:51

PR #46 opened: https://github.com/uvalib/mandala-navina/pull/46

Full slate of PRs from this session:

PR Branch Status
#44 docs/dev-db-decisions Ready — A/B/C decision record
#45 feat/user-migration Draft — held for on-dev work
#46 fix/group-relationship-delete-inherited-field Ready — the High 1b.2 bug fix

That's a productive session: the dev-DB discussion resolved and recorded, the user migration scaffolded, and a real High-priority bug in merged code fixed and verified.

Worth noting on merge order — #46 and #44 both touch docs/deferred/README.md but in different rows/regions, so they won't conflict regardless of order.

Ready to close out the ritual — the memory refresh (project-mandala-state is stale: predates today's decisions, the three new branches/PRs, and the resolved bug) and a session log. Want me to do both?

User — 2026-07-16 15:53

Document everything, refressh memory, and log the session.

Claude — 2026-07-16 15:53

Closing out the ritual. Docs are already updated (in the three PRs); now the memory refresh and session log. Let me first re-read the current memory files to make precise edits.

Claude — 2026-07-16 15:54

Now update the stale 1b.2 known-bug line and the 1b.1 status to reflect current reality:

Claude — 2026-07-16 15:54

Now the topic memory for the dev-DB workflow decisions (referenced by the links above):

Claude — 2026-07-16 15:54

Memory refreshed. Now the session log — checking the script's usage first: