Skip to content

Session Log: AV resequencing (ADR 018), sprint split, and Spike 7 progress

Date: 2026-09-04 Participants: Yuji Shinozaki, Than Grove, Xiaoming Wang, Claude Code Outcome: ADR 018 accepted (AV starts as a parallel track); AV split into Sprint 3 (core) and Sprint 4 (transcripts); Spike 11 scoped to the D7 pipeline; Spike 7 moved ○ Pending → ◐ Partial with a working D11 Kaltura module, a landed composer packaging fix, and the real D7 ingest workflow established. User uploads and multi-player support decided in scope.

⚠ This log is HAND-WRITTEN and abridged, not generated by scripts/save-session-log.py. The session read live production Kaltura credentials out of a database dump into tool output. Per the standing convention (docs/non-public-documentation.md), no raw transcript is published for a session that touched credential material, and no secret values appear anywhere in this repo. See "Credential exposure" below.

1. Context refresh and a carried-over question

Pulled main (two pulls: PR #183's B5 collection-viewing work, then Than's PR #184 resolving B2's action-icon/technical-modal follow-ups).

Answered the open question from the 2026-09-03 deferred note on dev-0's root-owned files/styles/: docker inspect shows sites/default/files is a bind mount from /mnt/data/mandala-drupal-0/, not a volume or the container's writable layer. So the live chown fix wrote through to the host and survives redeploys. Root cause (why it was root-owned originally) and staging/production parity remain open — each host's bind-mount source is independent.

2. ADR 018 — AV starts now, in parallel

Driver: Than closes out Sprint 2 today and is away for two weeks. He is the team's most up-to-speed person on both Phase 2 tracks (Texts, Sources), so those stall in his absence regardless. Yuji is better suited to AV.

Decision: supersede ADR 009's Phase 3 ordering only — not its risk analysis. The 2026-09-01 complexity comparison scoring AV hardest (3.8/5 vs Sources 3.2, Texts 2.0) stands unrevised; this is a resourcing decision, not a claim that AV got easier. AV begins with the two Phase 0 items nobody ever closed: Spike 7 (Kaltura) and the AV transcript format triage.

ADR 009's Status line was updated to point at 018 (metadata-only edit, matching the ADR 004 → 013 precedent); its content is untouched. Ratified by all three (Yuji, Than, Xiaoming), who were in the session. (This log and ADR 018's header both initially and wrongly recorded it as "accepted by Yuji alone, to be confirmed" — corrected same day.)

3. AV split into two sprints

The AV content-model audit established that field_transcript is a plain file field on the audio/video node, not a separate entity. So AV core can migrate without the transcript pipeline being solved — carrying field_transcript across inertly (download-only, no processing) in the meantime.

  • Sprint 3 — AV core: content type, Kaltura playback, PBCore/workflow → Paragraphs, the two extra OG access realms, collections/UI. Gated on Spike 7.
  • Sprint 4 — AV transcripts: Spike 11's scope. Gated on Spike 11 and on Sprint 3's nodes existing.

The dependency runs one way only, and the two gating spikes don't block each other, so both threads can be staffed independently.

4. Spike 11 scoped — two transcript systems exist, only one is in scope

Reading the real legacy code found two independent, non-identical transcript systems in production:

  1. The D7 Drupal pipeline (transcripts_ui + transcripts_apachesolr + transcripts_xslt) — source files in Toolbox .txt, SRT, or XML, transformed at runtime by shelling out to a Java Saxon XSLT processor into "TCUs" (Time-Coded Units). The tier mapping (tx/mb/ge/ft = transcription / morpheme breakdown / interlinear gloss / free translation) is multi-tier linguistic annotation, not captions. Has its own editor UI (transcripts_editor).
  2. The React app's own client-side viewer (mandala-om src/legacy/audiovideo.js, ~1,545 lines) reading a dedicated mandala-av Solr core, with its own sync/search/download — entirely decoupled from the D7 module.

Yuji: system 2 is out of scope for now. Spike 11 rewritten accordingly, with the system-1 findings recorded as "live evidence" so the spike doesn't re-derive them, and its central question reframed: reproduce the XSLT authoring pipeline as a live workflow, or convert existing transcripts once at migration time?

5. Spike 7 — Kaltura, ○ Pending → ◐ Partial

Module landscape. The D7-era kaltura module (7.x-3.2) has no D9/10/11 release and is Unsupported — a dead end. drupal/kaltura_media (1.0.4, Evolving Web) is a live candidate.

Verified on real Drupal 11, not inferred: patched core_version_requirement to add ^11, installed and enabled cleanly on 11.4.5; upgrade_status:analyze surfaced three non-blocking issues (a deprecated FileSystemInterface::EXISTS_REPLACE, a genuine pre-existing bug — an unimported FileException in a dormant catch branch — and a logger-serialization warning). Rendered the module's embed output using Spike 6's real production Kaltura IDs (partner 381832, uiconf 24762821, a real entry id) — structurally identical to the live D7 embed.

Packaging solved and landed (f383ff6, direct to main, inert — nothing references the module yet). composer-patches alone cannot fix this: the blocking drupal/core constraint lives in Composer metadata, evaluated before any file patch runs. The working fix is two-part — a local package-type repository override (listed before packages.drupal.org so it wins) redeclaring 1.0.4 with a corrected require, plus cweagans/composer-patches applying two real patches from a new drupal/patches/. Gotcha worth remembering: sourcing from git.drupalcode.org failed, because a git tag lacks the drupal.org packaging-script footer the tarball has, so a tarball-built patch won't apply — use the dist tarball.

6. Spike 7 — ingest, an error, and the correction

First pass concluded "there is no Drupal-side file-upload-to-Kaltura path", based on the real import/pull workflow (mb_kaltura_import_form → get_importable_entries(), a genuine Kaltura media->listAction() call filtered against already-linked entry ids → batch create_node_mediabase()).

Yuji challenged that, and it was wrong. There are two ingest paths:

  • Path A — bulk admin import (pull). Attach entries that already exist in Kaltura.
  • Path B — per-node upload (push). The node form's field widget opens a chunked upload modal (kaltura/nojs/chunked-upload/…), uploading browser-direct to Kaltura. Mandala actively supports it: a dedicated mb_kaltura/upload-keepalive route ("to be called by js during long uploads"), keepalive.js on both node forms pinging every 60s, and add-media-button visibility management.

Why it was missed (recorded because the same trap applies to the Texts/Sources archaeology): the search looked for server-side PHP upload calls in the custom module and treated a negative grep as proof of absence. The upload is client-side and lives in contrib — no server-side call could ever have appeared there. The signals were in things already listed and never opened (mb_upload_keepalive(), mb_kaltura_form_alter(), the all_media widget name).

Settled against real production data. Yuji supplied the known-good 2026-09-01 AV dump (gzip -t passes; gitignored). Streaming it, without a DB load: kaltura_chunked_uploader is enabled (status=1), and add_existing = 0 on both field_video and field_audio — which takes the uploader branch. So in production the node-form media button is exclusively an upload control, with no browse-existing option there at all.

Two incidental corrections to the AV content-model audit, both from the same dump: - Instances use field_kaltura_video / field_kaltura_audio, not the all_media widget the audit records (that's the field type's declared default, not the configured value). - There is no single player uiconf_id — at least 31832371, 48501 and 24762821 are in play across display settings, the React app, and the live embed.

Also found: PBCore auto-population from Kaltura is dead code — the call site is commented out and all five autopopulate/apply functions have zero invocations. Imported nodes get only title / collection / author / entry-id / tags; the rich metadata is hand-entered by catalogers. Nothing to replicate in D11, because it never worked in D7.

And root-caused the audit's open question #8: the 68 MISSING_TYPE nodes are create_node_mediabase()'s own fallback bundle string for Kaltura entries whose mediaType doesn't map to VIDEO/AUDIO, saved with no validation — a real import path, not corruption.

7. Scope decisions (Yuji)

  • User uploads are in scope. D11 builds an upload integration rather than accepting a KMC-upload-then-paste-entry-id workflow change.
  • All three known players, and more, must be supported — plus the wider set of Kaltura configuration elements.

Added AV10–AV13 and a design note. Uploads go browser-direct (uploadToken.add → chunked uploadToken.upload → media.add), matching D7 and keeping large masters out of PHP and the ALB; this needs a server-side Kaltura Session minting service (official kaltura/api-client-library) issuing short-TTL, upload-scoped sessions so the admin secret never reaches the browser.

The configuration surface, inventoried from the production dump per view mode, is wider than a player id: entry_widget, custom_cw (a separate uploader ui_conf, 4396241), delivery, player and thumbnail dimensions, rotate/stretch, custom_player — plus site-level variables and two ids mb_kaltura hardcodes (METADATA_PROFILE_ID, MB_MAIN_PLAYER_ID) that belong in config. ⚠ delivery is RTMP on the default display — Flash-era and dead; not to be ported blindly.

Secret delivery follows the established house pattern (confirmed by reading terraform-infrastructure): three layered env files merged by Ansible, with container_0.env.secret gitignored and committed only as a ccrypt .cpt, encrypted/decrypted via scripts/crypt-key.ksh / decrypt-key.ksh — the same mechanism already securing the SAML .pem.cpt keys. Kaltura's non-secret ids go in container_0.env.managed; the secrets go in the ccrypt file and should join required_env_vars so a missing secret fails the deploy loudly.

Credential exposure (no values in this repo)

D7's variable table stores kaltura_admin_secret and kaltura_secret in cleartext, so they are present in every production dump held locally. Values are deliberately not recorded here — ask Yuji; this belongs in the private docs repo per the standing convention. Same shape as the previously-flagged solrprod credential. Verified before every commit that no secret value reached a tracked file.

Dump hygiene: a third corrupt AV dump was found (~/mandala-prod-av-db_2023-12-05.sql.gz, fails gzip -t), after the two the audit recorded on 2026-06-11. The 2026-09-01 re-export is the only known-good copy. Always gzip -t before drawing a negative conclusion from an AV dump.

Open items

  • ~~ADR 018 needs Than's and Xiaoming's confirmation~~ — RESOLVED: ratified by all three in session. The earlier "Yuji alone" note was a recording error, corrected the same day in both this log and ADR 018's header.
  • ~~Texts/Sources coverage during Than's absence is undecided~~ — DECIDED 2026-09-04: both tracks are PAUSED rather than partially covered; capacity goes to the AV track. Explicitly temporary, revisit when Than returns. Recorded in roadmap.md Phase 2.
  • Spike 7 remaining: the Kaltura API auth/credential-validity check (does the partner admin secret still work against the live account) — effectively blocked on decrypting the secret anyway. A migration source plugin also remains.
  • Behavioural unknown: how often AV staff upload via the node form versus using Kaltura's KMC. A question for the content staff, not the codebase. (Note kaltura_last_imported = 2023-02-08 is written by the base contrib module's sync, which mb_kaltura bypassed — it is not evidence about the custom import page.)
  • Spike 11 cross-reference: mb_kaltura calls captionAsset->add(), i.e. Kaltura-native caption storage, separate from the XSLT pipeline. Not chased.
  • dev-0 files/styles/ root-ownership root cause, and whether staging/production share it.