Voxint architecture¶

How a run moves through the pipeline, the state machine and data model that back it, and the invariants each stage upholds. Pairs with gpu-contracts.md (the service wire contracts) and quality-gates.md (the thresholds the stages apply).

Pipeline shape¶

input ──▶ acquire ──▶ prepare ──▶ transcribe ──▶ diarize_embed ──▶ enhance_match ──▶ finalize
          (yt-dlp     (ffmpeg      (ASR)          (diarization +     (LLM enhance +
           URL         16 kHz,                     speaker            speaker matching)
           download;   gates)                      embeddings)
           no-op when
           local)

Six coarse stages run as self-contained (idempotent) stage functions, driven by a small engine (voxint.pipeline.engine). The engine owns everything the stages should not care about: state transitions, per-stage transactions, attempt bookkeeping, and crash recovery. Stage bodies own the science. Celery tasks are thin wrappers around the engine, with no orchestration logic hiding in task code.

The stages are partitioned into two execution lanes (see Execution lanes and queues): a GPU lane (acquire through diarize_embed) and a post lane (enhance_match, finalize), so a deployment that must serialize GPU work can still overlap one run's LLM enhancement with the next run's transcription.

ACQUIRE (STAGE_ORDER[0]) is the universal first stage: a successful no-op for local or uploaded media (source_url IS NULL), and a yt-dlp download for URL runs (voxint fetch / POST /fetch). Making it the first stage, rather than a special submit-time step, keeps the "every fresh run starts at STAGE_ORDER[0]" invariant intact, so legacy queued/current_stage=NULL rows route safely into the no-op. Download mechanics and the SSRF model are below.

State machine¶

Run state lives in Postgres (pipeline_runs.status + current_stage), guarded by compare-and-swap on an explicit revision column: every transition is an UPDATE … WHERE id = :id AND revision = :held that also increments revision. A worker holding a stale snapshot gets StaleRevisionError and must re-read. Lost updates are structurally impossible.

queued ──▶ running ──▶ completed ──▶ queued (restart: full or from a selected stage)
  ▲ ▲        │ ▲  │
  │ │        │ │  └──▶ awaiting_adjudication ──▶ running
  │ │        │ └──── running (stage advance)
  │ └────────┤          (lane handoff: running ──▶ queued, parked at the NEXT stage)
  │          ├──▶ paused ──▶ queued (resume) or cancelled
  │          ▼
  └──── failed ──▶ queued (requeue at same stage, or restart)

  cancelled ──▶ queued (restart: full or from a selected stage)
  queued ──▶ paused (pause before dispatch)

completed, failed, and cancelled are exit states for the normal flow, but none is fully terminal: all three can transition to queued via restart. failed can also requeue at its current stage (the existing retry path).

Restart comes in two forms. A full restart (from_stage=None) resets current_stage to None so the run re-processes from ACQUIRE. A stage-selective restart (from_stage=<stage>) preserves upstream outputs and eagerly deletes downstream outputs in the same transaction, then queues the run at the selected stage. Both forms bump processing_cycle, resetting the retry budget for the new cycle. Prior StageRun rows from earlier cycles are preserved as history.

Stage-selective restart validates prerequisites (upstream stage completed and its data exists) before deleting anything. A stage-aware blocker matrix determines which adjudication states block each restart point: ENHANCE_MATCH and FINALIZE are safe for all runs (no blockers), while earlier stages retain full adjudication guards. See docs/operations.md for the blocker table.

Pausing is cooperative: a queued or running run can be driven to PAUSED. A running run finishes its current stage first (the worker observes the transition via StaleRevisionError at the next stage boundary), then no further stages start. From PAUSED, a run resumes (PAUSED to QUEUED, re-enqueued for worker pickup) or is cancelled.

A requeued run retries the stage it was interrupted in: earlier stages are not re-run and nothing is skipped. A lane handoff is the one deliberate exception to "queued keeps the stage": the stage that just committed is complete, so the run parks queued at its successor, waiting for the other lane's worker to pick it up.

Validation covers the full (status, stage) tuple, not status membership alone: a run cannot start at the wrong stage, advance backwards or by more than one stage, complete mid-pipeline, or requeue at an unrelated stage. The handoff transition is validated to park at exactly next_stage(current).

Stage claims and recovery¶

CAS alone decides whose database write survives; it cannot stop two workers from both invoking a GPU call. So before executing a stage body, a worker claims the stage by committing a running row in stage_runs carrying its worker id and a lease; the (pipeline_run_id, stage, attempt) unique constraint arbitrates ties. A worker that finds an unexpired claim yields without executing.

Workers that die mid-stage leave the run running with a claim whose lease eventually expires. recover_interrupted_runs sweeps only expired claims (a healthy worker three hours into a transcription is never robbed), marks the interrupted attempt failed, and requeues the run through the same transition rules. Stage bodies remain at-least-once for non-transactional effects (filesystem, GPU services) and must be idempotent.

Execution lanes and queues¶

The six stages are partitioned into two lanes (GPU_SEGMENT / POST_SEGMENT in voxint.db.models, a contiguous split of STAGE_ORDER):

Lane	Stages	Queue	Task	Bound by
GPU	`acquire`, `prepare`, `transcribe`, `diarize_embed`	`celery` (default)	`voxint.run_pipeline`	the ASR / diarization / embedding services
Post	`enhance_match`, `finalize`	`post`	`voxint.finish_pipeline`	the LLM endpoint and the database

Both tasks drive the same engine; each passes its lane as execute_run(stages=...). When the GPU lane completes diarize_embed, the engine commits the finished claim and the running ──▶ queued(enhance_match) handoff in one transaction, then the task publishes voxint.finish_pipeline. The durable queued row is the handoff: if the worker dies (or the broker is down) after the commit but before the publish, the recovery sweep finds a stale queued run and re-publishes it, routing by current_stage through the shared pipeline_task_for_stage helper that every publisher (API, CLI, sweep, handoff) uses. A task delivered to the wrong lane is a pure no-op: it takes no entry CAS and creates no claims.

Commit-before-publish¶

Every submit function in voxint.ingest.service (submit_upload, submit_url, submit_media_item, submit_media_item_if_new) returns a SubmissionResult carrying run_id and a publish() method. The caller commits the session (creating the durable QUEUED row), then calls result.publish() to send the run to the broker. publish() returns False on a broker outage and never raises, so a down Redis degrades to "run stays QUEUED for the recovery sweep" rather than a failed request. The ingest module never imports Celery at module level; the lazy import lives inside SubmissionResult.publish().

The LLM-bound post-run jobs (voxint.generate_run_asset, voxint.research_speaker) are also routed to the post queue, so they never serialize behind GPU work. So are the beat sweeps (recovery, GC, notify, watch): the recovery sweep is the fallback that republishes a handed-off run whose finish publication was lost, so on a split deployment it must never sit queued behind a multi-hour GPU segment.

Both queues are declared in the Celery app (task_queues), and a worker started without -Q consumes both. The default single-worker deployment therefore behaves exactly as before; splitting the lanes is an opt-in deployment choice. A deployment whose model services share one GPU runs two workers: the GPU lane at --concurrency=1 (only one run touches the GPU at a time) and the post lane at a small concurrency of its own, so the GPU no longer idles while a previous run's LLM enhancement is in flight. See operations.md for the override recipe.

Data model (alembic revisions 0001–0064)¶

Table	Role
`app_settings`	single-row instance configuration set by the first-run setup wizard: onboarding-complete flag, custom vocabulary, LLM-enhancement toggle/endpoint, guided-tutorial state (revision 0006), and (revision 0021) one nullable column per in-UI-editable feature flag (`enrichment_names_enabled`, `enrichment_names_llm_enabled`, `enrichment_run_assets_enabled`, `enrichment_run_assets_autogenerate`, `voxint_web_research`, `enrichment_web_research_enabled`, `ytdlp_enabled`, `source_authority_domains`, `web_search_base_url`, `web_search_api_key`). These resolve row-over-env through `app_settings.resolve_effective_<flag>` (NULL/blank inherits the environment default, a stored value overrides it, following the `llm_` tri-state precedent); `web_search_api_key` is a credential handled like `llm_api_key` (plaintext at rest, resolver-only, never rendered/logged). The cross-flag invariants live in one `validate_effective_flags` shared with the boot-time config validator. Revision 0029* adds a `corrections` JSONB list: the operator's console-authored deterministic correction rules (#84, edited at Settings → Corrections). Unlike the row-over-env flags above, corrections are not resolved live: at submit they are unioned onto the run's resolved pack and frozen into `pipeline_runs.domain_pack`, so #82 composition and #83 provenance read them off the immutable snapshot unchanged (vocabulary is live-unioned; corrections must be per-run-frozen). Registered media folders and per-folder domain-pack selection live in the `media_folders` relation (since #153, revision 0040; the legacy `app_settings` columns were dropped in revision 0046)
`projects`	named groupings of media folders with project-scoped config (revision 0040, #153). Vocabulary and corrections overrides follow ADR 0002 per-field replacement. `archived_at` (revision 0064, #477) makes a project inert: its folders stay linked, but new runs in those folders resolve config as if the project were absent, learned-corrections capture stops, and new quote saves are refused. Three choke points enforce inertness: `ingest/service._folder_and_project`, `adjudication/learned_corrections._resolve_project`, and `api/saved_quotes.save_quote`
`media_items`	media identity, one row per source file. `source_path` (UNIQUE) is already present for local/uploaded media, pre-assigned and materialized by ACQUIRE for URL runs; a nullable, non-unique `source_url` records URL provenance (revision 0005)
`media_source_metadata`	write-once acquisition context, 0-or-1 per media item (revision 0009): normalized extractor fields (title, uploader/channel, description, upload date, source-claimed duration, tags, canonical URL, extractor name/version) plus a bounded, allowlisted, schema-versioned `raw` JSONB subset and `acquired_at`. Context, not identity: nothing here feeds attribution, and a MediaItem is per-acquisition, so a snapshot can never rewrite the context a past adjudication was made against
`pipeline_runs`	execution state + CAS revision, plus the reviewer claim (token, holder, expiry), the operator's free-text `operator_notes` (revision 0009: human input, kept structurally apart from scraped metadata, edited last-write-wins outside the CAS), and the write-once `domain_pack` JSONB snapshot resolved at submit (revision 0017: the exact pack the run was transcribed with, read by the worker and enrichment; `NULL` on pre-0017 runs)
`stage_runs`	per-stage attempt ledger and execution claim (worker id, lease, status, timing, error, metrics). The `metrics` JSONB's first documented consumer is `model_identity` (configurable pipeline models): for a stage that calls a model service, the model, revision, engine, and `decode_config_hash` read from its `/healthz` immediately before the attempt, written in the same transaction that completes the claim so a failed attempt cannot overwrite a later successful one. See Changing pipeline models
`audio_artifacts`	derived files (preprocessed audio, chunks, exports); `reclaimed_at`/`reclaimed_bytes` record GC reclamation of the preprocessed-audio intermediate (issue #15); the row survives as an audit stamp after its file is unlinked
`audio_chunks`	chunk boundaries for long-file processing
`transcript_segments`	raw ASR text (immutable) + `enhanced_text` beside it + `suspect` soft-tag; two GIN expression indexes (`english` tsvectors over each text variant separately, revision 0008) back the `/runs` transcript search. Both variants stay searchable; enhancement never shadows raw. Revision 0028 adds the deterministic-correction trail written by the `enhance_match` dual pass (#82): `correction_trace` JSONB (either `[]` or the `{version, input_base, entries}` envelope, `input_base` recording a `raw` vs `llm` base) and `corrector_version`, written only when the final text materially differs from raw, reset atomically on every re-enhance, and read back at review time for the #83 provenance surface (`NULL` `corrector_version` = legacy/unversioned, never recomputed)
`diarization_turns`	run-scoped observation ledger, one row per turn: interval, label, overlap, and the window's embedding outcome (vector + space, or an auditable `skip_reason`)
`speakers`	the grown speaker roster, with a curation lifecycle: `merged_into_id`/`merged_at` (merge tombstones) and `deleted_at` (reversible archive) (revision 0007)
`speaker_embeddings`	`vector(192)` + `embedding_space` tag; enrollment rows carry provenance (source run, label, and a unique link to the human decision that created them)
`speaker_assignments`	machine proposals (method, confidence, grounded flag; `llm_hint` rows carry `proposed_name`, method-shape CHECKs keep the two shapes disjoint)
`match_candidates`	observational match-decision evidence, one row per diarization label (revision 0032, issue #113): the matcher's decision (accepted \| rejected \| ineligible), top candidate, and the cosine / margin / vote-agreement / eligibility behind it, kept even for the near-misses that yield no proposal. Diagnostic only, it feeds no attribution; it is the raw material the offline harness scores (epic #112). Rewritten wholesale each `enhance_match` run beside the proposals
`auto_enroll_evidence`	auto-enrollment diagnostic evidence (revision 0058, issue #434): per-label similarity, margin, vote agreement, top candidate, and roster size at decision time, persisted for every auto-enrollment evaluation including near-misses. Retained for threshold tuning and debugging without re-running the pipeline
`adjudication_decisions`	immutable human ledger (insert-only, idempotency key). Revision 0051 adds a nullable `user_id` FK to `users` with asymmetric CHECK constraints: `NOT NULL` when `multi_user_at_write` is true, `NULL` otherwise, so the column records attribution without breaking single-operator rows
`users`	operator accounts (revision 0051, role constraint widened in revision 0057): `username` (UNIQUE, lowercase-only), Argon2id `password_hash`, `role` (`admin`, `reviewer`, or `viewer`), `disabled_at` (soft-disable), `created_at`. The first user created is forced to admin
`auth_sessions`	session tokens (revision 0051): sha256 `token_hash`, `user_id` FK, `created_at`, `expires_at`. Revoked on disable, role change, or password change
`enrichment_producer_runs`	one row per completed enrichment-producer invocation (revision 0010): producer key + version, XOR target scope (speaker \| run \| run+label), declared `covered_fields`, monotonic per-scope `generation` (allocated under an advisory lock; supersession compares generations, never wall clock), derived `outcome` (`'none'` = "we looked and found nothing", reviewable information), bounded schema-versioned `config` snapshot, idempotency key
`enrichment_candidates`	immutable machine-derived claims (revision 0010): suggestions about identity, never identity. Claim field/value, producer-local score + components. No stored review state; effective state is derived (decision > supersession stamp > proposed). A trigger blocks DELETE and every UPDATE except the write-once `superseded_by_producer_run_id` stamp
`enrichment_candidate_evidence`	1:many field-level provenance per claim (revision 0010): a `media_source_metadata` column/`raw.` key, a transcript segment (+ timestamp), or a fetched URL, so one claim can cite several sources together; append-only (trigger)
`profile_review_decisions`	append-only human trail for enrichment claims (revision 0010), deliberately separate from `adjudication_decisions`: accepting a bio is a different act from ruling on who spoke. UNIQUE per candidate (terminal accept/reject); corrections arrive as fresh candidates from a producer re-run
`run_enrichment_assets`	immutable run-level assets (revision 0012, issue #41): one successful summary/topics/entity-mentions generation per row, keyed (run, kind, generation) with a monotonic per-kind generation under an advisory lock. Whole documents, not per-field claims: no review lifecycle, and regenerate supersedes (write-once stamp, trigger-enforced like `enrichment_candidates`). Carries producer + version, model, schema-versioned payload and config snapshot, and a `source_content_hash` over the canonical generation inputs (transcript text + #36 metadata + operator notes + each segment's attributed speaker, resolved through the shared `display_name`), so re-adjudicating, renaming, or merging a speaker marks the run's assets stale, staleness is recomputable, and a prompt/model upgrade never masquerades as a source change
`run_asset_jobs`	mutable orchestration for one asset-generation attempt (revision 0012): queued → running (guarded claim) → succeeded \| failed \| cancelled, one active job per (run, kind) via a partial unique index, deadline-aware cancel. Failed/cancelled jobs record NO asset and consume no generation; the three kinds fail independently by construction
`segment_embeddings`	immutable transcript-search chunks (revision 0033, issue #121): one row per embedded paragraph, keyed (run, `embedding_space`, `generation`, `chunk_index`). Carries the chunk's span, dominant `speaker_label`, `text_rendering`, the exact embedded `chunk_text`, a `content_hash` over that string, and the 384-dim `embedding` vector. Chunks are paragraph-derived and ephemeral (boundaries shift when a correction, split, or speaker ruling changes), so the span and text live ON the row, not behind a segment FK. `generation` is monotonic per (run, space): a re-embed publishes a whole new generation atomically and the old one is replaced, never half-seen. No ANN index in v1: exact cosine scan is sub-second at single-operator scale (add HNSW only on measured latency evidence)
`embedding_jobs`	mutable orchestration for one index build per (run, `embedding_space`) (revision 0033): a dedicated lane, deliberately not the LLM-coupled `run_asset_jobs` family (embedding needs no LLM client and no `llm_enabled` gate). It reuses the proven lifecycle patterns only: queued → running (guarded claim UPDATE, duplicate delivery no-ops) → succeeded \| failed \| cancelled, one active job per (run, space) via a partial unique index. `source_content_hash` is the staleness detector (a run is stale when its recomputed resolved-transcript hash differs from the current generation's). A succeeded job is the generation manifest (it records the `generation` it published; the vectors live in `segment_embeddings`); a failed/cancelled job publishes no generation

Three invariants worth naming:

Raw is forever. Enhancement writes enhanced_text; it never touches raw_text.
Named ≠ grounded. An LLM-proposed name is not grounded until it has embedding-level evidence or a human ruling; a CHECK constraint enforces that only a cosine proposal with a concrete speaker can claim grounded, and machine proposals are never merged into the human ledger. The ledger itself is append-only at the database level (a trigger rejects UPDATE/DELETE) and writes go through one idempotent-replay operation. The savepoint-adopt-or-conflict skeleton shared by the ledger, annotations, drafts, and run-asset writers is extracted into voxint.idempotency.savepoint_adopt_or_conflict().
One embedding space at a time. Cosine similarity is only meaningful within a single embedding_space; all vector SQL lives in one module and always filters by space.
Drafts are suggestions about identity, not identity. The enrichment_* tables hold machine-derived claims as reviewable drafts; nothing there (accepted or not) feeds attribution, mutates speakers.display_name/notes, or writes the adjudication ledger. Writes go through the single sanctioned writers in voxint.enrichment (drafts.py for producers, review.py for the human trail); each new producer run atomically supersedes only its own older still-proposed claims within the fields it covered.

Enrichment producers (issue #38: `names.offline`)¶

Producers live under voxint/enrichment/producers/ in three layers: pure pattern extraction (name_patterns.py: a bounded, versioned regex inventory over metadata and transcript text, capitalization-independent with explicit false-positive guards), pure per-target aggregation/scoring (name_scoring.py: max-reliability base plus small corroboration/diversity/ seed bonuses capped at 0.95; no frequency reward; run_label claims are built from self-introductions only, so a title mention can never create or inflate a cluster-identity claim), and DB orchestration (names.py).

names.offline always invokes at run scope. A run-scope invocation may emit both run-level and run_label-level candidates, and supersession keys on the invocation scope, so every rerun cleanly retires the prior generation. Its idempotency key is an input signature (producer/pattern/scoring versions + domain-pack seeds + exact metadata/segment content): identical inputs short-circuit to the stored row before fresh timestamps are minted, changed inputs mint a new key and a superseding generation, and outcome='none' is recorded only after a successful scan (read failures raise). Invocation is operator-triggered only, via voxint enrich names or the workbench's claim-gated button, never a pipeline stage.

The roster page (/speakers) can rename, merge, archive/restore speakers and remove bad enrollment embeddings, all through speakers.roster, and none of it ever writes the decision ledger:

A speaker is active while merged_into_id and deleted_at are both NULL. The one active predicate (roster.active_speaker_clause) governs matching centroids, the workbench assign dropdown, and the decide route. Merged and archived speakers stop attracting proposals and decisions.
Merge B→A repoints B's speaker_embeddings, speaker_assignments, and speaker_profiles rows to A (A's own profile values win a per-field conflict) and keeps B as a tombstone (merged_into_id = A), so historical ledger FKs stay valid. Readers canonicalize through the merge map at read time: an old assign(B) decision renders as A while the ledger row keeps B forever. Writes collapse chains to depth 1; readers still follow chains defensively and fail loudly on a cycle.
Archive is reversible (deleted_at) and deletes the speaker's cosine assignments: stale machine grounding must not outlive the operator's verdict. Embeddings and human decisions are preserved; restore does not resurrect the purged proposals (matching re-proposes on future runs).

A speaker's current profile (bio, affiliation, link) lives in speaker_profiles (issue #159): one row per field, carrying provenance (manual or enrichment with the accepted candidate's id). The single write funnel is enrichment.review.record_profile_decision, which materializes an accepted claim under the canonical speaker's row lock; manual edits and merge repointing (speakers/profile.py, speakers/roster.py) take the same lock, so an accept, an edit, and a merge can never interleave on one speaker. A replayed accept fills an absent field or refreshes its own value only; it never reverses a later manual edit. Draft-claim history stays in the immutable enrichment tables. The speakers overview (/speakers) and detail (/speakers/{id}) pages read this table plus per-speaker aggregates folded from effective resolution (speakers/aggregate.py: one canonical newest completed run per recording, human rulings over automatic matches), with voice-match tiers graded against the live matching gates (speakers/tiers.py). The review console partitions labels through the three-band policy (speakers/policy.py, MatchBand); LabelState carries candidateSpeakerId / candidateSpeakerName plus matchSimilarity, matchMargin, and matchVoteAgreement, and auto-enrolled labels are banded from live evidence like unresolved labels. The rail derives its groups and copy in the pure frontend/src/lib/speaker-bands.ts module. - Removing an embedding hard-deletes the derived centroid (the minting decision and the raw diarization_turns vectors survive) and deletes all of that speaker's cosine assignments, because assignments carry no centroid lineage, so narrower invalidation would be a guess. - Names stay globally unique across every lifecycle state. Re-creating an archived name is refused with restore guidance; enrollment replay validates against durable provenance (run, label, operator), never the mutable display_name, so a rename can never break a replayed enrollment POST.

Transcript semantic search (the embedding spine, issue #121)¶

Exact full-text search over transcript_segments finds a passage only when the operator already knows a word in it. The embedding spine adds meaning-based retrieval: finished transcripts are embedded and stored as vectors, so a later query can rank passages by similarity. The data spine builds and maintains the index; the ranked "Meaning" query (below) reads it. The producer reads finished transcript text only. It never touches ASR, diarization, or TitaNet, so it does not trip the numerics parity gate.

The embedder. voxint.embeddings.onnx_embedder runs paraphrase-multilingual-MiniLM-L12-v2 as a vendored ONNX graph on the onnxruntime CPU provider, mean-pooling the token states and L2-normalizing in numpy. The BERT backbone is the whole ONNX graph; pooling and normalization are reproduced in Python, not baked in. It ships without torch, transformers, or sentence-transformers (a contract test, tests/contracts/test_text_embedding_deps.py, keeps that closure clean), so the "no torch/TF/CUDA in the app image" boundary holds. The output space is minilm-multi-l12-onnx-fp32-mean-v1, 384-dim; it is never compared against the 192-dim TitaNet speaker space. The tokenizer bakes truncation.max_length=128, so chunking counts tokens with a separate non-truncating count_tokens and keeps each chunk within MAX_SEQUENCE_TOKENS. Correctness is a measured equivalence gate (tests/parity/test_text_embedding.py, per-vector cosine >= 0.9999 against references generated once in a throwaway sentence-transformers env), not reasoning.

The producer. enrichment/producers/segment_embeddings.py resolves the run's finished transcript (attributed_transcript → paragraphize_transcript), splits it into token-bounded paragraph chunks, embeds each, and writes segment_embeddings rows. Chunks carry their own span and text because paragraph boundaries move when a correction, split, or speaker ruling changes.

The job lane. enrichment/embedding_jobs.py owns the lifecycle (create_jobs → claim → execute_job → atomic publish → cancel), mirroring the run_asset_jobs patterns without the LLM coupling. A build reads the transcript under a REPEATABLE READ snapshot before the CPU-bound embed, then publishes the whole run's new generation and deletes the prior one in one transaction under an advisory lock, so a concurrent reader sees the old generation whole or the new one whole, never a mix. Staleness is a whole-transcript content hash: a run is stale when its recomputed resolved-transcript hash differs from the current generation's, which is what runs_needing_embeddings scans for. A force-cancel fences an in-flight build so its publish cannot resurrect a superseded generation.

Producing an index. When SEMANTIC_INDEX_AUTOGENERATE is on, the worker's finalize hook (worker/tasks.py) enqueues a build as a run completes, so search covers new recordings with no manual step. It is best-effort: the completed run is never affected by an enqueue failure, and a run left unindexed is picked up by a later voxint embed backfill. The finalize hook and the backfill CLI both check minilm_artifacts_available() first and skip honestly when the weights are absent (a native install that never fetched the asset), rather than enqueuing a doomed job per run. voxint embed backfill drives the same lane synchronously for a whole-corpus catch-up with no broker. See semantic-search.md for the operator commands and the weights requirement, and gpu-contracts.md for the model services this spine sits alongside.

The query path. api/meaning_query.py serves the ranked /search "Meaning" mode, a distinct surface from the chronological /runs keyset browse (a GET /runs render is byte-stable except for the added Exact/Meaning tab strip). The query is embedded first, with the same in-process singleton, never a Celery round-trip. Three arms then run over segment_embeddings inside one short read-only REPEATABLE READ transaction: a vector arm (pgvector cosine_distance nearest neighbours, bounded by a candidate limit, the first pgvector SQL in the repo), a lexical arm (a simple-config full-text match with a mandatory @@ predicate so the tail is not padded with zero-rank non-matches), and an exact-quote arm (its own strpos query, not ILIKE, so a % or _ in the phrase is literal; all hits, since a literal can rank outside both bounded arms). The one snapshot is the correctness guard: two independent READ COMMITTED statements could straddle a concurrent publish and mix an old generation's rows from one arm with a new generation's from another. Because a publish deletes every prior generation in the transaction it commits the new one, the table holds exactly one generation per (run, space) at any committed read point, so search reads segment_embeddings directly and never resolves the current generation through embedding_jobs. The vector and lexical arms are fused with reciprocal rank fusion (k=60); exact-quote hits float above the fused order; a per-run cap is applied while walking the final order before the top-k truncation. Each result carries a jump_url (/runs/{id}/transcript?t=SECONDS); the read-only transcript island reads ?t= on load and scrolls the matching line into view with a brief highlight, with no audio seek. There is no ANN index in v1: the bounded exact scans are sub-second at single-operator scale, and an HNSW index is added only on measured latency evidence. Two tri-state runtime toggles (semantic_index_enabled, semantic_index_autogenerate) live under Settings > Semantic search; they depend on nothing else and validate through their own self-contained invariant (semantic_index_flags_ok: autogenerate rides on the feature), deliberately outside the EffectiveFlags web.

URL ingestion & SSRF (the ACQUIRE stage)¶

voxint fetch <url> / POST /fetch register a URL as a MediaItem.source_url and queue a run; the pipeline's first stage, ACQUIRE, downloads it with yt-dlp on the worker (a no-op when source_url IS NULL, meaning local/uploaded media). URL ingestion is an authenticated admin egress capability (ytdlp_enabled, on by default), not a sandbox, and is documented as such.

Two SSRF gates, one policy. A submitted URL is checked at two independent points that share a single per-address rule (media.netcheck.ip_is_public). That rule is stricter than the stdlib is_global: it rejects IPv6 site-local (fec0::/10) and unwraps IPv4-in-IPv6 embeddings (deprecated ::a.b.c.d, RFC 6052 NAT64 64:ff9b::/96, IPv4-mapped/6to4/Teredo) to judge the embedded IPv4, so [64:ff9b::127.0.0.1] is refused, which is_global alone would pass. The two gates:

String gate (submit time). ingest.validate_ingest_url requires an absolute http/https URL with a plain host, no embedded credentials, no whitespace/control chars, under a length ceiling, and (for an IP literal) a public address. It deliberately does not resolve DNS: a name that looks public now can rebind before the worker fetches it.
Resolved-host gate (download time). media.netcheck.assert_host_resolves_public re-resolves the host (A + AAAA) in the worker immediately before the download and rejects it (via the same ip_is_public) if any resolved address is non-public, closing the rebind-after-submit window for DNS names. It fail-closes on an unresolvable/empty/unparseable result. On refusal the run parks FAILED @ acquire for a manual Requeue, with a host-only (URL-free) error.

yt-dlp lockdown (media.ytdlp, verified against yt-dlp 2026.07.04): the argv runs with --no-config, --no-plugin-dirs (no local/remote plugin loading), --no-exec (no post-processor command), --no-playlist --max-downloads 1, a size cap, and hard wall-clock timeouts; file:// URLs are refused by yt-dlp's own default (we never pass --enable-file-urls). --proxy is passed always: the configured ytdlp_proxy when set, otherwise an empty value that means "explicit direct connection", so an ambient HTTP(S)_PROXY in the worker env can never silently reroute egress. --cookies is passed only when ytdlp_cookies_file is set. Both are treated as credentials, scrubbed verbatim from any surfaced error.

Source metadata capture (issue #36) rides the SAME invocation: --write-info-json --clean-info-json --no-write-playlist-metafiles with a typed infojson: output pinning source.info.json, never a second --dump-json call, which would double bot-block exposure and could describe different upstream state than the downloaded bytes. The stage sanitizes the info-JSON through a strict allowlist (media.source_metadata; secret-bearing keys like formats/http_headers/cookies are never copied), publishes the sanitized snapshot as a hash-addressed sidecar (source.<sha256>.metadata.v1.json, linked BEFORE the media file so a crash between publish and DB commit replays to a repaired row without re-downloading), and inserts the write-once media_source_metadata row. The raw info-JSON never leaves the attempt temp dir. Capture is best-effort: a missing/malformed/oversized info-JSON logs a warning and never fails an otherwise-valid acquisition. Surfaced on the run detail page, the runs browser (title), and GET /runs/{id}/export.json, a versioned object envelope (run + source_metadata + operator_notes + the same segment objects as the pinned bare-array /review/{id}/export.json, which stays frozen).

Residual: needs network policy, not a userland check. yt-dlp re-resolves the host independently when it connects, and its generic extractor follows HTTP redirects and constructs URLs. So a host that rebinds between our re-resolution and yt-dlp's fetch, an HTTP redirect to a private address, or an extractor-constructed private URL is beyond these gates. Closing that requires running the worker where it has no route to RFC1918 / link-local / the cloud metadata endpoint (egress firewall or a dedicated egress). The resolved-host gate raises the bar and closes the literal / rebind-at-check-time holes; it is not a substitute for egress control.

The opt-in compose.ytdlp-egress.yaml overlay (issue #16) productizes that network policy without a config knob: it routes yt-dlp's always-passed --proxy through a small filtering forward proxy (voxint.media.egress_proxy, the same image) that re-applies ip_is_public at the connection boundary and connects only to the vetted public IP. Because the proxy makes the outbound connection, the rebind window is closed and redirect / extractor destinations that resolve to private space are refused (for yt-dlp's own HTTP(S) traffic). It is deliberately not a sandbox: a helper yt-dlp spawns that ignores the proxy, or the worker's routable network, still wants a host-level egress firewall. See docs/operations.md, "Restricted URL-download overlay".

CSRF. Four mutation forms (POST /submit, /fetch, /runs/{id}/requeue, and POST /review/{id}/claim) carry a stateless, action-bound HMAC token (api.csrf, keyed by csrf_secret, independent of the Basic-auth password); a missing/mis-signed token is refused before any state change. When console_media_enabled is on, POST /submit and POST /fetch redirect to /media (303) before reaching the CSRF-protected handler; the /media/submit and /media/fetch routes carry their own CSRF actions. /claim needs its own because claiming is what mints the run's claim token: it has no unguessable token of its own yet. The remaining review-workbench mutations (release, decision, enroll) are instead gated by that per-run claim token. Since v0.27.0 the CSRF secret is auto-generated and persisted to DATA_DIR/csrf_secret on first start, so forms survive restarts without manual configuration; an explicit CSRF_SECRET env var overrides the persisted value.

Startup reconciler. The app lifespan runs reconcile_orphaned_incoming once at startup, scanning media_root/incoming/ for files with no committed MediaItem row (crash orphans from the os.replace-before-commit window) and removing them.

Requeue guard. requeue_failed_run raises RunArchivedError before checking the FAILED status, so an archived run cannot be requeued from any surface (the route-level guard was already present; the service-level guard covers the CLI).

Web research egress (issue #39)¶

voxint.research is the second outbound-fetch capability, and the THIRD consumer of the single egress policy in media.netcheck (ip_is_public, the shared string gate parse_http_url, and the fail-closed resolver core resolve_public_addresses), one module to audit for every path that leaves the box. It is off by default (VOXINT_WEB_RESEARCH=false) and deliberately independent of the LLM capability: configuring an LLM never implies egress, enabling retrieval never requires an LLM, and a config validator coupling the two is contract-tested absent. When off, both operations return structured disabled outcomes before any DNS or socket work.

Two operations, built as a library for the future research loop (issue #40) plus a feature-gated CLI (voxint research search|read):

web_search. One bounded query to a pluggable provider (SearchProvider protocol; SearxNG built in, its base URL being operator-configured egress in the same trust class as LLM_BASE_URL). Everything the provider RETURNS is untrusted: result URLs pass the shared string gate before being surfaced (refused ones are dropped and counted), titles/snippets are sanitized and capped.
read_url. A hardened single-page fetcher that CLOSES, for its own path, the redirect/rebinding residual documented above for yt-dlp (which it can't close because yt-dlp owns its connections; this fetcher owns its own). Every hop (the submitted URL and each redirect target, 301/302/303/307/308 only, Location resolved against the logical URL) is re-gated and re-resolved fail-closed, and the connection is pinned to a vetted address: the request URL's host is rewritten to the vetted IP while the Host header and TLS SNI carry the canonical (IDNA-encoded once) hostname, on a FRESH client per attempt so no keepalive connection can cross host identities. The checked address is the connected address. Responses must be identity-encoded (compressed responses are refused, removing the decompression-bomb class rather than bounding it), the streamed byte count is authoritative over Content-Length, and only text/html, application/xhtml+xml, and text/plain are readable. Extraction is stdlib-only (html.parser; no C parser on attacker bytes) and strips invisible-instruction characters (Unicode tag block, zero-width, bidi, C0/C1); retrieved content is data, never instructions.

Both operations take a mandatory bounded-identifier Attribution and an atomic per-invocation ResearchBudget (quotas enforced IN the tools; a spent budget yields a structured budget_exhausted outcome the #40 loop concludes from; quota is charged only after validation and the concurrency slot, so a refusal that performed no network work never burns budget). Every outbound request logs one attribution line (feature, reason, host, verdict, bytes, duration), and no ERROR detail or log line ever carries a URL, query string, or redirect Location (media.redaction throughout). The one deliberate exception: FetchOutcome.final_url on a successful read is provenance, the fragment-free logical URL actually read, which evidence records (#40's UrlEvidence) require and which may carry a query; consumers store it deliberately and never echo it into shared logs (the CLI prints it query-stripped). Timing caveat: the total wall clock bounds every HTTP operation via remaining-time propagation, but blocking DNS resolution cannot be hard-interrupted. DNS is the one non-hard-bounded step.

Web-research speaker enrichment (issue #40)¶

The web_researcher producer is the consumer #39 was built for: an operator-initiated, per-speaker research job that drives a bounded LLM tool loop (voxint.research.agent) and quarantines everything it finds in the #37 draft layer for field-by-field human review. Gated by ENRICHMENT_WEB_RESEARCH_ENABLED=false, which requires both VOXINT_WEB_RESEARCH and LLM_ENABLED at startup (fail-closed validator) and is re-checked in the worker so queued jobs cannot outlive a capability shutdown.

The orchestrator owns everything. The loop is a hand-rolled strict-JSON action protocol over the plain /chat/completions transport (HttpLLMClient.chat_json), with no provider function-calling and no agent framework, so budgets, the allowed-tool set, and evidence rules live in auditable application code, never in a prompt. Each round the model either requests up to RESEARCH_MAX_ACTIONS_PER_ROUND actions from exactly three tools (web_search, read_url, read-only query_existing_speakers) or concludes; anything outside the closed schema gets one repair attempt, then the job fails, never a silent found=false.
Budgets are hard. Retrieval quotas and the wall clock ride #39's ResearchBudget (exhaustion is a structured tool result); rounds are counted by the loop, and after the last round the model gets exactly one tools-disabled conclude request. All budgets snapshot onto the job row: the preview the operator approved is the contract.
Retrieved pages are hostile data. Page text reaches the model only as a JSON-encoded, untrusted-marked tool result capped at a 4k-char excerpt; read_url accepts only URLs from this job's own search results or the operator-stored seed URLs (copied exactly), so an injected page cannot steer fetches; and the server-side conclusion gate (the actual security boundary) drops any claim whose source is not a server-issued id of a page actually read, whose snippet does not locate verbatim (NFKC + casefold + whitespace-collapsed) in that page's kept text, or whose value is generic ("the host", "Speaker 2"). Surviving claims become speaker-scoped bio/affiliation/link candidates with UrlEvidence.
Jobs are durable, honest state. research_jobs holds queued → running (guarded claim; a duplicate Celery delivery no-ops) → succeeded | failed | cancelled, plus progress counters the console polls (htmx, 3 s while active) and a cooperative cancel_requested flag the loop re-reads between rounds. A confident found=false records an authoritative outcome='none' producer run; transport/LLM/contract failures and cancellation record nothing: a failure must never retire prior drafts. No automatic retries and no recovery sweep, deliberately: hidden re-execution of a non-deterministic web loop is worse than a visible stall the operator can cancel and restart.
Idempotency is per job, not per input. The producer-run key is web_researcher:speaker:{speaker_id}:{job_id}, one durable execution. Web research is non-deterministic, so an input-derived key would wrongly suppress deliberate re-research; an intentional rerun is a new job minting a new generation that supersedes still-proposed prior claims.

Provider seams¶

ASR, diarizer, embedder, and LLM sit behind typed protocols (voxint.clients.base). The GPU services speak versioned HTTP (/v1/transcribe, /v1/diarize, /v1/embed, /healthz) and share a MEDIA_ROOT volume with the workers, no multipart uploads. The LLM stage targets any OpenAI-compatible endpoint and is optional (LLM_ENABLED=false by default); enhancement is best-effort: bounded ID-keyed batches, one retry, a circuit breaker, and a wall-clock budget inside the stage lease, with failures degrading to NULL enhanced_text rather than failing the run (see docs/quality-gates.md). As with retrieved web content above, transcript text is data, never instructions: the enhancement prompt is hardened so a segment that reads like a command is enhanced as content, not obeyed. Speaker matching always runs and its invariant violations DO fail the stage. Test fakes satisfy the same protocols, which is how the end-to-end contract tests run without a GPU.

Domain-specific vocabulary and prompts are their own seam: a domain pack (voxint.domain_packs) supplies ASR vocabulary hints, name seeds, and LLM prompt fragments; a neutral meeting/podcast pack ships as the default. Selection is per run, resolved once at submit and frozen onto the run as a JSON snapshot (pipeline_runs.domain_pack, revision 0017): the pipeline worker and the offline name producer both read that snapshot, never the live env, so late enrichment can never diverge from what transcription used and a manifest edited on disk afterward never changes a past run's result. An unmapped folder, an upload, or a URL takes the default pack (DOMAIN_PACK_PATH, else the bundled generic). Several named packs may live under DOMAIN_PACKS_DIR (one child folder per pack, resolved by manifest name); voxint.domain_packs.registry is the shared resolver, and a NULL snapshot (a run predating revision 0017) falls back to the current default at execution time.

Console 2.0 P2a (#153) makes membership and configuration relational. The per-folder pack now lives on media_folders.domain_pack, and a run resolves its owning folder from the persisted media_items.media_folder_id, not a re-inferred path prefix, so reused or relocated media keep the folder they were created with. Vocabulary and corrections resolve per field, each taking its first present layer in the order explicit CLI/sidecar pack, then the project (projects.vocabulary / projects.corrections), then the folder pack, then the global baseline (the default pack unioned with app_settings). A project field is nullable: NULL inherits the layer below, an empty list is explicitly none and wins. Pack identity (name, prompt fragments, name seeds) always comes from the resolved pack; a project replaces only its two fields. An explicit or folder-pack layer no longer inherits the global glossary and corrections, the one deliberate behavior change; a run with no project and no folder pack still resolves to the default pack unioned with app_settings, byte-identical to before. Because vocabulary used to be unioned live at run start, a P2a snapshot carries a config_resolution_version: 2 key and the worker branches on it: version 2 uses the frozen effective vocabulary as-is, an absent key (every run predating #153) keeps the exact live-union path, so no pipeline_runs row is rewritten and requeuing an old run reproduces its original result. The full resolution spec is the P2a addendum to ADR 0002. The media_folders relation is authoritative, edited through the folder browser on the setup wizard's media step and under Settings → Media folders. The pre-P2a app_settings.media_folders and app_settings.folder_domain_packs columns were dropped in revision 0046. An archived project is skipped: its folders resolve as if unassigned for new runs, learned-corrections capture and new quote saves are off, history and membership stay live.

Console 2.0 P2b (#154) makes /media operable behind the same CONSOLE_MEDIA_ENABLED flag. Upload and URL fetch move onto the page (each may choose a settings folder that sets which vocabulary and corrections apply without moving the file, per the ADR 0002 addendum), a folder panel registers or unregisters folders, and a multi-select drives non-destructive bulk actions: assign a settings folder, re-run transcription, and archive or restore the latest run. Every bulk route prevalidates the whole selection before any write and then either applies atomically or reports each item's outcome; a deliberate refusal (a stale baseline, an unreadable sidecar, no run to archive) is a reported skip, not an aborted batch. Re-run is a two-step advisory flow: a preview resolves the config a fresh run would freeze through the read-only preview_effective_config seam and captures a per-file latest-run baseline, and the confirm step row-locks the selection, re-verifies each baseline, and mints one fresh run per surviving file in a single transaction, so a double-confirm creates at most one run per file. Archive and restore carry each selected row's render-time latest-run id as a baseline and act on that exact run, skipping on drift, so a double-submit is idempotent rather than sliding onto the next-older run. The archived view (/media?archived=1) lists files whose latest run is archived so bulk unarchive has a target. All routes are always registered (the route inventory is stable across the flag flip) and 404 until the flag is on; flipping it also points the sidebar Media link and the Home "Add media" action at /media instead of the legacy /runs upload. No schema migration ships in P2b.

Review console¶

Adjudication is post-hoc: runs complete normally and the console works a queue over COMPLETED runs. A run needs review while any diarization label has neither an effective human decision nor a grounded cosine proposal. (AWAITING_ADJUDICATION stays in the state machine, reserved for a future flow that genuinely blocks downstream processing; nothing enters it today.)

One resolver (adjudication/resolver.py) settles attribution at read time for the workbench, the queue, and the transcript export alike: effective human decision (newest ledger row per label; corrections are appends) beats grounded cosine beats nothing. llm_hint names render as evidence, never as identity; exclude suppresses attribution, never text.
Runs search (GET /runs, revision 0008): transcript full-text (q=, websearch_to_tsquery over per-segment english tsvectors of raw AND enhanced text separately, so the search document is one segment), a speaker facet answered by a SQL mirror of the resolver (speaker_attributed_exists: effective decision or grounded cosine, merge tombstones expanded via roster.alias_ids; archived speakers stay offered, marked), source-substring and UTC date facets. Everything AND-composes with status/review and the (created_at, id) keyset cursor; results stay newest-first, no relevance ranking; matching runs get one escaped ts_headline snippet (first matching segment).
Media detail page (GET /media/{id}/editor, issue #156): a read-only entry point into a media item's best run. Selects the latest completed run by default (?run= overrides, validated against the media item). Renders the transcript with the speaker palette and a verified-progress counter, a run chooser, and a metadata rail. Claim-token verification degrades to read-only when the token is stale or absent. No mutations: existing /review/{run_id}/* endpoints remain the only write surface.
Reviewer slot: claim columns on pipeline_runs, guarded by the same CAS revision as pipeline transitions. The claim token is an opaque per-claim secret required on every mutation; a re-claim rotates it, so a stale tab gets 409 instead of acting on a slot someone else holds. Claims expire on a TTL, so an abandoned tab never dams the queue.
Decisions POST through the existing idempotent ledger append. Each rendered form carries a fresh server-issued nonce as the idempotency key: htmx retries are harmless replays, new submissions are new (superseding) rulings.
Enrollment turns an unmatched voice into a roster identity atomically: a speakers row, one duration-weighted centroid in speaker_embeddings (same eligibility rules and centroid math as matching, imported from speakers/matching.py so they cannot drift), and the assign ruling. Raw per-turn vectors stay in diarization_turns; the centroid is re-derivable. Provenance columns plus a unique constraint on the source decision make duplicate enrollment structurally impossible.
Inline speaker merge (adjudication/merge.py, issue #54): the over-split fix ("these labels are one voice in this recording") as a workbench action instead of a roster-page trip. It is run-local: it records one assign ruling per label to a single survivor (an existing active speaker, or a newly enrolled one via the same enroll_new_speaker path) and never calls roster.merge_speakers; a later deliberate roster merge still unifies these rulings at read time, so deferring the global act loses nothing. It is a composite mutation done atomically under the run's claim lock, with deterministic child idempotency keys ({nonce}:{labelset-digest}:{label}) so one operator nonce backs several ledger rows, a replay returns the original outcome, and reusing the nonce for a different label set collides loudly rather than half-applying. preview_merge computes the exact impact server-side (advisory client counts are never trusted); the claim token proves ownership, not content-version, so apply re-checks each label's expected effective-ruling id and returns 409 if it drifted since the preview.
Two-scope relabel (issue #54 Phase B): a ruling can target ONE transcript segment instead of the whole (run, label). Storage stays the one immutable ledger, a nullable adjudication_decisions.transcript_segment_id (NULL = the historical label scope), not a second table, with a new segment-only inherit decision as the append-only reset (the ledger is insert-only, so "undo this override" is a new row, never an UPDATE). The writer derives the segment's label server-side; a CHECK keeps inherit segment-only. Every label-scope query filters transcript_segment_id IS NULL so a segment override never leaks into label resolution: effective_decisions (the source label_states reads), the _label_unresolved / speaker_attributed_exists SQL mirrors, and the web-research seeds. Read-time precedence is scope-local: newest within a scope, then segment beats label beats grounded machine, never comparing timestamps across scopes. segment_states resolves the active per-segment overrides (newest assign per segment; a newest inherit means none, so the segment follows its label live, never a frozen copy) and canonicalizes speaker ids through the same merge tombstones as the label path; attributed_transcript overlays it, so the HTML page and every export agree. Deliberate v1 limit: speaker search and the queue stay label-scoped (a segment-only speaker does not surface there).
Export picker (issues #52, #65, #375): every built transcript format is reachable from the workbench, the transcript page, and the editor through one shared Jinja fragment (fragments/export_menu.html): TXT, Markdown (.md), SubRip (.srt), WebVTT (.vtt), JSON, and RTTM. Pure HTML (no island): each option is a plain <a> whose href carries the query, so the menu works with JavaScript off. A single "Download transcript" <details> button opens a compact format list defaulting to the reviewed (corrected) text variant. The enhanced and raw variants are in a nested "Other text variants" disclosure, keeping the primary list short while preserving every combination. TXT and Markdown both offer a timestamp-free reading copy (?timestamps=false), a timestamps=... keyword the CLI mirrors (voxint export --no-timestamps); an integration test asserts the download is byte-identical to the CLI for both settings. The flag is inert for SRT/VTT (cue timing is structural) and JSON (keys are a frozen contract). The picker offers a Read on screen link above the format list. RTTM carries raw diarization labels only, so it takes no variant.
Read mode + Markdown (issue #65): a server-rendered on-screen reading view (GET /runs/{id}/transcript?read=1, no island) and a .md export both render from the same attributed_transcript seam through one grouping helper, paragraphize_transcript (in src/voxint/adjudication/transcript.py), which merges adjacent same-speaker lines into paragraphs. to_markdown writes ## speaker headings over > blockquotes with format_timespan time ranges, and funnels through render_transcript for CLI/route byte parity; read mode renders the same paragraphs via Jinja autoescape. So the two surfaces cannot drift from each other or from the other exports. Deliberately not built: a speaker-name omission toggle (caption guidance keeps speaker IDs; anonymization belongs in the roster, not the exporter) and a plain-.txt default flip (the timestamped default is a golden-pinned contract; the reading copy is surfaced in the UI instead), both parked for a real user story.
Triage & correction (issues #53/#58): the transcript flags low-confidence segments (persisted transcript_segments.confidence = exp(avg_logprob), below a configurable threshold) as "uncertain", a non-background cue so it never collides with the active-line highlight or the speaker accent. Per-segment operator workflow state lives in segment_review_states (mutable, one row per segment, UPSERT latest-wins), deliberately NOT the append-only adjudication ledger (orthogonal to speaker attribution) and NOT columns on the immutable transcript_segments: a verified mark (feeds an "N of M" counter) and an operator corrected_text. A correction is written beside raw_text, never over it; one shared effective_text selector (corrected → enhanced → raw, by IS NOT NULL) makes the default transcript view and every text export agree. ?text=raw is always the untouched ASR evidence, ?text=enhanced the pipeline text without corrections. Corrections are full-text-searchable (a partial FTS index over corrected_text, never coalesced with the raw/enhanced renderings) and feed enrichment (the name miners and run-asset generators read the same effective_text), both as settled in the provenance design note. Editing text clears the verified mark in the same transaction (edited text must be re-verified); reverting to the pipeline wording clears the correction. Both writes are claim-gated; the UPSERT is idempotent without a nonce. This operator corrected_text is distinct from a domain pack's automatic corrections (#82): those are deterministic literal substitutions the pipeline applies, recorded per segment in correction_trace (above) and surfaced separately in the console as "corrected by domain pack" (#83); an operator edit supersedes that marker for the segment. See domain-packs.md.
Auth: dual-mode, controlled by VOXINT_MULTI_USER. In single-operator mode (the default): HTTP Basic (constant-time compare) on every route but /healthz, fragments and media included; operator identity comes only from credentials. In multi-user mode: Argon2id password hashing (argon2-cffi), session cookies (voxint_session, sha256-hashed tokens in auth_sessions), login/logout pages with per-action CSRF tokens, and three roles (admin, reviewer, viewer). The _resolve_identity dependency in deps.py branches once; downstream code uses CurrentUserDep (the resolved AuthContext) and OperatorDep (the username string) regardless of mode. AdminDep gates every route that mutates installation-wide state, including the settings page. The login redirect target (?next=) is validated by _validate_next(), which rejects protocol-relative paths, backslash normalization, control characters, and scheme-bearing URLs to prevent open-redirect attacks. Adjudication routes pass user_id through to record_decision, apply_merge, and enroll_new_speaker so the immutable ledger carries per-user attribution. See operations.md for setup. The template layer enforces a second gate via shell.can_write, a boolean in the shell context processor (deps._shell_template_context): True for admin and reviewer, False for viewer and unauthenticated. Templates wrap mutation controls (upload, claim, edit, delete, archive buttons and forms) in {% if shell.can_write %} so viewers see a clean read-only console without controls that would 403 on click. The editor island derives its own read-only state from the claim token (viewers cannot obtain one). OOB htmx fragment renders in speakers.py inject shell manually because they bypass the normal template context. Startup refuses to bind off-loopback with the default password.
Response headers: one shared middleware seam (_apply_security_headers) stamps a deliberately minimal set on every response, and re-applies it on an unhandled 500. Referrer-Policy: no-referrer on every response (the review token rides in the URL, so no navigation or subresource may leak it in a Referer); Cache-Control: no-store on every token-sensitive response (/review/* and /media/{uuid}/editor*, where claim tokens ride in the URL or response body; the path classifier is _is_token_sensitive_path in app.py); X-Content-Type-Options: nosniff on every response (#103), so the browser honours the declared Content-Type instead of sniffing operator-controlled transcript exports or first-party assets into HTML. X-Frame-Options and a content-security policy are intentionally left out: neither is proportionate for a loopback, Basic-authed, single-operator console, and a real CSP for the htmx-plus-islands page would put the JavaScript-off fallback at risk.
Media: audio streams through a gate that requires the file to be DB-referenced, to resolve inside MEDIA_ROOT (symlink escapes rejected), and to carry a decodable audio stream per ffprobe (bounded subprocess, cached per path/size/mtime). Single-range HTTP semantics: 206/416, open-ended and suffix forms; multipart ranges are ignored per RFC.

Plugins (epic #136)¶

Plugins are for greenfield features with standalone surfaces. Features whose state is rendered by a core page stay in core. Translation, LLM enrichment, and semantic search therefore remain native. Synthdetect is the reference plugin.

The framework lives in src/voxint/plugins/. base.py defines the contract and contribution records. registry.py validates and orders plugins. hooks.py dispatches lifecycle events. deps.py provides bounded route dependencies. media.py exposes confined run audio. boot.py and doctor.py integrate startup checks and diagnostics.

Concrete plugin classes register in the BUILTIN tuple in discover.py. Plugins may import core. Core imports concrete plugin classes only in discover.py. The pipeline never imports voxint.plugins.

Framework seams cover settings sections, run-detail panels, routes, Celery task modules and routing, recoverable job lanes, post-completion hooks, and CLI commands. Plugin routes that mutate installation-wide state depend on AdminDep. Read-only routes do not require it.

Feature gates use tri-state settings. Pydantic Settings supplies the environment default. Nullable AppSettings columns supply per-installation overrides. resolve_effective_* functions select the active value at runtime. The base gate fails closed.

See the plugin author guide for package layout, hook selection, job safety, migrations, compose overlays, tests, and the shipping checklist.

Frontend islands (issue #48)¶

Jinja owns every page; interactive regions are React islands mounted into server-rendered markup. This extends the review console; it is not a new subsystem and adds no page routing.

Vite, not Astro (settled decision). Plain Vite v6 multi-entry + React 19
Tailwind v3, vite build only. Astro's value is its own page/SSR rendering, content collections, and .astro format, all unused here, since Jinja renders every page and mounting into a foreign template engine still means hand-writing createRoot(el).render(...) against data-* points. Vite's multi-entry + manifest.json output is the first-class workflow for compiling independent TS/TSX entries into content-hashed bundles that some other system embeds; it has no server-runtime concept, so nothing can reach for a Node adapter that would violate "no Node at operator runtime". It also keeps the npm supply-chain surface (every npm audit line) smaller. Trade-off (honest): if voxint ever wants genuinely server-rendered .astro pages we'd migrate then (cheap, because the React components are framework-agnostic beyond their mount call), and we accept hand-writing the ~20-line manifest→<script>/<link> lookup on the Python side as the price of not carrying an unused meta-framework.
Progressive enhancement is the contract. The server HTML is the fallback, fully usable with JS disabled or the asset route unbuilt; islands are additive and replace only their region's visual role once hydrated. An island that fails to hydrate degrades one region, never the page; the server markup inside its mount div stays visible. The transcript page demonstrates this: a native <audio> plus the segment list, active segment highlighted on timeupdate, over the same {% for ln in lines %} loop the JS-off page renders.
Auth-aware asset route, never a StaticFiles mount. Bundles serve through GET /static/app/{path} carrying the operator auth dependency on every byte; a mount would bypass it, and "everything but /healthz authenticates" is absolute. The route resolves+contains the untrusted path (traversal/symlink escapes 404 before any filesystem read) and sets immutable caching only for Vite-hashed filenames. A contract test pins the absence of any StaticFiles import/instantiation.
Build-stage boundary. The Dockerfile's node:22-slim stage builds the bundles and is then discarded; only dist/ is COPYed into the Python image before uv sync --no-editable, so the wheel packages the static tree. No Node binary ships, empirically verifiable via docker history / command -v node in the runtime image.
Mount convention #49–#59 extend, not reinvent. base.html pulls one shared module (main.ts) that scans for [data-island] nodes and dynamically imports only the bundles present; a page carries an island by emitting <div data-island="name" data-props='{...}'> with a server-rendered fallback inside. Adding an island never edits base.html. Islands read props via readProps() and call voxint's own routes through the shared api-client.ts apiFetch, whose ApiError mirrors FastAPI's {detail} shape, the seam #54/#55 consume for capability-aware responses.
Command palette (issue #162). Ctrl/Cmd+K opens a global search island (palette.ts) that queries commands (sidebar destinations, per-page actions), entities (media, speakers, projects by name), and transcript passages (semantic + lexical). The island loads lazily on first interaction. A "Recent" section (localStorage, capped at five) shows recently opened recordings when the query is empty. The palette is enabled by default (CONSOLE_PALETTE_ENABLED).
Per-turn playback + fail-closed seek gating (issues #49/#55). The media-editor island owns the audio player, transcript, walk cursor, and speaker rail. Per-line playback calls TranscriptPlayer.playSegment(). Hear this voice finds the first transcript segment for the label and moves the walk cursor there, which seeks and starts playback. The rail receives this callback only when the playback capability allows seeking.
The fail-closed capability contract (issue #55). api/playback.py's playback_capability() is the seek predicate: seek_enabled is true only when the media is actually servable, the duration is finite and positive, every transcript interval is well-formed, and no interval runs past duration + 0.05s (a fixed tolerance, absorbing float noise without scaling on long files). It accumulates every applicable reason with plain-language messages the islands show in a visible banner, never a bare tooltip. Media servability reuses resolve_servable_media(), the single seam GET /media itself calls, so capability can never advertise seeking while /media would 404/410.
Follow-along highlight + per-speaker colors (issues #50/#47). The transcript-player island keeps the active line in view as playback advances: a callback ref on the active <p> plus a scrollIntoView({ block: "nearest" }) (never smooth, and it never moves DOM focus). Following is a boolean that starts on; a single passive window scroll listener flips it off on any manual scroll (wheel/touch/keyboard/scrollbar all emit real scroll events). A short programmatic-scroll guard (a performance.now() timestamp armed before each auto-scroll) makes the listener ignore the events the auto-scroll itself emits, so following isn't self-cancelled. The lone "Resume following" control renders only while paused-from-following, next to the speed control; clicking it re-enables following and re-centers the active line. No always-on checkbox, no status dot. Per-speaker identity color is assigned by the pure api/speaker_colors.py speaker_palette(): a deterministic, order-independent map from the run's canonical label universe to curated palette indices [0, 8). That universe (_run_label_universe) is the union of the run's diarization-turn and transcript-segment labels, so even a transcript-only label (a segment whose label has no turn) gets a color. Both the transcript route and _workbench_context derive the palette from that same universe, so a label's color agrees across the transcript page, the JS-off fallback, the workbench label cards, and the walk/editing segment header. The color is rendered identically on every surface as a spk-N class → a CSS left-border accent (light/dark variants in base.html), and it is supplemental only: the speaker's display name (<strong> prefix on each transcript row and in the segment header's .me-speaker-identity group) is the primary, non-color identity cue (accessibility: never color alone). The raw diarization label (.spk-badge) is shown on the workbench, label cards, and the segment header (when it differs from the display name) but not on the editor transcript rows, where it duplicated the display name.
Waveform drag-to-select (issue #502). The WaveformStrip supports click-to-seek (existing) and click-and-drag range selection: the selected region renders as a theme-aware accent overlay, and a "Play selection" control appears for bounded playback. Pointer Events with setPointerCapture handle mouse, touch, and pen; a 5px distance threshold distinguishes click from drag. Selection state is controlled by the parent; transient drag state is local.
Searchable speaker combobox (issue #513). All four speaker-assignment dropdowns use a shared SpeakerCombobox component: type-ahead filtering, arrow-key navigation, ARIA combobox role, and an inline "Create [name]" option. Digit-key direct-assign (1-9) is preserved. The speaker roster refreshes live on roster-affecting endpoints (enroll, merge, undo) via an optional speakers snapshot in _labels_response. Shared rankItems / moveActive helpers are extracted into combobox.ts from the palette.

Worker orchestration (P3)¶

Two Celery tasks drive a run through the engine: voxint.run_pipeline owns the GPU segment (acquire through diarize_embed) and voxint.finish_pipeline owns the post segment (enhance_match, finalize), with a validated RUNNING -> QUEUED handoff between them (see Execution lanes and queues). Within its segment each task drives every stage itself rather than task-per-stage (task-per-stage would open an unclaimed window between handoffs that recovery misreads as a crash; the engine already resumes an interrupted run at its current stage, and the single segment boundary is covered by the same claim validation and recovery sweep as a crash). Failure handling is two-lane:

Transient (retryable service errors: saturated, model_unavailable, transport failures): the failed attempt stays in the stage_runs ledger, the run is CAS-requeued at the same stage (against the exact revision that failure produced, so a stale callback can never requeue a newer failure), and the task retries itself with exponential backoff. The attempt budget (STAGE_MAX_ATTEMPTS) counts transient service failures from the persisted ledger (restarts and broker loss never reset it); lease-expiry interruptions don't eat it, and the sweep applies the same ceiling to crash loops separately.
Deterministic (inference_failed, protocol violations, bad media): the run stays FAILED for the failure lane; voxint requeue is the explicit human override.

A beat task (voxint.recovery_sweep, every RECOVERY_SWEEP_SECONDS) requeues runs whose stage lease expired and re-enqueues QUEUED runs whose task evaporated with the broker (QUEUED_RUN_STALE_SECONDS grace so pending retry countdowns aren't stepped on). Duplicate enqueues are safe by design: claims and CAS arbitrate.

A second, opt-in beat task (voxint.gc_sweep, issue #15) reclaims the large normalized-audio intermediate for old terminal runs when MEDIA_RETENTION_ENABLED: it unlinks artifacts/{run_id}/normalized.wav and stamps the audio_artifacts row (reclaimed_at/reclaimed_bytes; the row is kept as an audit record). File reclamation only: source media, transcript, diarization, and the decision ledger are never touched, so a reclaimed run stays re-processable from source. Rows are claimed oldest-first with FOR UPDATE ... SKIP LOCKED, so overlapping sweeps neither double-count nor clobber a byte measurement. See operations.md for tuning.

Timeout ordering that must hold: HTTP client timeout (GPU_HTTP_TIMEOUT_SECONDS) < stage lease (STAGE_LEASE_SECONDS; two stages carry their own longer lease: DIARIZE_EMBED_LEASE_SECONDS because it makes one diarization call plus several sequential embedding batches, and ACQUIRE_LEASE_SECONDS (3 h) because a URL download runs under a wall-clock ACQUIRE_TIMEOUT_SECONDS (2 h) that must itself sit below the lease by a cleanup margin, validated at startup) < Redis visibility timeout (CELERY_VISIBILITY_TIMEOUT_SECONDS, which covers a whole run and is sized to the six-stage worst case, 48 h).