Maintenance, performance and backups
PostgreSQL stores studies, frozen plans, original source bytes/hashes, checkpoints, candidates and review history. Redis holds queued work and disposable chart caches. A run export is not a full database backup.
Diagnose before changing resources
Use the run's Performance and Execution tabs to distinguish network delay, retries, parsing, database persistence and analysis. More memory does not accelerate a network-bound stage. PostgreSQL settings are exposed in compose.yaml; size them for the host and concurrent workers. work_mem can be allocated multiple times per query/session, not once for the whole database.
docker compose ps
docker compose logs --tail 100 web worker controller
Do not restart workers to refresh documentation or frontend assets. Relevant chart caches invalidate on committed changes; Refresh charts forces recalculation. Cached results do not replace source evidence.
Database backup
Pause active runs through the frontend and wait for them to settle. Record which runs should later resume. Stop application writers, including the controller, then preserve the database and .env securely:
docker compose stop controller worker web
docker compose exec -T postgres pg_dump -U irtf -d irtf -Fc > /tmp/irtf-backup.dump
docker compose up -d web worker controller
Use your configured database/user if different. Verify backups through a restore into a separate test installation. Restore into a deliberately selected empty/replacement database with application writers stopped; do not overwrite a live research database casually. Resume only the intended runs after checking restored state.
Deploy updates
For a documentation/frontend-only update with compatible schema:
docker compose build web
docker compose up -d --no-deps web
For engine/model changes, pause and settle active work, back up, build the image, apply required migrations, and deploy compatible web/worker/controller versions. prepare_database also prepares imported-data indexes; migrate alone does not perform every backfill. Review release-specific work rather than assuming every update needs a full reanalysis.
The agenda-adoption and independent-author-intersection signals use the existing candidate payload and nullable base relationship; they do not require a schema migration by themselves. Deploy compatible code first, then explicitly run the relevant saved-data discovery action on an idle run. Opening Results or rebuilding the image does not reclassify saved candidates. Follow existing-run discovery operations, and inspect the returned coverage status before interpreting changed counts.
Research source artifacts and reviews remain authoritative. Operational cache data can expire; original artifacts and database volumes must be preserved. Never use docker compose down --volumes as an update procedure.
Large artifacts
Migration 0011_artifact_chunks adds storage for artifact parts. New files larger
than 16 MiB are saved as ordered parts in the same transaction as the artifact
metadata and checkpoint. The SHA-256 and size describe the complete original
file; downloads, exports, restoration and indexing reconstruct and verify those
bytes. Existing artifacts remain readable without a bulk conversion.
Deploy the migration and the compatible web, worker and controller image together.
Do not run an older application image after saving chunked artifacts. The reverse
migration must not be used while chunked artifacts exist: it would remove their
parts. PostgreSQL backups must include portal_artifactchunk as well as
portal_artifact (a full pg_dump includes both).
The opt-in PostgreSQL regression test writes and verifies a file larger than 1 GiB in a temporary test database:
LARGE_ARTIFACT_TEST=1 python web/manage.py test portal.test_artifact_chunks
Candidate review assignments
Migration 0012_review_assignments only adds allocation and assignment tables and constraints. It does not backfill or modify studies, frozen plans, candidates, evidence or reviews. Existing collectors and report code remain compatible with those additions; deploy the migration before the updated web image. A frontend deployment does not itself distribute candidates. The owner explicitly previews and applies an allocation inside a run.