Development and repository layout
| Location | Responsibility |
|---|---|
code/src/irtf_to_ietf/ |
Reusable collection/discovery engine and authoritative review schema |
web/portal/ |
Django models, review workflow, continuous execution and UI |
web/portal/static/ |
Bundled frontend assets, including the discovery flowchart |
docs/ |
Current method, workflow and operational reference |
code/tests/, web/portal/test*.py |
Engine and application verification |
The engine supports Python 3.11+; the web image specifies its runtime in Dockerfile. Dependencies are pinned in project requirement files. No Node build is required.
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r web/requirements.txt
python -m pip install --no-build-isolation --no-deps ./code
python -m unittest discover -s code/tests
TEST_SQLITE=1 python web/manage.py test portal
SQLite is a lightweight test option. Verify database-specific behavior against PostgreSQL:
docker compose run --rm --no-deps web python web/manage.py test portal --noinput
docker compose run --rm --no-deps web python web/manage.py check
docker compose run --rm --no-deps web python web/manage.py makemigrations --check --dry-run
Tests use a dedicated test database; do not run concurrent PostgreSQL suites against the same test database. Use synthetic data, never production reviews as fixtures.
Django migrations define the application schema. The engine's optional SQL model is separate. collectstatic runs during the image build. The same application image supports web and background services; maintenance explains deployment.
Documentation navigation is defined in web/portal/documents.py. Each current page appears once; legacy reader URLs redirect to the consolidated replacement. The clean distribution includes current docs, not local backups, secrets or research datasets. Use the repository's packaging tools and tests before sharing a build.
Local pgAdmin for testing
Set PGADMIN_EMAIL and a random PGADMIN_PASSWORD in your local .env, then start the optional service:
docker compose -f compose.yaml -f compose.pgadmin.yaml up -d --no-deps pgadmin
The existing PostgreSQL service must already be running. Open http://localhost:5050 and sign in with those credentials. The IRTF to IETF (local) server is preconfigured for database irtf; its database password is loaded from POSTGRES_PASSWORD. Browse Databases → irtf → Schemas → public → Tables to inspect records. This connects to the application's current database, so edits affect the running application.
The interface binds only to 127.0.0.1; PGADMIN_PORT changes its port. Login state and preferences are saved in the separate pgadmin_data volume. The service does not start with the default Compose configuration and does not restart automatically. Stop it after testing:
docker compose -f compose.yaml -f compose.pgadmin.yaml stop pgadmin
Do not use down -v to stop pgAdmin: that command also removes application database volumes.
Collection batches with a fully expanded relational checkpoint restore only pages of unfinished tasks and the document catalogues used to suppress duplicate discovery. Every restored artifact is checksum verified. Completed source pages remain in PostgreSQL for analysis and export. A checkpoint with pending expansion restores all metadata so interrupted expansion can be replayed. Compare restoration and total batch seconds in the run performance view to measure the effect.