feat: modernize Python repo to uv + pyproject.toml by farhan · Pull Request #1340 · openedx/edx-proctoring

farhan · 2026-09-24T10:03:14Z

Important

Written with the assistance of Claude Code, human-reviewed before posting.

Modernize edx-proctoring
Part of openedx/public-engineering#506

Note

Translations: after the src/ move, the openedx-translations extraction job will commit new source strings under translations/edx-proctoring/src/..., which transifex.yml and edx-platform's atlas pull don't read. Existing translations are unaffected; only new or changed strings pause until openedx/openedx-translations#82365 is fixed (the workflow will strip src/). Same situation as the already-merged DoneXBlock and RecommenderXBlock.

Summary

Publishes to PyPI: https://pypi.org/project/edx-proctoring/ (and npm: @edx/edx-proctoring)
Move the package into a src/ layout (src/edx_proctoring/)
Replace setup.py/setup.cfg with pyproject.toml (PEP 621 static metadata)
Switch from pip-compile to uv with PEP 735 dependency groups; commit uv.lock
Retain pylint/isort/pycodestyle/pydocstyle as on master
Coverage config moved from .coveragerc into [tool.coverage.run] (pyproject.toml:189)
Update CI (ci.yml, migrations-check.yml) to use astral-sh/setup-uv; SHA-pin all actions
Replace publish-pypi.yml with release.yml (python-semantic-release + OIDC trusted publishing), gated on ci.yml
Set requires-python = ">=3.12" (pyproject.toml:8)

Removed/Updated

Deleted files: setup.py, setup.cfg, requirements/, .coveragerc, openedx.yaml (legacy OEP-2 metadata, removed rather than ported)

Renamed/replaced: .github/workflows/publish-pypi.yml → .github/workflows/release.yml

Where setup.cfg / .coveragerc settings went:

Setting	New location
`[isort]`	`[tool.isort]` (`pyproject.toml:203`)
`[pycodestyle]`, `[pydocstyle]` (full ignore list), `[doc8]`	`tox.ini:72-85`
`[tool:pytest]`	`[tool.pytest.ini_options]` (`pyproject.toml:199`)
`.coveragerc`	`[tool.coverage.run]` (`pyproject.toml:189`)
`[bdist_wheel] universal = 1`	Dropped, so the wheel tag is now `py3` instead of `py2.py3`

Removed Makefile targets:

Target	Reason
`$(COMMON_CONSTRAINTS_TXT)`	pip-compile helper that downloaded `common_constraints.txt`; constraints now come from `edx_lint write_uv_constraints`

Updated Makefile targets:

Target	Change
`upgrade`	`edx_lint write_uv_constraints pyproject.toml` + `uv lock --upgrade` (no `--locked`, because this target is meant to change the lockfile)
`requirements`	`uv sync --locked --group dev` instead of pip-sync
`requirements-test`	`uv sync --locked --group test` instead of `pip install -r`
`quality-python`	Runs pylint/pycodestyle/isort directly instead of `tox -e quality`; pylint runs with `PYTHONPATH=.` so `mock_apps` resolves under `src/`
`test-python`	`python -Wd -m pytest ...` directly (no `pip install -e .` step)
`coverage`, `lint-js`, translation targets	Paths updated to `src/edx_proctoring`

Python 3.12 only

Master's CI matrix and classifiers were already 3.12-only, but master had no python_requires and still listed py311 in the tox envlist. This PR drops py311 from the envlist and sets requires-python = ">=3.12".

Versioning

[Dynamic] setuptools-scm with dynamic = ["readme", "version"]. Master had a PyPI publish workflow, and python-semantic-release now sets the version at release time from git tags. PSR only creates the tag and never pushes a commit; publish-npm.yml sets the npm version from the release tag (npm version --no-git-tag-version) before npm publish, so package.json:version is not bumped in git.

Important Notes

Translations: see the note at the top (openedx-translations#82365).
npm package: package.json:5 (main) and package.json:14 (files) now point at src/edx_proctoring/static. Without this change, npm pack produced only 3 files and no JS. Verified: npm pack --dry-run lists the same 47 files as master, and require.resolve('@edx/edx-proctoring') resolves to src/edx_proctoring/static/index.js, which exports the handlerWrapper that mockprock uses.
Release token: release.yml follows the sample-plugin standard with one exception. gh release create uses GH_TOKEN: ${{ secrets.REQUIREMENTS_BOT_GITHUB_TOKEN }} (release.yml:44). Releases created with GITHUB_TOKEN don't start other workflows, and .github/workflows/publish-npm.yml runs on release: published. There is no secrets: inherit on run_ci and no fetch-depth: 0.
CI on master: ci.yml no longer has a push: master trigger. release.yml calls it as its first job (run_ci), so CI still runs on every merge, and nothing is released unless CI passes. The workflow name Python CI, the matrix env names django42-drflatest/django52-drflatest, and the Codecov upload from django42-drflatest (ci.yml:53) match master.
No release commits: PSR has no version_variables, so it never pushes a version-bump commit to master. Such a push would be rejected, because master requires the openedx/cla check. The npm version comes from the release tag in publish-npm.yml instead, the same approach as openedx-atlas.
v6.0.0 was never published: the v6.0.0 tag and GitHub release exist, but PyPI and npm are still on 5.2.0. python-semantic-release reads tags, so it will treat 6.0.0 as released, and the first automated release will be 6.0.1 or 6.1.0. Maintainers: if 6.0.0 should ship as-is, the tag and release need to be re-cut before this merges (publish-npm.yml uses --allow-same-version, so a re-cut v6.0.0 still publishes).
version_check: src/edx_proctoring/scripts/version_check.py:13 now skips .dev versions. CI uses a shallow checkout, so setuptools-scm resolves to 0.0.0.dev0 there and the check is skipped.
fail-fast: fail-fast: false was dropped from the CI matrix, per the modernization convention. One failing job now cancels the rest.
Lockfile: every uv sync uses --locked (ci.yml:47, migrations-check.yml:32, Makefile:37, Makefile:41), so a stale uv.lock fails fast. --locked is never used on uv run.
Constraints: djangorestframework<3.17 and edx-lint<6.2.0 are pinned in [tool.edx_lint] uv_constraints (pyproject.toml:181, pyproject.toml:186), with the reason for each inline.
Wheel parity: the built wheel has the same 288 package files, the same Requires-Dist, and a byte-identical entry_points.txt compared with master. src/edx_proctoring/locale is a git-tracked symlink to ./conf/locale, and the wheel still builds cleanly because package data is listed explicitly (include-package-data = false). The long description is now the README only: CHANGELOG.rst is deprecated and has a note prepended.
Path fixes for src/: .pii_annotations.yml:1 source_path, .tx/config file paths, .eslintignore, karma.conf.js, and the GitHub source links in docs/.
Docs: the release section of docs/developing.rst now describes the PSR flow but keeps the manual edx-platform upgrade steps. In docs/pull_request_template.md, the conventional-commit item (linking OEP-0051) replaces the version-bump/changelog items in the pre-merge checklist, and the Post-Merge tag step is removed. The mkvirtualenv step is dropped from README.rst.
No docs tox env: docs/ has RST files but no Sphinx conf.py and no make docs target, so there is no [testenv:docs] and no docs job in CI.

Out of Scope

Item	Notes	Next action
`celery50` / `drflatest` tox factors	`celery{50}-django{42,52}-drflatest` (`tox.ini:3`) is kept for parity with master's env names, but these factors are now labels only. Celery and DRF come from `uv.lock`, not per-factor deps.	Follow-up PR to simplify env names (check branch protection first)
DRF capped `<3.17`	DRF 3.17 changed `ListSerializer.errors` to an index-keyed dict, and several tests assert the old list shape.	Update tests, then drop the pin
`.tx/config` legacy	The `edx-proctoring-js` resource's `source_file` points at `django.po` (`.tx/config:12`), the same as on master. This PR only updated the path.	Confirm whether `.tx/config` is still used; fix or remove it separately
`translations-django42/52` tox envs	tox 4 doesn't map these onto `[testenv:translations]`, so they run the unit tests rather than `make validate_translations`. Same on master.	Follow-up PR
npm publishing	`publish-npm.yml` still needs the bot token described above. Publishing npm from `release.yml` would let the release use `GITHUB_TOKEN`.	Follow-up, if maintainers prefer

Testing Notes

Note: PR creator has tested the PR via CI and building the packages manually. No manual testing has been done to test the repo features. It will be great if the repo owner could test it.

Important

npm release flow needs critical review. publish-npm.yml now runs on the release created by release.yml and sets the npm version from the release tag (npm version --no-git-tag-version) instead of a version-bump commit. Please review this flow carefully, and verify after merge that the first release publishes @edx/edx-proctoring to npm with the correct version.

🤖 Generated with Claude Code

irfanuddinahmad

Before merging, a PyPI owner needs to add a trusted publisher for this repo with workflow release.yml. Earlier releases used an API token, so the first OIDC publish will fail without it.

Also needs a maintainer decision before merge: the v6.0.0 tag and GitHub release exist, but the PyPI and npm publish runs failed, so both are still on 5.2.0. Either re-cut 6.0.0 or accept that the first automated release will be 6.0.1/6.1.0.

irfanuddinahmad · 2026-10-13T17:28:37Z

+      - name: Create GitHub Release with Assets
+        if: steps.release.outputs.released == 'true'
+        env:
+          GH_TOKEN: ${{ secrets.REQUIREMENTS_BOT_GITHUB_TOKEN }}


Please move npm publishing into release.yml as a publish_to_npm job, as sample-plugin does, and delete publish-npm.yml. Then gh release create can go back to secrets.GITHUB_TOKEN; we don't use custom org tokens in release.yml.

irfanuddinahmad · 2026-10-13T17:28:37Z

+exclude conftest.py
+exclude .gitignore
+exclude tox.ini
+exclude CHANGELOG.rst


Please remove exclude CHANGELOG.rst from MANIFEST.in. The 5.2.0 sdist ships CHANGELOG.rst; dropping it from the long description is fine, but the file should keep shipping.

irfanuddinahmad · 2026-10-13T17:28:37Z

@@ -0,0 +1,84 @@
+name: Release


Please replace release.yml with sample-plugin's current release.yml, dropping only the monorepo lines (directory:, the tutor build step, the extra publish jobs, packages-dir:). The pins and logic already match, but the workflow name, step names, run: | style and comments differ; keeping it identical makes future syncs a plain diff.

irfanuddinahmad

This needs a rebase: master has 3 new commits (including the 6.1.1 bump), and uv lock --check fails on this branch, so please re-lock after rebasing.

irfanuddinahmad · 2026-10-16T13:42:53Z

-    runs-on: ${{ matrix.os }}
+    name: ${{ matrix.toxenv }}
+    runs-on: ubuntu-latest
    strategy:


Please restore fail-fast: false under strategy:. It was on master, and this migration should keep CI behaviour unchanged.

irfanuddinahmad · 2026-10-16T13:42:53Z

-      - name: Install requirements
-        run: pip install -r requirements/ci.txt
+      - name: Run tox
+        run: uv run tox -e ${{ matrix.toxenv }}


Please use uv run --locked tox -e ${{ matrix.toxenv }}. Without --locked, uv run silently re-locks when uv.lock is stale instead of failing. It also matches i18n-tools, codejail-includes and platform-plugin-aspects.

I think --locked with sync commands is enough.

farhan · 2026-10-20T09:43:30Z

@irfanuddinahmad The PR is available for the next pass.

irfanuddinahmad

LGTM, thanks!

irfanuddinahmad · 2026-10-21T11:36:23Z

Before merging, an npm owner of @edx/edx-proctoring also needs to add a trusted publisher (repo openedx/edx-proctoring, workflow release.yml), otherwise publish_to_npm fails on the first release.

feanil · 2026-10-29T19:30:40Z


      - name: Setup node
-        uses: actions/setup-node@f1f314fca9dfce2769ece7d933488f076716723e # v1.4.6
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0


This needs a rebase, it conflicts with master in this file and publish-npm.yml after #1344 moved actions/setup-node to v7.1.0. Taking master's setup-node pin here and keeping publish-npm.yml deleted should resolve it, release.yml already uses the newer pin.

Fixed in 78fa600. Rebased; ci.yml and release.yml use the setup-node v7.1.0 pin, publish-npm.yml stays deleted.

feanil · 2026-10-29T19:30:40Z

+# When running on a PR branch without the release tag, setuptools-scm returns
+# a dev version (e.g. "0.0.0.dev0"). Skip the check in that case — there is
+# nothing to validate until the release tag is reachable.
+if python_version.endswith('.dev0') or '.dev' in python_version:


With the release setting the npm version from the tag, package.json never gets bumped in git, so this check can't catch anything anymore. In CI it always skips:

Version check skipped (dev version detected: 0.0.0.dev0).

and locally, once there's a release tag after this lands, it fails for anyone with a full clone:

ERROR: Version mismatch...

Delete the script along with the version_check tox env and its entry in the CI matrix. Then fallback_version in pyproject.toml can go back to 0.0.0, since this script was the only thing that needed the .dev0.

Fixed in 78fa600. Removed version_check.py, its tox env and CI entry; fallback_version is back to "0.0.0".

feanil · 2026-10-29T19:30:40Z

+To install edx-proctoring (requires `uv <https://docs.astral.sh/uv/>`_), create and activate a virtualenv,
+then install. Run the ``make`` commands below with the virtualenv activated:


uv sync ignores an activated virtualenv unless it's .venv, so with any other venv active make install warns VIRTUAL_ENV=... does not match the project environment path .venv and will be ignored, creates .venv, and then ./manage.py migrate fails with ImportError: Couldn't import Django. Point people at uv venv and source .venv/bin/activate before make install instead.

Fixed in 78fa600. README now runs uv venv and source .venv/bin/activate before make install.

Move packaging to pyproject.toml with a src/ layout, uv dependency groups and python-semantic-release, per public-engineering#506. - The wheel keeps master's contents, including the test modules edx-platform imports. - publish-pypi.yml is renamed to release.yml and runs ci.yml before releasing, since ci.yml no longer runs on push to master. - Coverage measures src/edx_proctoring by path and uses */ omit globs, so the measured files match master. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

farhan mentioned this pull request Sep 24, 2026

Modernize Python repo | Django Apps & Plugins openedx/public-engineering#512

Open

25 tasks

farhan force-pushed the farhan/modernize-python-repo branch 4 times, most recently from 8bc6212 to 81b40aa Compare October 8, 2026 07:35

farhan marked this pull request as ready for review October 9, 2026 14:56

farhan force-pushed the farhan/modernize-python-repo branch 2 times, most recently from 69a6924 to 8761f6b Compare October 10, 2026 05:55

irfanuddinahmad reviewed Oct 13, 2026

View reviewed changes

farhan force-pushed the farhan/modernize-python-repo branch from e357d3c to 290a72e Compare October 16, 2026 13:31

irfanuddinahmad reviewed Oct 16, 2026

View reviewed changes

farhan force-pushed the farhan/modernize-python-repo branch 4 times, most recently from 64da666 to 62bfbb0 Compare October 20, 2026 08:14

farhan requested a review from irfanuddinahmad October 20, 2026 09:43

irfanuddinahmad approved these changes Oct 20, 2026

View reviewed changes

farhan force-pushed the farhan/modernize-python-repo branch from 62bfbb0 to 5b81113 Compare October 27, 2026 07:43

feanil reviewed Oct 29, 2026

View reviewed changes

farhan force-pushed the farhan/modernize-python-repo branch from 8580f27 to 78fa600 Compare October 30, 2026 06:25

farhan requested a review from feanil October 30, 2026 07:46

feanil changed the title ~~chore: modernize Python repo to uv + pyproject.toml~~ feat: modernize Python repo to uv + pyproject.toml Nov 3, 2026

farhan force-pushed the farhan/modernize-python-repo branch from 78fa600 to d2f2b2e Compare November 3, 2026 15:46

feanil merged commit c82548a into openedx:master Nov 3, 2026
14 checks passed

feanil mentioned this pull request Nov 3, 2026

feat: show the course escalation email on onboarding pages #1360

Merged

2 tasks

		To install edx-proctoring (requires `uv <https://docs.astral.sh/uv/>`_), create and activate a virtualenv,
		then install. Run the ``make`` commands below with the virtualenv activated:

Conversation

farhan commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Removed/Updated

Python 3.12 only

Versioning

Important Notes

Out of Scope

Testing Notes

Uh oh!

irfanuddinahmad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

farhan commented Oct 20, 2026

Uh oh!

irfanuddinahmad left a comment

Choose a reason for hiding this comment

Uh oh!

irfanuddinahmad commented Oct 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

farhan commented Sep 24, 2026 •

edited

Loading