Activity

1. marcverhagen

   added

   ✨NNew feature or requestNew feature or request

   on Oct 6, 2025
2. clams-bot

   added this to infraon Oct 6, 2025
3. github-project-automation

   moved this to Todo in infraon Oct 6, 2025
4. clams-bot

   assigned

   keighrim

   on Oct 7, 2025
5. keighrim commented on Oct 8, 2025

   

   keighrim

   on Oct 8, 2025

   Member

   More actions

   more documentation needed for details like

   • what's "generated" when doing make package ?
   • how to upload to pypi?
   • how the documentation (gh-pages) is published?
6. keighrim commented on Oct 8, 2025

   

   keighrim

   on Oct 8, 2025

   Member

   More actions

   On top of the additional documentation needs, @marcverhagen and I just had a conversation on migrating setup.py+Makefile to pyproject.toml+python -m build workflow. I'll experiment with the idea within the open PR (#329)
7. keighrim commented on Oct 29, 2025

   

   keighrim

   on Oct 29, 2025

   Member

   More actions

   Here's some finding and proposal from my research into the migration problem

   ---

   migration scenario: from setup.py to pyproject.toml

   summary

   Do we need setup.py?
   With requires-python = ">=3.10", setup.py is technically unnecessary but pragmatically useful during migration.

   All Python 3.10+ users have modern pip and setuptools that can build packages directly from pyproject.toml using PEP 517/518 standards. However, maintaining setup.py temporarily supports legacy workflows and developer habits while the ecosystem transitions to modern build tools.

   general assessment of the current python packaging landscape

   Python 3.10 was released in October 2021 (4yr ago), which means:

   1. pip
      • Python 3.10+ distributions ship with pip >= 21.x (pip install . and pip install -e .)
      • pip 19.0 (January 2019) added PEP 517/518 support (python -m build support )
      • All Python 3.10+ users have pip that can build from pyproject.toml alone
   2. setuptools
      • setuptools 61.0 (March 2022) added full pyproject.toml metadata support
      • Python 3.10 users can easily install modern setuptools
      • We can declare requires = ["setuptools>=61.0", ...] to make sure the build environment is correct

   why we kept using setup.py

   Despite Python 3.10+ making setup.py technically unnecessary, this project continues to use it for the following reasons:

   1. GitHub Actions CI Pipeline Integration

      • Our CI/CD pipeline uses Makefile which internally calls python setup.py sdist and related commands
      • Migrating the entire build pipeline would require updating multiple workflow files and testing
      • The CI Pipeline is also shared with clams-python project, so when one migrates, both need to be updated simultaneously

   2. Developer Habits and Familiarity

      • Simply, I'm accustomed to using python setup.py develop for development installations
      • Existing documentation and scripts reference setup.py when they do (although as the original body of this issue goes, documentation rarely says anything about actually running make or setup.py)

   3. Build-Time Code Generation (Most Critical)

      • The vocabulary package and other python code must be generated at build time by fetching and processing external vocabulary definitions
      • This code generation logic is currently implemented in setup.py using custom command classes
      • Setup.py's prep_ext_files decorator hooks into setuptools' sdist and develop commands to execute generation logic before packaging
      • Migrating this would require implementing a custom build backend or setuptools plugin

   So the main roadblock to removing setup.py is the build-time code generation logic. If we can migrate that logic to somehow work with pyproject.toml, we can safely remove setup.py, and replace setup.py + Makefile with pyproject.toml + python -m build commands.

   migration plan (using setuptools's "entry points")

   Here's the result of my research on the migration path.

   1. Setuptools provides a finalize_distribution_options entry point specifically designed for pre-build hooks. Looks like it's the recommended approach for running code generation before packaging.

   2. We add entry point to pyproject.toml:

      [project.entry-points."setuptools.finalize_distribution_options"]
      build_hooks = "build_tools.hooks:setup_hooks"

   3. Separately implement the hook function in somewhere like build_tools/hooks.py:

      def setup_hooks(dist):
          """
          Called by setuptools during build preparation.
      
          This function runs before building wheels/sdists and can:
          - Generate Python code (vocabulary package)
          - Fetch external resources
          - Validate build environment
      
          :param dist: setuptools Distribution object
          """
          from build_tools.generation import generate_vocabulary
      
          generate_vocabulary()
          # Other build-time generation logic...

   References:

   • Setuptools Extension Points
   • Entry Points Specification
   • PEP 517 - Build System Interface

   This approach eliminates the need for setup.py while maintaining the same build-time code generation functionality.
8. keighrim commented on Apr 17, 2026

   

   keighrim

   on Apr 17, 2026

   Member

   More actions

   The PR #381 supersedes the plan drafted in the comment above. Instead of a minimal setup.py-shim migration, it executes the full pyproject.toml + setuptools-scm conversion: vocabulary moves to the clams-vocabulary PyPI dependency (no more spec-YAML codegen, and it helped a lot to depart from complex setup.py implementation), build-time schema fetching is replaced by a committed mmif/res/mmif.json + test-time drift detection, and specver is also maintained in pyproject ([project.urls]) . See updated CONTRIBUTING.md for the current build/release workflow: this should comprehensively address the issue originally raised.
9. keighrim

   closed this as completedin #381on Apr 22, 2026
10. github-project-automation

    moved this from Todo to Done in infraon Apr 22, 2026
11. clams-bot

    unassigned

    keighrim

    on Apr 22, 2026