Skip to content

[Feature] Add ASan/LSan coverage for Python bindings and C++ integration #1027

Closed

Description

@ColinLeeo

Problem

C++ sanitizer tests alone do not exercise native resource ownership through the Python bindings. The Python test suite needs an opt-in sanitizer build and CI job covering the complete Python-to-native call chain:

Python tests -> Cython extensions -> C wrapper -> C++ shared library.

This is particularly useful for detecting ownership mistakes across language boundaries, such as allocated return buffers that must be released by the binding, invalid frees, use-after-free, and result-buffer lifetime errors.

Proposed scope

  • Add a coordinated sanitizer build mode for the C++ shared library and all Python native extensions, including adapter sources.
  • Instrument both compilation and linking, preserving debug information and frame pointers. Check that optimization flags or extension-specific overrides do not discard sanitizer settings.
  • Add a Linux Python CI job with AddressSanitizer and LeakSanitizer enabled. Memory-leak detection must remain enabled, rather than relying only on ASan memory-access checks.
  • Use the same compiler/runtime for the shared library and extensions; document early runtime loading when using a standard Python interpreter.
  • Verify that tests import the sanitizer-built extensions and corresponding C++ shared library. Keep sanitizer artifacts separate from normal release wheels.
  • Exercise resource ownership in isolated subprocesses that terminate normally so that exit-time leak detection runs.

Suggested coverage, using synthetic test fixtures:

  • Tree and Table reads through both dataset and low-level result APIs.
  • Numeric values and STRING/TEXT/BLOB buffers, including nulls.
  • Full consumption, early termination, exception cleanup, repeated open/close, and reader/result lifetime interactions.
  • Arrow buffer ownership and release.

An import-only baseline can help identify interpreter or third-party noise. Any necessary suppressions should be narrowly scoped; do not suppress TsFile modules or disable leak detection globally.

Rebuilding CPython with ASan can be a later extension. The initial requirement is to instrument and test the Python bindings together with the C++ library.

Acceptance criteria

  • Build logs verify sanitizer instrumentation in the C++ library and every Python native extension.
  • A temporary positive-control memory error/leak causes the Python sanitizer job to fail with a useful allocation stack and nonzero exit status.
  • Ownership/lifetime regression tests run with leak detection enabled and fail when a corresponding defect is introduced.
  • Normal Python/C++ tests and release packaging remain unaffected when sanitizer mode is disabled.
  • Document build/run commands, runtime loading, supported platforms, symbolization, and suppression policy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions