Conversation
Add `svs_index_load()` and `svs_index_save()` API implementation for static Vamana index
Done: - [x] Create dynamic index with specified block size (default block size should be supported) - [x] Initialized with a dataset and labels list - [x] Add labeled vectors to a dynamic index - [x] Remove vectors by labels - [x] Check if a label exists - [x] Compute distance for label - [x] Get vector by label - [x] Consolidate/compact dynamic index - [x] Implement Save/Load
Adds `svs_index_get_num_threads` / `svs_index_set_num_threads` to the C API, enabling dynamic inspection and resizing of the search threadpool after index construction. ### ThreadPoolBuilder - Added `get_threads_num()` — delegates to the custom pool's `size()` op when `kind == CUSTOM`, otherwise returns the stored count - Added `resize(n)` — updates stored thread count; throws `std::invalid_argument` for `n == 0`, `SINGLE_THREAD`, or `CUSTOM` kinds (surfaced as `SVS_ERROR_INVALID_ARGUMENT` through `wrap_exceptions`) ### Index wrappers (`index.hpp`) - `Index` stores a `ThreadPoolBuilder`; `get_num_threads()` is pure-virtual — implemented in `IndexVamana` and `DynamicIndexVamana` by delegating to the wrapped `svs::Vamana` / `svs::DynamicVamana` instance, so the value reflects actual runtime state - `set_num_threads(n)` calls `pool_builder.resize(n)` then rebuilds and installs the threadpool via `set_threadpool()` ### C API (`svs_c.cpp` / `svs_c.h`) - Both entry points validate `index->impl` non-null before dereferencing (consistent with existing handle-check pattern) - Public header documents supported kinds and expected error codes for unsupported configurations
Resolve cmake version compatibility issue caused by using DOWNLOAD_EXTRACT_TIMESTAMP which is introduced in v.3.24 This PR fixes #317
…306) This pull request introduces a comprehensive C API test suite for the SVS project, leveraging the Catch2 testing framework. It adds new test files covering all major C API functionalities, integrates automated test building and execution into the CMake build system, and improves error handling and testability for dynamic index operations. **C API Test Infrastructure and Test Coverage:** * Added a new directory of C API tests using Catch2, with individual test files for error handling, algorithm configuration, storage, search parameters, index building, and dynamic index operations. **Dynamic Index Error Handling:** * Refactored `svs_index_dynamic_delete_points` to improve error handling.
- Introduced `svs_id_filter_interface` to define filtering operations. - Implemented `svs_index_search_topK` to support an optional ID filter for search operations. - Updated existing search functions to use the new filtered search capabilities. - Added a new source file `filtered_search.hpp` containing the logic for filtered top-K search. - Modified existing samples and tests to demonstrate and validate the new filtering functionality. - Marked the previous `svs_index_search` function as deprecated, directing users to use `svs_index_search_topK` instead.
…n) (#354) ## Summary Exposes memory accounting in the **C API** for the Valkey-search integration: - `svs_index_get_memory_usage(index, size_t* out_bytes, err)` — total allocated bytes. - `svs_index_get_memory_breakdown(index, svs_memory_breakdown_t* out, err)` — `{graph_bytes, data_bytes, metadata_bytes}` component split. ~~- `svs_index_element_size(index, size_t* out_bytes, err)` — bytes per stored vector.~~ (keep at data level) All follow the existing C API conventions (out-param + `svs_error_h`, `wrap_exceptions`), matching the Phase-A design in the memory-accounting contract (intel-innersource #333). ## Layers - **C API** (`bindings/c`): the three functions + `svs_memory_breakdown_t` in `svs_c.h`; interface virtuals + concrete overrides in `src/index.hpp`; impls in `src/svs_c.cpp`. - **Core / orchestrator**: brings in `get_memory_breakdown()` (`MemoryBreakdown` struct + capacity-based `svs::data::detail::dataset_allocated_bytes` helper) on `VamanaIndex` / `MutableVamanaIndex` and through the orchestrator, plus an `element_size()` accessor parallel to `dimensions()`. This mirrors the approved public PR #345 so the C API can build and test standalone; once #345 lands on `dev/c-api`, this reduces to just the C API layer. ## Tests `bindings/c/tests/c_api_index.cpp` (static) and `c_api_dynamic_index.cpp` (dynamic): usage > 0, breakdown total == usage, `graph_bytes`/`data_bytes` > 0 (metadata > 0 for dynamic), `element_size == sizeof(float) * dimensions`, and null-arg handling. Both test cases pass (84 / 166 assertions). Related: builds on #345; memory-accounting contract in intel-innersource #333 / #326.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scalable Vector Search C API bindings.
This pull request introduces a new C API binding for the project, providing C language access to the core Scalable Vector Search (SVS) functionality. The changes add a complete build system for the C API, define the public C API headers, and include a sample program to demonstrate usage. The most important changes are grouped below:
C API Design Document
bindings/SVS_C_API_Design.mddocument which describes the design proposal for the Scalable Vector Search (SVS) C API including: architecture overview, core components design, naming conventions, usage rules, draft API reference, etc.C API Implementation and Build System:
bindings/cdirectory with aCMakeLists.txtto build the shared librarysvs_c_api, set up installation rules, and link dependencies such as OpenMP and the core SVS library.CMakeLists.txtto include the new C API bindings in the build process.c_apiConfig.cmake.in) for downstream projects to find and use the C API library.C API Public Headers:
svs_c.handsvs_c_config.hinbindings/c/include/svs/c_api/, defining the C API's types, enums, opaque handles, and functions for error handling, algorithm configuration, storage configuration, index building, searching, and result management. [1] [2]Samples and Demonstration:
samplesdirectory with aCMakeLists.txtto build a simple example (c_api_simple) demonstrating how to use the new C API.