Problem Description
When uploading benchmark results, SDGym generates dataset details that include row counts. For large datasets, this currently requires loading the dataset data, which can be slow and may crash due to memory usage (this was observed with rel-bench datasets).
Expected behavior
SDGym should compute accurate row counts for dataset details without loading the full dataset into pandas or saving the dataset locally. This should make benchmark result uploads more reliable for large datasets while preserving the existing Dataset_Details.xlsx output.
Additional context
This should be tested with rel-bench datasets.
Problem Description
When uploading benchmark results, SDGym generates
dataset detailsthat include row counts. For large datasets, this currently requires loading the dataset data, which can be slow and may crash due to memory usage (this was observed withrel-benchdatasets).Expected behavior
SDGym should compute accurate row counts for dataset details without loading the full dataset into pandas or saving the dataset locally. This should make benchmark result uploads more reliable for large datasets while preserving the existing
Dataset_Details.xlsxoutput.Additional context
This should be tested with rel-bench datasets.