In attempting to fix phoneme non-contrasts in #398, I added a few new features (so far 5 of them). This has a substantial effect on the file size of data/phoible.csv, because the file has 105484 rows, so each new feature adds a minimum of ~827 kiB (more if there are lots of contour feature values in the new columns). I don't know how this snuck up on us, as GitHub usually warns about files bigger than 50 MB and rejects starting at 100 MB; in any case I can no longer push the aggregated data to that PR:
remote: error: File data/phoible.csv is 123.83 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
Possible fixes:
- use git-lfs or similar
- stop storing the raw data in the repo altogether, and distribute it only through the website. Anyone wanting to get unreleased version of the data must build it locally from source.
- start encoding 0 values as an empty string. Our own scripts and the scripts of users will need to be updated to continue working.
- don't store the data as one big CSV any more; use tables and build from tables with standard joins (AKA, back to how it was in the old days)
- something else?
thoughts @bambooforest?
In attempting to fix phoneme non-contrasts in #398, I added a few new features (so far 5 of them). This has a substantial effect on the file size of
data/phoible.csv, because the file has 105484 rows, so each new feature adds a minimum of ~827 kiB (more if there are lots of contour feature values in the new columns). I don't know how this snuck up on us, as GitHub usually warns about files bigger than 50 MB and rejects starting at 100 MB; in any case I can no longer push the aggregated data to that PR:Possible fixes:
thoughts @bambooforest?