SheetSage-Pro

Bilingual lyric-aligned lead-sheet transcription from music audio.

SheetSage-Pro takes a song (a local audio file or a YouTube link) and produces an editable lead sheet: melody, chord symbols, bar lines from detected downbeats, section labels, and Mandarin Chinese or English lyrics aligned to the notes they are sung on. Output formats are JSON, MusicXML and MIDI.

The pipeline chains pretrained models for each task and integrates their outputs on one metrical grid:

Stage	Model
Source separation	Mel-Band RoFormer
Beat / downbeat tracking	Beat This!
Structure and metre	All-In-One (+ SongPrep labels)
Melody transcription	GAME
Chord recognition	BTC (default), ISMIR2019-LVCR, or ChordMini BTC-CL
Lyric transcription	SongPrep
Lyric alignment	Phonsa (Mandarin), SOFA (English)

Installation

Requires Linux, conda, and a CUDA GPU (about 12 GB VRAM).

git clone https://github.com/pingw220/sheetsage-pro.git
cd sheetsage-pro

# 1. Create the environment and install the CLI
conda env create -f envs/environment.yml          # -> sheetsage-pro
conda activate sheetsage-pro
pip install -e .

# 2. External repositories and checkpoints
# Clone each model repository under external/ and download its checkpoint into
# checkpoints/ as listed in configs/tool_paths.yaml.

# 3. Provision the internal model environments (one command)
bash scripts/setup_envs.sh
bash scripts/verify_envs.sh

You only ever work inside sheetsage-pro. The external models run in private conda environments that setup_envs.sh creates, because their dependency pins conflict (for example All-In-One requires CUDA 11.8 builds while Beat This! requires torch 2.5); the pipeline switches to them per stage automatically. configs/tool_paths.yaml is the single place that says where each external repository and environment lives; edit it to match your machine.

Usage

From a YouTube link:

sheetsage-pro run-youtube "https://www.youtube.com/watch?v=..." --language zh   # Mandarin song
sheetsage-pro run-youtube "https://www.youtube.com/watch?v=..." --language en   # English song

From a local audio file, optionally with your own lyrics (skips transcription):

sheetsage-pro run-audio song.wav --language en --lyrics lyrics.txt --out outputs/song

Batch mode (many songs, models loaded once)

Single-song runs reload every stage's model, so most of their wall time is loading. For collections, the stage-batched runner keeps one process per stage and loads each model once; the melody-and-chords path then takes well under a minute per song on one GPU.

Write a manifest CSV with example_id and wav_path columns (one row per song), then run the stages in order over the same work directory:

M=my_songs.csv; W=work/my_batch
python scripts/batch/run_pro_component.py --task melody --stage prepare --manifest $M --work-root $W
python scripts/batch/run_pro_component.py --task melody --stage beat    --manifest $M --work-root $W
python scripts/batch/run_pro_component.py --task melody --stage game    --manifest $M --work-root $W --language zh
python scripts/batch/run_pro_component.py --task melody --stage lvcr    --manifest $M --work-root $W
python scripts/batch/run_pro_component.py --task melody --stage post    --manifest $M --work-root $W --chord-backend lvcr

Predictions land in $W/pred/<variant>/<example_id>.json with the raw, pitch-repaired and quantised melody variants side by side, plus the chord track. Stages are shardable (--shard-index/--num-shards) and resumable (--resume). Lyric transcription also has a batch path that loads the 7B model once for a whole job list:

python scripts/run_songprep.py --jobs-json jobs.json --output-json out.json --task all
# jobs.json: [{"song_id": "a", "audio": "a.wav"}, ...]

Each run writes an exports/ folder:

exports/leadsheet++.json    # full structured lead sheet
exports/lead_sheet.mid      # MIDI (melody, chords, click)
exports/lead_sheet.musicxml # MusicXML
exports/lyrics.txt          # lyrics used
exports/run_report.md       # per-stage report

--language selects the lyric aligner (zh -> Phonsa, en -> SOFA); there is no automatic language detection.

Demo

demo/ contains two complete runs, one per language, produced with the exact commands shown in their READMEs:

• demo/zh_love_confession/ -- Jay Chou, 告白氣球 (Love Confession)
• demo/en_blinding_lights/ -- The Weeknd, Blinding Lights

Evaluation

On public benchmarks against Sheet Sage (same audio, same scorer): note F1@50ms on GTSinger 0.112 -> 0.295, maj/min chord WCSR on GuitarSet 0.673 -> 0.904. Lyric alignment on GTSinger: Mandarin syllable onsets within a median 24 ms, English word onsets within 33 ms. Details are in the ISMIR 2026 late-breaking demo paper.

License

MIT (see LICENSE). The wrapped external models, repositories and checkpoints (Mel-Band RoFormer, Beat This!, All-In-One, GAME, the chord recogniser, SongPrep, Phonsa, SOFA, and their weights) are downloaded separately by the setup scripts and keep their own licenses; consult each project before commercial use.