MIDI to Piano Roll Video Converter

A powerful Python program that converts MIDI files into beautiful piano roll learning videos, complete with falling notes, colored keys, and synchronized audio.

Features

🎹 Piano Roll Visualization

88-key piano keyboard with proper white/black key positioning
Falling notes with customizable colors for each note
Real-time key press visualization
Professional-looking video output

🎵 Advanced MIDI Processing

Support for multi-track MIDI files
Automatic note extraction and timing
Handles complex polyphonic music
Color-coded notes by pitch (C=Red, D=Orange, E=Yellow-Green, etc.)

🎬 High-Quality Video Output

Customizable resolution (HD, Full HD, 4K)
Adjustable frame rate (30, 60 FPS)
Professional MP4 output with H.264 encoding
Optional audio synchronization

🔧 Flexible Configuration

Adjustable note falling speed and piano height
Multiple color modes (by note, by track, or single color)
Optional guide lines with octave markers showing piano key positions
Optional note names displayed on piano keys
Time indicator and custom corner text
Repeated note handling and end padding
Command-line interface for easy batch processing
Python API for programmatic usage

🔮 Future Development

Tempo detection and indicator
Time signature detection and indicator
Hand separation model/heuristics

Installation

Prerequisites

Python 3.7+
FFmpeg (for video encoding)
FluidSynth (optional, for MIDI to audio conversion)

Install Dependencies

# Clone or download this repository
cd midi2video

# Install Python dependencies
pip install -r requirements.txt

# Install system dependencies (Ubuntu/Debian)
sudo apt update
sudo apt install ffmpeg fluidsynth

# Install system dependencies (macOS with Homebrew)
brew install ffmpeg fluidsynth

# Install system dependencies (Windows)
# Download FFmpeg from https://ffmpeg.org/download.html
# Download FluidSynth from https://www.fluidsynth.org/

Quick Start

Command Line Usage

# Basic conversion
python midi2video.py your_song.mid

# Custom output filename
python midi2video.py your_song.mid -o my_video.mp4

# HD video with custom settings
python midi2video.py your_song.mid --width 1280 --height 720 --fps 60

# Faster note falling speed (does not affect tempo)
python midi2video.py your_song.mid --speed 300

# Generate without audio
python midi2video.py your_song.mid --no-audio

# Show guide lines for better note positioning visibility
python midi2video.py your_song.mid --guide-lines

# Show note names on piano keys for educational purposes
python midi2video.py your_song.mid --note-names

# Combine both features for maximum learning value
python midi2video.py your_song.mid --guide-lines --note-names

# Use different color modes
python midi2video.py your_song.mid --color-mode track    # Color by MIDI track
python midi2video.py your_song.mid --color-mode single   # Single color for all notes

# Customize piano height
python midi2video.py your_song.mid --piano-height 0.08   # Small piano (more falling area)
python midi2video.py your_song.mid --piano-height 0.20   # Large piano (better visibility)

Python API Usage

from midi2video import PianoRollVideoGenerator

# Create generator with all features
generator = PianoRollVideoGenerator(
    width=1920,
    height=1080,
    fps=30,
    falling_speed=200.0,      # Note falling speed in pixels/second
    piano_height_percent=0.20,  # 20% of video height for larger piano
    show_guide_lines=True,    # Enable guide lines
    show_note_names=True,     # Enable note names
    show_time_text=True,      # Enable time indicator
    color_mode="track",       # Color by MIDI track
    repeated_note_gap=0.05,   # Gap for repeated notes
    corner_text="Piano Tutorial",  # Custom corner text
    end_padding=2.0           # 2 seconds padding at end
)

# Load MIDI and generate video
generator.load_midi("your_song.mid")
generator.create_video("output.mp4")

Command Line Options

positional arguments:
  midi_file             Input MIDI file path

optional arguments:
  -h, --help            Show help message
  -o, --output OUTPUT   Output video file path
  --width WIDTH         Video width in pixels (default: 1920)
  --height HEIGHT       Video height in pixels (default: 1080)
  --fps FPS             Frames per second (default: 30)
  --speed SPEED         Note falling speed (does not affect tempo) in pixels/second (default: 200)
  --piano-height PIANO_HEIGHT
                        Height of piano keyboard as decimal of video height (default: 0.12)
  --audio AUDIO         Audio file to use instead of MIDI
  --soundfont SOUNDFONT SoundFont file for better audio quality
  --no-audio            Generate video without audio
  --guide-lines         Show subtle vertical guide lines for piano keys
  --note-names          Show note names on piano keys
  --color-mode {note,track,single}
                        Color mode: 'note' (by note name), 'track' (by MIDI track), 'single' (one color)
  --time-text           Show time indicator text on video
  --repeated-note-gap GAP
                        Gap duration in seconds for repeated notes (default: 0.05, use 0 to disable)
  --end-padding PADDING Additional seconds to add after the last note ends (default: 0.0)
  --corner-text TEXT    Text to display in the top-right corner of the video

Examples

Example 1: Basic Video

python midi2video.py classical_piece.mid

Creates classical_piece.mp4 with default settings.

Example 2: High Quality with Custom Audio

python midi2video.py jazz_song.mid \
  -w 1920 --height 1080 --fps 60 \
  --audio jazz_song.wav \
  -o jazz_hq.mp4

Example 3: Fast-Paced Video

python midi2video.py fast_piece.mid \
  --speed 400 \
  --fps 60

Example 4: Video with Guide Lines

python midi2video.py song.mid \
  --guide-lines \
  -w 1920 --height 1080

Creates a video with subtle vertical guide lines showing exactly where each piano key is positioned. C notes have darker lines to mark octave boundaries.

Example 5: Educational Video with Note Names

python midi2video.py song.mid \
  --note-names \
  --guide-lines \
  -w 1920 --height 1080

Creates a comprehensive learning video with both guide lines and note names displayed on the white piano keys.

Example 6: Multi-Track Color Visualization

python midi2video.py orchestral_piece.mid \
  --color-mode track \
  --guide-lines \
  -w 1920 --height 1080

Perfect for multi-instrument MIDI files - each track/instrument gets its own color, making it easy to follow different parts.

Example 7: Custom Piano Height

python midi2video.py song.mid \
  --piano-height 0.08 \
  --note-names \
  -w 1920 --height 1080

Creates a video with a smaller piano keyboard (8% of video height), giving more space for the falling notes area. Great for fast pieces or when you want to focus on the note patterns.

Example 8: Video with Time Text and Corner Text

python midi2video.py song.mid \
  --time-text \
  --corner-text "Piano Tutorial" \
  --end-padding 3.0

Creates a video with time indicator and custom corner text, plus 3 seconds of padding after the last note.

Example 9: Handle Repeated Notes

python midi2video.py song.mid \
  --repeated-note-gap 0.1 \
  --color-mode single

Creates a video with a 0.1-second gap between repeated notes and uses a single color for all notes.

Guide Lines Feature

When enabled with --guide-lines, the program displays subtle vertical reference lines:

Line Type	Color	Purpose
Regular boundaries	Light gray `(40,40,40)`	White key edges
Octave markers	Darker gray `(80,80,80)`	Left edge of C notes
Black key centers	Medium gray `(60,60,60)`	Center of black keys

The lighter lines at C notes create clear octave separations.

Color Modes Feature

The program offers three distinct color modes to suit different visualization needs:

🎵 Note Mode (Default: --color-mode note): Each note name gets its own color
🎼 Track Mode (--color-mode track): Each MIDI track/instrument gets its own color
🎨 Single Mode (--color-mode single): All notes use the same color

Audio Generation

The program can automatically generate audio from MIDI files using FluidSynth:

Default SoundFont: Uses system default
Custom SoundFont: Specify with --soundfont path/to/soundfont.sf2
External Audio: Use --audio to provide pre-recorded audio
No Audio: Use --no-audio for silent videos

Getting Better Audio Quality

For better audio quality:

Download a high-quality SoundFont (e.g., FluidR3_GM.sf2)

Use the --soundfont option:

python midi2video.py song.mid --soundfont FluidR3_GM.sf2

Troubleshooting

Common Issues

"FFmpeg not found"

Install FFmpeg and ensure it's in your PATH
On Windows, download from https://ffmpeg.org/

"FluidSynth failed"

Install FluidSynth: sudo apt install fluidsynth (Linux)
Use --no-audio to generate video without audio
Provide external audio with --audio

"MIDI file not supported"

Ensure the MIDI file is valid
Try with a different MIDI file
Check that the file contains piano notes in the range A0-C8

Getting MIDI Files

Free MIDI files can be found at:

Technical Details

Architecture

MIDI Parser: Uses pretty_midi for robust MIDI parsing
Video Renderer: OpenCV for frame generation
Video Encoder: MoviePy with FFmpeg backend
Audio Processing: FluidSynth for MIDI to audio conversion

Key Algorithms

Note Positioning: Calculates exact pixel positions for 88 piano keys
Falling Animation: Time-based physics simulation
Color Mapping: Musical note to color translation
Frame Generation: Real-time rendering pipeline

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues for:

Bug fixes
Performance improvements
New features
Better documentation

License

This project is released under the MIT License. See LICENSE file for details.

Support

If you encounter any issues or have questions:

Check the troubleshooting section above
Search existing issues on GitHub
Create a new issue with detailed information about your problem

Name		Name	Last commit message	Last commit date
.gitignore		.gitignore
README.md		README.md
midi2video.py		midi2video.py
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation