💾 sqlite-shadow-sync

Safe, zero-lock, automatic background shadow backups for SQLite in Node.js. Safe for Write-Ahead Log (WAL) mode, with built-in compression, background watching, and integrity verification checks.

🎬 Demo

---

✨ Features

• 🔒 Safe WAL Backups: Monitors transaction states and locks on Write-Ahead Log (-wal) logs to copy the database safely without partial page transactions.
• 🚀 Zero-Lock / Non-Blocking: Does not lock the primary thread or block write requests on SQLite during operation.
• 🛠️ Active & Passive Synchronization:
  • Active: Integrates natively with better-sqlite3 and sqlite3 APIs for safe active connection page copies.
  • Passive: Automatically watches the filesystem for changes and runs safe background checkpoints without requiring an active connection handle.
• 📦 Gzip Compression: Streaming GZip compressor keeps memory usage at zero while saving up to 80% storage space.
• 🔄 Auto-Rotation: Cleans up older historical backups automatically (configurable maxBackups).
• 🛡️ Integrity Checks: Validates database signatures before committing each backup file, avoiding corrupted shadows.

---

📦 Installation

npm install sqlite-shadow-sync

(Optional peers: better-sqlite3 or sqlite3 for active backups)

---

🚀 Usage

1. File-Watcher / Passive Mode (Zero-Connection)

Runs passively in the background. It watches the SQLite .db file, waits for a silent period with no active write changes (the idle timeout), and backs it up safely.

import { SqliteShadowSync } from "sqlite-shadow-sync";

const shadow = new SqliteShadowSync({
  dbPath: "./data/prod.db",
  backupDir: "./backups",
  watch: true, // Enable filesystem watching
  idleTimeoutMs: 3000, // Wait 3s after last write before copying
  compress: true, // Gzip compression (.db.gz)
  maxBackups: 10, // Keep last 10 backups
  onBackupSuccess: (path) => console.log(`✓ Backup saved: ${path}`),
  onBackupFailure: (err) => console.error(`✗ Backup failed:`, err),
});

// Stop background watchers when application shuts down
// shadow.destroy();

2. Time-Interval Mode

Performs automated backups on a custom timing interval loop.

const shadow = new SqliteShadowSync({
  dbPath: "./data/prod.db",
  backupDir: "./backups",
  intervalMs: 1000 * 60 * 60 * 2, // Backup every 2 hours
  compress: true,
});

3. Active Connection Backup (Manual trigger)

Trigger backups manually or from server tasks. If you provide an active SQLite connection (e.g. from better-sqlite3), it leverages native database backup handlers.

import Database from "better-sqlite3";
import { SqliteShadowSync } from "sqlite-shadow-sync";

const db = new Database("prod.db");

const shadow = new SqliteShadowSync({
  dbPath: "./prod.db",
  backupDir: "./backups",
});

// Trigger backup using active DB connection handles
await shadow.backup({ dbConnection: db });

---

⚙️ API Configuration Reference

Option	Type	Default	Description
dbPath	string	Required	Absolute or relative path to the source SQLite database file.
backupDir	string	Required	Directory where shadow backup files are saved.
intervalMs	number	undefined	Triggers a backup at the specified interval in milliseconds.
maxBackups	number	10	Maximum backup count to maintain. Older files are automatically rotated out. (Set 0 for unlimited).
compress	boolean	true	Stream compress backup files using high-level gzip. (.db.gz)
watch	boolean	false	Enable standard FSWatcher to detect database file writes.
idleTimeoutMs	number	2000	Delay in milliseconds to wait for writes to quiet down before copying.

---

📄 License

MIT