diff --git a/Other/swap_out-loop.md b/Other/swap_out-loop.md new file mode 100644 index 0000000..d342fa3 --- /dev/null +++ b/Other/swap_out-loop.md @@ -0,0 +1,180 @@ +# Economically Optimized Lightning Loop Out Tool (`swap_out-loop.py`) + +## 1. Overview + +`Other/swap_out-loop.py` is an interactive CLI tool designed to help Lightning Network node operators rebalance channels with high local liquidity by executing the most economically advantageous **Loop Out** submarine swaps via Lightning Labs Loop (`litloop` / `loopd`). + +Rather than simply finding channels with high local balances, `swap_out-loop.py` scores and ranks candidate channels based on their **Total Economic Cost per sat**, incorporating: +1. **Loop Server Service Fee**: The fee charged by Lightning Labs Loop for the swap. +2. **On-Chain Sweep Fee**: The miner fee required to sweep the on-chain HTLC into your LND wallet (or custom cold-storage address). +3. **Off-Chain LN Routing Fee**: The actual network fees incurred across intermediate hops to route payments to the Loop server node (`021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d`). +4. **Opportunity Cost (Foregone Profit)**: Draining a channel with a high local fee rate (e.g. 500 ppm) forfeits future routing profits, whereas draining a low-fee channel (e.g. 5 ppm) preserves profit margins. + +--- + +## 2. Key Features + +- **Hybrid Sizing**: + - Specify an explicit target amount via `--amt ` (e.g., `--amt 2000000`). + - Omit `--amt` to let the tool dynamically calculate the exact drain amount needed per channel to bring it down to a healthy equilibrium (default: 50% local ratio), respecting Loop's limits (250,000 to 240,000,000 sats). +- **Two-Stage Route Validation & Real Liquidity Probing**: + - **Stage 1 (`lncli queryroutes`)**: Fast graph traversal to verify route existence and compute theoretical off-chain hop fees to the Loop server node. + - **Stage 2 (Mandatory Live Prepay Probe)**: Probes the route with a random 32-byte fake hash via `lncli sendpayment`. Proves actual downstream liquidity without risking funds. Ghost liquidity paths from gossip graphs that fail live probing are automatically filtered out. + - **Direct 2-Hop Route Fallback**: If multi-hop routes fail due to intermediate bottlenecks or dead gossip nodes, the tool automatically checks if the candidate peer maintains a direct channel to the Loop node (`021c97a9...`), constructing the route via `lncli buildroute` and probing it with `lncli sendtoroute` to accurately capture the exact verified route fee. +- **Concurrent Worker Probing (`--workers `)**: + - Probes candidates sequentially by default for zero downstream HTLC collision, or in parallel via `--workers 2` using `ThreadPoolExecutor` with thread-safe output formatting, cutting scanning time in half. +- **Interactive Terminal UI (Arrow-Key Navigation)**: + - Native Python implementation (`termios` and `tty`) with zero Node/npm/npx dependencies. + - Navigate candidates with `↑` / `↓` (or `k` / `j`) arrow keys, showing real-time highlighted selection and cost breakdown. + - Press `[Enter]` to select and confirm, or `[q]` / `[Esc]` to abort. + - Automatically falls back to a clean PrettyTable and numbered prompt in non-interactive/piped environments. +- **Economical Sweep Timing**: + - Enforces a minimum confirmation target of 6 blocks (default: 9) to prevent overpaying for fast on-chain sweeps. + - Omits `--fast` so Loop's swap server batches the on-chain HTLC publication, reducing chain fees. +- **Native Single Source of Truth (SOT) Accounting**: + - Eliminates custom redundant SQLite databases by directly leveraging Loop daemon's native database (`~/.loop/mainnet/loop_sqlite.db`) as the authoritative Source of Truth. + - Configurable via `loop_db_path` in `config.ini` (with automatic fallback to `litloop listswaps` RPC). + - Swaps are automatically tagged on initiation with `--label "Loop-Out: ()"` to preserve peer and channel metadata directly in `loop.db`. + - Displays historical swaps, settled costs, and realized PPM with `python Other/swap_out-loop.py --history`. + - Supports spreadsheet bookkeeping via `--history --csv [optional_path]`. +- **Interactive Monitoring with Safe Detach**: + - Launches `litloop monitor` directly in the foreground. + - Node operators can press `[Ctrl+C]` at any time to detach without aborting the swap; background `loopd` continues processing automatically. + +--- + +## 3. Prerequisites & Setup + +### Debian NUC Configuration +The tool is designed to run directly on the node host (e.g., Debian NUC) where `lnd`, `litd` (or `loopd`), and `LNDg` are configured: +1. **LND**: Accessible via `lncli` or configured via `[lnd]` in `config.ini`. +2. **Loop Daemon**: Running via Lightning Terminal (`litd`) or standalone `loopd`. + - On Debian NUC, the user alias is: + ```bash + alias litloop="loop --rpcserver=localhost:8443 --tlscertpath=~/.lit/tls.cert" + ``` + - `swap_out-loop.py` automatically detects and uses `litloop` or falls back to `loop` with `~/.lit/tls.cert`. +3. **LNDg**: Channel balances and fee rates are queried via LNDg REST API. + +### Configuration (`config.ini`) +Ensure the following sections are configured in `../config.ini`: +```ini +[credentials] +lndg_username = your_lndg_username +lndg_password = your_lndg_password + +[lndg] +lndg_api_url = http://localhost:8889 + +[no-swapout] +# Add pubkeys you want to filter out for swapout outputs, comma-separated +swapout_blacklist = pubkey1,pubkey2 + +[loop] +# Path or alias to litloop/loop command. Defaults to litloop or loop +loop_command = litloop +# Path to loop's local SQLite database (Source of Truth for history and swap accounting) +loop_db_path = ~/.loop/mainnet/loop_sqlite.db +# Public key of the Lightning Labs Loop server node +loop_pubkey = 021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d +# Default confirmation target for sweep transaction (minimum 6 for economical sweep) +conf_target = 9 +# Default maximum local fee rate (ppm) to consider for looping out (opportunity cost cap) +max_local_fee_ppm = 100 +# Minimum local balance ratio (percentage) on candidate channel to consider +min_local_balance_ratio = 60 +# Minimum channel capacity (sats) +min_capacity = 3000000 +# Target equilibrium local balance ratio (percentage) for dynamic sizing +target_local_ratio = 50 +# Number of concurrent workers for route probing (default: 1, recommendation: 1-2) +workers = 1 +# Timeout in seconds for individual route prepay probes (default: 15) +probe_timeout = 15 +``` + +--- + +## 4. Usage & Command-Line Flags + +```bash +# Basic run with dynamic sizing and interactive arrow-key selection +python3 Other/swap_out-loop.py + +# Specify an explicit swap amount (e.g., 5,000,000 sats) +python3 Other/swap_out-loop.py --amt 5000000 + +# Fast parallel route evaluation with 2 workers +python3 Other/swap_out-loop.py --amt 8000000 --workers 2 + +# Dry-run simulation (probes routes and fetches live quotes without spending funds) +python3 Other/swap_out-loop.py --dry-run + +# Custom thresholds (minimum 5M capacity, max 50 ppm local fee, 70% min local ratio) +python3 Other/swap_out-loop.py --capacity 5000000 --fee-limit 50 --min-ratio 70 + +# Sweeping to an external cold-storage address with custom 12-block confirmation target +python3 Other/swap_out-loop.py --amt 3000000 --conf-target 12 --dest-addr bc1q... + +# Non-interactive / headless automation (automatically selects top-ranked candidate) +python3 Other/swap_out-loop.py --amt 2000000 --auto-approve + +# View historical loop-out operations from Loop's SOT database +python3 Other/swap_out-loop.py --history --limit 15 + +# Export historical swaps directly to CSV +python3 Other/swap_out-loop.py --history --csv +``` + +### CLI Flag Reference + +| Flag | Type | Default | Description | +|---|---|---|---| +| `--amt` | `int` | `None` | Specific amount in sats to loop out (`250,000` to `240,000,000`). If omitted, dynamically calculated. | +| `-c`, `--capacity` | `int` | Config / `3M` | Minimum channel capacity in sats. | +| `-f`, `--fee-limit` | `int` | Config / `100` | Maximum local outbound fee rate in ppm. | +| `-r`, `--min-ratio` | `float` | Config / `60%` | Minimum local balance ratio percentage. | +| `--conf-target` | `int` | Config / `9` | Confirmation target for on-chain sweep (minimum `6`). | +| `-w`, `--workers` | `int` | Config / `1` | Number of concurrent worker threads for route probing (recommended: `1` or `2`). | +| `--probe-timeout` | `int` | Config / `15` | Timeout in seconds for individual route prepay probes. | +| `--skip-prepay-probe` | `flag` | `False` | Skip active prepay probing and rely on queryroutes theoretical fees (not recommended). | +| `--max-routing-fee`| `int` | Auto buffer | Upper limit on off-chain routing fees in satoshis. | +| `--dest-addr` | `str` | LND wallet | Custom destination address for swept on-chain funds. | +| `--dry-run` | `flag` | `False` | Simulates candidate selection, live quotes, and route probes without executing. | +| `--auto-approve` | `flag` | `False` | Automatically executes the top-ranked candidate without interactive prompt. | +| `--history` | `flag` | `False` | Prints table of historical swaps from Loop's Source of Truth database. | +| `--csv` | `str` | `None` | Export history to CSV file (defaults to `data/loop_out_history.csv` if no path provided). | +| `--limit` | `int` | `30` | Maximum number of historical swaps to display with `--history`. | +| `-p`, `--pubkey` | `flag` | `False` | Shows remote node pubkeys in candidate table. | + +--- + +## 5. Economic Scoring Formula + +For each candidate channel routing amount $A$: +$$\text{Loop Fee} = \text{Server Fee} + \text{On-Chain Sweep Fee}$$ +$$\text{Routing Fee} = \text{LN Hop Fees to Loop Node}$$ +$$\text{Opportunity Cost} = \text{round}\left(A \times \frac{\text{Local Fee Rate (ppm)}}{1,000,000}\right)$$ +$$\text{Total Cost} = \text{Loop Fee} + \text{Routing Fee} + \text{Opportunity Cost}$$ +$$\text{Net Effective PPM} = \text{round}\left(\frac{\text{Total Cost}}{A} \times 1,000,000\right)$$ + +Candidates are sorted ascending by **Net Effective PPM**. + +--- + +## 6. Accounting & Data Persistence (Source of Truth) + +Rather than keeping a separate desynchronized database, swap records are read directly from Loop daemon's native SQLite database: +- **Location**: `~/.loop/mainnet/loop_sqlite.db` (configurable via `loop_db_path` in `config.ini`). +- **RPC Fallback**: If the database file is not directly mounted or accessible, queries automatically fall back to `litloop listswaps`. + +Each swap initiated by `swap_out-loop.py` is tagged with `--label "Loop-Out: ()"` to preserve full channel and peer metadata in Loop's own database. + +To view past operations: +```bash +# Formatted table view +python3 Other/swap_out-loop.py --history + +# Export to CSV spreadsheet +python3 Other/swap_out-loop.py --history --csv +``` diff --git a/Other/swap_out-loop.py b/Other/swap_out-loop.py index 0d4e2a6..66bdb00 100644 --- a/Other/swap_out-loop.py +++ b/Other/swap_out-loop.py @@ -1,247 +1,1379 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +""" +swap_out-loop.py: Economically optimized Loop Out liquidity rebalancing via Lightning Labs Loop. + +Features: +1. Discovers channel candidates with high local liquidity and low outbound fees via LNDg API. +2. Supports hybrid sizing: explicit --amt or dynamic per-channel equilibrium calculations. +3. Two-stage route validation: lncli queryroutes (graph fee evaluation) followed by + prepay probing (lncli sendpayment with a fake hash) to prove live downstream liquidity. +4. Total economic cost scoring: Loop server fee + on-chain sweep fee + off-chain routing fee + + foregone local routing revenue (opportunity cost ppm). +5. Interactive arrow-key CLI terminal menu (termios/tty) for intuitive candidate selection. +6. Execution via litloop / loop with minimum 6-block conf_target for economical sweeping. +7. Foreground litloop monitor streaming with safe Ctrl+C detachment. +8. Persistent accounting record store in SQLite (data/loop_out_history.db) with auto-synced CSV export. +""" + import os -import requests +import sys import json +import time +import datetime +import math +import binascii +import subprocess +import argparse import configparser +import sqlite3 +import csv +import logging +from typing import List, Dict, Any, Tuple, Optional +import requests from prettytable import PrettyTable -import argparse -# Get the path to the parent directory -parent_dir = os.path.dirname(os.path.abspath(__file__)) - -# Construct the path to the config.ini file -config_file_path = os.path.join(parent_dir, "..", "config.ini") -config = configparser.ConfigParser() -config.read(config_file_path) - -# API endpoint URL -api_url = config["lndg"]["lndg_api_url"] + "/api/channels?limit=500&is_open=true" - -# Authentication credentials -username = config["credentials"]["lndg_username"] -password = config["credentials"]["lndg_password"] - -# File path for storing data. This export can be used for swap-out -file_path = os.path.join(parent_dir, "..", "data", "low-fee-high-local.log") -# File path for storing BOS tags. Create symlink to homedir with ln -s ~/.bos bos -file_path_to_bos = os.path.join(parent_dir, "..", "bos", "tags.json") - -# Remote pubkey to ignore. Add pubkey or reference in config.ini if you want to use it. -ignore_remote_pubkeys = config["no-swapout"]["swapout_blacklist"].split(",") - -parser = argparse.ArgumentParser(description="Script to manage swap-out candidates.") -parser.add_argument( - "-b", - "--bos", - action="store_true", - help="Export bos tags.json file for easy probing.", -) -parser.add_argument( - "-e", - "--file-export", - action="store_true", - help="Write into defined file.log for easy pickup of swap-out automations like litd.", -) -parser.add_argument( - "-p", - "--pubkey", - action="store_true", - help="Show remote pubkey instead of channel ID in the table.", -) -parser.add_argument( - "-c", - "--capacity", - type=int, - default=5000000, - help="Set the capacity threshold for swap-out candidates.", -) -parser.add_argument( - "-f", - "--fee-limit", - type=int, - default=60, - help="Maximum local fee rate for swap-out candidates.", -) -args = parser.parse_args() - -# Set the CAPACITY_THRESHOLD based on the parsed argument -CAPACITY_THRESHOLD = args.capacity - - -def get_all_channels_info(): - all_channels_info = [] +# Global Loop Constants +LOOP_PUBKEY_DEFAULT = "021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d" +MIN_LOOP_OUT_SATS = 250_000 +MAX_LOOP_OUT_SATS = 240_000_000 +DEFAULT_CONF_TARGET = 9 +MIN_CONF_TARGET = 6 + +# ANSI Color Codes +class Colors: + HEADER = "\033[95m" + OKBLUE = "\033[94m" + OKCYAN = "\033[96m" + OKGREEN = "\033[92m" + WARNING = "\033[93m" + FAIL = "\033[91m" + ENDC = "\033[0m" + BOLD = "\033[1m" + HIGHLIGHT = "\033[1;30;46m" # Bold black on cyan background for menu cursor + + +def print_color(text: str, color_code: str = "", bold: bool = False): + """Prints colorized text to stdout.""" + if bold: + print(f"{color_code}{Colors.BOLD}{text}{Colors.ENDC}") + else: + print(f"{color_code}{text}{Colors.ENDC}") + + +def setup_logger(project_root: str) -> logging.Logger: + """Sets up rotating file logger.""" + logger = logging.getLogger("swap_out_loop") + logger.setLevel(logging.INFO) + logs_dir = os.path.join(project_root, "logs") + os.makedirs(logs_dir, exist_ok=True) + log_file = os.path.join(logs_dir, "swap_out-loop.log") + + if not logger.handlers: + fh = logging.FileHandler(log_file) + formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") + fh.setFormatter(formatter) + logger.addHandler(fh) + return logger + + +def load_config() -> Tuple[configparser.ConfigParser, str]: + """Loads config.ini from parent directory.""" + parent_dir = os.path.dirname(os.path.abspath(__file__)) + project_root = os.path.dirname(parent_dir) + config_file_path = os.path.join(project_root, "config.ini") + + config = configparser.ConfigParser() + if os.path.exists(config_file_path): + config.read(config_file_path) + return config, project_root + + +def run_command( + command_args: List[str], + timeout: int = 60, + expect_json: bool = False, + dry_run: bool = False, + dry_run_output: str = "", +) -> Tuple[bool, Any, Optional[str]]: + """Runs a subprocess command securely with error handling.""" + if dry_run: + if expect_json: + return True, (dry_run_output if isinstance(dry_run_output, dict) else {}), None + return True, dry_run_output, None + try: - response = requests.get(api_url, auth=(username, password)) - if response.status_code == 200: - data = response.json() - if "results" in data: - results = data["results"] - for result in results: - channel_info = { - "chan_id": result.get("chan_id", ""), - "local_balance": result.get("local_balance", 0), - "capacity": result.get("capacity", 0), - "remote_pubkey": result.get("remote_pubkey", ""), - "local_fee_rate": result.get("local_fee_rate", 0), - } - all_channels_info.append(channel_info) - else: - print(f"API request failed with status code: {response.status_code}") + process = subprocess.run( + command_args, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + timeout=timeout, + check=False, + ) + stdout = process.stdout.strip() + stderr = process.stderr.strip() + + if process.returncode != 0: + err_msg = stderr or stdout or f"Command failed with code {process.returncode}" + if expect_json and stdout: + try: + data = json.loads(stdout) + return False, data, err_msg + except json.JSONDecodeError: + pass + return False, stdout, err_msg + + if expect_json: + try: + data = json.loads(stdout) + return True, data, None + except json.JSONDecodeError as jde: + return False, stdout, f"JSON parse error: {jde}" + + return True, stdout, None + + except subprocess.TimeoutExpired: + return False, None, f"Command timed out after {timeout}s: {' '.join(command_args)}" except Exception as e: - print(f"Error: {e}") - return all_channels_info + return False, None, f"Execution error: {e}" + + +def get_config_val(config: Any, section: str, option: str, fallback: Any = "") -> Any: + """Helper to safely read from ConfigParser or dict.""" + if isinstance(config, configparser.ConfigParser): + return config.get(section, option, fallback=fallback) + elif isinstance(config, dict): + sec = config.get(section, {}) + if isinstance(sec, dict): + return sec.get(option, fallback) + return config.get(option, fallback) + return fallback + + +def get_lnd_connection_params(config: Any) -> List[str]: + """Extracts LND connection parameters from config for lncli.""" + params = [] + rpc = get_config_val(config, "lnd", "rpcserver", "").strip() + tls = get_config_val(config, "lnd", "tlscertpath", "").strip() + mac = get_config_val(config, "lnd", "macaroonpath", "").strip() + if rpc: + params.append(f"--rpcserver={rpc}") + if tls: + params.append(f"--tlscertpath={os.path.expanduser(tls)}") + if mac: + params.append(f"--macaroonpath={os.path.expanduser(mac)}") + return params + + +def resolve_loop_command(config: configparser.ConfigParser) -> List[str]: + """Resolves loop / litloop command line structure.""" + loop_cmd = get_config_val(config, "loop", "loop_command", "litloop").strip() + + # If litloop is installed or aliased + if loop_cmd == "litloop": + # Check if litloop binary exists in path + if subprocess.run(["which", "litloop"], stdout=subprocess.PIPE, stderr=subprocess.PIPE).returncode == 0: + return ["litloop"] + # Check if loop is in PATH + if subprocess.run(["which", "loop"], stdout=subprocess.PIPE, stderr=subprocess.PIPE).returncode == 0: + # Fallback to loop binary with litd connection settings + tlscert = os.path.expanduser("~/.lit/tls.cert") + if os.path.exists(tlscert): + return ["loop", "--rpcserver=localhost:8443", f"--tlscertpath={tlscert}"] + return ["loop"] + return ["litloop"] + + # If custom command specified + return loop_cmd.split() + +def fetch_channels_lndg(config: configparser.ConfigParser) -> List[Dict[str, Any]]: + """Fetches open active channels from LNDg API.""" + if not config.has_section("lndg") or not config.has_section("credentials"): + return [] -def terminal_output(): + lndg_url = config.get("lndg", "lndg_api_url", fallback="http://localhost:8889").rstrip("/") + username = config.get("credentials", "lndg_username", fallback="") + password = config.get("credentials", "lndg_password", fallback="") + + api_url = f"{lndg_url}/api/channels?limit=1000&is_open=true&is_active=true" try: - response = requests.get(api_url, auth=(username, password)) - - if response.status_code == 200: - data = response.json() - if "results" in data: - results = data["results"] - - table = PrettyTable() - if args.pubkey: - table.field_names = [ - "Alias", - "Is Active", - "Capacity", - "Local Balance", - "Local PPM", - "AR Out Target", - "Auto Rebalance", - "Pubkey", - ] - else: - table.field_names = [ - "Alias", - "Is Active", - "Capacity", - "Local Balance", - "Local PPM", - "AR Out Target", - "Auto Rebalance", - "Channel ID", - ] - - sorted_results = sorted( - results, - key=lambda x: (x.get("local_balance", 0) / x.get("capacity", 1)), - reverse=True, - ) + resp = requests.get(api_url, auth=(username, password), timeout=15) + if resp.status_code == 200: + data = resp.json() + return data.get("results", []) + except Exception as e: + print_color(f"Warning: Failed to fetch channels from LNDg API: {e}", Colors.WARNING) + return [] + + +def filter_and_size_candidates( + channels: List[Dict[str, Any]], + target_amt: Optional[int] = None, + min_capacity: int = 3_000_000, + max_fee_rate: int = 100, + min_local_ratio: float = 60.0, + target_local_ratio: float = 50.0, + blacklist: Optional[List[str]] = None, +) -> List[Dict[str, Any]]: + """ + Filters channels with high local liquidity & low outbound fee rates. + Sizes proposed swap amount dynamically or against explicit target_amt. + """ + if blacklist is None: + blacklist = [] + + candidates = [] + for ch in channels: + if not ch.get("is_active", True) or not ch.get("is_open", True): + continue - for result in sorted_results: - alias = result.get("alias", "") - remote_pubkey = result.get("remote_pubkey", "") - is_active = result.get("is_active", "") - capacity = result.get("capacity", "") - local_fee_rate = result.get("local_fee_rate", "") - local_balance = result.get("local_balance", "") - ar_out_target = result.get("ar_out_target", "") - auto_rebalance = result.get("auto_rebalance", "") - channel_id = result.get("chan_id", "") - - if ( - local_fee_rate <= args.fee_limit - and remote_pubkey not in ignore_remote_pubkeys - and local_balance > CAPACITY_THRESHOLD - ): - local_balance_ratio = (local_balance / capacity) * 100 - if args.pubkey: - table.add_row( - [ - alias, - is_active, - capacity, - f"{local_balance_ratio:.2f}%", - local_fee_rate, - ar_out_target, - auto_rebalance, - remote_pubkey, - ] - ) - else: - table.add_row( - [ - alias, - is_active, - capacity, - f"{local_balance_ratio:.2f}%", - local_fee_rate, - ar_out_target, - auto_rebalance, - channel_id, - ] - ) - - print(table) + pubkey = ch.get("remote_pubkey", "") + if pubkey in blacklist: + continue + + capacity = int(ch.get("capacity", 0)) + local_balance = int(ch.get("local_balance", 0)) + local_fee_rate = int(ch.get("local_fee_rate", 0)) + + if capacity < min_capacity: + continue + if local_fee_rate > max_fee_rate: + continue + + local_ratio = (local_balance / capacity) * 100 if capacity > 0 else 0 + if local_ratio < min_local_ratio: + continue + + # Dynamic or fixed sizing + if target_amt is not None and target_amt > 0: + proposed_amt = target_amt + # Verify channel has sufficient liquidity leaving reasonable reserve (5% or 100k) + reserve = max(100_000, int(capacity * 0.05)) + if local_balance - proposed_amt < reserve: + continue else: - print(f"API request failed with status code: {response.status_code}") + # Rebalance channel down to target equilibrium ratio (e.g. 50%) + target_local_balance = int(capacity * (target_local_ratio / 100.0)) + proposed_amt = local_balance - target_local_balance + + # Clamp proposed amount to Loop limits + if proposed_amt < MIN_LOOP_OUT_SATS: + continue + if proposed_amt > MAX_LOOP_OUT_SATS: + proposed_amt = MAX_LOOP_OUT_SATS + candidate = { + "chan_id": str(ch.get("chan_id", "")), + "alias": ch.get("alias", f"Node_{pubkey[:8]}"), + "remote_pubkey": pubkey, + "capacity": capacity, + "local_balance": local_balance, + "local_ratio": local_ratio, + "local_fee_rate": local_fee_rate, + "proposed_amt": proposed_amt, + } + candidates.append(candidate) + + # Sort primarily by local_ratio descending, then local_fee_rate ascending + candidates.sort(key=lambda x: (x["local_ratio"], -x["local_fee_rate"]), reverse=True) + return candidates + + +def parse_loop_quote_output(output_text: str, amt: int) -> Dict[str, int]: + """Parses output from `litloop quote out`.""" + quote = { + "send_offchain": amt, + "receive_onchain": amt, + "estimated_onchain_fee": 0, + "service_fee": 0, + "total_loop_fee": 0, + } + + for line in output_text.splitlines(): + line_clean = line.strip() + if "Send off-chain:" in line_clean: + val = line_clean.split(":")[1].replace("sat", "").strip() + if val.isdigit(): + quote["send_offchain"] = int(val) + elif "Receive on-chain:" in line_clean: + val = line_clean.split(":")[1].replace("sat", "").strip() + if val.isdigit(): + quote["receive_onchain"] = int(val) + elif "Estimated on-chain fee:" in line_clean: + val = line_clean.split(":")[1].replace("sat", "").strip() + if val.isdigit(): + quote["estimated_onchain_fee"] = int(val) + elif "Loop service fee:" in line_clean: + val = line_clean.split(":")[1].replace("sat", "").strip() + if val.isdigit(): + quote["service_fee"] = int(val) + elif "Estimated total fee:" in line_clean: + val = line_clean.split(":")[1].replace("sat", "").strip() + if val.isdigit(): + quote["total_loop_fee"] = int(val) + + # If total_loop_fee was parsed but components were not (standard vs verbose output) + if quote["total_loop_fee"] > 0 and quote["service_fee"] == 0 and quote["estimated_onchain_fee"] == 0: + quote["service_fee"] = max(0, quote["total_loop_fee"] - 200) + quote["estimated_onchain_fee"] = min(quote["total_loop_fee"], 200) + elif quote["total_loop_fee"] == 0: + quote["total_loop_fee"] = quote["service_fee"] + quote["estimated_onchain_fee"] + + return quote + + +def get_loop_quote( + loop_cmd_parts: List[str], + amt: int, + conf_target: int = 9, + dry_run: bool = False, +) -> Dict[str, int]: + """Queries litloop quote out for cost estimates.""" + if dry_run: + # Realistic simulation: 0.1% loop server fee + ~200 sat sweep fee + service_fee = int(amt * 0.001) + onchain_fee = 200 + return { + "send_offchain": amt, + "receive_onchain": amt - service_fee - onchain_fee, + "estimated_onchain_fee": onchain_fee, + "service_fee": service_fee, + "total_loop_fee": service_fee + onchain_fee, + } + + cmd = list(loop_cmd_parts) + ["quote", "out", "-v", "--conf_target", str(conf_target), str(amt)] + success, output, _ = run_command(cmd, timeout=20) + if success and isinstance(output, str): + return parse_loop_quote_output(output, amt) + + # Fallback to standard quote if -v unsupported + cmd_std = list(loop_cmd_parts) + ["quote", "out", "--conf_target", str(conf_target), str(amt)] + succ_std, out_std, _ = run_command(cmd_std, timeout=20) + if succ_std and isinstance(out_std, str): + return parse_loop_quote_output(out_std, amt) + + # Default fallback estimate if offline + service_fee = int(amt * 0.001) + return { + "send_offchain": amt, + "receive_onchain": amt - service_fee - 200, + "estimated_onchain_fee": 200, + "service_fee": service_fee, + "total_loop_fee": service_fee + 200, + } + + +def query_route_to_loop( + config: configparser.ConfigParser, + dest_pubkey: str, + amt: int, + outgoing_chan_id: str, + dry_run: bool = False, +) -> Tuple[bool, int, int]: + """ + Stage 1: Checks route viability and off-chain routing fees via lncli queryroutes. + Returns (success, routing_fee_sats, hops_count). + """ + lncli_path = get_config_val(config, "paths", "lncli_path", "lncli") + lnd_params = get_lnd_connection_params(config) + + cmd = [ + lncli_path, + *lnd_params, + "queryroutes", + "--dest", + dest_pubkey, + "--amt", + str(amt), + "--outgoing_chan_id", + str(outgoing_chan_id), + ] + + success, output, _ = run_command( + cmd, + timeout=25, + expect_json=True, + dry_run=dry_run, + dry_run_output={"routes": [{"total_fees": "40", "hops": [{"chan_id": outgoing_chan_id}]}]}, + ) + + if success and isinstance(output, dict) and output.get("routes"): + best_route = output["routes"][0] + routing_fee = int(best_route.get("total_fees", 0)) + hops_count = len(best_route.get("hops", [])) + return True, routing_fee, hops_count + + return False, 0, 0 + + +def probe_direct_route_to_loop( + config: Any, + remote_pubkey: str, + dest_pubkey: str, + amt: int, + timeout: int = 15, +) -> Tuple[bool, int, int, Optional[str]]: + """ + Attempts to probe direct 2-hop route: Local -> Peer -> Loop. + Uses lncli buildroute and lncli sendtoroute with a fake payment hash. + Returns (success, verified_fee, hops_count, error_msg). + """ + if not remote_pubkey: + return False, 0, 0, "No remote pubkey provided" + + lncli_path = get_config_val(config, "paths", "lncli_path", "lncli") + lnd_params = get_lnd_connection_params(config) + + # 1. Build direct route + build_cmd = [ + lncli_path, + *lnd_params, + "buildroute", + "--amt", + str(amt), + "--hops", + f"{remote_pubkey},{dest_pubkey}", + ] + succ_build, out_build, err_build = run_command(build_cmd, timeout=timeout, expect_json=True) + if not succ_build or not isinstance(out_build, dict) or "route" not in out_build: + return False, 0, 0, err_build or "Direct channel to Loop not available or buildroute failed" + + route = out_build["route"] + direct_fees = int(route.get("total_fees", 0)) + hops_count = len(route.get("hops", [])) + + # 2. Probe direct route using sendtoroute with fake payment hash + fake_payment_hash = binascii.hexlify(os.urandom(32)).decode() + probe_cmd = [ + lncli_path, + *lnd_params, + "sendtoroute", + f"--payment_hash={fake_payment_hash}", + "-", + ] + try: + process = subprocess.run( + probe_cmd, + input=json.dumps(out_build), + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + timeout=timeout, + ) + if process.stdout: + out_probe = json.loads(process.stdout) + failure_code = out_probe.get("failure", {}).get("code", "") + if failure_code in ["INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS", "INCORRECT_PAYMENT_DETAILS"]: + return True, direct_fees, hops_count, None + for htlc in out_probe.get("htlcs", []): + code = htlc.get("failure", {}).get("code", "") + if code in ["INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS", "INCORRECT_PAYMENT_DETAILS"]: + return True, direct_fees, hops_count, None + err = out_probe.get("failure", {}).get("code") or "Direct route lacked liquidity" + return False, 0, 0, err except Exception as e: - print(f"Error: {e}") + return False, 0, 0, str(e) + + return False, 0, 0, "Failed to verify direct route" + + +def send_prepay_probe( + config: Any, + dest_pubkey: str, + amt: int, + outgoing_chan_id: str, + remote_pubkey: str = "", + timeout: int = 20, + skip_probe: bool = False, +) -> Tuple[bool, int, int, Optional[str]]: + """ + Stage 2: Live route probe using a random 32-byte fake hash. + Proves downstream liquidity without risking funds. + Returns (success, verified_routing_fee, hops_count, error_detail). + """ + if skip_probe: + return True, 0, 0, None + + lncli_path = get_config_val(config, "paths", "lncli_path", "lncli") + lnd_params = get_lnd_connection_params(config) + fake_payment_hash = binascii.hexlify(os.urandom(32)).decode() + + cmd = [ + lncli_path, + *lnd_params, + "sendpayment", + "--dest", + dest_pubkey, + "--amt", + str(amt), + "--payment_hash", + fake_payment_hash, + "--outgoing_chan_id", + str(outgoing_chan_id), + "--timeout", + f"{timeout}s", + "--json", + ] + + _, output, _ = run_command(cmd, timeout=timeout + 10, expect_json=True) + + if isinstance(output, dict): + # 1. Check if any attempt reached Loop destination + htlcs = output.get("htlcs", []) + for htlc in htlcs: + failure = htlc.get("failure", {}) + code = failure.get("code", "") + if code in ["INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS", "INCORRECT_PAYMENT_DETAILS"]: + route = htlc.get("route", {}) + verified_fee = int(route.get("total_fees", 0)) + hops_count = len(route.get("hops", [])) + return True, verified_fee, hops_count, None + + # Check top-level failure reason + failure_reason = output.get("failure_reason", "") + payment_error = output.get("payment_error", "") + acceptable_signals = [ + "INCORRECT_PAYMENT_DETAILS", + "INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS", + "FAILURE_REASON_INCORRECT_PAYMENT_DETAILS", + ] + if any(sig in failure_reason for sig in acceptable_signals) or any( + sig in payment_error for sig in acceptable_signals + ): + if htlcs: + last_route = htlcs[-1].get("route", {}) + verified_fee = int(last_route.get("total_fees", 0)) + hops_count = len(last_route.get("hops", [])) + return True, verified_fee, hops_count, None + return True, 0, 0, None + + # 2. Multi-hop probe failed. If peer has a direct channel to Loop, probe the direct channel! + if remote_pubkey: + succ_dir, fee_dir, hops_dir, err_dir = probe_direct_route_to_loop( + config, remote_pubkey, dest_pubkey, amt, timeout=timeout + ) + if succ_dir: + return True, fee_dir, hops_dir, None + + err_detail = "Insufficient liquidity on route to Loop" + if isinstance(output, dict): + err_detail = output.get("failure_reason") or output.get("payment_error") or err_detail + return False, 0, 0, err_detail + + +def calculate_economic_cost( + amt: int, + service_fee: int, + onchain_fee: int, + routing_fee: int, + local_fee_rate: int, +) -> Dict[str, Any]: + """Computes full economic breakdown including opportunity cost.""" + # Opportunity cost: lost potential routing fee from draining local liquidity + opportunity_cost = round((amt * local_fee_rate) / 1_000_000) + total_cost = service_fee + onchain_fee + routing_fee + opportunity_cost + effective_ppm = round((total_cost / amt) * 1_000_000) if amt > 0 else 0 + + return { + "server_fee": service_fee, + "onchain_fee": onchain_fee, + "routing_fee": routing_fee, + "opportunity_cost": opportunity_cost, + "total_cost": total_cost, + "effective_ppm": effective_ppm, + } + + +def get_loop_db_path(config: Any) -> Optional[str]: + """ + Returns the configured or discovered path to Loop's SQLite database (Source of Truth). + Checks [loop] -> loop_db_path and [paths] -> loop_db_path in config.ini, + falling back to default ~/.loop/mainnet/loop_sqlite.db. + """ + db_path = "" + if hasattr(config, "get"): + if config.has_section("loop"): + db_path = config.get("loop", "loop_db_path", fallback="") + if not db_path and config.has_section("paths"): + db_path = config.get("paths", "loop_db_path", fallback="") + + if db_path: + expanded = os.path.expanduser(db_path.strip()) + if os.path.exists(expanded): + return expanded + return expanded + + default_path = os.path.expanduser("~/.loop/mainnet/loop_sqlite.db") + if os.path.exists(default_path): + return default_path + + return None + + +LOOP_STATE_MAP = { + 0: "INITIATED", + 1: "HTLC_PUBLISHED", + 2: "SUCCESS", + 3: "FAILED", + 4: "FAILED", + 5: "INVOICE_SETTLED", + 6: "SUCCESS", + 7: "FAILED", + 8: "HTLC_PUBLISHED", + 9: "PREIMAGE_REVEALED", +} -def write_bos_tags(): +def fetch_loop_history_from_db(db_path: str, limit: int = 50) -> List[Dict[str, Any]]: + """Reads Loop Out swaps directly from Loop's SQLite database (Source of Truth).""" + if not os.path.exists(db_path): + return [] + + conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) + conn.row_factory = sqlite3.Row + cur = conn.cursor() + query = """ + SELECT + hex(s.swap_hash) AS swap_id, + s.initiation_time, + s.amount_requested AS amount, + s.label, + lo.outgoing_chan_set, + lo.dest_address, + su.update_state, + COALESCE(su.server_cost, 0) AS server_cost, + COALESCE(su.onchain_cost, 0) AS onchain_cost, + COALESCE(su.offchain_cost, 0) AS offchain_cost + FROM loopout_swaps lo + JOIN swaps s ON lo.swap_hash = s.swap_hash + LEFT JOIN ( + SELECT swap_hash, update_state, server_cost, onchain_cost, offchain_cost, + ROW_NUMBER() OVER (PARTITION BY swap_hash ORDER BY update_timestamp DESC) as rn + FROM swap_updates + ) su ON lo.swap_hash = su.swap_hash AND su.rn = 1 + ORDER BY s.initiation_time DESC + LIMIT ?; + """ + cur.execute(query, (limit,)) + rows = [] + for r in cur.fetchall(): + server_fee = int(r["server_cost"]) + onchain_fee = int(r["onchain_cost"]) + routing_fee = int(r["offchain_cost"]) + total_cost = server_fee + onchain_fee + routing_fee + amt = int(r["amount"]) + ppm = int((total_cost * 1_000_000) / amt) if amt > 0 else 0 + + state_code = r["update_state"] + state_str = LOOP_STATE_MAP.get(state_code, f"STATE_{state_code}" if state_code is not None else "INITIATED") + + raw_time = str(r["initiation_time"]) + formatted_time = raw_time.split(".")[0].replace(" +0000 UTC", "") + + rows.append({ + "swap_id": r["swap_id"].lower(), + "initiation_time": formatted_time, + "amount": amt, + "label": r["label"] or "", + "outgoing_chan_set": r["outgoing_chan_set"] or "", + "server_fee": server_fee, + "onchain_fee": onchain_fee, + "routing_fee": routing_fee, + "total_cost": total_cost, + "effective_ppm": ppm, + "status": state_str, + }) + conn.close() + return rows + + +def fetch_loop_history_from_cli(config: Any, limit: int = 50) -> List[Dict[str, Any]]: + """Queries loop listswaps via RPC CLI as fallback.""" + loop_cmd = resolve_loop_command(config) + cmd = list(loop_cmd) + ["listswaps"] + succ, out, _ = run_command(cmd, timeout=30, expect_json=True) + if not succ or not isinstance(out, dict) or "swaps" not in out: + return [] + + raw_swaps = [s for s in out.get("swaps", []) if s.get("type") == "LOOP_OUT"] + raw_swaps.sort(key=lambda s: int(s.get("initiation_time", 0)), reverse=True) + + rows = [] + for s in raw_swaps[:limit]: + amt = int(s.get("amt", 0)) + server_fee = int(s.get("cost_server", 0)) + onchain_fee = int(s.get("cost_onchain", 0)) + routing_fee = int(s.get("cost_offchain", 0)) + total_cost = server_fee + onchain_fee + routing_fee + ppm = int((total_cost * 1_000_000) / amt) if amt > 0 else 0 + + ns_time = int(s.get("initiation_time", 0)) + if ns_time > 0: + formatted_time = datetime.datetime.fromtimestamp(ns_time / 1e9, tz=datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S") + else: + formatted_time = "UNKNOWN" + + chan_set = s.get("outgoing_chan_set", []) + chan_str = ",".join(str(c) for c in chan_set) if isinstance(chan_set, list) else str(chan_set) + + rows.append({ + "swap_id": s.get("id", "").lower(), + "initiation_time": formatted_time, + "amount": amt, + "label": s.get("label", ""), + "outgoing_chan_set": chan_str, + "server_fee": server_fee, + "onchain_fee": onchain_fee, + "routing_fee": routing_fee, + "total_cost": total_cost, + "effective_ppm": ppm, + "status": s.get("state", "UNKNOWN"), + }) + return rows + + +def fetch_loop_history(config: Any, limit: int = 50) -> Tuple[List[Dict[str, Any]], str]: + """ + Fetches Loop Out history. First tries direct SQLite read from the configured + or discovered loop_db_path (Source of Truth). If unavailable, falls back to RPC CLI (listswaps). + Returns (swaps_list, source_description). + """ + db_path = get_loop_db_path(config) + if db_path and os.path.exists(db_path): + try: + swaps = fetch_loop_history_from_db(db_path, limit=limit) + return swaps, f"Loop SQLite DB SOT ({db_path})" + except Exception: + pass + + swaps = fetch_loop_history_from_cli(config, limit=limit) + return swaps, "Loop Daemon RPC (listswaps)" + + +def export_history_to_csv(swaps: List[Dict[str, Any]], csv_path: str) -> None: + """Exports swap history records to CSV file.""" + if not swaps: + return + os.makedirs(os.path.dirname(os.path.abspath(csv_path)), exist_ok=True) + with open(csv_path, "w", newline="", encoding="utf-8") as f: + writer = csv.writer(f) + writer.writerow(swaps[0].keys()) + for s in swaps: + writer.writerow(list(s.values())) + + +def read_single_keypress() -> str: + """Reads a single keypress from standard input in raw mode.""" + import termios + import tty + + fd = sys.stdin.fileno() + old_settings = termios.tcgetattr(fd) try: - response = requests.get(api_url, auth=(username, password)) - if response.status_code == 200: - data = response.json() - if "results" in data: - results = data["results"] - # Filter and sort results based on the same criteria - filtered_sorted_results = [ - result - for result in sorted( - results, - key=lambda x: ( - x.get("local_balance", 0) / x.get("capacity", 1) - ), - reverse=True, - ) - if result.get("local_fee_rate", 0) <= args.fee_limit - and result.get("remote_pubkey", "") not in ignore_remote_pubkeys - and result.get("local_balance", 0) > CAPACITY_THRESHOLD - ] - # Extract remote_pubkey from filtered and sorted results - remote_pubkeys = [ - result.get("remote_pubkey", "") - for result in filtered_sorted_results - ] - - tags_data = { - "tags": [ - { - "alias": "swap-candidates", - "id": "454d13aff835eeb91de6183684a208cd7e3d4cc19d025fab84f6838c4575cdae", - "nodes": remote_pubkeys, - } - ] - } - - with open(file_path_to_bos, "w") as file: - json.dump(tags_data, file, indent=2) - - print(f"Tags data written to {file_path_to_bos}") + tty.setraw(fd) + ch = sys.stdin.read(1) + if ch == "\x1b": # Escape sequence + ch2 = sys.stdin.read(1) + if ch2 == "[": + ch3 = sys.stdin.read(1) + return f"\x1b[{ch3}" + return "\x1b" + return ch + finally: + termios.tcsetattr(fd, termios.TCSADRAIN, old_settings) + + +def print_candidates_table(candidates: List[Dict[str, Any]]) -> None: + """Prints a formatted summary table of candidates with fee and cost breakdown.""" + table = PrettyTable() + table.field_names = [ + "#", + "Alias", + "Channel ID", + "Local %", + "Swap Size", + "Loop Fee", + "Sweep Fee", + "Route Fee", + "Opp Cost", + "Total Cost", + "Net PPM", + ] + table.align = "r" + table.align["Alias"] = "l" + table.align["#"] = "c" + for idx, c in enumerate(candidates, 1): + table.add_row([ + f"[{idx}]", + c["alias"][:20], + c["chan_id"], + f"{c['local_ratio']:.1f}%", + f"{c['proposed_amt']:,}", + f"{c['server_fee']:,}", + f"{c['onchain_fee']:,}", + f"{c['routing_fee']:,}", + f"{c['opportunity_cost']:,}", + f"{c['total_cost']:,}", + f"{c['effective_ppm']:,}", + ]) + print(table) + + +def interactive_menu_select(candidates: List[Dict[str, Any]]) -> Optional[Dict[str, Any]]: + """ + Renders an interactive CLI terminal menu with arrow-key navigation. + Falls back to text input if terminal is non-interactive. + """ + if not candidates: + return None + + if not sys.stdin.isatty(): + # Non-interactive fallback + print("\nAvailable Candidates:") + print_candidates_table(candidates) + try: + choice = input(f"\nSelect candidate [1-{len(candidates)}] or [q] to cancel: ").strip().lower() + if choice.isdigit() and 1 <= int(choice) <= len(candidates): + return candidates[int(choice) - 1] + except (EOFError, KeyboardInterrupt): + pass + return None + + current_idx = 0 + total = len(candidates) + + while True: + # Clear screen segment and render + print("\033[2J\033[H", end="") # Clear screen and move to top-left + print_color("=== Lightning Loop Out - Economic Channel Selection ===", Colors.HEADER, bold=True) + print_color("Use [↑/k] and [↓/j] to navigate, [Enter] to select, [q] to cancel.\n", Colors.OKCYAN) + + table = PrettyTable() + table.field_names = [ + "Sel", + "Alias", + "Channel ID", + "Local %", + "Swap Size", + "Loop Fee", + "Sweep Fee", + "Route Fee", + "Opp Cost", + "Total Cost", + "Net PPM", + ] + table.align = "r" + table.align["Alias"] = "l" + table.align["Sel"] = "c" + + for idx, c in enumerate(candidates): + is_selected = idx == current_idx + sel_mark = "▶" if is_selected else " " + + row = [ + sel_mark, + c["alias"][:20], + c["chan_id"], + f"{c['local_ratio']:.1f}%", + f"{c['proposed_amt']:,}", + f"{c['server_fee']:,}", + f"{c['onchain_fee']:,}", + f"{c['routing_fee']:,}", + f"{c['opportunity_cost']:,}", + f"{c['total_cost']:,}", + f"{c['effective_ppm']:,}", + ] + + if is_selected: + # Highlight active row in color + row = [f"{Colors.OKGREEN}{Colors.BOLD}{val}{Colors.ENDC}" for val in row] + table.add_row(row) + + print(table) + print() + selected_cand = candidates[current_idx] + print_color( + f"Active: {selected_cand['alias']} | Swap: {selected_cand['proposed_amt']:,} sats | Total Cost: {selected_cand['total_cost']:,} sats ({selected_cand['effective_ppm']} ppm)", + Colors.OKBLUE, + bold=True, + ) + + key = read_single_keypress() + if key in ("\x1b[A", "k", "K"): # Up + current_idx = (current_idx - 1) % total + elif key in ("\x1b[B", "j", "J"): # Down + current_idx = (current_idx + 1) % total + elif key in ("\r", "\n", " "): # Enter or Space to select + return candidates[current_idx] + elif key in ("q", "Q", "\x1b"): # Quit or Escape + print_color("\nLoop Out selection cancelled.", Colors.WARNING) + return None + + +def execute_loop_out( + config: Any, + channel_id: str, + amt: int, + conf_target: int = 9, + max_routing_fee: int = 0, + dest_addr: Optional[str] = None, + alias: str = "", + dry_run: bool = False, +) -> Dict[str, Any]: + """Initiates the Loop Out swap using litloop / loop, tagging with label.""" + label = f"Loop-Out: {alias} ({channel_id})" if alias else f"Loop-Out: {channel_id}" + if dry_run: + fake_swap_id = "dry-run-swap-" + binascii.hexlify(os.urandom(16)).decode() + print_color(f"\n[DRY RUN] Simulated litloop out execution for channel {channel_id}:", Colors.WARNING, bold=True) + print_color( + f' Command: litloop out --amt {amt} --channel {channel_id} --conf_target {conf_target} --label "{label}" --force', + Colors.WARNING, + ) + return {"success": True, "swap_id": fake_swap_id, "dry_run": True} + + loop_cmd = resolve_loop_command(config) + cmd = list(loop_cmd) + [ + "out", + "--amt", + str(amt), + "--channel", + str(channel_id), + "--conf_target", + str(conf_target), + "--label", + label, + "--force", + ] + + if max_routing_fee > 0: + cmd.extend(["--max_swap_routing_fee", str(max_routing_fee)]) + if dest_addr: + cmd.extend(["--addr", dest_addr]) + + print_color(f"\nInitiating Loop Out: {' '.join(cmd)}", Colors.OKBLUE, bold=True) + success, output, err = run_command(cmd, timeout=120) + + if not success or not output: + print_color(f"Failed to initiate loop out: {err}", Colors.FAIL, bold=True) + return {"success": False, "error": err, "dry_run": False} + + # Extract swap ID from output if available + swap_id = "unknown_swap_id" + for line in output.splitlines(): + if "Swap initiated" in line or "Swap ID" in line or "swap hash" in line.lower(): + parts = line.split() + if len(parts) > 1: + swap_id = parts[-1].strip(".:,") + + print_color("✓ Swap Initiated Successfully!", Colors.OKGREEN, bold=True) + print_color(output, Colors.OKCYAN) + return {"success": True, "swap_id": swap_id, "output": output, "dry_run": False} + + +def evaluate_single_candidate( + c: Dict[str, Any], + config: Any, + loop_cmd: List[str], + loop_pubkey: str, + conf_target: int, + probe_timeout: int, + skip_prepay_probe: bool, + print_lock: Optional[Any] = None, +) -> Optional[Dict[str, Any]]: + """Evaluates a single candidate channel by querying quote, route, and prepay probe.""" + chan_id = c["chan_id"] + amt = c["proposed_amt"] + alias = c["alias"] + + def log(msg: str): + if print_lock: + with print_lock: + print(msg) else: - print(f"API request failed with status code: {response.status_code}") - except Exception as e: - print(f"Error: {e}") + print(msg) + + log(f" → Checking {alias[:20]} ({chan_id}) for {amt:,} sats...") + + # 1. Fetch Loop Quote (read-only query) + quote = get_loop_quote(loop_cmd, amt, conf_target=conf_target, dry_run=False) + + # 2. Stage 1: queryroutes (read-only query) + route_ok, route_fee, hops = query_route_to_loop(config, loop_pubkey, amt, chan_id, dry_run=False) + if not route_ok: + log(f" ✗ Queryroutes found no route to Loop node for {alias[:20]}.") + return None + + # 3. Stage 2: Prepay probe with fake hash (proves actual live liquidity without spending funds) + probe_ok, verified_fee, verified_hops, probe_err = send_prepay_probe( + config=config, + dest_pubkey=loop_pubkey, + amt=amt, + outgoing_chan_id=chan_id, + remote_pubkey=c.get("remote_pubkey", ""), + timeout=probe_timeout, + skip_probe=skip_prepay_probe, + ) + if not probe_ok: + log(f" ✗ Prepay probe failed for {alias[:20]}: {probe_err}") + return None + + actual_fee = verified_fee if verified_fee > 0 or skip_prepay_probe else route_fee + actual_hops = verified_hops if verified_hops > 0 or skip_prepay_probe else hops + log(f" ✓ Route verified with live liquidity for {alias[:20]} ({actual_hops} hops, {actual_fee:,} sat routing fee).") + + # 4. Economic cost breakdown + econ = calculate_economic_cost( + amt=amt, + service_fee=quote["service_fee"], + onchain_fee=quote["estimated_onchain_fee"], + routing_fee=actual_fee, + local_fee_rate=c["local_fee_rate"], + ) + + c_eval = dict(c) + c_eval.update(quote) + c_eval.update(econ) + return c_eval + + +def monitor_loop(config: configparser.ConfigParser): + """Streams litloop monitor to terminal with detachment instructions.""" + loop_cmd = resolve_loop_command(config) + cmd = list(loop_cmd) + ["monitor"] + + print_color("\n" + "=" * 70, Colors.OKCYAN) + print_color("Streaming Loop Monitor (Real-Time Swaps Progress):", Colors.OKCYAN, bold=True) + print_color("NOTE: Sweeping requires ~6-9 on-chain block confirmations (~30-60+ min).", Colors.WARNING) + print_color("You can safely press [Ctrl+C] to detach anytime; loopd runs in the background.", Colors.OKGREEN) + print_color("=" * 70 + "\n", Colors.OKCYAN) + + try: + subprocess.run(cmd) + except KeyboardInterrupt: + print_color("\nDetached from Loop Monitor. Background loopd will continue processing.", Colors.OKGREEN, bold=True) + + +def parse_arguments() -> argparse.Namespace: + """Configures CLI argument parser.""" + parser = argparse.ArgumentParser( + description="Economically optimized Loop Out liquidity rebalancing via litloop." + ) + parser.add_argument( + "--amt", + type=int, + default=None, + help=f"Specific amount to loop out in satoshis ({MIN_LOOP_OUT_SATS:,} to {MAX_LOOP_OUT_SATS:,}). If omitted, dynamically calculated per channel.", + ) + parser.add_argument( + "-c", + "--capacity", + type=int, + default=None, + help="Minimum channel capacity in satoshis (default from config or 3,000,000).", + ) + parser.add_argument( + "-f", + "--fee-limit", + type=int, + default=None, + help="Maximum local fee rate in ppm to consider for looping out (default from config or 100).", + ) + parser.add_argument( + "-r", + "--min-ratio", + type=float, + default=None, + help="Minimum local liquidity ratio percentage to qualify as candidate (default 60%%).", + ) + parser.add_argument( + "--conf-target", + type=int, + default=None, + help="Confirmation target in blocks for on-chain sweep transaction (minimum 6, default 9).", + ) + parser.add_argument( + "--max-routing-fee", + type=int, + default=0, + help="Maximum off-chain swap routing fee in satoshis.", + ) + parser.add_argument( + "--dest-addr", + type=str, + default=None, + help="Custom on-chain Bitcoin address for swept funds (defaults to LND internal wallet).", + ) + parser.add_argument( + "-w", + "--workers", + type=int, + default=None, + help="Number of concurrent worker threads for route probing (default from config or 1). Recommended max: 2.", + ) + parser.add_argument( + "--probe-timeout", + type=int, + default=15, + help="Timeout in seconds for individual route prepay probes (default: 15s).", + ) + parser.add_argument( + "--skip-prepay-probe", + action="store_true", + help="Skip active prepay probing and rely on queryroutes theoretical fees (not recommended).", + ) + parser.add_argument( + "--dry-run", + action="store_true", + help="Simulate route probing and quote calculation without initiating any real swap.", + ) + parser.add_argument( + "--auto-approve", + action="store_true", + help="Automatically select and initiate the top (#1) most economical candidate without interactive prompt.", + ) + parser.add_argument( + "--history", + action="store_true", + help="Display past Loop Out swaps from Loop's database (Source of Truth).", + ) + parser.add_argument( + "--csv", + nargs="?", + const="default", + default=None, + help="Export historical swaps to CSV file (default: data/loop_out_history.csv).", + ) + parser.add_argument( + "--limit", + type=int, + default=30, + help="Maximum number of historical swaps to display (default: 30).", + ) + parser.add_argument( + "-p", + "--pubkey", + action="store_true", + help="Display remote pubkeys in candidate table.", + ) + return parser.parse_args() + + +def display_history(config: Any, limit: int = 30, csv_path: Optional[str] = None): + """Prints table of historical loop-out operations from the Source of Truth.""" + swaps, source = fetch_loop_history(config, limit=limit) + if not swaps: + print_color(f"No past Loop Out operations found via {source}.", Colors.WARNING) + return + + print_color(f"\n=== Lightning Loop Out - Historical Swaps ===", Colors.HEADER, bold=True) + print_color(f"Source of Truth: {source}\n", Colors.OKCYAN) + + table = PrettyTable() + table.field_names = [ + "Time (UTC)", + "Swap ID", + "Label / Channel", + "Amount (sat)", + "Server Fee", + "Onchain Fee", + "Route Fee", + "Total Cost", + "Net PPM", + "Status", + ] + table.align = "r" + table.align["Time (UTC)"] = "l" + table.align["Swap ID"] = "l" + table.align["Label / Channel"] = "l" + table.align["Status"] = "c" + + for s in swaps: + disp_label = s.get("label", "") + if not disp_label: + chans = s.get("outgoing_chan_set", "") + disp_label = (chans[:24] + "...") if len(chans) > 24 else chans + + status = s.get("status", "UNKNOWN") + status_colored = status + if status == "SUCCESS": + status_colored = f"{Colors.OKGREEN}{status}{Colors.ENDC}" + elif status == "FAILED": + status_colored = f"{Colors.FAIL}{status}{Colors.ENDC}" + elif status in ["INITIATED", "HTLC_PUBLISHED", "PREIMAGE_REVEALED"]: + status_colored = f"{Colors.WARNING}{status}{Colors.ENDC}" + + table.add_row( + [ + s.get("initiation_time", ""), + s.get("swap_id", "")[:12] + "...", + disp_label[:28], + f"{s.get('amount', 0):,}", + f"{s.get('server_fee', 0):,}", + f"{s.get('onchain_fee', 0):,}", + f"{s.get('routing_fee', 0):,}", + f"{s.get('total_cost', 0):,}", + f"{s.get('effective_ppm', 0):,}", + status_colored, + ] + ) + print(table) + + if csv_path: + export_history_to_csv(swaps, csv_path) + print_color(f"\n✓ Exported {len(swaps)} records to CSV: {csv_path}", Colors.OKGREEN) def main(): - terminal_output() - - if args.bos: - write_bos_tags() - - if args.file_export: - # Remove chan_ids = get_chan_ids_to_write() - # Remove if chan_ids: - # Remove with open(file_path, 'w') as file: - # Remove file.write(','.join(chan_ids) + '\n') - # Remove print(f"Channel-ID data written to {file_path}") - pass + args = parse_arguments() + config, project_root = load_config() + logger = setup_logger(project_root) + + if args.history: + csv_target = None + if args.csv is not None: + csv_target = ( + args.csv + if args.csv != "default" + else os.path.join(project_root, "data", "loop_out_history.csv") + ) + display_history(config, limit=args.limit, csv_path=csv_target) + return + + # Configuration defaults + loop_pubkey = config.get("loop", "loop_pubkey", fallback=LOOP_PUBKEY_DEFAULT) + min_capacity = args.capacity or config.getint("loop", "min_capacity", fallback=3_000_000) + max_fee_ppm = args.fee_limit if args.fee_limit is not None else config.getint("loop", "max_local_fee_ppm", fallback=100) + min_local_ratio = args.min_ratio if args.min_ratio is not None else config.getfloat("loop", "min_local_balance_ratio", fallback=60.0) + target_local_ratio = config.getfloat("loop", "target_local_ratio", fallback=50.0) + conf_target = args.conf_target or config.getint("loop", "conf_target", fallback=DEFAULT_CONF_TARGET) + + if conf_target < MIN_CONF_TARGET: + print_color(f"Warning: Minimum conf-target is {MIN_CONF_TARGET} for economical sweeping. Adjusting to {MIN_CONF_TARGET}.", Colors.WARNING) + conf_target = MIN_CONF_TARGET + + blacklist_raw = config.get("no-swapout", "swapout_blacklist", fallback="") + blacklist = [pk.strip() for pk in blacklist_raw.split(",") if pk.strip()] + + print_color("=== Lightning Loop Out - Economic Liquidity Optimizer ===", Colors.HEADER, bold=True) + if args.dry_run: + print_color("[!] RUNNING IN DRY-RUN SIMULATION MODE (No funds will move)", Colors.WARNING, bold=True) + + print(f"Fetching channels from LNDg API...") + channels = fetch_channels_lndg(config) + if not channels: + print_color("No channels found or could not connect to LNDg API. Exiting.", Colors.FAIL) + sys.exit(1) + + print(f"Filtering candidates (min_capacity: {min_capacity:,} sat, max_fee: {max_fee_ppm} ppm, min_ratio: {min_local_ratio}%)...") + candidates = filter_and_size_candidates( + channels=channels, + target_amt=args.amt, + min_capacity=min_capacity, + max_fee_rate=max_fee_ppm, + min_local_ratio=min_local_ratio, + target_local_ratio=target_local_ratio, + blacklist=blacklist, + ) + + if not candidates: + print_color("No suitable candidate channels found matching criteria.", Colors.WARNING) + sys.exit(0) + + print_color(f"Found {len(candidates)} channel candidates. Fetching quotes and probing paths to Loop server...", Colors.OKBLUE) + + # Route probing & quote evaluation on top candidates + loop_cmd = resolve_loop_command(config) + evaluated_candidates = [] + top_candidates = candidates[:8] + workers = args.workers or config.getint("loop", "workers", fallback=1) + workers = max(1, workers) + + if workers > 1: + import concurrent.futures + import threading + print_color(f"Probing {len(top_candidates)} candidate channels using {workers} concurrent workers...", Colors.OKCYAN) + print_lock = threading.Lock() + with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as executor: + futures = [ + executor.submit( + evaluate_single_candidate, + c=c, + config=config, + loop_cmd=loop_cmd, + loop_pubkey=loop_pubkey, + conf_target=conf_target, + probe_timeout=args.probe_timeout, + skip_prepay_probe=args.skip_prepay_probe, + print_lock=print_lock, + ) + for c in top_candidates + ] + for future in concurrent.futures.as_completed(futures): + res = future.result() + if res: + evaluated_candidates.append(res) + else: + for c in top_candidates: + res = evaluate_single_candidate( + c=c, + config=config, + loop_cmd=loop_cmd, + loop_pubkey=loop_pubkey, + conf_target=conf_target, + probe_timeout=args.probe_timeout, + skip_prepay_probe=args.skip_prepay_probe, + print_lock=None, + ) + if res: + evaluated_candidates.append(res) + + if not evaluated_candidates: + print_color("\nNo candidates passed both queryroutes and prepay probing.", Colors.FAIL, bold=True) + sys.exit(1) + + # Rank ascending by effective_ppm + evaluated_candidates.sort(key=lambda x: x["effective_ppm"]) + + # Selection + selected = None + if args.auto_approve: + print("\nEvaluated Candidates:") + print_candidates_table(evaluated_candidates) + selected = evaluated_candidates[0] + print_color(f"\nAuto-approved top candidate: {selected['alias']} ({selected['chan_id']})", Colors.OKGREEN, bold=True) + else: + selected = interactive_menu_select(evaluated_candidates) + + if not selected: + print_color("Operation cancelled. No swap initiated.", Colors.WARNING) + sys.exit(0) + + # Max routing fee budget + max_rf = args.max_routing_fee or int(selected["routing_fee"] * 1.5) + 50 + + # Execute + res = execute_loop_out( + config=config, + channel_id=selected["chan_id"], + amt=selected["proposed_amt"], + conf_target=conf_target, + max_routing_fee=max_rf, + dest_addr=args.dest_addr, + alias=selected["alias"], + dry_run=args.dry_run, + ) + + if res.get("success"): + logger.info( + f"Swap initiated for {selected['alias']} ({selected['chan_id']}): " + f"amt={selected['proposed_amt']} sats, total_cost={selected['total_cost']} sats" + ) + + if not args.dry_run: + monitor_loop(config) if __name__ == "__main__": diff --git a/README.md b/README.md index 13ce459..a1db599 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ Manage on-chain UTXOs, submarine swaps, and dynamic channel fee pricing. | [`Other/rebalance_guard.py`](Other/rebalance_guard.py) | `cronjob` / `CLI tool` | **Standalone Rebalance Guard**: Audits all open LNDg channels across both native Auto-Fees and `fee_adjuster.py`. Protects channels with active inbound discounts by setting `ar_out_target = 100%`, and automatically restores baseline targets using **Dynamic Hysteresis** once liquidity recovers. | | [`Other/boltz_swap-out.py`](Other/boltz_swap-out.py) | `CLI tool` | Automates submarine swap-outs through the Boltz exchange (Lightning to Liquid L-BTC). | | [`Other/swap_wallet.py`](Other/swap_wallet.py) | `CLI tool` | Batches automated payouts or drain payments over Lightning to a designated Lightning Address. | -| [`Other/swap_out-loop.py`](Other/swap_out-loop.py) | `CLI tool` | Orchestrates continuous swap-outs for rebalancing large liquidity sinks. | +| [`Other/swap_out-loop.py`](Other/swap_out-loop.py) | `CLI tool` | Economically optimized Loop Out liquidity rebalancing via Lightning Labs Loop (`litloop` / `loopd`) with route probing, economic scoring, arrow-key interactive menu, and SQLite accounting. See [documentation](Other/swap_out-loop.md). | | [`Other/lnd_utxo_consolidator.py`](Other/lnd_utxo_consolidator.py) | `CLI tool` | Safely consolidates fragmented on-chain LND UTXOs during low-mempool fee environments. | --- diff --git a/config.ini.example b/config.ini.example index da7bbba..d749847 100644 --- a/config.ini.example +++ b/config.ini.example @@ -94,4 +94,28 @@ day_of_week = friday [lnd] rpcserver = your_lnd_host:port tlscertpath = /path/to/your/lnd/tls.cert -macaroonpath = /path/to/your/lnd/admin.macaroon \ No newline at end of file +macaroonpath = /path/to/your/lnd/admin.macaroon +[loop] +# Path or alias to litloop/loop command. Defaults to litloop or loop +loop_command = litloop +# Path to loop's local SQLite database (Source of Truth for history and swap accounting) +loop_db_path = ~/.loop/mainnet/loop_sqlite.db +# Public key of the Lightning Labs Loop server node +loop_pubkey = 021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d +# Default confirmation target for sweep transaction (minimum 6 for economical sweep) +conf_target = 9 +# Default maximum local fee rate (ppm) to consider for looping out (opportunity cost cap) +max_local_fee_ppm = 100 +# Minimum local balance ratio (percentage) on candidate channel to consider +min_local_balance_ratio = 60 +# Minimum channel capacity (sats) +min_capacity = 3000000 +# Target local balance ratio (percentage) when calculating dynamic loop-out size +target_local_ratio = 50 +# Number of concurrent workers for route probing (default: 1 for zero downstream collision, 2 for parallel) +workers = 1 +# Timeout in seconds for individual route prepay probes (default: 15) +probe_timeout = 15 +# Loop daemon connection parameters (used if loop_command is 'loop' instead of 'litloop') +# rpcserver = localhost:8443 +# tlscertpath = ~/.lit/tls.cert diff --git a/tests/Other/__init__.py b/tests/Other/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/Other/test_swap_out_loop.py b/tests/Other/test_swap_out_loop.py new file mode 100644 index 0000000..96c3a17 --- /dev/null +++ b/tests/Other/test_swap_out_loop.py @@ -0,0 +1,517 @@ +import json +import configparser +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +""" +Unit tests for Other/swap_out-loop.py +""" + +import os +import sys +import tempfile +import sqlite3 +import csv +from unittest.mock import patch, MagicMock +import pytest + +import importlib.util +from pathlib import Path + +script_path = Path(__file__).resolve().parent.parent.parent / "Other" / "swap_out-loop.py" +spec = importlib.util.spec_from_file_location("swap_out_loop", str(script_path)) +swap_out_loop = importlib.util.module_from_spec(spec) +sys.modules["swap_out_loop"] = swap_out_loop +spec.loader.exec_module(swap_out_loop) + + +@pytest.fixture +def sample_channels(): + return [ + { + "chan_id": "111111111111111111", + "capacity": 10_000_000, + "local_balance": 8_500_000, + "remote_balance": 1_500_000, + "local_fee_rate": 5, + "remote_pubkey": "02aaaabbbbcccc1111222233334444555566667777888899990000aaaabbbbcccc", + "alias": "Cheap-High-Liquidity-Node", + "is_active": True, + "is_open": True, + }, + { + "chan_id": "222222222222222222", + "capacity": 10_000_000, + "local_balance": 9_000_000, + "remote_balance": 1_000_000, + "local_fee_rate": 450, # High fee (high opportunity cost) + "remote_pubkey": "02bbbbccccdddd1111222233334444555566667777888899990000bbbbccccdddd", + "alias": "Expensive-High-Liquidity-Node", + "is_active": True, + "is_open": True, + }, + { + "chan_id": "333333333333333333", + "capacity": 10_000_000, + "local_balance": 3_000_000, # Low local balance (30%) + "remote_balance": 7_000_000, + "local_fee_rate": 10, + "remote_pubkey": "02ccccddddeeee1111222233334444555566667777888899990000ccccddddeeee", + "alias": "Low-Local-Balance-Node", + "is_active": True, + "is_open": True, + }, + { + "chan_id": "444444444444444444", + "capacity": 2_000_000, # Below min capacity + "local_balance": 1_800_000, + "remote_balance": 200_000, + "local_fee_rate": 10, + "remote_pubkey": "02ddddeeeeffff1111222233334444555566667777888899990000ddddeeeeffff", + "alias": "Small-Channel-Node", + "is_active": True, + "is_open": True, + }, + { + "chan_id": "555555555555555555", + "capacity": 10_000_000, + "local_balance": 8_000_000, + "remote_balance": 2_000_000, + "local_fee_rate": 15, + "remote_pubkey": "02blacklist1111111111111111111111111111111111111111111111111111111", + "alias": "Blacklisted-Node", + "is_active": True, + "is_open": True, + }, + { + "chan_id": "666666666666666666", + "capacity": 10_000_000, + "local_balance": 8_000_000, + "remote_balance": 2_000_000, + "local_fee_rate": 15, + "remote_pubkey": "02inactive11111111111111111111111111111111111111111111111111111111", + "alias": "Inactive-Node", + "is_active": False, # Inactive + "is_open": True, + }, + ] + + +def test_filter_and_size_candidates_dynamic(sample_channels): + """Test dynamic sizing and filtering with default thresholds.""" + blacklist = ["02blacklist1111111111111111111111111111111111111111111111111111111"] + candidates = swap_out_loop.filter_and_size_candidates( + channels=sample_channels, + target_amt=None, + min_capacity=3_000_000, + max_fee_rate=100, + min_local_ratio=60.0, + blacklist=blacklist, + ) + + # Only Cheap-High-Liquidity-Node should qualify + assert len(candidates) == 1 + c = candidates[0] + assert c["chan_id"] == "111111111111111111" + assert c["alias"] == "Cheap-High-Liquidity-Node" + # Dynamic swap amount brings local balance down to 50% (5,000,000), so 8,500,000 - 5,000,000 = 3,500,000 + assert c["proposed_amt"] == 3_500_000 + + +def test_filter_and_size_candidates_fixed_amount(sample_channels): + """Test candidate filtering when an explicit target amount is passed.""" + blacklist = ["02blacklist1111111111111111111111111111111111111111111111111111111"] + # User specifies 4,000,000 sats + candidates = swap_out_loop.filter_and_size_candidates( + channels=sample_channels, + target_amt=4_000_000, + min_capacity=3_000_000, + max_fee_rate=500, # allow expensive one as well + min_local_ratio=60.0, + blacklist=blacklist, + ) + # Both Cheap and Expensive qualify and have proposed_amt == 4,000,000 + assert len(candidates) == 2 + for c in candidates: + assert c["proposed_amt"] == 4_000_000 + + +def test_parse_loop_quote_output_verbose(): + sample_quote_output = """ +Send off-chain: 2000000 sat +Receive on-chain: 1997799 sat + +Estimated on-chain fee: 153 sat +Loop service fee: 2048 sat +Estimated total fee: 2201 sat + +No show penalty (prepay): 30000 sat +Conf target: 9 block +CLTV expiry delta: 0 block +Publication deadline: 2026-09-20 16:41:19 +0200 CEST +""" + quote = swap_out_loop.parse_loop_quote_output(sample_quote_output, 2_000_000) + assert quote["send_offchain"] == 2_000_000 + assert quote["receive_onchain"] == 1_997_799 + assert quote["estimated_onchain_fee"] == 153 + assert quote["service_fee"] == 2048 + assert quote["total_loop_fee"] == 2201 + + +def test_parse_loop_quote_output_standard(): + sample_standard_output = """ +Send off-chain: 3500000 sat +Receive on-chain: 3496000 sat +Estimated total fee: 4000 sat +""" + quote = swap_out_loop.parse_loop_quote_output(sample_standard_output, 3_500_000) + assert quote["send_offchain"] == 3_500_000 + assert quote["receive_onchain"] == 3_496_000 + assert quote["total_loop_fee"] == 4000 + + +def test_calculate_economic_cost(): + """Verify total economic cost calculation and effective PPM.""" + result = swap_out_loop.calculate_economic_cost( + amt=2_000_000, + service_fee=2_000, + onchain_fee=200, + routing_fee=50, + local_fee_rate=20, + ) + assert result["server_fee"] == 2_000 + assert result["onchain_fee"] == 200 + assert result["routing_fee"] == 50 + assert result["opportunity_cost"] == 40 + assert result["total_cost"] == 2_290 + assert result["effective_ppm"] == 1145 + + +def test_queryroutes_parsing_success(): + """Test parsing lncli queryroutes JSON.""" + mock_routes = { + "routes": [ + { + "total_time_lock": 967983, + "total_fees": "120", + "total_amt": "2000120", + "hops": [ + {"chan_id": "111111111111111111", "pub_key": "02aaa..."}, + {"chan_id": "999999999999999999", "pub_key": "021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d"}, + ], + } + ] + } + with patch.object(swap_out_loop, "run_command") as mock_run: + mock_run.return_value = (True, mock_routes, None) + success, fee, hops = swap_out_loop.query_route_to_loop( + config={}, + dest_pubkey="021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d", + amt=2_000_000, + outgoing_chan_id="111111111111111111", + dry_run=False, + ) + assert success is True + assert fee == 120 + assert hops == 2 + + +def test_prepay_probe_success(): + """Test prepay probe detecting INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS as proof of viable route.""" + mock_output = { + "failure_reason": "FAILURE_REASON_INCORRECT_PAYMENT_DETAILS", + "payment_error": "incorrect or unknown payment details", + "htlcs": [ + { + "failure": {"code": "INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS"}, + "route": {"total_fees": "120", "hops": [{"chan_id": "111"}, {"chan_id": "222"}]}, + } + ], + } + with patch.object(swap_out_loop, "run_command") as mock_run: + mock_run.return_value = (True, mock_output, None) + success, fee, hops, err = swap_out_loop.send_prepay_probe( + config={}, + dest_pubkey="021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d", + amt=2_000_000, + outgoing_chan_id="111111111111111111", + ) + assert success is True + assert fee == 120 + assert hops == 2 + assert err is None + + +def test_prepay_probe_channel_failure(): + """Test prepay probe failing due to lack of intermediate liquidity.""" + mock_output = { + "failure_reason": "FAILURE_REASON_NO_ROUTE", + "payment_error": "temporary channel failure", + } + with patch.object(swap_out_loop, "run_command") as mock_run: + mock_run.return_value = (False, mock_output, "Probe failed: no route") + success, fee, hops, err = swap_out_loop.send_prepay_probe( + config={}, + dest_pubkey="021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d", + amt=2_000_000, + outgoing_chan_id="111111111111111111", + ) + assert success is False + assert "no_route" in err.lower() or "no route" in err.lower() or "temporary channel failure" in err.lower() + + +def test_probe_direct_route_to_loop_success(): + """Test probe_direct_route_to_loop building direct route and probing with fake hash.""" + build_output = { + "route": { + "total_fees": "12206", + "hops": [{"pub_key": "03peer"}, {"pub_key": "02loop"}], + } + } + probe_output = { + "failure": {"code": "INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS"} + } + with patch.object(swap_out_loop, "run_command", return_value=(True, build_output, None)), patch("subprocess.run") as mock_subproc: + mock_subproc.return_value.stdout = json.dumps(probe_output) + success, fee, hops, err = swap_out_loop.probe_direct_route_to_loop( + config={}, + remote_pubkey="03peer", + dest_pubkey="02loop", + amt=4_880_000, + ) + assert success is True + assert fee == 12206 + assert hops == 2 + assert err is None + + +def test_prepay_probe_fallback_to_direct_route(): + """Test send_prepay_probe falling back to direct route when multi-hop probe fails.""" + fail_output = { + "failure_reason": "FAILURE_REASON_NO_ROUTE", + "payment_error": "temporary channel failure", + } + with patch.object(swap_out_loop, "run_command", return_value=(False, fail_output, "temporary failure")), patch.object(swap_out_loop, "probe_direct_route_to_loop", return_value=(True, 12206, 2, None)) as mock_dir: + success, fee, hops, err = swap_out_loop.send_prepay_probe( + config={}, + dest_pubkey="021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d", + amt=4_880_000, + outgoing_chan_id="111111111111111111", + remote_pubkey="03peer", + ) + assert success is True + assert fee == 12206 + assert hops == 2 + assert err is None + mock_dir.assert_called_once() + + +def test_get_loop_db_path(): + """Test determining loop_db_path from config.ini or default path.""" + config = configparser.ConfigParser() + config.add_section("loop") + config.set("loop", "loop_db_path", "/tmp/custom_loop_sqlite.db") + path = swap_out_loop.get_loop_db_path(config) + assert path == "/tmp/custom_loop_sqlite.db" + + +def test_fetch_loop_history_from_db_sot(): + """Test querying Loop Out history directly from Loop's SQLite DB (Source of Truth).""" + with tempfile.TemporaryDirectory() as tmpdir: + db_path = os.path.join(tmpdir, "loop_sqlite.db") + conn = sqlite3.connect(db_path) + cur = conn.cursor() + cur.execute(""" + CREATE TABLE swaps ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + swap_hash BLOB, + initiation_time TIMESTAMP, + amount_requested BIGINT, + label TEXT + ); + """) + cur.execute(""" + CREATE TABLE loopout_swaps ( + swap_hash BLOB PRIMARY KEY, + dest_address TEXT, + outgoing_chan_set TEXT + ); + """) + cur.execute(""" + CREATE TABLE swap_updates ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + swap_hash BLOB, + update_timestamp TIMESTAMP, + update_state INTEGER, + server_cost BIGINT, + onchain_cost BIGINT, + offchain_cost BIGINT + ); + """) + hash1 = bytes.fromhex("11223344556677889900aabbccddeeff11223344556677889900aabbccddeeff") + cur.execute("INSERT INTO swaps (swap_hash, initiation_time, amount_requested, label) VALUES (?, ?, ?, ?)", + (hash1, "2026-09-20 14:00:00", 5000000, "Loop-Out: block-iad-1 (896468114071224320)")) + cur.execute("INSERT INTO loopout_swaps (swap_hash, dest_address, outgoing_chan_set) VALUES (?, ?, ?)", + (hash1, "bc1ptest", "896468114071224320")) + cur.execute("INSERT INTO swap_updates (swap_hash, update_timestamp, update_state, server_cost, onchain_cost, offchain_cost) VALUES (?, ?, ?, ?, ?, ?)", + (hash1, "2026-09-20 14:45:00", 2, 5000, 153, 12206)) + conn.commit() + conn.close() + + swaps = swap_out_loop.fetch_loop_history_from_db(db_path, limit=10) + assert len(swaps) == 1 + s0 = swaps[0] + assert s0["swap_id"] == "11223344556677889900aabbccddeeff11223344556677889900aabbccddeeff" + assert s0["amount"] == 5000000 + assert s0["server_fee"] == 5000 + assert s0["onchain_fee"] == 153 + assert s0["routing_fee"] == 12206 + assert s0["total_cost"] == 17359 + assert s0["effective_ppm"] == 3471 + assert s0["status"] == "SUCCESS" + assert "block-iad-1" in s0["label"] + + +def test_fetch_loop_history_from_cli_fallback(): + """Test querying loop listswaps via RPC CLI when SQLite DB direct access is not used.""" + mock_listswaps = { + "swaps": [ + { + "id": "aabbccddeeff", + "type": "LOOP_OUT", + "amt": "4000000", + "state": "SUCCESS", + "initiation_time": "1726830000000000000", + "cost_server": "4000", + "cost_onchain": "150", + "cost_offchain": "8000", + "label": "Loop-Out: Sunny Sarah", + "outgoing_chan_set": ["1055691691402854401"], + }, + { + "id": "112233", + "type": "LOOP_IN", + "amt": "20000000", + "state": "SUCCESS", + } + ] + } + with patch.object(swap_out_loop, "run_command", return_value=(True, mock_listswaps, None)): + swaps = swap_out_loop.fetch_loop_history_from_cli(config={}, limit=10) + assert len(swaps) == 1 + assert swaps[0]["swap_id"] == "aabbccddeeff" + assert swaps[0]["amount"] == 4000000 + assert swaps[0]["total_cost"] == 12150 + assert swaps[0]["status"] == "SUCCESS" + + +def test_export_history_to_csv(): + """Test exporting parsed history to CSV file.""" + with tempfile.TemporaryDirectory() as tmpdir: + csv_path = os.path.join(tmpdir, "test_history.csv") + sample_swaps = [ + { + "swap_id": "test1234", + "initiation_time": "2026-09-20 14:00:00", + "amount": 5000000, + "label": "Loop-Out: test", + "outgoing_chan_set": "123456", + "server_fee": 5000, + "onchain_fee": 150, + "routing_fee": 1000, + "total_cost": 6150, + "effective_ppm": 1230, + "status": "SUCCESS", + } + ] + swap_out_loop.export_history_to_csv(sample_swaps, csv_path) + assert os.path.exists(csv_path) + with open(csv_path, "r", encoding="utf-8") as f: + reader = list(csv.DictReader(f)) + assert len(reader) == 1 + assert reader[0]["swap_id"] == "test1234" + assert reader[0]["total_cost"] == "6150" + + +def test_dry_run_execution(): + """Verify that dry_run mode does not invoke litloop out or send real funds.""" + with patch.object(swap_out_loop, "run_command") as mock_run: + result = swap_out_loop.execute_loop_out( + config={}, + channel_id="111111111111111111", + amt=2_000_000, + conf_target=9, + max_routing_fee=100, + dest_addr=None, + dry_run=True, + ) + assert result["dry_run"] is True + assert result["success"] is True + mock_run.assert_not_called() + + +def test_interactive_menu_select_non_tty(): + """Verify fallback to text input when stdin is not a TTY.""" + sample_candidates = [ + { + "alias": "Node-A", + "chan_id": "111", + "proposed_amt": 2000000, + "effective_ppm": 1100, + "local_ratio": 80.0, + "server_fee": 2000, + "onchain_fee": 150, + "routing_fee": 10, + "opportunity_cost": 40, + "total_cost": 2200, + }, + { + "alias": "Node-B", + "chan_id": "222", + "proposed_amt": 3000000, + "effective_ppm": 1150, + "local_ratio": 75.0, + "server_fee": 3000, + "onchain_fee": 150, + "routing_fee": 20, + "opportunity_cost": 60, + "total_cost": 3230, + }, + ] + + with patch('sys.stdin.isatty', return_value=False): + with patch('builtins.input', return_value='1'): + selected = swap_out_loop.interactive_menu_select(sample_candidates) + assert selected is not None + assert selected['chan_id'] == '111' + + with patch('builtins.input', return_value='q'): + selected = swap_out_loop.interactive_menu_select(sample_candidates) + assert selected is None + + +def test_evaluate_single_candidate_success(): + """Test evaluate_single_candidate successfully validating quote, route, and prepay probe.""" + candidate = { + "chan_id": "111", + "proposed_amt": 2_000_000, + "alias": "Test-Peer", + "local_fee_rate": 10, + "remote_pubkey": "03peer", + } + mock_quote = {"service_fee": 2000, "estimated_onchain_fee": 150} + with patch.object(swap_out_loop, "get_loop_quote", return_value=mock_quote), patch.object(swap_out_loop, "query_route_to_loop", return_value=(True, 25, 2)), patch.object(swap_out_loop, "send_prepay_probe", return_value=(True, 25, 2, None)): + res = swap_out_loop.evaluate_single_candidate( + c=candidate, + config={}, + loop_cmd=["litloop"], + loop_pubkey="02loop", + conf_target=9, + probe_timeout=15, + skip_prepay_probe=False, + ) + assert res is not None + assert res["chan_id"] == "111" + assert res["service_fee"] == 2000 + assert res["routing_fee"] == 25 + assert res["total_cost"] == 2195