Spark swaps: payment timeout, no waiting on create, a Transactions tab and slow exits

docs/rebalance-algorithm.md

@@ -340,7 +340,7 @@ which one gets drained can differ between runs on an otherwise unchanged node.
 | `REBALANCE_INITIAL_RETRY_DELAY_SECONDS` | `60` | First retry delay. |
 | `REBALANCE_RETRY_BACKOFF_MULTIPLIER` | `2.0` | `delay = 60 · 2^(attempt−2)` ⇒ 60 s, 120 s. |
 | `REBALANCE_AMOUNT_BACKOFF_RATIO` | `0.8` | Retries shrink the amount by this factor per attempt. |
-| `REBALANCE_MAX_PARTS` | `32` | Max MPP shards. |
+| `SEND_PAYMENT_MAX_PARTS` | `32` | Max MPP shards of a payment sent with SendPaymentV2: rebalances and Spark swaps. `REBALANCE_MAX_PARTS` is still read as a fallback. |
 | `REBALANCE_DEFAULT_MAX_FEE_PCT` | `0.05` | Fallback cap when none is supplied (0.05 %). |
 | `REBALANCE_DEFAULT_RETRY_MAX_FEE_PCT` | `0.05` | Fallback retry cap. |
 | `REBALANCE_RECONCILE_TERMINAL_WINDOW_HOURS` | `24` | How far back `MonitorRebalancesJob` reconciles. |

docs/spark-swaps.md

@@ -15,23 +15,24 @@ The Spark client is the [NSpark](https://github.com/orklabs/nspark) .NET SDK, fr
 2. A swap that would bring the wallet over its max balance, counting what its swaps not paid yet will still bring in, is refused.
 3. The Spark wallet issues an invoice through the SSP. An invoice without a receive request is refused before anything is paid: the swap's funds are found only through the transfer the SSP reports for it.
 4. The swap is recorded before anything is paid: its payment hash, destination wallet and the Spark identity that will receive it. No address is reserved yet: the swap needs one only when it exits.
-5. The node pays the invoice. The payment is not cancellable. A definitive failure fails the swap. An unknown outcome, such as a broken payment stream, is left to the monitor.
+5. The node pays the invoice in the background: creation returns as soon as the swap is recorded, and the swap shows as pending on the Swaps page meanwhile. LND keeps trying routes for up to the node's Spark payment timeout (its liquidity settings on the Nodes page, else `SPARK_PAYMENT_TIMEOUT_MINUTES`), splitting the payment into up to `SEND_PAYMENT_MAX_PARTS` parts. The payment is not cancellable. Once paid, its routing fee is saved; a definitive failure fails the swap. An unknown outcome, such as a broken payment stream or NodeGuard restarting mid-payment, is left to the monitor, which finds the payment by its hash.
 
 **Payment.** `MonitorSwapsJob` runs every 10 minutes, or every minute in a dev environment. It looks the payment up on the node by its hash. The swap fails if the payment failed, or if the node never saw it and the invoice has expired. Once the payment has settled, the swap is `SparkSwapExitJob`'s.
 
 **Exit.** `SparkSwapExitJob` runs every `SPARK_EXIT_INTERVAL_MINUTES` (10 by default, 1 in a dev environment). Each paid swap exits on its own, from the Spark wallet it was paid into (found by its identity, whichever wallet the node uses now), to its own destination address, in saved steps: paid, then leaves attributed, then exit sent, then payout confirmed.
 1. **Leaves attributed.** A Spark balance is made of leaves of fixed sizes, and the payment arrives as one inbound transfer whose leaves add up to what it brought in. The job claims it and records on the swap the transfer (the one the SSP reports for the swap's receive request), its leaves and their total. A transfer is given to one swap only, and a swap only ever takes its own: while the SSP reports none, the swap waits, and nothing is matched by amount. Only a claimed transfer is attributed: one whose claim failed is claimed again on the next run. A transfer that can never be claimed leaves the swap waiting until the overdue alert.
 2. **Exit sent.** The job reserves the swap's address in the destination wallet and saves it on the swap, then exactly those leaves exit to it. No leaf is split or swapped with the SSP, so the remote signer can sign the exit too. The exit fee is capped by `SPARK_MAX_EXIT_FEE_SATS`, and the exit's txid is saved right away.
+   - The exit goes at `SPARK_EXIT_SPEED`, `SLOW` by default: the cheapest, at the cost of time. The SSP quotes each speed separately, a user fee plus an L1 broadcast fee at the fee rate that speed aims for: `250 × sats_per_vbyte + 750` in all ([Spark fees](https://docs.spark.money/wallets/estimate-fees), [Withdraw to L1](https://docs.spark.money/wallets/withdraw-to-l1)). Spark doesn't promise a confirmation time for any speed; under high fees a `SLOW` exit can take hours.
    - Leaves near their timelock floor are renewed first. A renewal keeps a leaf's id (the operators renew it in place), so the swap still exits its own leaves. If they can't move yet (a renewal failed, or they are frozen), the swap waits and tries again on the next run, and the overdue alert below flags it. No other leaves are ever taken in their place.
    - If the leaves already exited (a crash before the txid was saved), the job waits for the payout instead of exiting again.
-3. **Payout confirmed.** The swap completes when the destination wallet sees a confirmed payout to the reserved address. Matching on the address, rather than the exit's txid, survives the SSP fee-bumping the exit. The payout is checked before exiting, so an exit whose txid was not saved never runs twice.
+3. **Payout confirmed.** The exit's leaves stay the wallet's (owned, locked) until its transaction confirms; the swap stays pending meanwhile and still counts against the node's Max swaps in flight. The swap completes when the destination wallet sees a confirmed payout to the reserved address. Matching on the address, rather than the exit's txid, survives the SSP fee-bumping the exit. The payout is checked before exiting, so an exit whose txid was not saved never runs twice.
 
 All attributions run before any exit, and exits go strictly one at a time, oldest swap first. This is separate from Max swaps in flight: that limit decides how many swaps a node may have open; the job only orders their exits. With Max swaps in flight at 3, three swaps are paid in parallel, and the next run exits the first, then the second, then the third.
 
 **Fees and accounting.** What the swap delivered is what landed on-chain: its payout, shown as "Received on-chain" on the Swaps page. The fees add up to the amount minus the payout, plus the routing fee:
 - **Routing fee:** what the node paid on Lightning. It includes Spark's fixed 15 bps (0.15%), which Spark charges to the sender on the Lightning route through the invoice's route hints ([Spark fees](https://docs.spark.money/wallets/estimate-fees)); receiving into Spark is free.
 - **Service fee:** shown as Spark's 15 bps "in routing fee", not added to the total. Only if the payment brought in less than the amount is that shortfall recorded as a service fee.
-- **On-chain fee:** the move from Spark to L1: the SSP's exit fee and the L1 broadcast, what the swap's leaves held minus the payout.
+- **On-chain fee:** the move from Spark to L1: the SSP's exit fee and the L1 broadcast at the exit speed, what the swap's leaves held minus the payout.
 
 Each fee shows its BTC amount and its percentage of the swap, with ppm and USD in the tooltip.
 
@@ -40,7 +41,7 @@ Each fee shows its BTC amount and its percentage of the swap, with ppm and USD i
 Spark swaps are trust-minimized, not trustless the way Loop's are. A Loop swap is an on-chain HTLC: either the node's Lightning payment fails or the destination is paid on-chain, and only the Loop server's liveness is trusted. A Spark swap goes through three steps:
 
 1. **Lightning into Spark is atomic.** The operators hold shares of the invoice's preimage and release it to the SSP only when the SSP transfers the Spark leaves to the Spark wallet's identity. A threshold of operators (two of the three on mainnet: Lightspark, Breez and Flashnet) could release it without that transfer.
-2. **While the sats are on Spark**, from the payment until the exit confirms (minutes), they rely on the same operator threshold, which co-signs every leaf transfer. If the operators disappear, a unilateral exit with the leaves' pre-signed refund transactions still recovers the sats, but it is slow (timelocks) and costs on-chain fees. NSpark can snapshot what that exit needs (`RecoveryService.GetRecoverySnapshotAsync`); NodeGuard doesn't automate it.
+2. **While the sats are on Spark**, from the payment until the exit confirms (minutes at `FAST`, possibly hours at `SLOW`), they rely on the same operator threshold, which co-signs every leaf transfer. If the operators disappear, a unilateral exit with the leaves' pre-signed refund transactions still recovers the sats, but it is slow (timelocks) and costs on-chain fees. NSpark can snapshot what that exit needs (`RecoveryService.GetRecoverySnapshotAsync`); NodeGuard doesn't automate it.
 3. **The exit to on-chain is atomic.** Before signing anything, NSpark checks that the SSP's exit transaction pays the reserved address at least the amount minus the fee cap, and that the connector transaction spends it. The operators release the leaves to the SSP only once that exit transaction confirms.
 
 So the exposure is the sats in transit, while they are on Spark. That is why a Spark wallet is transit only: its max balance caps it, and the guardrails alert when sats get stuck or an exit is overdue.
@@ -62,7 +63,7 @@ Spark wallets let NodeGuard use Spark without the remote signer, for instance fo
 - **Storage.** The mnemonic is encrypted with ASP.NET Data Protection (purpose `NodeGuard.SparkWallet.Mnemonic.v1`). The key ring is NodeGuard's own, in Postgres (`DataProtectionKeys`), so it is shared by every replica and survives restarts and redeploys. This protects against a dump of the `Wallets` table alone, not of the whole database: the key ring itself is stored unencrypted. If the key ring is lost or NodeGuard's application name changes, the mnemonics can't be decrypted, and the words shown at creation are the only way back. A wallet whose seed doesn't decrypt is reported unavailable.
 - **Using one.** Nodes page ▸ liquidity settings ▸ Spark wallet, next to the Spark weight. A Spark weight above 0 needs a Spark wallet. Several nodes may share one. A node's Spark wallet can't change while the node has a Spark swap in flight. Until a node has one, the New Swap dialog lists no node for Spark and says so.
 - **Max balance.** Each Spark wallet has its own max balance (Edit max balance), the most it may hold counting a new swap and its swaps not paid yet. Empty means `SPARK_MAX_BALANCE_SATS`. For a mainnet test, keep it small (1-2 M sats).
-- **Withdraw all** exits everything the wallet holds to a new address of an on-chain wallet, the fee capped by `SPARK_MAX_EXIT_FEE_SATS`. It is refused while the wallet has a swap in flight, so a swap's sats never leave with the rest.
+- **Withdraw all** exits everything the wallet holds to a new address of an on-chain wallet, at `SPARK_EXIT_SPEED`, the fee capped by `SPARK_MAX_EXIT_FEE_SATS`. It is refused while the wallet has a swap in flight, so a swap's sats never leave with the rest.
 - **Archive** is allowed only for an empty wallet that no node uses.
 - On mainnet, NodeGuard logs a warning at startup while Spark wallets exist.
 
@@ -86,6 +87,8 @@ While Spark is enabled, invalid settings stop NodeGuard at startup.
 | `SPARK_MAX_EXIT_FEE_SATS` | `20000` | Highest SSP exit fee accepted per swap |
 | `SPARK_MAX_BALANCE_SATS` | `10000000` | Most sats a Spark wallet may hold, counting a new swap and its swaps not paid yet, unless it has its own max balance |
 | `SPARK_EXIT_INTERVAL_MINUTES` | `10`, `1` in a dev environment | How often paid swaps exit on-chain |
+| `SPARK_EXIT_SPEED` | `SLOW` | `FAST`, `MEDIUM` or `SLOW`: the fee rate swap exits and Withdraw all aim for. `SLOW` is the cheapest and confirms last; the overdue alert waits 6, 12 or 24 hours accordingly |
+| `SPARK_PAYMENT_TIMEOUT_MINUTES` | `10` | How long LND keeps trying routes for a swap's payment, 1 to 25 (the invoice lasts 30 minutes); a node can set its own in its liquidity settings |
 | `SPARK_OPERATOR_CERTS_DIR` | none; refused on mainnet | Self-signed operator certificates to pin (regtest) |
 | `SPARK_SIGNER_RIE_URL` | none; refused on mainnet | Reach a locally run signer image through the Lambda emulator |
 
@@ -106,8 +109,8 @@ With the remote signer, `REMOTE_SIGNER_ENDPOINT` must be the Function URL with n
 ## Guardrails and alerts
 
 The swap monitor checks each Spark wallet after each pass. Each alert below is audited once and logged with structured fields:
-- `SparkBalanceStuck`: for an hour, a Spark wallet has held more than its swaps in flight account for. Until a swap exits, what it brought in (or its amount, until that is known) is its own and never counts as stuck.
-- `SparkExitOverdue`: a swap was paid six hours ago and still has no confirmed payout.
+- `SparkBalanceStuck`: for an hour, a Spark wallet has held more than its swaps in flight account for. Until a swap completes, what it brought in (or its amount, until that is known) is its own and never counts as stuck, including while its exit waits to confirm.
+- `SparkExitOverdue`: a swap was paid and still has no confirmed payout after 6 hours at `FAST`, 12 at `MEDIUM` or 24 at `SLOW` (`SPARK_EXIT_SPEED`).
 
 Each wallet's balance is shown in its details on the Wallets page and exported as the `nodeguard.spark.balance` gauge (meter `NodeGuard.Spark`), tagged `spark.wallet` with the wallet's id (`remote-signer` for the remote signer's).
 

src/Data/Models/Node.cs

@@ -145,6 +145,12 @@ public class Node : Entity
         /// </summary>
         public int? SparkWalletId { get; set; }
 
+        /// <summary>
+        /// How long LND keeps trying routes for this node's Spark swap payments, in minutes; null means
+        /// SPARK_PAYMENT_TIMEOUT_MINUTES
+        /// </summary>
+        public int? SparkPaymentTimeoutMinutes { get; set; }
+
         #endregion Automatic Swap Out Configuration
 
         #region Routing Engine

src/Data/Repositories/Interfaces/ISwapOutRepository.cs

@@ -57,4 +57,7 @@ public interface ISwapOutRepository
    /// The Spark transfers already given to a swap, whatever its status: a transfer belongs to one swap only
    /// </summary>
    Task<HashSet<string>> GetSparkTransferIdsAsync();
+
+   /// <summary>The Spark swaps paid into the Spark wallet with this identity, whatever their status</summary>
+   Task<List<SwapOut>> GetSparkSwapsByIdentityAsync(string identityPublicKey);
 }

src/Data/Repositories/SwapOutRepository.cs

@@ -214,5 +214,16 @@ public async Task<HashSet<string>> GetSparkTransferIdsAsync()
 
          return transferIds.ToHashSet(StringComparer.OrdinalIgnoreCase);
       }
+
+      public async Task<List<SwapOut>> GetSparkSwapsByIdentityAsync(string identityPublicKey)
+      {
+         await using var context = await _dbContextFactory.CreateDbContextAsync();
+         var identity = identityPublicKey.ToLowerInvariant();
+
+         return await context.SwapOuts
+            .AsNoTracking()
+            .Where(s => s.Provider == SwapProvider.Spark && s.SparkIdentity != null && s.SparkIdentity.ToLower() == identity)
+            .ToListAsync();
+      }
    }
 }

src/Helpers/Constants.cs

@@ -199,10 +199,11 @@ public class Constants
     public static double REBALANCE_AMOUNT_BACKOFF_RATIO = 0.8;
 
     /// <summary>
-    /// Maximum number of partial payments (MPP shards) LND may split a rebalance into. 1
-    /// disables splitting; higher values let LND complete an amount across several routes.
+    /// Maximum number of partial payments (MPP shards) LND may split a payment sent with SendPaymentV2 into:
+    /// rebalances and Spark swap payments (SEND_PAYMENT_MAX_PARTS; REBALANCE_MAX_PARTS still read as a
+    /// fallback). 1 disables splitting; higher values let LND complete an amount across several routes.
     /// </summary>
-    public static uint REBALANCE_MAX_PARTS = 32;
+    public static uint SEND_PAYMENT_MAX_PARTS = 32;
 
     /// <summary>
     /// How far back the MonitorRebalancesJob sweeps for already-terminal-but-possibly-wrong
@@ -608,8 +609,9 @@ static Constants()
         var rebProbeBackoff = Environment.GetEnvironmentVariable("REBALANCE_AMOUNT_BACKOFF_RATIO");
         if (rebProbeBackoff != null) REBALANCE_AMOUNT_BACKOFF_RATIO = double.Parse(rebProbeBackoff, NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture);
 
-        var rebMaxParts = Environment.GetEnvironmentVariable("REBALANCE_MAX_PARTS");
-        if (rebMaxParts != null) REBALANCE_MAX_PARTS = uint.Parse(rebMaxParts);
+        var sendPaymentMaxParts = Environment.GetEnvironmentVariable("SEND_PAYMENT_MAX_PARTS")
+                                  ?? Environment.GetEnvironmentVariable("REBALANCE_MAX_PARTS");
+        if (sendPaymentMaxParts != null) SEND_PAYMENT_MAX_PARTS = uint.Parse(sendPaymentMaxParts);
 
         var rebReconcileWindow = Environment.GetEnvironmentVariable("REBALANCE_RECONCILE_TERMINAL_WINDOW_HOURS");
         if (rebReconcileWindow != null) REBALANCE_RECONCILE_TERMINAL_WINDOW_HOURS = int.Parse(rebReconcileWindow);