The problem: Depleted channels drastically lower the likelihood of successful payments. Without probing, any given link has roughly a 50% chance of fulfilling a routing request.
Question: Can we reasonably do something about this?
This post collects potential solutions — please view it as a reference and starting point. I’m happy for feedback, refinements, and critiques.
Some ideas included might be philosophically undesirable but deserve technical consideration.
1. Two Broad Strategies
Protocol changes (requiring updates)
Voluntary node behavior changes
2. Protocol Changes
Changes would modify the LN protocol, sometimes needing broad adoption.
Sharing liquidity information
Extend the gossip protocol to share liquidity info, either locally (e.g. the friend of a friend network) or network-wide.
Even partial signals (e.g., liquidity at channel ends) could be helpful.
Pros:
(Min-cost flow) routing becomes much easier as the uncertainty decreases.
Cons:
Honest signaling enforcement, bandwidth, and privacy.
Note: (Fee Free) offchain rebalancing schemes might also be enabled or supported on a protocol level. In particular offchain rebalancing can be modeled as a circulation problem instead of just finding a single rebalancing cycle.
Hierarchical topologies
Organize Lightning like real transport systems (compare with Highway, Trainsystems,…):
Spanning tree based “core” node networks (spanning trees don´t deplete but split if a link breaks)
Peripheral nodes connect strategically
Use sparse graphs or skip topologies
4. Conclusion
This is an overview, not a final prescription.
Many solutions interact and involve trade-offs between implementation overhead, privacy, scalability, and decentralization.
Feedback, missing ideas, and criticisms welcome! (:
Rather than use gossip to share liquidity info, another potential solution which I briefly describe in this post is to add a query message to get path(s) from a peer.
This improves the likelihood of successful payment because each hop can inform the requester of a feasible outbound channel set without revealing any of their channel balances.
Depending on implementation, this potentially allows nodes to change their routing policies (e.g fees) on a per transaction basic, which improves their control over liquidity flow.
Of course, this approach has a lot of implications, so I want to create follow-up on my previous post with details regarding an implementation and a cost/benefit analysis, but I think it’s actually a relatively simple way to solve a lot of existing problems.
Can you detail that a bit more? We’ve been using htlc_maximum_msat in eclair as a control valve like you suggested a few years ago, and it seems to be working quite nicely (I’m not sure how exactly we could quantify that “it works”, but at least we’ve implemented the mechanism and haven’t seen any specific issues with it).
I don’t know if other implementations have done it, but it’s nice because it doesn’t require any protocol changes: you only need to update your implementation, and other nodes on the network will respect the htlc_maximum_msat value you dynamically set. It creates a bit of a (reasonable) channel_update spam though.
Sorry I may have shortened my initial post a bit too much with respect to the control valves section. As you pointed out the semantics of htlc_maximum_msat are clear in the sense that your node will not accept an hltc of a larger amount and indeed sending nodes will respect this in path or flow finding (though it is not clear what happens if a node tries to add a second HTLC with the same preimage that also respects htlc_maximum_msat).
However I have seen various motivations of why node operators select a specific value for htlc_maximum_msat for example
I have seen node operators signaling the amount of liquidity that they have left in the channel (or a fraction of the remaining liquidity, such that they do not have to update the value after each successful routing).
In my initial Blog article I suggested to use a markov process such that two peers that share a channel converge to a ratio of htlc_maximum_msat values so that the expected flow through the channel is balanced - given the demand. That of course is a very different semantic than just selecting a fraction of the liquidity
I am happy to hear that you an confirmed that a version of my suggestion works for you. I’d love to see some quantification for this. Maybe if it has sufficient interest from your end we could out of band discuss how we could measure and quantify this. Maybe it gives insights on weather this is sufficient or if other steps are necessary.
Yes that was exactly why I advocated for htlc_maximum_msat initially. It works out of the box without protocol upgrade which is extremely nice! However as in my first reply a protocol wide agreement on what nodes signal with their value may make this mechanism even stronger.
This makes me think of @ZmnSCPxj 's Forwardable Peerswaps proposal. Essentially a way to voluntarily optimize on-chain swaps, “forwarding” them from net senders of liquidity to net receivers and replenishing liquidity across the whole route.
This would take what would be a local peerswap rebalancing one channel and have it rebalance many channels at once, with each participating node benefiting. Should at least be considered in this discussion imo.
Nearly all proposals listed above require some erosion of privacy.
In addition, flow valves do not fix the core complaint of payment failure anyway — they just signal the payment failure earlier.
We have to accept that any solution to payment failure requires onchain action. Full stop. The point is not to set onchain actions to 0. The point is to keep the onchain actions low, which is sufficient for scaling purposes.
Forwardable peerswaps scales a single onchain action to multiple channels being rebalanced at once. Even if only a single channel is rebalanced, it potentially still scales multiple smaller in-Lightning payments over that channel to a single onchain action. It also does not erode existing privacy, as the details of in-Lightning payments are not changed (unlike local forwarding or centralized routing).
Another thing to consider is sidepools. Sidepools are multiparty channels, but are unpublished. This implies that existing routing is not changed at all (avoiding “rethinking routing itself”) and existing privacy is preserved (multiparty channels leak routing to all participants in the channel, but in sidepools, the multiparty construct is restricted to only rebalancing 2-party channels). Again, this implies onchain actions to move funds to/from sidepools, but again, the point of scaling is to have multiple in-Lightning payments be summarized into a few onchain actions, not eliminate onchain actions to 0.
@nick sure I missed to include peerwaps and I agree they may be useful. Thanks for the reminder!
I think we fully agree though I would like to be a bit more precise in language and differentiate between infeasible payments and payment failure:
Infeasible payments (e.g. those where even if all liquidity states where known the min cut from sender to recipient is below the payment amount) need on chain action
Feasible payments can still fail - most likely due to channel depletion. Those can be drastically reduced (as shown here) via off chain liquidity management without any additional on chain transaction.
Note that in particular if you follow the first route (with or without forwardable peerswap) you may improve your situation but this will affect the network and you may drastically worsen the situation of the network (you could also improve it, but the fact is that you don´t know). This is why I am thinking about solutions where nodes can collaborate to find an improved liquidity state that is beneficial for everyone.
This kind of collaboration is something that should also be considered when doing a peerswap or any other method of onchain / offchain swap.