This commit introduces a ban manager that marks peers as banned if
they send too many invalid channel announcements to us. Expired
entries are purged after a certain period of time (currently 48 hours).
This commit adds the ability to store closed channels by scid in
the database. This will allow the gossiper to ignore channel
announcements for closed channels without having to do any
expensive validation.
This commit expands the definition of the dust limit to take into
account commitment fees as well as dust HTLCs. The dust limit is now
known as a fee exposure threshold. Dust HTLCs are fees anyways so it
makes sense to account for commitment fees as well. The link has
been modified slightly to calculate dust. In the future, the switch
dust calculations can be removed.
This commit introduces a filterManager that uses our outbound peers'
feefilter values to determine an acceptable minimum feerate that
ensures successful transaction propagation. Under the hood, a moving
median is used as it is more resistant to shocks than a moving average.
This commit introduces a wrapper function fetchFundingTxWrapper
which calls fetchFundingTx in a goroutine. This is to avoid an issue
with pruned nodes where the router is attempting to stop, but the
prunedBlockDispatcher is waiting to connect to peers that can serve
the block. This can cause the shutdown process to hang until we
connect to a peer that can send us the block.
This is to avoid a potential race on WriteMessage and Flush internals.
Because there is no locking on WriteMessage and Flush, if we allow
writeMessage calls in Start after the writeHandler has started,
the writeMessage calls may call WriteMessage/Flush at the same time
that writeMessage calls from the writeHandler does. Since there is
no locking, internals like b.nextHeaderSend can race and cause
panics.
Without this the following could happen:
* InboundPeerConnected is called while we already have an inbound
connection with the peer. This calls removePeer which calls Disconnect.
* If the peer is starting up in Start, it may be sending messages
synchronously via SendMessage(true, ...). This eventually calls the
writeMessage function which will exit if disconnect is set to 1.
* Since Disconnect was called, disconnect will be 1 and writeMessage
will exit, causing writeHandler to exit.
* If there is more than 1 message being sent, later messages will
queue in queueHandler but be unable to get into sendQueue as the
writeHandler goroutine has exited.
* The synchronous sends will be waiting on the errChan indefinitely
and startReady will never get closed meaning Disconnect will never
proceed.
The end result is that the server's mutex will be held until shutdown.
Avoid this by using writeMessage to bypass the writeHandler goroutine.
Prior to this commit, taproot channels had a bug:
- If a disconnect happened before peer.AddNewChannel was called,
then the subsequent reconnect would call peer.AddNewChannel and
attempt the ChannelReestablish dance.
- peer.AddNewChannel would call NewLightningChannel with
populated nonce ChannelOpts. This in turn would call
InitRemoteMusigNonces which would create a new musig pair session
and set the channel's pendingVerificationNonce to nil.
- During the reestablish dance, ProcessChanSyncMsg would be called.
This would also call InitRemoteMusigNonces, except it would fail
since pendingVerificationNonce was set to nil in the previous
invocation.
To fix this, we add a new functional option to signal to the init logic
that it doesn't need to call InitRemoteMusigNonces in in
ProcessChanSyncMsg.
* htlcswitch/hop: use InvalidOnionVersion for replayed packets
The link will send an update_fail_malformed_htlc, so we need to set
the BADONION bit. Since there isn't a replay-specific error, we
set the failure code to InvalidOnionVersion which has the BADONION bit.
* release-notes: update for 0.17.1
This tests that the funding manager doesn't immediately consider
a coinbase transaction that is also a funding transaction usable
until the maturity has been reached.
When a block is disconnected due to a reorg, DisconnectBlockAtHeight
is called for the block height. Prior to this patch, it would delete
every SCID in the graph with a block height greater than the
disconnected height. This meant that a reorg would delete every
zero-conf channel edge from the graph. The fix simply iterates up
until the StartingAlias and deletes every SCID between the
disconnected height and the StartingAlias height.
This changes the call-sites in several places to use the *P2P variants
to not trigger an OOM on untrusted input. This makes the code safe with
the new tlv version. Note that the call-sites prior to this change were
also safe.
Previously, the Switch would not check waiting-close channels' fwdpkgs
for settles or fails to reforward. This could result in a force close
in a rare edge case if a restart occurred at the wrong time. Now,
waiting-close fwdpkgs are checked and the issue is avoided.
NewForce's Pause method doesn't reset the ticker, so a test flake
would occur in TestChannelLinkCancelFullCommitment where
PendingCommitTicker.Pause() was called, but the underlying timer was
still ticking. When PendingCommitTicker.Resume() was called, an
unlucky Ticks() call could end up firing, leading to the link being
shut down.
This fixes a bug where a caller would:
- call NotifyWhenOnline and pass in a channel
- the server's mutex is held and the pubStr is checked in peersByPub
- the peer object is in the peersByPub map
- the peer object has its quit channel closed
- QuitSignal is called in the select statement, and the function
returns
- the caller is still waiting for the channel to be sent on forever
since it was never sent on or added to the peerConnectedListeners
slice.
This patch fixes the above bug by adding the channel to the
peerConnectedListeners slice if the QuitSignal select case is called.
This is required by BOLT#07 as otherwise the counter-party will
discard the channel_update as they may not consider the channel
"ready" or reorg-safe. Most other implementations besides eclair
have work-arounds for this, but it is nice to be spec-compliant.
This updates the RPCAcceptor to send the correct commitment type
even if the zero-conf or scid-alias channel types are set. This also
adds two bools to the ChannelAcceptRequest struct that denotes whether
the funder set the zero-conf and scid-alias channel types.
With this change, transactions created via craftSweepTx will be
standard. Previously, p2wsh/p2pkh scripts passed in via SendCoins would
be weighted as p2wpkh scripts. With a feerate of 1 sat/vbyte,
transactions returned would be non-standard. Luckily, the critical
sweeper subsystem only used p2wpkh scripts so this only affected
callers from the rpcserver.
Also added is an integration test that fails if SendCoins manages
to generate a non-standard transaction. All script types are now
accounted for in getWeightEstimate, which now errors if an unknown
script type is passed in.
This removes the requirement that the zero-conf channel acceptor
flow use anchors. Also adds a fail-early check for minimum depth
zero in the non zero conf case. It would fail later, but it makes
more sense to fail immediately when receiving AcceptChannel.
This commit modifies the netann subsystem to use the peer's alias
for ChannelUpdates where appropriate (i.e. in case we are sending
the alias to the peer). It also modifies the loadActiveChannels
function in the peer package to handle upgrading a channel when the
scid-alias feature bit is turned on.
AddInvoice,AddHoldInvoice now issue invoices that include our
peer's aliases. Some extra sanity checks are included to ensure we
don't leak our confirmed SCID for a private channel.
feature-bit channels
This allows opening zero-conf chan-type, scid-alias chan-type, and
scid-alias feature-bit channels. scid-alias chan-type channels are
required to be private. Two paths are available for opening a zero-conf
channel:
* explicit chan-type negotiation
* LDK carve-out where chan-types are not used, LND is on the
receiving end, and a ChannelAcceptor is used to enable zero-conf
When a zero-conf channel is negotiated, the funding manager:
* sends a FundingLocked with an alias
* waits for a FundingLocked from the remote peer
* calls addToRouterGraph to persist the channel using our alias in
the graph. The peer's alias is used to send them a ChannelUpdate.
* wait for six confirmations. If public, the alias edge in the
graph is deleted and replaced (not atomically) with the confirmed
edge. Our policy is also read-and-replaced, but the counterparty's
policy won't exist until they send it to us.
When a scid-alias-feature channel is negotiated, the funding manager:
* sends a FundingLocked with an alias:
* calls addToRouterGraph, sends ChannelUpdate with the confirmed SCID
since it exists.
* when six confirmations occurs, the edge is deleted and re-inserted
since the peer may have sent us an alias ChannelUpdate that we are
storing in the graph.
Since it is possible for a user to toggle the scid-alias-feature-bit
to on while channels exist in the funding manager, care has been taken
to ensure that an alias is ALWAYS sent in the funding_locked message
if this happens.
This intent of this change is to prevent privacy leaks when routing
with aliases and also to allow routing when using an alias. The
aliases are our aliases.
Introduces are two maps:
* aliasToReal:
This is an N->1 mapping for a channel. The keys are the set of
aliases and the value is the confirmed, on-chain SCID.
* baseIndex:
This is also an N->1 mapping for a channel. The keys are the set
of aliases and the value is the "base" SCID (whatever is in the
OpenChannel.ShortChannelID field). There is also a base->base
mapping, so not all keys are aliases.
The above maps are populated when a link is added to the switch and
when the channel has confirmed on-chain. The maps are not removed
from if the link is removed, but this is fine since forwarding won't
occur.
* getLinkByMapping
This function is introduced to adhere to the spec requirements that
using the confirmed SCID of a private, scid-alias-feature-bit
channel does not work. Lnd implements a stricter version of the spec
and disallows this behavior if the feature-bit was negotiated, rather
than just the channel type. The old, privacy-leak behavior is
preserved.
The spec also requires that if we must fail back an HTLC, the
ChannelUpdate must use the SCID of whatever was in the onion, to avoid
a privacy leak. This is also done by passing in the relevant SCID to
the mailbox and link. Lnd will also cancel back on the "incoming" side
if the InterceptableSwitch was used or if the link failed to decrypt
the onion. In this case, we are cautious and replace the SCID if an
alias exists.
This allows the router to determine what is and isn't an alias from
lnd's definition of an alias. Any ChannelAnnouncement that has an
alias ShortChannelID field is not verified on-chain. To prevent a
DoS vector from existing, the gossiper ensures that only the local
lnd node can send its ChannelAnnouncements to the router with an
alias ShortChannelID.
An OptionalMsgField has been added that allows outside subsystems
to provide a short channel id we should insert into a ChannelUpdate
that we then sign and send to our peer.
When the gossiper receives a ChannelUpdate, it will query the
alias manager by the passed-in FindBaseByAlias function to determine
if the short channel id in the ChannelUpdate points to a known
channel. If this lookup returns an error, we'll fallback to using
the original id in the ChannelUpdate when querying the router.
The lookup and potential fallback must occur in order to properly
lock the multimutex, query the correct router channels, and rate
limit the correct short channel id. An unfortunate side effect of
receiving ChannelUpdates from our peer that reference on of our
aliases rather than the real SCID is that we must store this policy.
Yet it is not broadcast-able. Care has been taken to ensure the
gossiper does not broadcast *any* ChannelUpdate with an alias SCID.
The cachedNetworkMsg uses the new processedNetworkMsg struct. This
is necessary so that delete-and-reinsert in the funding manager
doesn't process a ChannelUpdate twice and end up in a deadlock since
the err chan is no longer being used.
This introduces the a store for managing all things alias-related.
There are two maps:
* baseToSet:
This stores the "base" short channel id as the key. The value is
the set of all aliases issued for this channel. The "base" SCID is
whatever is stored in the OpenChannel's ShortChannelID member. For
everything other than zero-conf channels, this is the confirmed SCID.
For zero-conf channels, this is the very first alias assigned. This is
used mostly by the Switch to retrieve a set of aliases and determine
if it is safe to forward.
* aliasToBase:
This stores the reverse mapping of baseToSet. Each key is an alias
SCID and the value is the "base" SCID. This is exclusively used by
the gossiper to determine if an alias in a ChannelUpdate our peer
sends actually references a channel we know of.
The functions make use of the above two maps:
* AddLocalAlias:
This persists the {alias, base} pair in the database. The baseToSet
map is populated. The aliasToBase is optionally populated depending on
where this function is called from. Upgrade cases, where the
scid-alias feature bit is toggled and channels already exist, will
not persist to the gossip map. This is mainly to simplify the tangle
of logic that would otherwise occur.
* GetAliases:
This fetches the set of aliases by using the passed-in base SCID. This
is used in the Switch and other places where the alias set is needed.
* FindBaseSCID:
This fetches the base given an alias. This is used in the gossiper to
determine validity of a peer's ChannelUpdate that contains an alias.
* DeleteSixConfs:
This removes the aliasToBase map entry for the given "base". This is
used when the gossiper mappings are no longer needed, i.e. when the
channel has six confirmations and is public.
* PutPeerAlias:
Stores the peer's alias.
* GetPeerAlias:
Fetches the peer's alias.
* RequestAlias:
Generates an alias for us in the range 16000000:0:0 and
16250000:16777215:65535
This extends the Reservation arguments to include whether a pending
channel open has negotiated the zero-conf channel type, the scid-alias
channel type, and/or the scid-alias feature bit. The result of those
negotiates are stored in the OpenChannel's ChanType. The arguments to
NewChannelReservation have also been simplified.
This introduces a BigSize migration that is used to expand the width
of the ChannelStatus and ChannelType fields. Three channel "types"
are added - ZeroConfBit, ScidAliasChanBit, and ScidAliasFeatureBit.
ScidAliasChanBit denotes that the scid-alias channel type was
negotiated for the channel. ScidAliasFeatureBit denotes that the
scid-alias feature bit was negotiated during the *lifetime* of the
channel. Several helper functions on the OpenChannel struct are
exposed to aid callers from different packages.
The RefreshShortChanID has been renamed to Refresh.
A new function BroadcastHeight is used to guard access to the
mutable FundingBroadcastHeight member. This prevents data races.
This allows the zero-conf and scid-alias feature bits to be toggled
using the config. The feature bits are off by default to protect users
from accidentally incurring the risk of a zero-conf channel.
This defines the zero-conf feature bit, the scid-alias feature bit,
the zero-conf channel type, and the scid-alias channel type. It also
defines the dependency "tree" that exists for the feature bits.
The scid-alias feature bit signals that the node requires an alias
short channel id to be sent in funding_locked. The scid-alias channel
type requires that the channel is private, in addition to some other
forwarding-related privacy measures.
With this, extra calls to RemoveLink will wait for the link to
fully stop. This is accomplished by a map that stores a single stop
channel that callers to RemoveLink will listen on. This map is not
consulted when the Switch is shutting down and calls Stop on each
individual link. Though that could be added in the future, it is
not necessary.
Without this, calls to DisconnectPeer would bypass the
peerTerminationWatcher and allow subsequent connect requests to
go through before the peer's links were fully shut down. This could
lead to force closes.
This prevents a panic where an insert is attempted on a nil map via
updateActiveHTLCs. This panic would occur if the channel arbitrator
was in a buggy state, possibly introduced by power loss or via
SIGKILL.
This can cause an intermittent panic otherwise if bbolt remaps itself
via munmap and mmap. From bbolt's documentation:
* Byte slices returned from Bolt are only valid during a transaction.
Once the transaction has been committed or rolled back then the memory
they point to can be reused by a new page or can be unmapped from
virtual memory and you'll see an unexpected fault address panic when
accessing it.
This commit fixes OOO notifications between backlog and non-backlog
events by having the non-backlog goroutines wait on a chan that
signals that backlog processing is complete.
This commit also replaces usage of the canceled atomic variable with
the cancelChan to signal that delivery of an event should no longer
occur. Atomics do not make perfect "sequence" points as the atomic
may be checked too early and the end-result of delivering to a stopped
ntfnQueue is the same. Using the cancelChan ensures that we do not
hang on sending to ntfnQueue.
Prior to this change, if SubscribeSingleInvoice or
SubscribeNotifications was called, it was possible that a state
change would never be delivered to the client. The sequence of
events would be:
- delivery of backlog events with invoice in the Open state
- invoice goes to the Accepted state, no client to notify
- client added to map
This is fixed by adding the client to the map first. However, with
this change alone it then becomes possible for notifications to be
delivered out of order. This is addressed in a following commit.
This fixes a nil-pointer-dereference that would occur if this was
called for a settled AMP invoice. Terms.PaymentPreimage is always
false for AMP invoices.
This allows Switch-initiated payments to be failed back if they don't
make it into a commitment. Prior to this commit, a Switch-initiated
HTLC could get "lost" meaning the circuit wouldn't get deleted except
if conditions were "right" and the network result store would never
be made aware of the HTLC's fate. Switch-initiated HTLC's are now
passed to the link's mailbox to ensure they can be failed back.
This change also special-cases the ErrDuplicateKeystone error from
OpenCircuits(...) so that callers of updateCommitTx() in the link
don't send an Error to the peer if they encounter the keystone error.
With the first async change, the keystone error should now always
be recoverable.
On startup, we'll check whether we have the coop close chan status
and have already broadcasted a coop close txn, and then make a
decision on whether to restart the process based on that.
The main idea is that NotifyContractUpdate adds the ContractUpdate to
a map called unmerged. It is populated in Start by shallow-copying the
activeHTLCs map values (htlcSet). The htlcSets underlying maps are not
copied, and so unmerged will just contain pointers to them. This should
be fine since unmerged will not modify them. At the call-sites of
activeHTLCs, it is updated to include the unmerged sets. This happens
with a mutex and should not cause any data race, even though it is
copying the underlying map pointers. No persistence should be
necessary since on restart, activeHTLCs and unmerged will just be
populated again.
Eugene Siegel