This commit adds the integration test testDataLossProtection, that
ensures that when a node loses state, the channel counterparty will
force close the channel, and they both can recover their funds.
This commit makes the chainwatcher attempt to dispatch a remote close
when it detects a remote state with a state number higher than our
known remote state. This can mean that we lost some state, and we check
the database for (hopefully) a data loss commit point retrieved during
channel sync with the remote peer. If this commit point is found in the
database we use it to try to recover our funds from the commitment.
This commit adds a check for the LocalUnrevokedCommitPoint sent to us by
the remote during channel reestablishment, ensuring it is the same point
as they have previously sent us.
This commit enumerates the various error cases we can encounter when we
compare our remote commit chain to the view the remote communicates to us
via msg.NextLocalCommitHeight.
We now compare this height to our remote tail and tip height, returning
relevant error in case of a unrecoverable desync, and re-send a
commitment signature (including log updates) in case we owe one.
This commit enumerates the various error cases we can encounter when we
compare our local commit chain to the view the remote communicates to us
via msg.RemoteCommitTailHeight.
We now compare this height to our local tail height (note that there's
never a local "tip" at this point), returning relevant error in case of
a unrecoverable desync, and re-send a revocation in case we owe one.
This commit defines a few new errors that we can potentially encounter
during channel reestablishment:
* ErrInvalidLocalUnrevokedCommitPoint
* ErrCommitSyncLocalDataLoss
* ErrCommitSyncRemoteDataLoss
in addition to the already defined errors
* ErrInvalidLastCommitSecret
* ErrCannotSyncCommitChains
This commit moves the responsibility for publishing the funding tx to
the network from the wallet to the funding manager. This is done to
distinguish the failure of completing the reservation within the wallet
and failure of publishing the transaction.
Earlier we could fail to broadcast the transaction, which would cause us
to fail the funding flow. This is not something we can do directly,
since the CompeteReservation call will mark the channel IsPending in the
databas.e
This commit makes sure we delete a pending channel from the set of
activeReservations within the fundingmanager immediately after the
channel is moved to the openChannelBucket in the DB. Previously we
wouldn't do this before the funding tx was confirmed, making it possible
that failing the funding flow at a later point would try to cancel a
non-existent reservation context.
In this commit we modify the integration tests slightly, by setting the
parties that gets breached during the breach tests to --nolisten. We do
this to ensure that once the data protection logic is in place, they
nodes won't automatically connect, detect the state desync and recover
before we are able to trigger the breach.
This commit makes sure the channels that are force closed also are put
into the state "waiting close" before the commitment transaction is
confirmed, and exits this state when it confirms.
This was previously not checked, as this check was added before the
"waiting close" state was introduced.
This commit fixes a flake within the integration tests, where we would
mine a set of blocks before checking if Bob's sweep tx was in the
mempool. Usually this would pass since the blocks were generated before
the tx hit the miner's mempool, but sometimes it was mined and then we
would check the mempool.
This commit fixes this by correctly waiting immediately for Bob to sweep
his funds, as they are not time locked.
This commit increases the time we wait for a spend client to notify a
mempool spend from 50ms to 10s. This is done to catch the case where
bitcoind would use up to 7 seconds before notifying about a mempool
spend, which wasn't caught by the test.
This commit fix a bug within the bitcoind notifier logic, which would
ignore the passed mempool argument, and notify spentness whether the
spending transaction was confirmed or not. The logic used to fix this is
similar to what is already done for the btcd backend.
This commit handles a racy condition within the breacharbiter's justice
tx procedure. For backends that have no mempool we would check if an
HTLC output was spent and then try broadcasting the justice tx, but this
would fail since we wouldn't detect the spend before it was in a block.
The result was that we would continuously attempt to broadcast the
transaction, effectively ending up in an endless (until the second-level
tx actually comfirmed) loop.
Instead we now register for spend notifications in case broadcasting the
transaction fails, and then wait for any of the notifications to be
sent before trying again.
This is a necessary step to be able to make lnd work well only with
confimed transactions, and was a better solution than introducing
timeouts within the broadcast loop (which complicates integration
tests).
This commit adds a poll 20 ms interval to WaitPredicate, similar to what
is done for WaitInvariant. This makes the predicate not being checked
super-rapidly, potentially filling the logs with useless info over the
wait predicate interval.
This commit adds the helper method shutdownAndAssert, and makes us shut
down nodes created temporarily during the test cases using defers
instead of at the end of the test. This makes it easier to remember to
shut them down, as it is done right after they are started.
We remove the internale broadcastMessage method, and instead handle the
mutex handling within BroadcastMessage. This lets us hold the mutex only
when neccessary.
This commit removes the sendToPeer method from the server, and instead
moves the necessary logic into SendToPeer. This let's us make the mutex
acquisition more fine-grained, only holding it while reading from the
peer map. Earlier it was required to be held during the whole call to
sendToPeer, as the method would access the map internally.
We check if the channel is FullySynced instead of comparing the local
and remote commit chain heights, as the heights might not be in sync.
Instead we call FullySynced which recently was modified to use compare
the message indexes instead, which is _should_ really be in sync between
the chains.
The test TestChanSyncOweRevocationAndCommitForceTransition is altered to
ensure the two chains at different heights before the test is started, to
trigger the case that would previously fail to resend the commitment
signature.
This commit changes cancelReservationCtx to gold the resMtx from start
to finish. Earlier it would lock at different times only when accessing
the maps, meaning that other goroutines (I'm looking at you
PeerTerminationWatcher) could come in and grab the context in between
locks, possibly leading to a race.
This commit moves the responsibility of sending a funding error on the
reservation error channel inside failFundingFlow, reducing the places we
need to keep track of sending it.
This commit adds a test which will restore a channel from an OpenChannel
struct at various stages of the state transation cycle, ensuring the
HTLC local and remote add heights are restored properly.
This commit fixes a bug which would cause the add heights of the HTLCs
in the update log to be set wrongly. At times, an add height could be
incorrecly set, leading to the HTLCs not being accounted for correctly
during evaluating the HTLC views. This was caused by the assumption that
if the HTLC was not on the pending remote commit, then it was locked in
on both the local and the remote commit, which is not always true.
Instead of making this assumption, we instead now inspect the three
commits: the local, remote and pending remote; and set the add heights
accordingly. This should ensure that HTLCs are subtracted from the
balances only when they are first added.
This commit adds an integration test that checks that in case a channel
counterparty tries to settle an HTLC with the wrong preimage, the
channel is failed and force closed.
This commit makes the peer aware of the LinkFailureErrors that can
happen during link operation, and making it start a goroutine to
properly remove the link and force close the channel.
Adds a new closure OnChannelFailure to the link config, which is called
when the link fails. This function closure should use the given
LinkFailureError to properly force close the channel, send an error to
the peer, and disconnect the peer.
This commit introduces a new error type LinkFailureError which is used
to distinguish the different kinds of errors that we can encounter
during link operation. It encapsulates the information necessary to
decide how we should handle the error.
The pending state definitin in ChannelCloseSummary was slightly changed
in such a way that channels that has had their commitment broadcasted
now is no longer considered "pending close". They now instead stay in
the open chan bucket with the ChanStatus "CommitmentBroadcasted" until
their commitment is confirmed. This commit updates the IsPending godoc
to reflect this.
We no longer have to mark the channel as fully closed in the database,
as it is done directly in the chainWatcher. Instead, we stop the watcher
and delete it from the set of active watchers.
This commit makes the dispatchCooperativeClose method mark the channel
fully closed directly, without registering for confirmation
notifications first. We can do this as recent changes to the
contractcourt changed the definition of a closed channel in the database
to have had its closing tx confirmed, and we only dispatch the
cooperative close once the transaction has 1 confirmation.
We also rename the markChanClosed method to notifyChanClosed, to more
clearly indicate that the ChainArbitrator no longer has to mark the
channel fully closed in the database.
This commit stops the chan closer from sending the potential coop close
transactions to the chainwatcher, as this is no longer needed. The
chainwatcher recently was modified to watch for any potential close, and
will because of this handle the close regardless of which one appears in
chain.
When the chancloser broadcast the final close transaction, we mark it as
CommitmentBroadcasted in the database.
Johan T. Halseth