Reduces the number of concurrent tranches of itests running to two on
ARM in an attempt to make the tests less flaky because of very high CPU
usage with the default 4 parallel tranches.
We'll want to re-use the abandon channel functionality for the batch
funding, as a cleanup in case the funding is aborted before publishing
any transaction.
We'll want to re-use the PSBT funding functionality from the wallet kit
sub server in the main RPC server. To be able to dynamically obtain an
instance of the wallet kit server we need to know its name under which
it registers itself in the list of sub server. We export the name so we
don't have to hard code it in the main server.
It seems that the GitHub CI runners share a CPU and therefore only have
very few ressources available. This causes the wallet unlocker and breach
arbiter tests to fail if we don't give them enough time to complete.
We add an additional test case to the on-chain fund recovery test that
tries restoring the same wallet from the extended master root key
instead of the seed.
In addition to creating a new wallet from an aezeed, we allow specifying
an exteded master root key as the main wallet key directly.
Because an exteded key (xprv) doesn't contain any information about the
creation time of the wallet, we must assume a birthday to start scanning
the chain from (if the user doesn't provide an explicit value). Since
lnd only uses SegWit addresses, it makes sense to
choose the date that corresponds to the first mainnet block that
contained SegWit transactions.
Restoring a wallet from an extended master root key will result in a
significantly longer initial wallet rescan time if the default value is
used.
Because we now get one more update per channel when closing it, we need
to update our test that looks at the close notifications sent by the
SubscribeChannelEvent RPC.
We want to make sure we can recover funds from a channel that was force
closed by the local node just before the node needed to be restored. We
add a special test case with specific assertions for that scenario.
With one more notification event being dispatched in the local force
close case we need to update one of the integration tests to account for
the additional message.
In case we force close a channel from our end the funds can be time
locked for up to 2 weeks worst case. If something happens to our node in
that time, we don't have any information on that channel anymore because
currently the channel is removed from the channel backup file the moment
the commitment transaction confirms.
Instead we want to keep the channel around in the SCB file until we've
actually swept the time locked funds.
It looks like lnd even supports recovering from such a case if the peer
is still online and can tell us what commit height to use for deriving
the commit point. If the peer isn't around anymore then external tools
could be used to brute force the commit height.
We might want to react to a channel being fully resolved after being
involved in a force close. For this we add a new callback and invoke it
where appropriate.
We'll want to be informed about a channel that's been fully resolved on
chain in case it was involved in a force close. We add a new event type
and emit method for it.
This commit uses the require library for the link nodes garbage collect
test and fixes a flake that was discovered while hunting for other
flakes. Some times the channel isn't updated fast enough so we can
detech it on first try. So we give it a few more tries to stabilize the
test.
This commit fixes an old flake in the neutrino anchor output tests. It
turns out that sometimes with Neutrino we don't have enough UTXOs in our
wallet to sweep both the local and remote anchor. This is very likely a
timing issue, we need to give the wallet more time to catch up with the
chain and process all transactions to find unspent outputs.
We address this two-fold: We add an additional UTXO to Alice. And then
we also make sure we detect both UTXOs properly after restarting.
From the error in the itest output log it is not clear whether scraping
the profile page caused a test to fail or whether it was just a
follow-up error. We make it a bit more clear with an added message.
To make it easier to see what tranche a failed test was running in, we
add the tranche index to the test name. This corresponds to the number
in the .logs-tranche<index> folder name so the logs can be found much
quicker.
Depends on btcsuite/btcwallet#757.
Pulls in the updated version of btcwallet and walletdb that have the DB
interface enhanced by their own View() and Update() methods with the
reset callback/closure supported out of the box. That way the global
package-level View() and Update() functions now become pure redirects.
Since we're now storing the content of multiple previously distinct
database files in etcd, we want to properly namespace them as not to
provoke any key collisions. We append a sub namespace to the given
global namespace in order to still support multiple lnd nodes using the
same etcd database simultaneously.
Because the btcwallet code uses the legacy walletdb interface we must
assume it is not fully concurrency safe. Therefore we make sure only a
single writer can be active at any given time for the wallet DB backend
when using etcd.
To have all the database backend related code in one place, we finally
also move the initialization of the wallet DB loader option into the
GetBackends() method.
The final database that needs to be made remote compatible is the
watchtower server and client database.
They are handled a bit differently because both of them are not always
active, only when specifically turned on in the config.
Even though the sphinx router's persistent replay log is not crucial in
the operation of lnd as its state can be re-created by creating a new
brontide connection, we want to make lnd fully stateless and therefore
have the option of not storing any state on disk.
The wallet and macaroon databases are the first files that are created
for a new node. When initializing those databases, the parent directory
is created if it does not exist.
With both of the databases now potentially being remote, that code that
creates the parent directory might never be executed and we need to do
that manually when parsing the configuration.
Otherwise we run into an error when we later try to create our default
macaroons in that folder and it doesn't exist.
The macaroon root keys should also be stored to the remote database if a
replicated backend such as etcd is used.
This commit refactors the macaroons service and wallet unlocker to
accept a kvdb backend directly instead of creating the bolt instance
automatically.
This commit gets rid of the concept of a local and remote database when
etcd is used. Instead the same backend is now used for both the
(previously renamed from local and remote DB) graph and channel state
databases.
This will make path finding extremely slow on etcd and will require
further optimizations and possibly a write-through cache for the graph
DB. But this is a requirement for making lnd itself fully stateless.
In order to separate our databases more clearly, we refactor the height
hint cache DB to use a kvdb backend instead of the channel DB instance
directly.
As a preparation to not have a local and remote version of the database
around anymore, we rename the variables into what their actual function
is. In case of the RPC server we even directly use the channel graph
instead of the DB instance. This should allow us to extract the channel
graph into its own, separate database (perhaps with better access
characteristics) in the future.
As a preparation to initialize more than just the channel database on
startup we introduce a new struct that holds a reference to each of our
database instances.
Since we don't really do anything with the errors we check in the itest
error whitelist but get a lot of failed Travis runs because of it, we
disable the check for the moment.
We bump the version of the healthcheck module to a version that does not
exist. But by doing so we can tag the merge commit of this PR with that
actual version and have everything working. That is faster than merging
the changes to the module, pushing a tag, then bumping the tag in
go.mod.
To reduce the likelyhood of the linter OOMing on the GitHub runner, we
exclude any generated code from being inspected and also tune the golang
garbage collector to be a bit more agressive.
To avoid a naming conflict with etcd, we rename our very generic
rpc.proto to lightning.proto to match the service name that's declared
within. This will break many external tutorials and possibly also our
API docs but the change needs to be done eventually.
Oliver Gugger