Commit Graph
100 Commits
Author SHA1 Message Date
Oliver Gugger 0a4962a18e itest: run sub itests correctly
To avoid the "Error outside of test" log and to properly terminate the
test if a sub test fails, we need to correctly invoke them using the
RunTestCase method.
2020-12-01 22:24:03 +01:00
Oliver Gugger 76d2c49a17 lnd: fix Onion v2 support for Neutrino backends
With this commit we make it possible to use an Onion v2 hidden service
address as the Neutrino backend.
This failed before because an .onion address cannot be looked up and
converted into an IP address through the normal DNS resolving process,
even when using a Tor socks proxy.
Instead, we turn any v2 .onion address into a fake IPv6 representation
before giving it to Neutrino's address manager and turn it back into an
Onion host address when actually dialing.
2020-11-30 22:42:57 +01:00
Oliver Gugger 6f3c8611f4 tor: convert onion v2 addrs into fake tcp6
If we use a chain backend that only understands IP addresses (like
Neutrino for example), we need to turn any Onion v2 host addresses into
a fake IPv6 representation, otherwise it would be resolved incorrectly.
To do this, we use the same fake IPv6 address format that bitcoind and
btcd use internally to represent Onion v2 hidden service addresses.
2020-11-30 22:42:57 +01:00
Oliver Gugger 94183e0f52 docker+docs: move dev Dockerfile to root, document dev build
To finally end the discussion what Dockerfile should be used for what
and whether we should build from local source or check out from git, we
place both Dockerfiles next to each other and explicitly document their
purpose.
2020-11-23 14:48:45 +01:00
Oliver Gugger 8fea653c61 release: skip version check in automated builds 2020-11-19 12:35:58 +01:00
Oliver Gugger 4246238ff5 GitHub: build release binaries and upload to release
This commit adds another GitHub workflow that is activated for each
pushed tag. The release binaries are compiled from that tag for all
supported architectures. A new release in the GitHub repository is then
drafted for the tag and the finished binary packages are uploaded to
that release.
2020-11-19 12:35:53 +01:00
Oliver Gugger 9494ad6fb1 GitHub: add workflow for building tagged docker images
We add a GitHub workflow that is triggered whenever a new version tag is
pushed. It will trigger a docker image build for that version and
automatically push it to the specified repo.
2020-11-17 20:30:35 +01:00
Oliver Gugger 30c2c0addc make: remove arch darwin-386 that is unsupported by go 1.15
As of go version 1.15.x, the darwin-386 architecture is no longer
supported. Because we use that go version on Travis to assert all
architectures can be built successfully, we have to remove this
architecture from the list.
2020-11-13 10:19:52 +01:00
Oliver Gugger 9bd8784ae8 GitHub: run cross compilation for all architectures
Because we now have conditionally compiled code that depends on the
architecture it is built for, we want to make sure we can build all
architectures that we also release. Since GitHub builds are very fast,
we can easily do this instead of only compiling for certain select
architectures.
2020-11-13 10:19:51 +01:00
Oliver Gugger 32ee527f6b server+htlcswitch: auto compact sphinx DB too 2020-11-13 10:19:51 +01:00
Oliver Gugger a8ef4fc158 channeldb: store and evaluate last compaction time 2020-11-13 10:19:51 +01:00
Oliver Gugger 505be0d8bc channeldb: implement compaction
This commit adds the compaction feature of the bbolt compact to our bolt
backend. This will try to open the DB in read-only mode and create a
compacted copy in a temporary file. Once the compaction was successful,
the temporary file and the old source file are swapped atomically.
2020-11-13 10:19:50 +01:00
Oliver Gugger 35c1fad517 server+healthcheck: rename function, add absolute disk space function
With this commit we rename the existing AvailableDiskSpace function to
its correct name AvailableDiskSpaceRatio as it only returns a ratio. We
then go ahead and add a new function that returns the actual number of
free bytes available on a file system.

This also fixes some comments and always returns an error instead of
panicking.
2020-11-13 10:19:50 +01:00
Oliver Gugger f8907fdb47 multi: add AutoCompact option to bolt backend
With this commit we thread the new AutoCompact flags all the way through
to the bolt backend.
2020-11-13 10:19:49 +01:00
Oliver Gugger 6131a53eb4 lncfg+channeldb: add autocompact flags to BoltConfig 2020-11-13 10:19:49 +01:00
Oliver Gugger 08042405c7 server: allow overwriting of DNS seeds 2020-11-13 10:07:03 +01:00
Oliver Gugger 1548528a3f lncfg+sample-lnd.conf: add DNS seed flag to chain cfg 2020-11-13 09:49:18 +01:00
Oliver Gugger c7eea13f95 Merge pull request #4763 from carlaKC/fix-build
lnd: return missing shutdown parameter
2020-11-12 08:16:38 +00:00
Oliver Gugger f51f247a1c docs: describe new itest make commands 2020-11-10 13:57:31 +01:00
Oliver Gugger 47e8882480 make+itest: allow the same test to be run in parallel 2020-11-10 13:57:31 +01:00
Oliver Gugger 72cacb9c5e Merge pull request #1288 from guggero/stateless-init
walletunlocker+lnd: add command line flag to allow passing admin macaroon after wallet creation
2020-11-07 12:39:04 +00:00
Oliver Gugger 0b9b7def15 docs: describe stateless wallet initialization 2020-11-07 11:24:35 +01:00
Oliver Gugger 24adf475ce lnd_test: add integration tests for stateless init 2020-11-07 11:24:35 +01:00
Oliver Gugger e7aa9256ab walletunlocker+lnd: implement stateless wallet initialization 2020-11-07 11:24:34 +01:00
Oliver Gugger 4b0b20c98b walletunlocker: use require library, fix linter issues
To prepare for new tests to be added, we first rewrite the existing
tests to use the require library and get rid of some smaller linter
issues.
2020-11-07 11:24:34 +01:00
Oliver Gugger b685a97fcd lnd: shutdown wallet unlocker after macaroon creation
Because we'll need to return the macaroon through the wallet unlocker
we cannot shut down its service before we have done so, otherwise
we'll end up in a deadlock. That's why we collect all shutdown
tasks and return them as a function that can be called after we've
initialized the macaroon service.
2020-11-07 11:24:34 +01:00
Oliver Gugger 71ba2a8e60 lnd: prepare for proper wallet unlocker cleanup
As a preparation for the next commit where we add proper wallet unlocker
shutdown handling, we move the calls that require cleanup down after the
creation of the wallet unlocker service itself.
2020-11-07 11:24:33 +01:00
Oliver Gugger 294cb4a966 macaroons: remember stateless init
To make sure no macaroons are created anywhere if the stateless
initialization was requested, we keep the requested initialization mode
in the memory of the macaroon service.
2020-11-07 11:24:33 +01:00
Oliver Gugger ae71d60715 macaroons: add ability to change the password or create a new root key 2020-11-07 11:24:33 +01:00
Oliver Gugger cd85e17b19 macaroons: use require for tests 2020-11-07 11:18:52 +01:00
Oliver Gugger 10673b9dfa lncli: add flags for stateless init to wallet unlocker commands 2020-11-07 11:18:52 +01:00
Oliver Gugger 4c8d374007 lnrpc: add arguments for stateless wallet initialization to wallet unlocker
This commit adds the --stateless_init flag to all three wallet unlocker
operations. Once you initialize a wallet stateless, you need to set
this flag for every further wallet unlocker operation. Otherwise you
risk non-encrypted macaroon information to leak to the underlying
system.
2020-11-07 11:18:52 +01:00
Oliver Gugger a606f462bc itest: move longest test to beginning
To make sure the test that takes the longest overall time is always
started first, independent of the number of test tranches we run, we
move it to the beginning of the list. Because that test involves a lot
of waiting, it allows us to play around with the number of tranches more
efficiently.
2020-11-04 11:03:37 +01:00
Oliver Gugger e6c47294fb itest: fix typo and formatting 2020-11-04 11:03:36 +01:00
Oliver Gugger ea4bb5dc5c itest: fix chanbackup restore flake
Updating the fee of the mock estimator _after_ starting carol turned out
to be flaky and could lead to the new fee not being picked up in time
for the force close. That lead to carol not cpfp'ing the force closed
transaction.
2020-11-04 11:03:34 +01:00
Oliver Gugger d4068e9892 travis+make: execute test groups in parallel 2020-11-04 11:03:33 +01:00
Oliver Gugger ca7564e4b4 itest: add flags for lnd executable 2020-11-04 11:03:31 +01:00
Oliver Gugger f358a4474d lntest: add log dir flag 2020-11-04 11:03:30 +01:00
Oliver Gugger 9bbf134237 itest: split tests into dynamic tranches 2020-11-04 11:03:29 +01:00
Oliver Gugger 891eb0d63d lntest: use nextAvailablePort for bitcoind 2020-11-04 11:03:27 +01:00
Oliver Gugger 5c04449dfd lntest: use nextAvailablePort for fee service 2020-11-04 11:03:26 +01:00
Oliver Gugger a50d337e42 lntest: lower initial port, add ApplyPortOffset function
To allow running multiple test tranches in parallel, we need a way to
make sure the TCP ports don't collide. We'll work with offsets for the
ports, using a different offset for each tranche.
2020-11-04 11:03:25 +01:00
Oliver Gugger eaf60a63e3 docs: update in-place rescan section 2020-11-04 09:12:24 +01:00
Oliver Gugger a7c81eb595 lnd: drop wallet transactions if flag is set
To force a full chain rescan, the wallet can now be instructed to drop
its full transaction history on startup.
2020-11-04 09:12:24 +01:00
Oliver Gugger 37817f192e config: add reset-wallet-transactions flag 2020-11-04 09:12:24 +01:00
Oliver Gugger d0c3666310 mod: update to latest btcwallet dep 2020-11-04 09:12:19 +01:00
Oliver Gugger 8b32285f48 Merge pull request #4721 from guggero/itest-flake-hunt
itest: cleanup and flake fix
2020-10-28 17:05:06 +00:00
Oliver Gugger 098598ec74 itest: sort error whitelist 2020-10-28 14:35:11 +01:00
Oliver Gugger fd684a8ffa itest: add new errors to whitelist 2020-10-28 14:35:10 +01:00
Oliver Gugger 34439fbc2a itest: cleanup multi-hop tests
As a preparation to fix an issue with the mempool wait, we clean up the
multi-hop itests a bit. We fix the formatting, use the require library
for assertions consistently and simplify some of the wait predicates.
Commonly used code is also extracted into functions.
2020-10-28 13:38:04 +01:00
Oliver Gugger d1b46211d8 itest: replace WaitForTxBroadcast with WaitForTxInMempool 2020-10-28 13:29:18 +01:00
Oliver Gugger 1714394add itest: fix send payment flake
In some cases the router isn't yet fully aware of all newly opened
channels. We need to give it some time to process the updates. Therefore
we add a wait for sending payments to give it a few more changes to
catch up.
2020-10-28 13:22:48 +01:00
Oliver Gugger e6b0141b10 itest: fix typos, formatting 2020-10-28 13:22:48 +01:00
Oliver Gugger ac16bd0fe3 Merge pull request #4699 from matheusdtech/failing-link-flake
itest: Fix flakes when payments cause chan closure
2020-10-28 09:10:35 +00:00
Oliver Gugger 747bd674dc Merge pull request #4716 from matheusdtech/fix-itest-comment
itest: Sync nodes to chain during singleHop tests
2020-10-27 09:36:46 +00:00
Oliver Gugger 9e4cfa0182 Merge pull request #4718 from yyforyongyu/fix-rpc-format
trivial: fix make rpc-format
2020-10-26 08:40:00 +00:00
Oliver Gugger d12f76fd6d Merge pull request #4626 from guggero/sample-conf-check
GitHub workflow: Check all command line flags are described in sample-lnd.conf
2020-10-16 13:12:22 +02:00
Oliver Gugger 7ea4caa635 github: add sample-lnd.conf to PR checklist 2020-10-16 08:53:16 +02:00
Oliver Gugger ab4f1c3b5a github+make: check all CLI flags are in sample-lnd.conf
We add a GitHub action to our workflow that makes sure all command line
flags of lnd that are available with the default build tags are
contained in the sample-lnd.conf file.
2020-10-16 08:53:16 +02:00
Oliver Gugger 96379f3503 sample-lnd.conf: remove extra whitespaces 2020-10-16 08:53:16 +02:00
Oliver Gugger b555357f9b sample-lnd.conf: add all flags without build tag
Because we'll add a check that enforces all command line flags that are
available with the default build tags to also be contained in the
sample-lnd.conf file, we first add all that currently are missing.
2020-10-16 08:53:15 +02:00
Oliver Gugger f98c74319c Merge pull request #4617 from guggero/no-txindex-itest
itest: add bitcoind test without txindex enabled
2020-10-09 15:51:20 +02:00
Oliver Gugger 4dcdfd98bb travis: add bitcoind test without txindex 2020-10-09 13:35:06 +02:00
Oliver Gugger 3e3618ae9a lntest: add build flag for disabling txindex on bitcoind
We create a new build flag for running the bitcoind tests without the
txindex enabled. We don't want this to be the default so we use a
negated build flag.
2020-10-09 13:35:04 +02:00
Oliver Gugger b7fc7c3dae lntest: use extraArgs, fix linter issues 2020-10-09 13:35:02 +02:00
Oliver Gugger 43c2031fa8 lntest: extract common bitcoind code into own file
To prepare for running multiple tests on bitcoind with different options
each time, we extract the common code into its own file.
2020-10-09 13:35:00 +02:00
Oliver Gugger 902d7dd812 Merge pull request #4643 from guggero/make-fuzz
Add fuzzing commands to Makefile
2020-10-09 11:03:30 +02:00
Oliver Gugger 41d8d56354 gitignore: ignore fuzzing artifacts 2020-10-09 09:53:56 +02:00
Oliver Gugger 1a1a9a6744 mod: add auto-added go mod changes
When running the make DEPGET target for the go-fuzz binaries, this entry
is auto added, even though we don't have a direct reference to it in our
code. Not sure why this is needed in the first place.
2020-10-09 09:53:56 +02:00
Oliver Gugger 5189cb799a make: add fuzz-build and fuzz commands 2020-10-09 09:53:55 +02:00
Oliver Gugger 916b75fd4e Merge pull request #4680 from guggero/travis-bitcoind-speedup
Travis+GitHub: update bitcoind to 0.20.1, extract bitcoind binary from docker image to speed up download
2020-10-09 09:43:52 +02:00
Oliver Gugger e8253479d5 Travis+GitHub: update to bitcoind 0.20.1, remove cache
With this commit we use the same version of bitcoind in all tests and
also remove the caches for the binary as docker has its own cache.
2020-10-08 21:30:11 +02:00
Oliver Gugger 785e797ef8 scripts: install bitcoind binary from docker image
Because the bitcoind mirror is extremely slow, we spend at least 2 to 3
minutes of each bitcoind related test on just downloading the binary. We
can achieve the same result by just pulling the docker image and
extracting the binary from that.
2020-10-08 21:29:51 +02:00
Oliver Gugger eb74011d5a itest: add error to whitelist 2020-10-08 20:39:11 +02:00
Oliver Gugger 843efc324b walletrpc: serialize TXID not outpoint
The txid_str field of a locked UTXO's outpoint shouldn't contain the
output index but only the reverse order hex serialized transaction ID.
2020-10-06 10:59:51 +02:00
Oliver Gugger 8e9f37b0fc lncli+docs: fix outpoint serialization in JSON output
This commit fixes the JSON output of the outpoint field of locked UTXOs
in the response of the PSBT fund command.
2020-10-06 10:59:50 +02:00
Oliver Gugger 0c0d114d18 mod: bump btcwallet dep to fix scriptSig in PSBT bug
To fix a bug where the scriptSig field was mistakenly set on nested
SegWit inputs of a PSBT, we bump the btcwallet dependency that contains
said fix.
2020-10-06 10:59:46 +02:00
Oliver Gugger f8e3b41d10 Merge pull request #4389 from guggero/psbt-signing
walletrpc: add new PSBT creation+signing RPCs
2020-10-03 11:47:55 +02:00
Oliver Gugger 84dfed3fe2 docs: update PSBT documentation 2020-10-03 10:34:57 +02:00
Oliver Gugger eb280fd248 itest: use new PSBT functions for funding flow
Now that we have all functions that we need to complete the whole
PSBT channel funding flow, we change the itest to use Dave's wallet
to fund the channel from Carol to Dave through a PSBT.
2020-10-03 10:34:56 +02:00
Oliver Gugger f114fb3c8d itest: use require library
We rewrite the test to use the require library to make it a
bit more condensed.
2020-10-03 10:34:56 +02:00
Oliver Gugger c206d062d5 lncli: add new PSBT wallet commands 2020-10-03 10:34:55 +02:00
Oliver Gugger 9a063355e2 walletrpc: disallow locking with internal ID
The internal lock ID that the wallet kit subserver uses to lock inputs
for itself shouldn't be allowed to be used when locking inputs manually
over the RPC.
2020-10-03 10:34:55 +02:00
Oliver Gugger 6229609be7 lnrpc+walletrpc: add RPCs for new PSBT methods 2020-10-03 10:34:53 +02:00
Oliver Gugger f947576f33 lnwallet+mock: add new PSBT methods 2020-10-03 10:34:39 +02:00
Oliver Gugger 9f5f98e736 btcwallet: use new base wallet methods 2020-10-03 10:34:39 +02:00
Oliver Gugger 936a83858b chanfunding: use util functions from psbt package 2020-10-03 10:34:39 +02:00
Oliver Gugger 8e0a162b8b mod: update to latest btcwallet version 2020-10-03 10:34:34 +02:00
Oliver Gugger fd962d322a Merge pull request #4650 from guggero/lncli-no-macaroons
lncli: fix macaroon usage when unlocking or using --no-macaroons
2020-09-26 10:01:32 +02:00
Oliver Gugger b28bf29c5f lncli: don't read macaroons when not using them
When we either don't use macaroons (because the global --no-macaroons
flag is set) or don't need them (for the wallet unlocker commands), we
don't try to read the file at all to avoid running into an error if the
file doesn't exist (which it doesn't in those two cases).
2020-09-25 15:54:10 +02:00
Oliver Gugger 44524fd2e5 lncli: fix go vet recommendations 2020-09-25 15:18:45 +02:00
Oliver Gugger ca679e0f0e lncli: move wallet unlocker commands into own file
This is a pure code move, we extract all wallet unlocker operations
(create, unlock, changepassword) into its own file.
2020-09-25 15:16:56 +02:00
Oliver Gugger cd41cd4d2d Merge pull request #4593 from guggero/itest-linter-fix
lntest: allow the main test files to be buildable w/o the rpctest build tag
2020-09-21 22:57:33 +02:00
Oliver Gugger 1558edbc3c travis+lint: fix memory usage
There is a setting to control how often the garbage collector is run.
Apparently this is a tradeoff between CPU and memory usage. If we can
limit the memory being used in that way, this allows us to use multiple
worker again, so overall this shouldn't be much slower than before.
2020-09-21 21:17:46 +02:00
Oliver Gugger 9f7d8dd92e lntest: make compilable without subserver build tags
To make it possible to compile the itests together with the other tests,
we don't want to use anything from the optional subservers.
2020-09-21 21:16:32 +02:00
Oliver Gugger ae7d69838e Merge pull request #4625 from guggero/letsencrypt-listen
lnd+config: allow Let's Encrypt listen IP to be set
2020-09-17 14:55:30 +02:00
Oliver Gugger 01ff012631 Merge pull request #4624 from mayankchhabra/patch-1
docs: remove duplicate "tor.streamisolation" option from sample-lnd.conf
2020-09-17 14:21:31 +02:00
Oliver Gugger d33ddffca4 sample-lnd.conf: change letsencryptport to letsencryptlisten 2020-09-17 12:31:01 +02:00
Oliver Gugger 85c2103f44 Merge pull request #4436 from yyforyongyu/accept-base-pubkey
signrpc: SharedKeyRequest accept raw pubkey
2020-09-17 11:38:00 +02:00
Oliver Gugger 00cb6fcc5d lnd+config: allow Let's Encrypt listen IP to be set
To make it possible to request a Let's Encrypt certificate by using a
different IP address where the port 80 might still be free, we add the
IP part to its configuration as well instead of just the port.
This makes it possible to use an IPv6 address for the ACME request if
all available IPv4 addresses already have their port 80 occupied.
2020-09-17 09:54:23 +02:00