To avoid running into the "server is still starting" error when trying
to close a channel, we first wait for the error to disappear before we
try closing the actual channel.
This commit replaces most of the hard coded 10, 15, 20 and 30 second
timeouts with the default timeout. This should allow darwin users to
successfully run the parallel itests locally as well.
It seems that our itests don't perform correctly in a high CPU usage
scenario such as running 6 tests in parallel. Some goroutines don't get
enough execution time and causes check to time out.
By reducing the total number of parallel tests, we hope to give all
goroutines some more breathing room.
In high CPU usage scenarios such as our parallel itests, it seems that
some goroutines just don't get any CPU time before our test timeouts
expire. By polling 10 times less frequently, we hope to reduce the
overall number of goroutines that are spawned because of the RPC
requests within the polling code.
To make sure we build the exact version of btcd that is referenced in
the project's go.mod file and to not overwrite any binary the user might
already have installed on the system, we compile btcd into an explicit
file in the itest directory.
This should also speed up invocations of "make itest-only" because the
test harness doesn't always compile btcd on its own.
We also fix a bug with the version parsing where adding a "replace"
directive in the go.mod would result in the awk commands to extract the
wrong version. Because we no longer use the DEPGET goal to build and
install btcd, using a replace directive now actually works for itests.
To remove the need to have an extra make goal for the Windows itests, we
instead add the flag windows=1 that sets the make variable EXEC_SUFFIX
to properly add the ".exe" suffix to all executable names.
To make the Makefile a bit easier to understand, we remove the implicit
ITEST goal/command variable and switch all itest execution over to
explicit goals in the main Makefile.
With the new btcd version we can specify our own listen address
generator function for any btcd nodes. This should reduce flakiness as
the previous way of getting a free port was based on just picking a
random number which lead to conflicts.
We also double the default values for connection retries and timeouts,
effectively waiting up to 4 seconds in total now.
To avoid the "Error outside of test" log and to properly terminate the
test if a sub test fails, we need to correctly invoke them using the
RunTestCase method.
With this commit we make it possible to use an Onion v2 hidden service
address as the Neutrino backend.
This failed before because an .onion address cannot be looked up and
converted into an IP address through the normal DNS resolving process,
even when using a Tor socks proxy.
Instead, we turn any v2 .onion address into a fake IPv6 representation
before giving it to Neutrino's address manager and turn it back into an
Onion host address when actually dialing.
If we use a chain backend that only understands IP addresses (like
Neutrino for example), we need to turn any Onion v2 host addresses into
a fake IPv6 representation, otherwise it would be resolved incorrectly.
To do this, we use the same fake IPv6 address format that bitcoind and
btcd use internally to represent Onion v2 hidden service addresses.
To finally end the discussion what Dockerfile should be used for what
and whether we should build from local source or check out from git, we
place both Dockerfiles next to each other and explicitly document their
purpose.
This commit adds another GitHub workflow that is activated for each
pushed tag. The release binaries are compiled from that tag for all
supported architectures. A new release in the GitHub repository is then
drafted for the tag and the finished binary packages are uploaded to
that release.
We add a GitHub workflow that is triggered whenever a new version tag is
pushed. It will trigger a docker image build for that version and
automatically push it to the specified repo.
As of go version 1.15.x, the darwin-386 architecture is no longer
supported. Because we use that go version on Travis to assert all
architectures can be built successfully, we have to remove this
architecture from the list.
Because we now have conditionally compiled code that depends on the
architecture it is built for, we want to make sure we can build all
architectures that we also release. Since GitHub builds are very fast,
we can easily do this instead of only compiling for certain select
architectures.
This commit adds the compaction feature of the bbolt compact to our bolt
backend. This will try to open the DB in read-only mode and create a
compacted copy in a temporary file. Once the compaction was successful,
the temporary file and the old source file are swapped atomically.
With this commit we rename the existing AvailableDiskSpace function to
its correct name AvailableDiskSpaceRatio as it only returns a ratio. We
then go ahead and add a new function that returns the actual number of
free bytes available on a file system.
This also fixes some comments and always returns an error instead of
panicking.
Because we'll need to return the macaroon through the wallet unlocker
we cannot shut down its service before we have done so, otherwise
we'll end up in a deadlock. That's why we collect all shutdown
tasks and return them as a function that can be called after we've
initialized the macaroon service.
As a preparation for the next commit where we add proper wallet unlocker
shutdown handling, we move the calls that require cleanup down after the
creation of the wallet unlocker service itself.
To make sure no macaroons are created anywhere if the stateless
initialization was requested, we keep the requested initialization mode
in the memory of the macaroon service.
This commit adds the --stateless_init flag to all three wallet unlocker
operations. Once you initialize a wallet stateless, you need to set
this flag for every further wallet unlocker operation. Otherwise you
risk non-encrypted macaroon information to leak to the underlying
system.
To make sure the test that takes the longest overall time is always
started first, independent of the number of test tranches we run, we
move it to the beginning of the list. Because that test involves a lot
of waiting, it allows us to play around with the number of tranches more
efficiently.
Updating the fee of the mock estimator _after_ starting carol turned out
to be flaky and could lead to the new fee not being picked up in time
for the force close. That lead to carol not cpfp'ing the force closed
transaction.
To allow running multiple test tranches in parallel, we need a way to
make sure the TCP ports don't collide. We'll work with offsets for the
ports, using a different offset for each tranche.
As a preparation to fix an issue with the mempool wait, we clean up the
multi-hop itests a bit. We fix the formatting, use the require library
for assertions consistently and simplify some of the wait predicates.
Commonly used code is also extracted into functions.
In some cases the router isn't yet fully aware of all newly opened
channels. We need to give it some time to process the updates. Therefore
we add a wait for sending payments to give it a few more changes to
catch up.
We add a GitHub action to our workflow that makes sure all command line
flags of lnd that are available with the default build tags are
contained in the sample-lnd.conf file.
Because we'll add a check that enforces all command line flags that are
available with the default build tags to also be contained in the
sample-lnd.conf file, we first add all that currently are missing.
We create a new build flag for running the bitcoind tests without the
txindex enabled. We don't want this to be the default so we use a
negated build flag.
When running the make DEPGET target for the go-fuzz binaries, this entry
is auto added, even though we don't have a direct reference to it in our
code. Not sure why this is needed in the first place.
Because the bitcoind mirror is extremely slow, we spend at least 2 to 3
minutes of each bitcoind related test on just downloading the binary. We
can achieve the same result by just pulling the docker image and
extracting the binary from that.
Now that we have all functions that we need to complete the whole
PSBT channel funding flow, we change the itest to use Dave's wallet
to fund the channel from Carol to Dave through a PSBT.
The internal lock ID that the wallet kit subserver uses to lock inputs
for itself shouldn't be allowed to be used when locking inputs manually
over the RPC.
Oliver Gugger