Due to a misunderstanding of how the gpg command line options work, we
didn't actually create detached signatures because the --clear-sign
flag would overwrite that. We update our verification script to now only
download the detached signatures and verify them against the main
manifest file.
We also update the signing instructions.
Instead of only allowing the installed versions of lnd and lncli to be
verified, we now also support specifying explicit paths to binaries that
we want to verify.
To fix an issue where the vendor.tar.gz in a release build had a
different hash if the mobile RPC stubs were in the mobile/ folder, we
clean those out first.
The culprit was the `google.golang.org/grpc/test/bufconn` package which
is currently only used in the mobile RPC stubs and nowhere else.
Therefore the vendor/module.txt was different when vendoring with the
generated mobile RPC stubs being around.
Because we now build a docker image for the RPC compilation, we can save
some execution minutes if we run the mobile RPC and code compilation check in the
same step of the CI workflow.
Now that we have a base docker image that has all our RPC compilation
dependencies installed, we can also run the mobile RPC compilation
there. This removes the need to install falafel and goimports on the
local machine.
Because we compile the REST code from the rest-annotations.yaml and no
longer import the annotations in the proto files, we don't need to
specify the custom import path anymore.
This commit aims to make it easier for developers to compile our
protobuf definitions. They now only need to have docker installed
instead of a whole set of binaries and libraries all pinned to very
specific versions.
When verifying the release signatures, we don't want to fail if a
signer's signature is not available in the gpg key ring. Instead we just
don't want to count the signature for now and still succeed if there's
at least one other good sig with a known key.
To avoid leaking any sensitive information like Docker Hub credentials
because of compromised actions repositories, we use our own, vendored
actions for all steps that potentially touch sensitive information.
The verification script makes sure the hashes of the binaries inside of
a docker image match those of an official release.
The script first downloads all signatures, validates them, then compares
the hashes of the installed binaries to those contained in the detached
signature files.
We add a new make goal called release-install that creates the same
reproducible binaries as the release script would create, but only for
the current OS/architecture. It then installs those binaries to the
system's GOBIN directory.
To allow easy verification of individual binaries (instead of just the
packag tarballs/zips), we also add the hashes of lnd and lncli binaries
to the manifest. We do the same in the docker build.
To prevent novice users from running non-release versions, we update the
installation instructions to point them towards our release binaries.
We also document the new Docker build helper that allows you to compile
lnd without needing to install golang.
A new top level feeurl option was added recently to replace the neutrino.feeurl
option. The new option was never added to the sample config file and the
text was never updated to reflect that the option is required for
neutrino on mainnet. We fix this and also add a valid mainnet example
URL to the sample config file.
A number of tricks need to be applied to the tar and zip commands
respectively to make sure they always produce deterministic archives.
This includes setting the file timestamps to a fixed date and setting
specific ownership attributes (tar) or no attributes at all (zip).
Apparently dropping the wallet transaction history only fully takes
effect after opening the wallet from scratch again. To do this cleanly,
we need to do it in the unlocker instead of lnd.
If a channel backup from an old node is restored and a channel contained
within that file was unconfirmed at the time of the backup, the whole
channel short ID is 0:0:0. Unfortunately we got stuck in just the first
case and didn't fall through to the second one. To avoid that, we flip
the case logic because for the second case, the block height in the
short channel ID cannot be zero.
To enable building docker images for ARM64 platforms as well,
we just need to specify the desired target platforms and the Docker
Buildx service will do the job for us (provided the base images support
the given platforms, which is the case for golang).
To avoid running into the "server is still starting" error when trying
to close a channel, we first wait for the error to disappear before we
try closing the actual channel.
This commit replaces most of the hard coded 10, 15, 20 and 30 second
timeouts with the default timeout. This should allow darwin users to
successfully run the parallel itests locally as well.
It seems that our itests don't perform correctly in a high CPU usage
scenario such as running 6 tests in parallel. Some goroutines don't get
enough execution time and causes check to time out.
By reducing the total number of parallel tests, we hope to give all
goroutines some more breathing room.
In high CPU usage scenarios such as our parallel itests, it seems that
some goroutines just don't get any CPU time before our test timeouts
expire. By polling 10 times less frequently, we hope to reduce the
overall number of goroutines that are spawned because of the RPC
requests within the polling code.
To make sure we build the exact version of btcd that is referenced in
the project's go.mod file and to not overwrite any binary the user might
already have installed on the system, we compile btcd into an explicit
file in the itest directory.
This should also speed up invocations of "make itest-only" because the
test harness doesn't always compile btcd on its own.
We also fix a bug with the version parsing where adding a "replace"
directive in the go.mod would result in the awk commands to extract the
wrong version. Because we no longer use the DEPGET goal to build and
install btcd, using a replace directive now actually works for itests.
To remove the need to have an extra make goal for the Windows itests, we
instead add the flag windows=1 that sets the make variable EXEC_SUFFIX
to properly add the ".exe" suffix to all executable names.
To make the Makefile a bit easier to understand, we remove the implicit
ITEST goal/command variable and switch all itest execution over to
explicit goals in the main Makefile.
With the new btcd version we can specify our own listen address
generator function for any btcd nodes. This should reduce flakiness as
the previous way of getting a free port was based on just picking a
random number which lead to conflicts.
We also double the default values for connection retries and timeouts,
effectively waiting up to 4 seconds in total now.
To avoid the "Error outside of test" log and to properly terminate the
test if a sub test fails, we need to correctly invoke them using the
RunTestCase method.
With this commit we make it possible to use an Onion v2 hidden service
address as the Neutrino backend.
This failed before because an .onion address cannot be looked up and
converted into an IP address through the normal DNS resolving process,
even when using a Tor socks proxy.
Instead, we turn any v2 .onion address into a fake IPv6 representation
before giving it to Neutrino's address manager and turn it back into an
Onion host address when actually dialing.
If we use a chain backend that only understands IP addresses (like
Neutrino for example), we need to turn any Onion v2 host addresses into
a fake IPv6 representation, otherwise it would be resolved incorrectly.
To do this, we use the same fake IPv6 address format that bitcoind and
btcd use internally to represent Onion v2 hidden service addresses.
To finally end the discussion what Dockerfile should be used for what
and whether we should build from local source or check out from git, we
place both Dockerfiles next to each other and explicitly document their
purpose.
This commit adds another GitHub workflow that is activated for each
pushed tag. The release binaries are compiled from that tag for all
supported architectures. A new release in the GitHub repository is then
drafted for the tag and the finished binary packages are uploaded to
that release.
We add a GitHub workflow that is triggered whenever a new version tag is
pushed. It will trigger a docker image build for that version and
automatically push it to the specified repo.
As of go version 1.15.x, the darwin-386 architecture is no longer
supported. Because we use that go version on Travis to assert all
architectures can be built successfully, we have to remove this
architecture from the list.
Because we now have conditionally compiled code that depends on the
architecture it is built for, we want to make sure we can build all
architectures that we also release. Since GitHub builds are very fast,
we can easily do this instead of only compiling for certain select
architectures.
This commit adds the compaction feature of the bbolt compact to our bolt
backend. This will try to open the DB in read-only mode and create a
compacted copy in a temporary file. Once the compaction was successful,
the temporary file and the old source file are swapped atomically.
With this commit we rename the existing AvailableDiskSpace function to
its correct name AvailableDiskSpaceRatio as it only returns a ratio. We
then go ahead and add a new function that returns the actual number of
free bytes available on a file system.
This also fixes some comments and always returns an error instead of
panicking.
Because we'll need to return the macaroon through the wallet unlocker
we cannot shut down its service before we have done so, otherwise
we'll end up in a deadlock. That's why we collect all shutdown
tasks and return them as a function that can be called after we've
initialized the macaroon service.
As a preparation for the next commit where we add proper wallet unlocker
shutdown handling, we move the calls that require cleanup down after the
creation of the wallet unlocker service itself.
To make sure no macaroons are created anywhere if the stateless
initialization was requested, we keep the requested initialization mode
in the memory of the macaroon service.
This commit adds the --stateless_init flag to all three wallet unlocker
operations. Once you initialize a wallet stateless, you need to set
this flag for every further wallet unlocker operation. Otherwise you
risk non-encrypted macaroon information to leak to the underlying
system.
To make sure the test that takes the longest overall time is always
started first, independent of the number of test tranches we run, we
move it to the beginning of the list. Because that test involves a lot
of waiting, it allows us to play around with the number of tranches more
efficiently.
Updating the fee of the mock estimator _after_ starting carol turned out
to be flaky and could lead to the new fee not being picked up in time
for the force close. That lead to carol not cpfp'ing the force closed
transaction.
To allow running multiple test tranches in parallel, we need a way to
make sure the TCP ports don't collide. We'll work with offsets for the
ports, using a different offset for each tranche.
Oliver Gugger