Commit Graph
100 Commits
Author SHA1 Message Date
Oliver Gugger 54a25146f4 rpcperms: add unique request ID
This commit adds a unique request ID that is the same for each gRPC
request and response intercept message or each request/response message
of a gRPC stream.
2021-11-08 15:01:26 +01:00
Oliver Gugger 9a28a4a9ff lnrpc+rpcperms+lntest: add unique MW message ID
This commit adds a new unique intercept message ID that is different for
each message sent to an RPC middleware. This will allow us to re-purpose
the existing request ID field into its original intention, which was to
have a unique ID for a full gRPC request/response pair or gRPC stream as
a whole.
2021-11-08 15:00:29 +01:00
Oliver Gugger 54584aabb6 Merge pull request #5930 from guggero/no-seed-backup-macaroon
config_builder: always send admin macaroon
2021-11-05 11:54:36 +01:00
Oliver Gugger 77e457a48e docs: update release notes 2021-11-05 10:50:07 +01:00
Oliver Gugger 08b196e236 config_builder: always send admin mac on chan 2021-11-05 10:49:37 +01:00
Oliver Gugger 87c5b76e4d lnd: simplify listener creation 2021-11-05 10:49:37 +01:00
Oliver Gugger eb2be7c7c0 lntest: count unconfirmed balance as well
It looks like in some cases (apparently mostly on Windows) a previous
sub test sometimes leaves some unconfirmed balance in the wallet. That
balance is confirmed in the next test when new coins are sent to the
wallet.
2021-11-02 20:36:45 +01:00
Oliver Gugger 066a83315e docs: update release notes 2021-11-02 20:36:43 +01:00
Oliver Gugger 2a6f128a43 itest: wait for channel to be properly closed
It looks like in some cases a channel is still being closed while we
already try to create a sweep output. In that case the pending closed
channel is still counted as anchor channel and a reserve output is
created. To make sure that doesn't happen, we make sure there are no
pending or open channels of any kind around before we create the sweep
transaction.
2021-11-02 20:36:27 +01:00
Oliver Gugger 2ca3eea3f7 itest: fix REST WebSocket ping/pong itest
This commit fixes a flake in the REST itest that was caused by an
invoice being created before the actual subscription client was
registered.
2021-11-02 20:36:26 +01:00
Oliver Gugger 32101db873 Merge pull request #5916 from blastshielddown/docs_5899
lnrpc: add public key detail to signrpc KeyDescriptor [skip ci]
2021-11-02 15:03:14 +01:00
Oliver Gugger 919b31a67b Merge pull request #5896 from schjonhaug/instructions-for-creating-swift-protoc-definitions
Added instructions on how to build proto definitions for Swift
2021-11-02 15:02:39 +01:00
Oliver Gugger 97786c72fc Merge pull request #5878 from halseth/filter-ascii-bbolt-name
kvdb/bolt_compact: filter out non-ASCII from bucket name before printing
2021-11-02 10:22:46 +01:00
Oliver Gugger e399814aea Merge pull request #5550 from bottlepay/postgres-run-channeldb
kvdb: run channeldb test suite on postgres
2021-11-02 10:21:12 +01:00
Oliver Gugger 5681c0eecf Merge pull request #5897 from lightningnetwork/config-parser-fix
Bug fixes for lightning-terminal
2021-10-26 19:49:15 +02:00
Oliver Gugger af09f11c1c rpcperms: don't intercept if no middleware is registered
If there is no middleware registered, we don't need to intercept any
call and therefore can skip the request and macaroon parsing section.
2021-10-26 17:48:10 +02:00
Oliver Gugger c02bf19fc5 config: fix reflection parsing in LiT
In the case where lnd's config struct is embedded inside another struct
(for example in lightning-terminal), the flag won't be found under its
original name. So we try to also look it up under the prefixed name.
2021-10-26 17:46:21 +02:00
Oliver Gugger 6450f0c080 Merge pull request #5895 from LN-Zap/upstream/fix/sweeps-missing-label
sweep: add label to sweep transactions
2021-10-26 15:25:55 +02:00
Oliver Gugger 8095901924 Merge pull request #5893 from guggero/waddrmgr-update
mod: update btcwallet dependency to fix crash
2021-10-25 15:13:55 +02:00
Oliver Gugger f045ee5e96 Merge pull request #5881 from alexbosworth/patch-6
lnrpc: log channel point in coop close warning
2021-10-25 11:30:08 +02:00
Oliver Gugger ad9c160bc9 Merge pull request #5845 from guggero/itest-fixes-rest
itest: fix build matrix, WebSocket test timeout and log file upload
2021-10-25 11:16:20 +02:00
Oliver Gugger bdf787b248 Merge pull request #5879 from lightningnetwork/go-get-deprecated
lnrpc: migrate `go get` to `go install`
2021-10-25 09:52:42 +02:00
Oliver Gugger 47879d95e8 mod: update btcwallet dependency to fix crash
Fixes #5864.
Updates the btcwallet dependency to the version that fixes the
concurrent map access crash.
2021-10-25 09:50:58 +02:00
Oliver Gugger 27632daa98 itest: test path finding without cache 2021-10-22 09:32:50 +02:00
Oliver Gugger 01015aced4 routing: run path finding tests against cache and DB
To make sure we don't hit any edge cases we refactor the path finding
tests to run both with and without the cache enabled.
2021-10-22 09:32:49 +02:00
Oliver Gugger 0a2ccfc52b multi: use single read transaction for path finding
This commit partially reverts bf27d05a.
To avoid creating multiple database transactions during a single path
finding operation, we create an explicit transaction when the cached
graph is instantiated.
We cache the source node to avoid needing to look that up for every path
finding session.
The database transaction will be nil in case of the in-memory graph.
2021-10-22 09:32:49 +02:00
Oliver Gugger 1fef2970cf channeldb: implement DB fallback for path finding
If the in-memory graph cache is disabled, we fall back to querying the
database.
2021-10-22 09:32:48 +02:00
Oliver Gugger f216da32f3 lnd+channeldb: add graph cache option to channeldb
With this commit we forward the config option for disabling the channel
graph cache as a boolean to the channeldb. But we invert its meaning to
make the flag easier to understand.
2021-10-22 09:32:48 +02:00
Oliver Gugger a2ad533136 docs+lncfg+sample-lnd.conf: add no-graph-cache option 2021-10-22 09:32:47 +02:00
Oliver Gugger 4e224fe0aa GitHub: fix matrix, package log files before upload
It turns out we were using the wrong matrix variable in the actual
make command and ran the same itest 6 times with no arguments, all
resulting in running the btcd test.

To avoid uploading too many files in individual requests, we zip them
first before uploading the zip itself.
2021-10-21 12:07:15 +02:00
Oliver Gugger 423de4d79a itest: defer close of done channel
The testing framework uses runtime.Goexit() when a test fails which
still allows the defer calls to execute. That's why we should use defer
to close the done channel to allow all goroutines to exit properly.
2021-10-21 11:09:52 +02:00
Oliver Gugger d43854aa34 docs: add remote signing documentation and release notes 2021-10-14 16:14:50 +02:00
Oliver Gugger a6ea019f56 lntest: add itest for remote signing 2021-10-14 16:14:49 +02:00
Oliver Gugger 36dc4b3abf itest: refactor tests to be re-used 2021-10-14 15:42:59 +02:00
Oliver Gugger 78b700387c cmd/lncli+walletrpc: add createwatchonly command 2021-10-14 15:42:58 +02:00
Oliver Gugger 1541b2ef1b multi: create and list all default internal accounts 2021-10-14 15:42:56 +02:00
Oliver Gugger ceb31f9034 multi: allow watch-only wallet creation 2021-10-14 15:42:55 +02:00
Oliver Gugger 6d339f31c0 rpcwallet: forward FinalizePsbt call over RPC 2021-10-14 15:42:53 +02:00
Oliver Gugger a3addcc927 multi: forward SendCoins call over RPC 2021-10-14 15:42:52 +02:00
Oliver Gugger 19db382e24 multi: forward address and import calls over RPC
In order to support the full range of on-chain functionality, including
importing watch-only accounts in the watch-only instance, we need to
forward some calls like creating new addresses or importing accounts to
the remote signing instance.
2021-10-14 15:42:51 +02:00
Oliver Gugger 9cae7ad3c2 multi: add remote RPC signing wallet implementation 2021-10-14 15:42:49 +02:00
Oliver Gugger 917cf4e99b signrpc: add compact_sig flag to SignMessage 2021-10-14 15:42:48 +02:00
Oliver Gugger 9fa9dd8e43 multi: extract key ring creation from chain control
To make it possible to supply our own implementation of a secret key
ring, we extract that part from the chain control and split the whole
chain control creation into two parts.
2021-10-14 15:42:47 +02:00
Oliver Gugger 1309c6afea multi: allow internal wallet to be watch-only 2021-10-14 15:42:45 +02:00
Oliver Gugger 6093393e2f multi: refactor SignMessage to specify hashing 2021-10-14 15:42:44 +02:00
Oliver Gugger 5d7e814ea8 Merge pull request #5674 from torkelrogstad/2021-08-26-http-mux
lntest: avoid global ServeMux
2021-10-13 10:45:57 +02:00
Oliver Gugger 00af978bf2 Merge pull request #5708 from guggero/remote-signer-extract-wallet
refactor: move towards more configurable implementation details
2021-10-08 14:23:03 +02:00
Oliver Gugger 0cfe0f43e7 Travis: bump ARM parallelity to 3 2021-10-08 12:22:09 +02:00
Oliver Gugger 82ae284577 docs: add release notes 2021-10-08 12:13:17 +02:00
Oliver Gugger e1da1f8941 multi: extract wallet initialization
With this commit we extract the wallet creation/unlocking and
initialization completely out of the main function. This will allow us
to use custom implementations in the future.
2021-10-08 12:13:15 +02:00
Oliver Gugger 047d8ea3bc config_builder+lnd: move wallet related code
We move some of the wallet related functions into the new file that
houses the new customizable implementations for some of our interfaces.
Since the next customizable interface will be the wallet, we move those
wallet and chain backend related helper functions.
NOTE: This is a pure code move.
2021-10-08 12:08:25 +02:00
Oliver Gugger f6d7e70f51 multi: extract database initialization 2021-10-08 12:08:25 +02:00
Oliver Gugger 0e279eb15a multi: refactor external subserver config
As a preparation for making more and more implementation details
configurable, we add a new ImplementationCfg struct that houses all the
interfaces that can be defined externally.
2021-10-08 12:08:23 +02:00
Oliver Gugger 140d5a8086 chainreg+lnd: split chain control initialization
As a preparation for extracting the wallet related initialization code,
we first need to separate the purely configuration related chain control
initialization from the wallet related code. We do that by splitting the
chain control into a partial and full struct that is initialized in two
parts. This also allows us to create the wallet configuration itself
outside of the chain control package and we need to thread through fewer
parameters through the chain control config.
2021-10-08 12:06:54 +02:00
Oliver Gugger 47f1b81a51 walletunlocker+lnd: move WalletUnlockParams
As a preparation for adding the wallet unlock params to the chain
control, we first need to move them out of the main package.
2021-10-08 12:06:54 +02:00
Oliver Gugger 862f712394 lnd+pilot: use minHTLCIn instead of whole config
To remove one more direct dependency to a variable in our main function,
we pass in the required parameter to the autopilot only instead of the
whole chain configuration.
2021-10-08 12:06:54 +02:00
Oliver Gugger cab800e1e7 chainreg: add chain source and channel constraints
As a preparation for separating the pure config related chain control
setup from the wallet related chain control, we store more information
in the chain control instance that is required for the wallet
initialization.
2021-10-08 12:06:53 +02:00
Oliver Gugger b22f51098a rpcperms+lnd: use macaroon service from interceptor chain
We'll refactor the wallet creation and unlock process in a following
commit and want to make it possible to not need a direct reference to
the macaroon service in our main function. Since we store it in the
interceptor chain anyway (if we're using macaroons in the first place),
we might as well use the instance there directly.
2021-10-08 12:06:53 +02:00
Oliver Gugger d12154154a chainreg+lnd: add block cache to chainreg config
The block cache size in the chainreg.Config previously wasn't used but
instead the block cache was passed in as a separate parameter. We
replace the cache size with the actual cache in the config to streamline
things somewhat.
2021-10-08 12:06:53 +02:00
Oliver Gugger e79d59dd4c multi: use key locator for lnwallet.MessageSigner
To simplify the message signing API even further, we refactor the
lnwallet.MessageSigner interface to use a key locator instead of the
public key to identify which key should be signed with.
2021-10-08 12:06:52 +02:00
Oliver Gugger afa03f22cc multi: use key locator only for signing messages
To simplify the API surface of a remote signer even more, we refactor
the SignMessage and SignMessageCompact calls to only accept a key
locator as we always know what key we're using for signing anyway.
2021-10-08 12:06:52 +02:00
Oliver Gugger 8b7c88537c multi: refactor SignDigestCompact into SignMessageCompact
To make it possible to use a remote lnrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
lnrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
2021-10-08 12:06:52 +02:00
Oliver Gugger 02757f6735 multi: refactor SignDigest into SignMessage
To make it possible to use a remote signrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
signrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
2021-10-08 12:06:51 +02:00
Oliver Gugger dd3719d5de Merge pull request #5832 from guggero/node-feature-log
channeldb: fix graph cache feature handling
2021-10-08 10:02:44 +02:00
Oliver Gugger 77bf6cfed3 channeldb: fix graph cache feature handling [skip ci]
Fixes #5830.
When a channel for a node is announced before the node itself is
announced on the network, it's possible that we have channels for a node
but no features defined yet. This was previously logged as a warning
which spammed the log unnecessarily.
2021-10-08 10:02:11 +02:00
Oliver Gugger c30152c1c0 Merge pull request #5842 from hsjoberg/gomobile-ios-xcframework
build: make gomobile iOS build to .xcframework folder
2021-10-08 09:58:03 +02:00
Oliver Gugger 0d2ea30ada docs: add release notes 2021-10-07 14:26:00 +02:00
Oliver Gugger 0fac6c400e itest: wait for node to fully start up 2021-10-06 16:29:49 +02:00
Oliver Gugger e9c9ec2e84 itest: exit goroutines on error 2021-10-06 16:29:44 +02:00
Oliver Gugger b925ebb92c GitHub: only upload logs on failure 2021-10-06 16:29:44 +02:00
Oliver Gugger 134be244b4 GitHub+Travis: move itests to GitHub Actions 2021-10-05 20:48:48 +02:00
Oliver Gugger c89637a4e2 itest: fix close of closed channel panic 2021-10-05 20:48:48 +02:00
Oliver Gugger 1774934666 multi: fix Postgres on Travis
Fixes the docker build that was caused by
https://github.com/docker-library/postgres/issues/884.
Using the alpine and version 13 image avoids the problem introduced
with postgres 14 and debian bullseye.
2021-10-05 20:48:48 +02:00
Oliver Gugger 6bc0862fdf itest: fix commitment_deadline context expired flake 2021-10-05 20:48:48 +02:00
Oliver Gugger bedba02a2d itest: enable slow mining by default
We now redirect the mineBlocks function to the mineBlocksSlow function
which waits after each mined block. To reduce the overall time impact of
using that function everywhere, we only wait 20 milliseconds instead of 50ms
after each mined block to give all nodes
some time to process the block. This will still slow down everything by a bit
but reduce flakes that are caused by different sub systems not being
up-to-date.
2021-10-05 20:48:47 +02:00
Oliver Gugger 98061dfd58 mod+lntest: disable stall handler in btcd mining node
The latest version of btcd allows its stall handler to be disabled. We
use that new config option to make sure the mining btcd node and the lnd
chain backend btcd node aren't disconnected if some test takes too long
and no new p2p messages are exchanged.
2021-10-05 20:48:47 +02:00
Oliver Gugger d688a2a7f5 GitHub: download key from git repo, update Roasbeef's key
With this commit we update the release template to download Roasbeef's
key from the git repository via GitHub.
We also update the key mentioned in the template to the new signing key.
[skip ci]
2021-10-05 10:06:40 +02:00
Oliver Gugger 51d19dad87 Merge pull request #5363 from guggero/psbt-no-final-tx
Allow skipping `PsbtFinalize` step during channel funding to support external broadcast
2021-10-04 12:37:51 +02:00
Oliver Gugger 692ea25295 Merge pull request #5642 from guggero/in-memory-graph
In-memory graph cache for faster pathfinding
2021-10-04 11:20:23 +02:00
Oliver Gugger 3e7df3d20d docs: add release notes 2021-10-04 11:17:58 +02:00
Oliver Gugger bf60b20def docs: add example for PSBT batch funding 2021-10-04 11:17:09 +02:00
Oliver Gugger 57253c0c05 itest: add test for fully external PSBT funding 2021-10-04 11:17:08 +02:00
Oliver Gugger 1608faf199 multi: allow skipping the PSBT finalize step
The FundingPsbtFinalize step is a safety measure that assures the final
signed funding transaction has the same TXID as was registered during
the funding flow and was used for the commitment transactions.
This step is cumbersome to use if the whole funding process is completed
external to lnd. We allow the finalize step to be skipped for such
cases. The API user/script will need to make sure things are verified
(and possibly cleaned up) properly.
2021-10-04 11:17:08 +02:00
Oliver Gugger d4136002c1 lnwallet: only set funding TX witness if we publish
During the final part of the channel funding negotiation we only need to
assemble the full funding TX with the witness if we are going to publish
the transaction ourselves. If the final funding TX is published
externally we don't need this information. This will make it possible to
skip the verify process for fully externally funded PSBT channels.
2021-10-04 11:17:08 +02:00
Oliver Gugger 9b04ae45e3 chanfunding: fix swapped godoc comments 2021-10-04 11:17:07 +02:00
Oliver Gugger fa8e8e6f99 Merge pull request #5761 from naveensrinivasan/naveen/feat/fix-mongo-CVE
mod:replace mongo driver to address CVE-2021-20329
2021-10-04 10:43:20 +02:00
Oliver Gugger c43b9e4fe7 Merge pull request #5759 from naveensrinivasan/naveen/feat/fix-GO-2021-0053
mod:replace gogo/protobuf for GO-2021-0053
2021-10-01 15:45:54 +02:00
Oliver Gugger 036041d671 Merge pull request #5665 from LN-Zap/upstream/feat/single-utxo-reserved
lnd: use change output index from authoring TX to check reserved value
2021-10-01 09:51:28 +02:00
Oliver Gugger 0b4e03f5fc multi: add golang 1.17 compatible build tags
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md

The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
2021-09-29 17:31:37 -07:00
Oliver Gugger 7aa9661d42 lntest: use errgroup for node startup
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
2021-09-29 17:31:32 -07:00
Oliver Gugger 493262e253 itest: fix flake in update_channel_status itest
This commit fixes a flake in the channel status update itest that
occurred if Carol got a channel edge update for a channel before it
heard of the channel in the first place. To avoid that, we wait for
Carol to sync her graph before sending out channel edge or policy
updates.

As always when we touch itest code, we bring the formatting and use of
the require library up to date.
2021-09-29 17:50:01 +02:00
Oliver Gugger 6240851f93 channeldb: optimize memory usage of initial cache fill
With this commit we use an optimized version of the node iteration that
causes fewer memory allocations by only loading the part of the graph
node that we actually need to know for the cache.
2021-09-29 17:00:06 +02:00
Oliver Gugger a5202a89e6 docs: add release notes 2021-09-29 17:00:05 +02:00
Oliver Gugger a95a3728b5 lnd+channeldb: pre-allocate cache size
To avoid the channel map needing to be re-grown while we fill the cache
initially, we might as well pre-allocate it with a somewhat sane value
to decrease the number of grow events.
2021-09-29 17:00:05 +02:00
Oliver Gugger bf27d05aa8 routing+server: use cached graph interface 2021-09-29 17:00:05 +02:00
Oliver Gugger 1d1c42f9ba multi: use minimal policy in cache 2021-09-29 17:00:04 +02:00
Oliver Gugger 15d3f62d5e multi: use cache for source channels 2021-09-29 17:00:04 +02:00
Oliver Gugger d6fa912188 multi: further decouple graph
To further separate the channel graph from the channel state, we
refactor the AddrsForNode method to use the graphs's public methods
instead of directly accessing any buckets. This makes sure that we can
have the channel state cached with just its buckets while not using a
kvdb level cache for the graph.
At the same time we refactor the graph's test to also be less dependent
upon the channel state DB.
2021-09-29 17:00:03 +02:00
Oliver Gugger ddea833d31 multi: extract address source into interface
As a preparation to have the method for querying the addresses of a node
separate from the channel state, we extract that method out into its own
interface.
2021-09-29 17:00:03 +02:00
Oliver Gugger c1f686f860 channeldb+funding: move opening channel state to DB
The funding manager doesn't need to know the details of the underlying
storage of the opening channel state, so we move the actual store and
retrieval into the channel database.
2021-09-29 17:00:02 +02:00