Commit Graph
100 Commits
Author SHA1 Message Date
Oliver Gugger be2566cf26 Merge pull request #5053 from guggero/verify-count
verify-install: add keys to repo, pin signatures to keys
2021-09-27 18:28:05 +02:00
Oliver Gugger 421add062c docs: add release notes 2021-09-27 15:49:12 +02:00
Oliver Gugger d7d1574095 scripts: verify version in manifest
To make a downgrade attack harder, we also check that the version string
is contained in the manifest, on the same line as the hash.
2021-09-27 15:49:12 +02:00
Oliver Gugger 5bafff2cf2 scripts: allow packaged release archive to be verified
Instead of only allowing to verify installed/unpacked binaries, we also
allow a single binary or packaged release archive to be verified.
2021-09-27 15:49:11 +02:00
Oliver Gugger 5afed8ab62 docker: add keys to docker image
Because the PGP keys are no longer downloaded from Keybase but are used
from the repo directly, we also have to copy them to the Docker image so
we can run the script without needing to supply them through a volume.
2021-09-27 15:49:11 +02:00
Oliver Gugger 6d0368a189 scripts: pin signatures to keys
With this commit we add a new restriction that checks that the
username in the signature file (manifest-<username>-<version>.sig) actually
does have a signing key and that the signature was created with that key.
The signature is only counted towards the minimum of 5 signatures if
that check is successful.
The changes in this commit were inspired by @kixunil in #5048.
2021-09-27 15:49:11 +02:00
Oliver Gugger 0984638a74 scripts: make sure signature files have unique names
To fix a simple attack where the same file would be uploaded multiple
times under the same name, we make sure we only count unique file names.
2021-09-27 15:49:10 +02:00
Oliver Gugger f6c66ca980 scripts: add signing keys to repo
Instead of importing the keys from Keybase, we add all the signing keys
to the repository. In addition to having the key file present each key's
ID must also be added to the verification script. This acts as a double
check that the correct key is added and makes a key change more
explicit.
2021-09-27 15:49:08 +02:00
Oliver Gugger 4c8bf9c28d scripts: extract functions
As a preparation to make the script easier to understand, we extract
some of the sub tasks into functions.
2021-09-27 15:48:19 +02:00
Oliver Gugger f60a1ba0ce Merge pull request #5699 from guggero/lncli-deletepayment
lncli: add deletepayments command
2021-09-27 13:41:41 +02:00
Oliver Gugger 57d203c9f1 docs: add release notes 2021-09-27 12:34:53 +02:00
Oliver Gugger 348405994c cmd/lncli: add deletepayments command
To give the CLI user the option to delete a single or multiple payments
in one command, we expose the DeletePayment and DeleteAllPayments
RPCs to the command line as well. Due to the similarity of the two RPCs
we combine them into a single command with multiple flags.

To make the command a bit more safe to run without arguments, we
consciously switch the logic of the RPC flag "failed_payments_only"
which is false by default into a "--include_non_failed" in the CLI which
is false by thefault. So a user running the command without knowing what
they are doing are only deleting failed payments by default, not all of
the payments.
2021-09-27 12:34:53 +02:00
Oliver Gugger 23d7065655 Merge pull request #5790 from arshbot/add-arshbot-verify
scripts: add arshbot key to verify script
2021-09-27 12:28:58 +02:00
Oliver Gugger 765d043cc0 mod+lncli: remove deprecated package [skip ci]
The use of golang.org/x/crypto/ssh/terminal was marked as deprecated
since that code was moved to its own repo golang.org/x/term.
With this commit we use the new package name directly, which was already
pulled in under its new name anyway.
2021-09-24 09:22:47 +02:00
Oliver Gugger 1d103f1f55 cmd/lncli: group all Mission Control cmds into own file
Having one file per sub command seems a bit too excessive and isn't
really implemented for any of the other commands. So we just group the
Mission Control commands into their own file for now.
2021-09-23 16:59:41 +02:00
Oliver Gugger 28c25d7e9c cmd/lncli: group Mission Control cmds into own category
To make the "Payments" category a bit less overloaded and to move the
Mission Control configuration commands away from the "root" category, we
create the new "Mission Control" category that we move the commands
into. We do this in a single commit so the next one where we move them
into the same file can be a pure code move without any additional
changes.
2021-09-23 16:59:41 +02:00
Oliver Gugger da418ef46b cmd/lncli: move all payment related cmds to single file
With this commit we move all commands that can be found within the
"Payments" category into its own file called cmd_payments.go. The only
exception are the Mission Control related commands which we are going to
move to their own category and file in the next commit.
2021-09-23 16:59:40 +02:00
Oliver Gugger e686a70b96 Merge pull request #5765 from sime/patch-1
docs: Remove pruned node warning.
2021-09-23 09:17:50 +02:00
Oliver Gugger 8d8db52723 Merge pull request #5772 from guggero/use-kvdb-1.0.3
mod: update kvdb to kvdb/v1.0.3 [skip ci]
2021-09-21 15:25:04 +02:00
Oliver Gugger 1ca332c611 mod: update kvdb to kvdb/v1.0.3 [skip ci] 2021-09-21 12:27:42 +02:00
Oliver Gugger 3f7909b48f doc: add release notes 2021-09-20 17:04:39 +02:00
Oliver Gugger 4b43e977b2 lntest: add RPC middleware itests 2021-09-20 17:04:39 +02:00
Oliver Gugger 0cc260bfc7 itest: add _test suffix to test file 2021-09-20 17:04:38 +02:00
Oliver Gugger efe5f6ae90 multi: add RPC middleware interception
With the middleware handler in place, we now need to add a new gRPC
interceptor to the interceptor chain that will send messages to the
registered middlewares for each event that could be of interest to them.
2021-09-20 17:04:29 +02:00
Oliver Gugger 75ca574790 lnrpc+rpcperms: add middleware handler
With this commit we introduce the concept of RPC middleware: A mechanism
similar to the existing channel or HTLC interceptors but this time for
gRPC messages themselves.
An RPC middleware can register itself to the main RPC server to get
notified each time a new gRPC request comes in, a gRPC response is sent
back or a streaming RPC is connected. The middleware can
validate/inspect incoming requests and modify/overwrite outgoing
responses.

Since this also opens the door for malicious software to interfere with
lnd in a negative way, we bind everything to macaroons with custom
caveat conditions: A middleware declares upon registration which custom
caveat name it can handle. Only client requests that send a macaroon
with that custom caveat will then be given to the middleware for
inspection. The only exception is if the middleware instead registers
to use the read-only mode. In that mode it will be able to intercept
all requests/responses, even those not made with a special encumbered
macaroon. But the middleware won't be able to alter responses in the
read-only mode. Therefore requests with the default, unencumbered macaroons
can never be modified by any middleware.
2021-09-20 13:12:49 +02:00
Oliver Gugger 918e021177 config+sample-lnd.conf+lncfg: add RPC middleware config 2021-09-20 13:05:47 +02:00
Oliver Gugger 8509d92070 log+rpcperms: add custom RPCP logger to rpcperms package
The custom RPC middleware logic that we are going to add in the next
commits will need to log under their own sub logger so we add one with a
new subsystem name.
2021-09-20 13:05:47 +02:00
Oliver Gugger 7b822f41df lncli: add custom caveats to bakemacaroon
With the new condition and checker in place, we can give the end user
the ability to add such a custom caveat to a baked macaroon.
There won't be an RPC counterpart for this operation since all first party
caveats currently are only added on the client side.
2021-09-20 13:05:47 +02:00
Oliver Gugger 045765111a multi: use safe copy for macaroons
Fixes #4383 by adding a new SafeCopyMacaroon function that correctly
clones all caveats and prevents modifications on the copy from affecting
the original.
2021-09-20 13:05:46 +02:00
Oliver Gugger 538175f487 macaroons: add custom caveat contraint and checker
The way the macaroon bakery library lnd uses works is that one has to
register a Checker method for each caveat name that should be supported.
Since we want to allow fully customizable custom caveats we add another
layer of naming to the caveat by splitting the condition of the "outer"
caveat into two pieces, the custom caveat name and the actual custom
caveat condition.
The custom Checker function only checks that the format is correct and
that there is a handler available for a custom condition. It does not
check the condition itself, however. If the passed in acceptor signals
acceptance of a custom caveat then the bakery accepts the macaroon as a
whole (given its signature, standard caveats and permissions are all
correct) and assumes that another component down the line will make sure
the actual custom condition of a caveat is valid.
2021-09-20 13:05:46 +02:00
Oliver Gugger 96ea4bf05e rpcserver+macaroons: extract RawMacaroonFromContext
We'll re-use the code for extracting a macaroon from a request context
later on so we extract it into its own exported function.
2021-09-20 13:05:33 +02:00
Oliver Gugger 4b7452a35e lnrpc+itest: fix write deadline issue with WS ping
Fixes #5680.
To make sure we're always reading from the WebSocket connection, we need
to always have an ongoing (but blocking) conn.ReadMessage() call going
on. To achieve this, we do the read in a separate goroutine and write to
a buffered channel. That way we can always read the next message while
the current one is being forwarded. This allows incoming ping messages
to be received and processed which then leads to the deadlines to be
extended correctly.
2021-09-20 12:21:03 +02:00
Oliver Gugger 5f94ebbd7d lnrpc: use request context in WebSocket proxy
The request context was not properly used to pass it along to the gRPC
endpoint which caused streaming calls to still be active on the gRPC
side even if the WS side already hung up.
We also issue an explicit close on the forwarding writer to signal when
the WS side was closed.
2021-09-20 12:07:46 +02:00
Oliver Gugger 29a8661517 Merge pull request #5640 from bhandras/kvdb-prefetch
kvdb+channeld: extend `kvdb` with `Prefetch` for prefetching buckets in one go and speed up payment control by prefetching payments on hot paths
2021-09-20 09:42:18 +02:00
Oliver Gugger d9f0f07142 Merge pull request #5748 from Roasbeef/flake-block-diff
routing: add wait.NoError to TestBlockDifferenceFix assertion [skip ci]
2021-09-20 08:25:27 +02:00
Oliver Gugger 5f43ef5816 Merge pull request #5751 from guggero/use-kvdb-1.0.2
mod: update kvdb to kvdb/v1.0.2 [skip ci]
2021-09-17 15:19:17 +02:00
Oliver Gugger 63233ff66c mod: update kvdb to kvdb/v1.0.2 [skip ci] 2021-09-17 13:08:08 +02:00
Oliver Gugger 583ccfeca4 Merge pull request #5611 from yyforyongyu/itest-flake-chan-open
itest: fix test flakes from open channel not found and tx not found in mempool
2021-09-17 09:51:17 +02:00
Oliver Gugger 44d73d7e3e Merge pull request #5743 from guggero/amp-string-nil
record: fix nil pointer in log string
2021-09-16 20:16:01 +02:00
Oliver Gugger dd3f3e7ce5 record: fix nil pointer in log string 2021-09-16 19:51:03 +02:00
Oliver Gugger 7d012ae581 Merge pull request #5738 from naveensrinivasan/naveen/fix/dns
mod: bump miekg/dns to next version
2021-09-16 09:42:16 +02:00
Oliver Gugger d9534ea108 Merge pull request #5405 from ErikEk/routing-updatechanpolicy-chan-check
routing: report invalid channels in updatechanpolicy call
2021-09-16 09:41:02 +02:00
Oliver Gugger 15ec974b15 Merge pull request #5734 from bhandras/itest_parallelism
build: fix itest parallelism
2021-09-15 20:15:29 +02:00
Oliver Gugger 3c15e3d81a Merge pull request #5686 from yyforyongyu/fix-health-check
healthcheck: quit checking when max attempts are reached
2021-09-15 17:41:44 +02:00
Oliver Gugger 03bce2129b Merge pull request #5732 from ellemouton/returnPingHanglerIfError
peer: return from pingHandler if error
2021-09-15 09:48:17 +02:00
Oliver Gugger 08c9d3fbdd Merge pull request #5304 from orbitalturtle/check-macaroon-rpcs
rpc: Bake and validate macaroons with external permissions
2021-09-15 09:42:38 +02:00
Oliver Gugger 4d89585a2d Travis: reduce ARM itest parallelism to 2
Reduces the number of concurrent tranches of itests running to two on
ARM in an attempt to make the tests less flaky because of very high CPU
usage with the default 4 parallel tranches.
2021-09-14 22:11:25 +02:00
Oliver Gugger 1ea6db1f26 Merge pull request #5725 from bottlepay/fix-amp-itest
lntest/itest: fix SendPaymentAMP test
2021-09-14 22:06:58 +02:00
Oliver Gugger 777abe5ff0 Merge pull request #5716 from guggero/fix-unit-tests
cluster: fix kvdb unit test
2021-09-13 15:59:31 +02:00
Oliver Gugger 55b3802387 cluster: fix kvdb unit test
It looks like #5702 was merged prematurely and broke the kvdb unit
tests.
2021-09-13 14:57:42 +02:00
Oliver Gugger 45343e4454 Merge pull request #5660 from LN-Zap/upstream/feat/delete-payment
add DeletePayment that allows to delete a specific payment or its failed HTLCs
2021-09-13 14:56:51 +02:00
Oliver Gugger 25a53f04f7 Merge pull request #5702 from bhandras/itest_etcd_logs
itest: save etcd logs along node logs in separate files
2021-09-13 11:50:35 +02:00
Oliver Gugger cc939cfa5d Merge pull request #5706 from Liongrass/patch-1
trivial: link from readme to builder's guide [skip ci]
2021-09-10 10:23:44 +02:00
Oliver Gugger 4337116bce Merge pull request #5697 from arshbot/export-selectHopHints
invoicesrpc: expose selectHopHints
2021-09-08 10:27:36 +02:00
Oliver Gugger d50ee834e1 Merge pull request #5688 from bhandras/recovery_itest_flake
itest: fix on-chain recovery flake caused by short timeout/btcd on ARM
2021-09-07 10:44:43 +02:00
Oliver Gugger f901d450a5 Merge pull request #5692 from guggero/migration-guide-arm
docs: update safety warning in migration guide [skip ci]
2021-09-06 11:57:26 +02:00
Oliver Gugger 8d7f5314a8 docs: update safety warning in migration guide [skip ci]
Fixes #5691.
With this commit we update the safety guidelines for migrating lnd data
between 32bit ARM and 64bit ARM devices.
2021-09-06 11:00:36 +02:00
Oliver Gugger e76c4c0e9b Merge pull request #5356 from guggero/batch-channel-open
Atomic batch channel funding
2021-09-03 19:56:56 +02:00
Oliver Gugger 180fba056f docs: add BatchOpenChannel to PSBT doc and release notes 2021-09-03 18:39:45 +02:00
Oliver Gugger 7a9ee48cf1 itest: add batch funding test 2021-09-03 18:39:45 +02:00
Oliver Gugger bb079f746b lncli: add batchopenchannel command 2021-09-03 18:39:44 +02:00
Oliver Gugger 37d0d651c4 rpcserver: extract abandonChan method
We'll want to re-use the abandon channel functionality for the batch
funding, as a cleanup in case the funding is aborted before publishing
any transaction.
2021-09-02 12:13:46 +02:00
Oliver Gugger ae2e9541a4 walletrpc: export wallet kit sub server name
We'll want to re-use the PSBT funding functionality from the wallet kit
sub server in the main RPC server. To be able to dynamically obtain an
instance of the wallet kit server we need to know its name under which
it registers itself in the list of sub server. We export the name so we
don't have to hard code it in the main server.
2021-09-02 12:13:46 +02:00
Oliver Gugger 84035f1fb2 funding: add batch funding function 2021-09-02 12:13:42 +02:00
Oliver Gugger 7c41d4b349 lnrpc: add batch channel open RPC types 2021-09-02 12:11:10 +02:00
Oliver Gugger 93d12cd9fc Merge pull request #5643 from bottlepay/updatepolicy-nested-tx
routing/localchans: fix nested db tx
2021-08-24 17:42:39 +02:00
Oliver Gugger ed5a3416bc Merge pull request #5623 from yyforyongyu/update-contrib-guide
docs: explicitly set line wrap rules for error messages [skip ci]
2021-08-24 17:29:18 +02:00
Oliver Gugger 2835cebe90 Merge pull request #5659 from guggero/unit-test-race-fix
multi: fix unit test for high CPU load envs
2021-08-24 15:16:28 +02:00
Oliver Gugger 998abd1886 Merge pull request #5657 from ErikEk/releasedoc-fix
docs: release note fix
2021-08-24 14:00:45 +02:00
Oliver Gugger 1f3a6e101d multi: fix unit test for high CPU load envs
It seems that the GitHub CI runners share a CPU and therefore only have
very few ressources available. This causes the wallet unlocker and breach
arbiter tests to fail if we don't give them enough time to complete.
2021-08-24 13:03:21 +02:00
Oliver Gugger 23cd231483 Merge pull request #4717 from guggero/xprv-wallet-init
walletunlocker: Allow wallet to be created from extended master root key (xprv)
2021-08-24 13:01:41 +02:00
Oliver Gugger c74c46147f Merge pull request #5646 from yyforyongyu/fix-context-leak
itest: properly handle the creation of timeout context
2021-08-24 10:44:36 +02:00
Oliver Gugger 648374ec2a Merge pull request #5354 from ErikEk/itest-include-btcd-backend-rotation-logs
itest: include compressed btcd backend logs from logrotate
2021-08-24 10:40:01 +02:00
Oliver Gugger 8632fc9233 docs: update release notes 2021-08-24 10:37:34 +02:00
Oliver Gugger 0822573743 itest: test wallet recovery from extended master root key
We add an additional test case to the on-chain fund recovery test that
tries restoring the same wallet from the extended master root key
instead of the seed.
2021-08-24 10:37:34 +02:00
Oliver Gugger 3fd944e7e4 lntest: allow node restore by extended root key
To allow testing restoring a node from an extended master root key, we
add an extra argument to the RestoreNodeWithSeed function.
2021-08-24 10:37:33 +02:00
Oliver Gugger bbd5980d42 lncli: add extended master root key to create command
To allow users to restore a wallet from an existing extended master root
key, we accept one of three answers when asking for an existing seed.
2021-08-24 10:37:33 +02:00
Oliver Gugger 370d056863 lncli: extract printCipherSeedWords 2021-08-24 10:37:33 +02:00
Oliver Gugger bdae6711af lncli: extract askRecoveryWindow 2021-08-24 10:37:32 +02:00
Oliver Gugger aa9435be84 lnd+walletunlocker: allow creating wallet from extended key
In addition to creating a new wallet from an aezeed, we allow specifying
an exteded master root key as the main wallet key directly.
Because an exteded key (xprv) doesn't contain any information about the
creation time of the wallet, we must assume a birthday to start scanning
the chain from (if the user doesn't provide an explicit value). Since
lnd only uses SegWit addresses, it makes sense to
choose the date that corresponds to the first mainnet block that
contained SegWit transactions.
Restoring a wallet from an extended master root key will result in a
significantly longer initial wallet rescan time if the default value is
used.
2021-08-24 10:37:32 +02:00
Oliver Gugger 6d70468ad5 lnrpc: add extended master key to walletunlocker 2021-08-23 11:26:00 +02:00
Oliver Gugger d329655bf6 mod: update to latest btcwallet dep 2021-08-23 11:25:39 +02:00
Oliver Gugger 951e2164fd Merge pull request #5536 from Kixunil/systemd-notify
Implemented systemd notify
2021-08-20 19:19:34 +02:00
Oliver Gugger 746a7dfbfc Merge pull request #5559 from de6df1re/lntest-itest-bugfix
itest: fix payment test error message
2021-08-20 19:01:13 +02:00
Oliver Gugger 9e8b9ccd4c Merge pull request #5615 from yyforyongyu/itest-fix-commitment-deadline
itest: fix commit deadline for neutrino backend
2021-08-19 16:40:14 +02:00
Oliver Gugger 958119a12a Merge pull request #5609 from mateuszmp/patch-2
Docs: Replacing deprecated import in JS readme
2021-08-16 10:24:14 +02:00
Oliver Gugger 5499a35987 Merge pull request #5395 from Crypt-iQ/fuzz_lnwire_patch_06162021
fuzz/lnwire: minor touch-ups, remove MaxPayloadLength
2021-08-12 09:30:38 +02:00
Oliver Gugger f7384a09e6 Merge pull request #5620 from szollo/master
docs: fix inconsistent naming of channel.backup [skip ci]
2021-08-12 09:26:13 +02:00
Oliver Gugger 07fa98fca5 Merge pull request #5614 from yyforyongyu/skip-release-note
trivial: skip release note for trivial fixes
2021-08-11 11:50:15 +02:00
Oliver Gugger dee9b9a9b1 Merge pull request #5603 from Crypt-iQ/peer_switch_refactor_08022021
htlcswitch+peer: introduce packetHandler, ChannelUpdateHandler interfaces for cleaner separation
2021-08-11 09:31:02 +02:00
Oliver Gugger a5e0ece64b Merge pull request #5616 from yyforyongyu/fix-context-timeout
itest: fix close channel context timeout
2021-08-11 09:19:17 +02:00
Oliver Gugger 527c6f06e2 docs: add release notes 2021-08-09 16:06:56 +02:00
Oliver Gugger f7943448b9 itest: fix channel open and close ntfn test
Because we now get one more update per channel when closing it, we need
to update our test that looks at the close notifications sent by the
SubscribeChannelEvent RPC.
2021-08-09 16:06:56 +02:00
Oliver Gugger bb4c754504 itest: test local force close restore scenario
We want to make sure we can recover funds from a channel that was force
closed by the local node just before the node needed to be restored. We
add a special test case with specific assertions for that scenario.
2021-08-09 13:55:37 +02:00
Oliver Gugger 294fba0bc5 itest: fix channel backup subscription test
With one more notification event being dispatched in the local force
close case we need to update one of the integration tests to account for
the additional message.
2021-08-09 13:55:35 +02:00
Oliver Gugger 06a693f10e lnrpc+rpcserver: add new fully resolved chan update event
This commit adds the new channel update event for a channel that is now
fully resolved to the event subscription RPC.
2021-08-09 13:55:34 +02:00
Oliver Gugger 5917486dc9 channel_notifier: keep chan in SCB on local foce close
In case we force close a channel from our end the funds can be time
locked for up to 2 weeks worst case. If something happens to our node in
that time, we don't have any information on that channel anymore because
currently the channel is removed from the channel backup file the moment
the commitment transaction confirms.
Instead we want to keep the channel around in the SCB file until we've
actually swept the time locked funds.

It looks like lnd even supports recovering from such a case if the peer
is still online and can tell us what commit height to use for deriving
the commit point. If the peer isn't around anymore then external tools
could be used to brute force the commit height.
2021-08-09 13:55:33 +02:00
Oliver Gugger 7e68cae8bd server+contractcourt: signal channel fully resolved
We might want to react to a channel being fully resolved after being
involved in a force close. For this we add a new callback and invoke it
where appropriate.
2021-08-09 13:55:31 +02:00
Oliver Gugger dcff9e5e26 channelnotifier: add event for fully resolved channel
We'll want to be informed about a channel that's been fully resolved on
chain in case it was involved in a force close. We add a new event type
and emit method for it.
2021-08-09 13:55:30 +02:00
Oliver Gugger 31063c269e Merge pull request #5509 from guggero/itest-flake-hunt
itest: neutrino anchor output flake fix
2021-08-05 17:44:53 +02:00