Commit Graph
100 Commits
Author SHA1 Message Date
Johan T. Halseth 2bef62b467 lnd: remove global var access from genCertPair
Instead pass the optional strings as slices to the method.
2019-07-22 09:26:25 +02:00
Johan T. Halseth 7838e6e1fa autopilot/agent_test: add more agent channel allocation testcases
Add tests for allocation when well within and well above channel budget.
2019-07-18 00:58:12 +02:00
Johan T. Halseth 68f7642564 autopilot/agent_test: wait for expected num chans and balance
Previously we waited only for the number of channels to become what we
expected, but this wasn't enough. Sometimes the agent had't yet updated
its internal balance, causing the test to fail with an unexpected
balance.
2019-07-18 00:58:12 +02:00
Johan T. Halseth bf7f392c11 autopilot: use subtractFees when funding channel
To make the autopilot able to account for fees, we let it use the
subtractFees option when opening channels.

This makes sure that each channel we attempt to open will eat at most
Amt out of our budget. Previously fees would eat into our funds in
addition, causing us to deplete our funds more than expected on each
channel opening.
2019-07-18 00:58:12 +02:00
Johan T. Halseth 0405703019 fundingmanager test: add TestFundingManagerFundAll
TestFundingManagerFundAll tests that we can initiate a funding request to
use the funds remaining in the wallet. This should produce a funding tx with
no change output.
2019-07-18 00:58:12 +02:00
Johan T. Halseth b9816259cb fundingmanager+server: define subtractFees
Let one initiate a funding request with the remaining funds in the
wallet.
2019-07-18 00:58:12 +02:00
Johan T. Halseth 87c8165f01 mock: make it possible to set list of returned utxos 2019-07-18 00:58:12 +02:00
Johan T. Halseth 2cd7d5a2a4 fundingmanager test: extract publishing logic into fundChannel, set funding fee during unit tests 2019-07-18 00:58:12 +02:00
Johan T. Halseth f15d81426c lnwallet/wallet: define SubtractFees for InitFundingReserveMsg
This commit adds a SubtractFees option to the funding request, letting
the caller specify that the fees should be deducted from the funding
amount. This paves the way for letting the funding manager spend up to a
given amount when creating a channel, like the rest of the funds in the
wallet.
2019-07-18 00:33:15 +02:00
Johan T. Halseth 4239f7d600 lnwallet/wallet: add coinSelectSubtractFees 2019-07-18 00:33:15 +02:00
Johan T. Halseth 1b2297c2b5 lnwallet/wallet: extract contribution setup into initOurContribution
This lets us easily call unlock() in case contribution setup fails.
2019-07-18 00:33:15 +02:00
Johan T. Halseth 44384a1b5b lnwallet: move coin selection before ChannelReservation
Now that coin selection is independent of creating the channel
reservation, we can move it first, in preparation for doing custom coin
selection.
2019-07-18 00:33:14 +02:00
Johan T. Halseth 98a3d04ba3 lnwallet: make selectCoinsAndChange return selected coins
This makes the method independent of the ChannelContribution struct.

We also add a function closure to the return of selectCoinsAndChange,
that let is unlock the selected output in case of error.
2019-07-18 00:33:14 +02:00
Johan T. Halseth fcf74debe6 lnwallet+funding: rename Capacity and FundingAmt
Instead use LocalFundingAmt and RemoteFundingAmt to make it clear who is
contributing funds.
2019-07-18 00:33:14 +02:00
Johan T. Halseth e716251805 server+funding: remove unused remoteFundingAmt 2019-07-18 00:33:14 +02:00
Johan T. Halseth 2f5c1a69fb lnwallet test: add TestCoinSelect 2019-07-18 00:33:14 +02:00
Johan T. Halseth 78da62c6f7 fundingmanager tests: run tests in parallel 2019-07-18 00:33:14 +02:00
Johan T. Halseth 452ee6aad4 fundingmanager+server: define MaxPendingChans. RejectPush in funding config
This commit makes the funding manager access the MaxPendingChannels and
RejectPush values from the fundingConfig instead of the global config
struct.

Done to avoid sharing state between tests.
2019-07-18 00:33:14 +02:00
Johan T. Halseth f069276970 Merge pull request #3306 from ottosuess/patch-4
invoicesrpc: fix route hint off-by-one-error
2019-07-12 12:28:15 +02:00
Johan T. Halseth e593c5e16f Merge pull request #3284 from wpaulino/bitcoind-zmq-unnecessary-timeouts
build: update dependencies to prevent unnecessary bitcoind ZMQ timeouts
2019-07-12 12:26:55 +02:00
Johan T. Halseth 0820b07c19 make: add -goroutinedump, rm pprof*.log to itest 2019-07-11 14:47:02 +02:00
Johan T. Halseth 8b64b14627 gitignore: add pprof files to gitignore 2019-07-11 14:44:50 +02:00
Johan T. Halseth 195a868455 lntest: add flag -goroutinedump 2019-07-11 14:44:44 +02:00
Johan T. Halsethandtaketa 8d5430dae3 lnd_test: add lndHarness to harnessTest, dump goroutines on Fatalf
Co-authored-by: taketa <853211b@gmail.com>
2019-07-11 14:41:41 +02:00
Johan T. Halsethandtaketa 314ba7db03 lntest/harness: add method SaveProfilesPages
SaveProfilesPages will write the active goroutines to files
pprof-n-*.log.

Co-authored-by: taketa <853211b@gmail.com>
2019-07-11 14:41:40 +02:00
Johan T. Halseth 6028722d71 lntest/node: make ProfilePort part of node config 2019-07-11 14:36:59 +02:00
Johan T. Halseth 7a5247b766 Merge pull request #3245 from breez/non_static_permissions
rpc: fix calculating RPC permissions.
2019-07-10 13:33:54 +02:00
Johan T. Halseth f0cbe46324 lnrpc: make BumpFee doc start with RPC name 2019-07-10 12:56:54 +02:00
Johan T. Halseth 25375d603c Merge pull request #3246 from tyzbit/fwdinghistory_3240
rpcserver: ForwardingHistory end_time defaults to time.Now()
2019-07-10 09:56:00 +02:00
Johan T. Halseth e775eddaee Merge pull request #3285 from cfromknecht/codeowners-typo
CODEOWNERS: fix aezeed typo
2019-07-10 09:53:46 +02:00
Johan T. Halseth b162595929 Merge pull request #3229 from halseth/fallback-hostname
lnd: fallback to localhost if failing getting hostname
2019-07-08 14:57:36 +02:00
Johan T. Halseth 601773958d lnd: fallback to localhost if failing getting hostname
Needed on some versions of Android.
2019-07-08 14:00:25 +02:00
Johan T. Halseth 8b83f03308 rpcserver+router: log terminal sendPayment error 2019-07-01 16:22:57 -07:00
Johan T. Halseth 7c4c64fffc lnrpc/rpc.proto: remove trailing spaces 2019-06-20 14:35:39 +02:00
Johan T. Halseth 97dda4f8c8 channeldb/migrations: skip migration for empty outgoing payments bucket
Previously the migration would fail if the source node was not set in
the database. Since we know that the source node must have been set
before making any payments, we check whether there actually are any
payments to migrate, and return early if not.
2019-06-18 21:14:16 +02:00
Johan T. Halseth 7df7449c47 Merge pull request #3187 from yaslama/UnknownAsDefaultAddressType
Use UnknownAddressType value as default in lnwallet.AddressType
2019-06-17 12:18:55 +02:00
Johan T. Halseth f98f452528 channeldb: avoid modifying bucket during cursor traversal
This is not safe according to bbolt documentation.
2019-06-14 21:09:03 +02:00
Johan T. Halseth ed8d635cf1 contractcourt/briefcase: avoid bucket modification in ForEach loop
Since the contents were deleted before the bucket was deleted, we just
delete the bucket immediately.
2019-06-14 21:09:02 +02:00
Johan T. Halseth b7a37728c0 channeldb/migrations: avoid modifying bucket during ForEach loop
bbolt documenation state that this is not safe and leads to undefined
behavior.
2019-06-14 21:09:02 +02:00
Johan T. Halseth f63c5d0170 Merge pull request #3182 from yaslama/addressTypeInitialization
Initialize addressType to UnknownAddressType
2019-06-14 10:32:01 +02:00
Johan T. Halseth dcaed7254e make: delete log files at new output file location 2019-06-12 16:44:21 +02:00
Johan T. Halseth 1ba99da7bd gitignore: ignore new output file location 2019-06-12 16:43:53 +02:00
Johan T. Halseth 386f8ece54 routing+zpay32: copy pubkeys before nilling Curve and spewing
Since nilling the pubkey curve will lead to a nil-pointer exception if
the key is later used for signature verification, we make sure to make a
copy before nilling and spewing.
2019-06-12 12:19:43 +02:00
Johan T. Halseth 00a86696a8 Merge pull request #3145 from federicobond/avoid-pubkey-serialization
discovery,fundingmanager: avoid serialization in NotifyWhenOnline
2019-06-12 10:50:35 +02:00
Johan T. Halseth 0aec3fd230 rpcserver: filter out non-suceeded payments, include payment status
Earlier versions of ListPayments only included completed payments. We
return to this behavior by ignore all other payments if the nonSucceeded
boolean is not set in the request.
2019-06-12 08:57:47 +02:00
Johan T. Halseth 6794708ed5 lnrpc: expose PaymentStatus for ListPayments
Also make it optional to list non-succeeded payments.
2019-06-12 08:51:51 +02:00
Johan T. Halseth 5485101f9f Merge pull request #3170 from cfromknecht/remove-router-reject-cache
routing/router: remove router-level reject cache
2019-06-11 21:52:27 +02:00
Johan T. Halseth e45d4d703a Merge pull request #2762 from halseth/reliable-payments-lookup-circuitmap
[reliable payments] persist htlcswitch pending payments
2019-06-08 18:05:30 +02:00
Johan T. Halseth 187c8bc0b5 config+autopilot: make conftarget configurable 2019-06-08 17:29:33 +02:00
Johan T. Halseth 9a7f24d2d3 autopilot: extract default fee estimate target conf into constant 2019-06-08 17:22:34 +02:00
Johan T. Halseth dd88015985 htlcswitch/switch test: add TestSwitchGetPaymentResult
TestSwitchGetPaymentResult tests that the switch interacts as expected
with the circuit map and network result store when looking up the result
of a payment ID. This is important for not to lose results under
concurrent lookup and receiving results.
2019-06-07 16:53:33 +02:00
Johan T. Halseth dd3abbc4ef htlcswitch/payment_result_test: add TestNetworkResultStore
TestNetworkResultStore tests that the networkResult store behaves as
expected, and that we can store, get and subscribe to results.
2019-06-07 16:53:33 +02:00
Johan T. Halseth f556b375ff lnd_test: add testHoldInvoicePersistence
testHoldInvoicePersistence tests that a sender to a hold-invoice, can be
restarted before the payment gets settled, and still be able to receive
the preimage.
2019-06-07 16:53:32 +02:00
Johan T. Halseth 2cc778d309 htlcswitch/switch: use paymentResultStore to keep track of results 2019-06-07 16:53:32 +02:00
Johan T. Halseth 2dea790b55 multi: make GetPaymentResult take payment hash
Used for logging in the switch, and when we remove the pending payments,
only the router will have the hash stored across restarts.
2019-06-07 16:53:32 +02:00
Johan T. Halseth 1febe1a6d5 htlcswitch/payment_result: add paymentResultStore
paymentResultStore is a persistent store where we keep track of all
received payment results. This is used to ensure we don't lose results
from payment attempts on restarts.
2019-06-07 16:53:32 +02:00
Johan T. Halseth df3f5d02ad htlcswitch/payment_result: add (de)serialization of networkResult + test 2019-06-07 16:53:32 +02:00
Johan T. Halseth f5dee02ff4 htlcswitch/mock: set SelfKey and mock ErrorSource 2019-06-07 16:42:25 +02:00
Johan T. Halseth 473afbcac5 Merge pull request #3163 from Roasbeef/spend-ntfn-fix
build: update to latest btcd version
2019-06-06 11:30:55 +02:00
Johan T. Halseth 6d52128da0 routing/router test: assert SendToRoute init values
This commit adds an assertion to the SendToRoute test that the payment
value stored to the DB during SendToRoute execution is the correct one.

This assertion would fail before the previous commit that fixed a
missing value initialization.
2019-06-04 10:34:59 +02:00
Johan T. Halseth 1161d87eec routing/router: correct SendToRoute's amount record in DB
Previously we would mistakenly use the payment value from the dummy
LightningPayment struct, which would obviously be 0 always. Now we
instead calculate the value from the given route.
2019-06-04 10:13:33 +02:00
Johan T. Halseth 3a44574625 chainntnfs/neutrinonotify: async call GetUtxo from RegisterSpendNtfn
Since GetUtxo is a potentially long running call, we would see
RegisterSpendNtfn block in some cases, in particular on starting the
chain watcher, causing lnd to hang on startup.

This commit makes the call to GetUtxo in a go routine, letting
RegisterSpendNtfn return immediately when the notification events are
created.
2019-06-04 09:58:04 +02:00
Johan T. Halseth 5882e54bf3 route test: add a simple unit test for the TotalFees method
Checks fees on 0, 1, and 2-hop routes.
2019-05-28 11:16:34 +02:00
Johan T. Halseth f829d33d6f route: return 0 fee from TotalFees for empty route.
Otherwise we would crash calling the method on empty routes.
2019-05-28 11:03:14 +02:00
Johan T. Halseth 28fdf9712d Merge pull request #3125 from halseth/make-fmt-router-rpc
routerrpc test: run make fmt
2019-05-28 09:43:28 +02:00
Johan T. Halseth 7cb25a5afb channeldb/control_tower test: add TestPaymentControlDeleteNonInFlight
TestPaymentControlDeleteNonInFlight checks that calling DeletaPayments
only deletes payments from the database that are not in-flight.
2019-05-27 20:19:00 +02:00
Johan T. Halseth 649ee875fd routing/router_test: add test case for ForwardingError on SendHTLC 2019-05-27 20:19:00 +02:00
Johan T. Halseth 589f0fcc5d control_tower: make initial bucket bucket creation stricter 2019-05-27 20:18:59 +02:00
Johan T. Halseth 95b4828780 channeldb: derive PaymentStatus implicitly from stored info 2019-05-27 20:18:59 +02:00
Johan T. Halseth d705b8a013 channeldb+multi: rename Grounded->Unknown, Completed->Succeeded 2019-05-27 20:18:59 +02:00
Johan T. Halseth d78d3f50b9 channeldb: move deprecated code to migration_09_legacy_serialization.go
And unexport deprecated code.
2019-05-27 20:18:59 +02:00
Johan T. Halseth 1b788904f0 channeldb+router: record payment failure reason in db 2019-05-27 20:18:59 +02:00
Johan T. Halseth 60e2367973 routing/router_test: add TestRouterPaymentStateMachine
TestRouterPaymentStateMachine tests that the router interacts as
expected with the ControlTower during a payment lifecycle, such that it
payment attempts are not sent twice to the switch, and results are
handled after a restart.
2019-05-27 20:18:59 +02:00
Johan T. Halseth 3f76bc0629 routing+server: define PaymentSessionSource 2019-05-27 20:18:59 +02:00
Johan T. Halseth f4306b1178 routing/payment_session+router: make PaymentSession interface 2019-05-27 20:18:58 +02:00
Johan T. Halseth adc4640f4f routing: export MissionControl 2019-05-27 20:18:58 +02:00
Johan T. Halseth 3323800e02 routing/router: resume payment state machine at startup
On startup the router will fetch the in-flight payments from the control
tower, and resume their execution.
2019-05-27 20:18:58 +02:00
Johan T. Halseth 677cb018c9 routing/control_tower: add FetchInFlightPayments 2019-05-27 20:18:58 +02:00
Johan T. Halseth de1bf8a518 routing/router: persist payment state machine
This commit makes the router use the ControlTower to drive the payment
life cycle state machine, to keep track of active payments across
restarts.  This lets the router resume payments on startup, such that
their final results can be handled and stored when ready.
2019-05-27 20:18:58 +02:00
Johan T. Halseth dd73c51a34 routing/payment_lifecycle: extract create attempt into method 2019-05-27 20:18:58 +02:00
Johan T. Halseth e9b2182cdc routing/payment_lifecycle: extract error handling into method 2019-05-27 20:18:58 +02:00
Johan T. Halseth 83bfaa4fb4 routing: extract payment flow into method on paymentLifecycle
This encapsulates all state needed to resume a payment from any point of
the payment flow, and that must be shared between the different stages
of the execution. This is done to prepare for breaking the send loop
into smaller parts, and being able to resume the payment from any point
from persistent state.
2019-05-27 20:18:57 +02:00
Johan T. Halseth ae7bf2cb7b routing/router: move sending and receiving payment result into loop 2019-05-27 20:18:57 +02:00
Johan T. Halseth 59c2557cc9 channeldb/control_tower test: test new payment behavior 2019-05-27 20:18:57 +02:00
Johan T. Halseth 1af1832ff7 channeldb/control_tower: add payment information during state changes
This commit gives a new responsibility to the control tower, letting it
populate the payment bucket structure as the payment goes through
its different stages.

The payment will transition states Grounded->InFlight->Success/Failed,
where the CreationInfo/AttemptInfo/Preimage must be set accordingly.

This will be the main driver for the router state machine.
2019-05-27 20:18:57 +02:00
Johan T. Halseth 6d80661bbb channeldb/payments: add StatusFailed 2019-05-27 20:18:57 +02:00
Johan T. Halseth bb4aadd16c channeldb/control_tower: remove non-strict option
Since we have performed a migration, the db should be in a consistent
state, and we can remove the non-strict option.
2019-05-27 20:18:57 +02:00
Johan T. Halseth b7189ba028 channeldb test: add TestOutgoingPaymentsMigration 2019-05-27 20:18:57 +02:00
Johan T. Halseth 693807cf6e channeldb/control_tower: add FetchSentPayments 2019-05-27 20:18:56 +02:00
Johan T. Halseth 801521ed2d channeldb/migration: add migration for new payment bucket structure
migrateOutgoingPayments moves the OutgoingPayments into a new bucket format
where they all reside in a top-level bucket indexed by the payment hash. In
this sub-bucket we store information relevant to this payment, such as the
payment status.

To avoid that the router resend payments that have the status InFlight (we
cannot resume these payments for pre-migration payments) we delete those
statuses, so only Completed payments remain in the new bucket structure.
2019-05-27 20:18:56 +02:00
Johan T. Halseth 178996f0d3 channeldb/payments+control_tower: split OutgoingPayments
This commit changes the format used to store payments within the
DB. Previously this was serialized as one continuous struct
OutgoingPayment, which also contained an Invoice struct we where only
using a few fields of. We now split it up into two simpler sub-structs
CreationInfo, AttemptInfo and PaymentPreimage.

We also want to associate the payments more closely with payment
statuses, so we move to this hierarchy:

There's one top-level bucket "sentPaymentsBucket" which contains a set
of sub-buckets indexed by a payment's payment hash. Each such sub-bucket
contains several fields:
paymentStatusKey -> the payment's status
paymentCreationInfoKey -> the payment's CreationInfo.
paymentAttemptInfoKey -> the payment's AttemptInfo.
paymentSettleInfoKey -> the payment's preimage (or zeroes for
non-settled payments)

The CreationInfo is information that is static during the whole payment
lifcycle. The attempt info is set each time a new payment attempt
(route+paymentID) is sent on the network. The preimage is information
only known when a payment succeeds.  It therefore makes sense to split
them.

We keep legacy serialization code for migration puproses.
2019-05-27 20:18:56 +02:00
Johan T. Halseth f022810f8b channeldb/codec: add privateKey serialization 2019-05-27 20:18:56 +02:00
Johan T. Halseth 2417f40532 channeldb: put payment status in new bucket
We move the payment status to a new bucket hierarchy. Old buckets and
fetch methods are kept around for migration purposes.
2019-05-27 20:18:56 +02:00
Johan T. Halseth d027e10201 htlcswitch+channeldb: move control tower to channeldb 2019-05-27 20:18:56 +02:00
Johan T. Halseth 6e102d64b9 Merge pull request #3101 from AdamISZ/make-unit-cover-pkg
Allow unit-cover target for make to filter by package
2019-05-27 13:18:45 +02:00
Johan T. Halseth 1df148d8ef routerrpc test: run make fmt 2019-05-27 12:33:43 +02:00
Johan T. Halseth 34032eff0b make: increase itest timeout to 40 min
Needed for neutrino itest on darwin.
2019-05-27 09:29:03 +02:00
Johan T. Halseth 991b4174fd lnd_test: fix spelling 2019-05-27 09:29:03 +02:00
Johan T. Halseth ca1e9dcecb lntest: skip zeroConf ListUnspent for neutrino 2019-05-27 09:29:03 +02:00
Johan T. Halseth 19d6cc00e5 travis: enable neutrino itest 2019-05-27 09:29:03 +02:00