Commit Graph
100 Commits
Author SHA1 Message Date
Johan T. Halseth 80ac8c1df0 lncli: add lncli autopilot commands
This commit adds commands 'status', 'enable' and 'disable' for
autopilot.

The commands will only be enabled for autopilotrpc builds of lncli.
2018-12-13 13:02:40 +01:00
Johan T. Halseth fdf903b8a9 gitignore: only match binaries in current folder, remove main* 2018-12-13 12:33:45 +01:00
Johan T. Halseth 8754635de5 rpcsercer+subservercfg: populate autopilot RPC config 2018-12-13 12:33:45 +01:00
Johan T. Halseth 28a92185cc lnrpc: add autopilotrpc driver 2018-12-13 12:33:44 +01:00
Johan T. Halseth 568c32d1fa lnrpc/autopilotrpc: add lnrpc autopilot server 2018-12-13 12:33:44 +01:00
Johan T. Halseth 00376230ff lnrpc+log: add autopilot rpc logger 2018-12-13 12:33:44 +01:00
Johan T. Halseth 413aad5efa lnrpc+subserver_cfg: add configs for autopilot sub-server 2018-12-13 12:33:44 +01:00
Johan T. Halseth db4bbe1c3a lnrpc: add autopilotrpc subservice
This commit adds a new service lnrpc/autopilot, that is to be used to
communicate with the running autopilot. Currently a RPC for getting the
status of the agent is included, as well as enabling/disabling at
runtime.
2018-12-13 12:33:38 +01:00
Johan T. Halseth cff42e06c8 pilot+lnd: let autopilot.Manager manage pilot-agent
This commit moves the responsibility of managing the life cycle of the
autopilot from main to the autopilot Manager. It utilizes the recently
introduced autopilot Manager, and just sets up the necessary interfaces
for the Manager to properly set up the required subscriptions when
starting the agent.
2018-12-13 12:26:29 +01:00
Johan T. Halseth 6310ed0f1c autopilot: add autopilot.Manager
This commit adds a new type Manager responsible for managing an
autopilot agent, making it possible to start and stop it at runtime.
2018-12-13 12:26:29 +01:00
Johan T. Halseth 3be292b36e lnd_test: use context with timeout instead of context.Background
To ensure RPC calls won't block indefinitely.
2018-12-13 08:34:57 +01:00
Johan T. Halseth f989b2ece8 lnd_test: let assertNumConnections set its own context 2018-12-13 08:33:13 +01:00
Johan T. Halseth 6a4edc8335 lnd_test: let assertNodeNumChannels set its own context 2018-12-13 08:33:12 +01:00
Johan T. Halseth 4db939069a lnd_test: use cancelable context for SendPayment 2018-12-13 08:31:58 +01:00
Johan T. Halseth fd347888bb lnd_test: make assertAmountPaid set its own context timeout
Let us avoid having to set it each time we call it.
2018-12-13 08:31:58 +01:00
Johan T. Halseth 8e57ed7025 autopilot/agent_test: let agent self-trigger retry
This ensures that it is done updating its pending connections map before
re-querying the heuristic.
2018-12-11 10:21:40 +01:00
Johan T. Halseth dda7355ad6 autopilot/agent_test: ensure mochHeuristics exits cleanly 2018-12-11 10:20:38 +01:00
Johan T. Halseth 71444e74ac Merge pull request #2033 from gitlikeagirl/ckc-hashinsendresponse
lnrpc: Add payment hash to SendResponse
2018-12-11 09:54:52 +01:00
Johan T. Halseth 0255db5120 autopilot/prefattach_test: ensure non-connected nodes get 0-score
This addition to the unit tests makes sure nodes that have no channels
in the graph are left out od the scored nodes, implicitly giving them a
score of 0.
2018-12-10 15:05:24 +01:00
Johan T. Halseth b9c33da3f8 autopilot/prefattach: skip nodes having no channels 2018-12-10 15:01:43 +01:00
Johan T. Halseth aeeff4714d autopilot/graph: define addRandNode 2018-12-10 15:01:43 +01:00
Johan T. Halseth 5224c94aaf autopilot/prefattach_test: assert scored nodes having addresses
This addtion to the test makes sure the scored nodes have their
addresses populated. This would fail without the previous commit.
2018-12-10 15:01:05 +01:00
Johan T. Halseth d9eef4625e autopilot/prefattach: populate Addrs field 2018-12-10 15:00:41 +01:00
Johan T. Halseth 55ba5fe654 autopilot/graph: add addresses to memNode 2018-12-10 14:56:54 +01:00
Johan T. Halseth 40db2dd5a5 autopilot/choice: return ErrNoPositive in case no choice can be made 2018-12-10 13:53:36 +01:00
Johan T. Halseth f1e8c8d5b5 autopilot/agent: move choice algorithms to new file choice.go 2018-12-10 13:53:31 +01:00
Johan T. Halseth 6130189d95 autopilot/interface+agent: remove Select 2018-12-06 14:26:28 +01:00
Johan T. Halseth b3d315298c autopilot/agent: use NodeScores to select channel candidates
This commit makes the autopilot agent use the new NodeScores heuristic
API to select channel candiates, instead of the Select API. The result
will be similar, but instead of selecting a set of nodes to open
channels to, we get a score based results which can later be used
together with other heuristics to choose nodes to open channels to.

This commit also makes the existing autopilot agent tests compatible
with the new NodeScores API.
2018-12-06 14:26:26 +01:00
Johan T. Halseth e84bd29836 autopilot/agent: add weightedChoice and chooseN algorithm
The algorithms will be used to select nodes at random from the weighted
distribution set by the node's scores given by the heuristic.
2018-12-06 14:26:05 +01:00
Johan T. Halseth be45697c6d autopilot/prefattach_test: use NodeScores API for prefAttach tests
This commit converts the existing unit tests of Select into tests of
NodeScores.
2018-12-06 14:26:05 +01:00
Johan T. Halseth 5e8e54083f autopilot/prefattach+interface: add API NodeScores
This commit adds a new method NodeScores to the AttachementHeuristic
interface. Its intended use is to score a set of nodes according to
their preference as channel counterparties.

The PrefAttach heuristic gets a NodeScores method that will score the
ndoes according to their number of already existing channels, similar to
what is done already in Select.
2018-12-06 14:26:05 +01:00
Johan T. Halseth 5ecc209c41 autopilot/interface+agent: remove NodeKey from AttachmentDirective
Instead parse the pubkey bytes only when needed.
2018-12-06 14:26:05 +01:00
Johan T. Halseth 89c3c5319f autopilot/agent: split opening logic into own method
This commit takes the logic after the autopilot agent has decided that it
needs to open more channels, and moves it into a new method openChan.
2018-12-06 14:26:03 +01:00
Johan T. Halseth 26810fe928 autopilot/agent: split attachement directive attempts into method 2018-12-06 14:25:22 +01:00
Johan T. Halseth fb10175ea5 autopilot/agent: add maps to agent struct
This commit moves the maps used by the controller loop to the Agent
struct, in preparation for breaking it up into smaller parts.
2018-12-06 14:24:18 +01:00
Johan T. Halseth 86e6d230f2 autopilot/agent: add attachment directive goroutine to wait group 2018-12-06 14:24:15 +01:00
Johan T. Halseth 35f4ec84d1 autopilot/prefattach: use HeuristicConstraints
This commit makes the pref attach heuristic and the agent use the
HeuristicConstraints internally.
2018-12-06 11:25:30 +01:00
Johan T. Halseth e98d859882 autopilot: define HeuristicConstraints
This commit defines a new struct HeuristicConstraints that will be used
to keep track of the initial constraints the autopilot agent needs to
adhere to. This is currently done in the ConstrainedPrefAttachement
heuristic itself, but this lets us share these constraints and common
method netween several heuristics.
2018-12-06 11:20:09 +01:00
Johan T. Halseth 70aeda1589 channeldb/graph: don't create node bucket in PruneGraph
Instead return ErrSourceNodeNotSet directly. This would be returned from
the call pruneGraphNodes -> sourceNode anyway.
2018-12-06 10:22:16 +01:00
Johan T. Halseth 9d467d3534 channeldb/graph: reuse nodes bucket in deleteLightningNode
Lets us avoid passing the tx from pruneGraphNodes
2018-12-06 10:22:16 +01:00
Johan T. Halseth 1b9ca06563 channeldb/graph: reuse nodes bucket when fetchin source node 2018-12-06 10:22:16 +01:00
Johan T. Halseth 86d40636cd channeldb/graph: don't create bucket in delEdgeUpdateIndexEntry
We are deleting what's in the bucket, so if it doesn't exist, just
return early with a non-error.
2018-12-06 10:22:16 +01:00
Johan T. Halseth 84de318553 channeldb/graph: don't create bucket in UpdateEdgePolicy
instead return ErEdgeNotFound, which would be returned anyway when
querying the edgeIndex for the channel.
2018-12-06 10:22:15 +01:00
Johan T. Halseth 9722323161 channeldb/graph: don't create node bucket in DeleteChannelEdge
Instead return ErrGraphNodeNotFound directly. If the node bucket was
created it would be empty, and the call delChannelByEdge ->
fetchChanEdgePolicies -> fetchChanEdgePolicy ->
deserializeChanEdgePolicy -> fetchLightningNode would return this error
anyway.
2018-12-06 10:22:15 +01:00
Johan T. Halseth 351f4aab79 channeldb/graph: don't create buckets in DeleteChannelEdge
Instead return ErrEdgeNotFound if buckets don't exist. This would be
returned anyway, when the chanIndex is checked for the channel point in
question.
2018-12-06 10:22:15 +01:00
Johan T. Halseth 5d8d99b7bc channeldb/graph: don't create bucket in UpdateChannelEdge
It would return ErrEdgeNotFound when edge not found in bucket anyway.
2018-12-06 10:22:15 +01:00
Johan T. Halseth 81fe6e73ed router+graph: return ErrGraphNodesNotFound if no nodes to prune
Avoids creating a bucket unneccessarily.
2018-12-06 10:22:15 +01:00
Johan T. Halseth 87df6b71c5 router test: only cleanup if context creation succeeds 2018-12-06 10:22:15 +01:00
Johan T. Halseth 3d7bbd6d72 channeldb/graph: write max 80 sig len for nodes 2018-12-05 09:31:26 +01:00
Johan T. Halseth 5dcd2a4530 gossiper: validate our own node announement 2018-12-05 09:31:25 +01:00
Johan T. Halseth 349551373d Merge pull request #2257 from yancyribbens/apk-docker-dependencies-fix
docker: fix dockerfile dependency bug
2018-12-05 08:26:53 +01:00
Johan T. Halseth 0ba4fb8605 make: don't hardcode commit date
As it won't correspond to the commit hash, include the whole version
string.
2018-12-04 14:09:33 +01:00
Johan T. Halseth bafc44c14b make: get btcd commit hash from go.mod
Since go.sum also contains historical commits, we must fetch currenct
commit hash from go.mod.
2018-12-04 13:50:51 +01:00
Johan T. Halseth 5f8425cb03 routing/pathfind_test: check error before attempting cleanup 2018-12-04 12:20:45 +01:00
Johan T. Halseth 67b018476c routing/pathfind test: add TestPathSourceEdgesBandwidth
This commit adds a new test that checks that the bandwidth hints are
considered correclty for local channels, and that disable flags are
ignored in this case.
2018-12-04 12:20:45 +01:00
Johan T. Halseth 0b6a19866b routing/pathfind: pass ignore disable flags for local channels
To decouple our own path finding from the graph state, we don't consider
the disable bit when attempting to use local channels. Instead the
bandwidth hints will be zero for local inactive channels.

We alos modify the unit test to check that the disable flag is ignored
for local edges.
2018-12-04 12:20:45 +01:00
Johan T. Halseth 20b9d5d000 routing: define graphParams and restrictParams
To avoid the findPath parameter list getting out of hand, we define new
structs that wraps the mandatory and optional parameters to findPath.
2018-12-04 12:20:44 +01:00
Johan T. Halseth bea31310e1 lnd_test: make sure miner have seen tx before mining
This commit makes mineBlocks take an argument to check the number of txs
expected in the mempool before mining. By using waitForTxInMempool
before mining blocks we ensure that the txs in question have actually
propagated to the miner's mempool before mining the blocks.
2018-11-29 12:33:16 +01:00
Johan T. Halseth cdcb7105b3 lnd_test: assert existence of unconfirmed tx in mempool 2018-11-29 12:31:40 +01:00
Johan T. Halseth b07499f227 Merge pull request #2224 from halseth/itest-timeouts
lnd_test: increase test timeouts
2018-11-28 08:22:46 +01:00
Johan T. Halseth 7e48b012e2 Merge pull request #2230 from halseth/itests-tryconnect-peer
lntest/harness: retry ConnectPeer of chain backend still syncing
2018-11-28 08:21:08 +01:00
Johan T. Halseth 0011bcdad1 Merge pull request #2229 from halseth/global-defaultcsv
lnd_test: make tests use global defaultCSV
2018-11-28 08:19:58 +01:00
Johan T. Halseth 8854db5191 gitignore: ignore temp backend logs folder 2018-11-27 10:21:19 +01:00
Johan T. Halseth 4d05e6bbd8 lnd_test+lntest: keep btcd debug logs from test run
Useful for debugging 🤷
2018-11-27 10:21:19 +01:00
Johan T. Halseth 3a2fd51f2f lnd_test+lntest: move copyFile() -> lntest.CopyFile() 2018-11-27 10:21:17 +01:00
Johan T. Halseth b389d5f6b0 lntest/harness: retry ConnectPeer of chain backend still syncing 2018-11-27 10:15:00 +01:00
Johan T. Halseth ec76a25530 lnd_test: make test use globale defaultCSV 2018-11-27 10:11:32 +01:00
Johan T. Halseth 62e37ec840 lnd_test: use global defaultTimeout instead of test specific timeouts 2018-11-27 10:01:02 +01:00
Johan T. Halseth f13e5472d5 lnd_test: use global minerMempoolTimeout when waiting for miner to see TXs 2018-11-27 09:59:46 +01:00
Johan T. Halseth ea32a13fdb lnd_test: use global channelCloseTimeout when closing channels 2018-11-27 09:59:31 +01:00
Johan T. Halseth 864d77d82e lnd_test: use global channelOpenTimeout when opening channels 2018-11-27 09:59:29 +01:00
Johan T. Halseth 5c294601f0 lnwallet/channel: remove startingHeight from commitmentChain
Field is not being used.
2018-11-23 08:58:45 +01:00
Johan T. Halseth b1a35fc8f4 channeldb/migrations_test: add TestMigrateOptionalChannelCloseSummaryFields 2018-11-21 13:36:42 +01:00
Johan T. Halseth a9bd6100ff htlcswitch/link: remove handled TODO 2018-11-21 10:28:57 +01:00
Johan T. Halseth 0c4948b40b lnd test: add offline scenario to testDataLossProtection
This adds the scenario where a channel is closed while the node is
offline, the node loses state and comes back online. In this case the
node should attempt to resync the channel, and the peer should resend a
channel sync message for the closed channel, such that the node can
retrieve its funds.
2018-11-21 10:28:57 +01:00
Johan T. Halseth 9e81b1fe53 chain_watcher: poll for commit point in case of failure
We pool the database for the channel commit point with an exponential
backoff. This is meant to handle the case where we are in process of
handling a channel sync, and the case where we detect a channel close
and must wait for the peer to come online to start channel sync before
we can proceed.
2018-11-21 10:28:57 +01:00
Johan T. Halseth 0dd7eed64e peer: define resendChanSyncMsg
This method is used to fetch channel sync messages for closed channels
from the db, and respond to the peer.
2018-11-21 10:28:56 +01:00
Johan T. Halseth da3f515f12 channeldb/db_test: add TestFetchClosedChannelForID 2018-11-21 10:28:56 +01:00
Johan T. Halseth 45b132a954 channeldb/db: define FetchClosedChannelForID
FetchClosedChannelForID is used to find the channel close summary given
only a channel ID.
2018-11-21 10:28:56 +01:00
Johan T. Halseth a9f93b2988 contractcourt/chain_watcher: add channel sync message to CloseChannelSummary 2018-11-21 10:28:56 +01:00
Johan T. Halseth 676a1b1407 lnwallet+link: make ChanSyncMsg take channel state as arg
This lets us get the channel reestablish message without creating the LightningChannel struct first.
2018-11-21 10:28:56 +01:00
Johan T. Halseth 0b9a323fcb channeldb/channel: add LastChanSync field to CloseChannelSummary
This commit adds an optional field LastChanSyncMsg to the
CloseChannelSummary, which will be used to save the ChannelReestablish
message for the channel at the point of channel close.
2018-11-21 10:28:43 +01:00
Johan T. Halseth 46c961aa17 channeldb/migrations: migrate ChannelCloseSummary 2018-11-21 10:27:58 +01:00
Johan T. Halseth 28b15dcbbb channeldb/channel: write boolean to indicate presence of ChannelCloseSummary fields 2018-11-21 10:27:58 +01:00
Johan T. Halseth 149a8ce94f channeldb/legacy_serialization: add deserializeCloseChannelSummaryV6
This commit adds a new file legacy_serialization.go, where a copy of the
current deserializeCloseChannelSummary is made, called
deserializeCloseChannelSummaryV6.

The rationale is to keep old deserialization code around to be used
during migration, as it is hard maintaining compatibility with the old
format while changing the code in use.
2018-11-21 10:12:05 +01:00
Johan T. Halseth 780e1bbead peer: correct variable name, print error 2018-11-21 10:12:04 +01:00
Johan T. Halseth 13098a595a lnd test: refactor testDataLossProtection
This extracts part of the test into a new helper method timeTravel,
which can be used to easily reset a node back to a state where channel
state is lost.
2018-11-21 10:12:04 +01:00
Johan T. Halseth 0ca7c8c5f8 lnd_test: define helper getChanInfo 2018-11-21 10:12:04 +01:00
Johan T. Halseth eb2f832bba lnd_test: define createPayReqs helper method 2018-11-21 10:12:04 +01:00
Johan T. Halseth e1d8b07735 lnd_test: remove unused math/rand rependency 2018-11-21 10:07:56 +01:00
Johan T. Halseth d6d833c242 Merge pull request #2185 from babonet13/patch-1
Add EXPOSE directive for 9735 (p2p) & 10009 (rpc)
2018-11-19 08:48:20 +01:00
Johan T. Halseth a6b3ef3f08 Merge pull request #2178 from grunch/removes-duplicated-default-msg
removes duplicated default message on queryroutes command
2018-11-19 08:44:47 +01:00
Johan T. Halseth aceda7fbcb travis: enable caches
This commit sets the GOCACHE environment variable, and enables caching
on travis.

GOCACHE will tell the compiler (only go 1.10+) where to store build and
test artifacts. Caching this directory will significantly speed up
succeeding builds and test runs.

We also cache vendor/ as this will speed up the call to 'make dep'.

A few dependencies in the GOPATH are cached, as calls to 'go get' for
these will be sped up. Note that we don't cache the lnd directory, as it
will conflict with the changes in the PR being built.
2018-11-16 11:08:54 +01:00
Johan T. Halseth b4aeb83dbb travis: split up into 3 builds
This commit distributes the CI tests into 3 independend builds, by
splitting the integration test run and unit test coverage.

To better handle the extra cases, we define a build matrix with the
three build types (RACE and LINT, ITEST, COVER).
2018-11-16 11:08:16 +01:00
Johan T. Halseth d3c9be1f32 travis: only run on latest go 2018-11-16 11:07:03 +01:00
Johan T. Halseth b64d55cdbf make: remove make travis 2018-11-16 11:04:51 +01:00
Johan T. Halseth 9e950b4739 travis: execute make travis step by step
Instead of calling 'make travis' directly, we call each step. This lets us
better track how much time is spent on each.

Also note that we execute 'itest-only' instead of 'itest', and instead
execute the dependencies (btcd, build) manually first.

We explicitly set the ITEST environment variable, for readability, and
define a new COVER. This is currently true when ITEST=true to keep the
existing build configuration, but will later be configured to be
independent.
2018-11-16 11:04:51 +01:00
Johan T. Halseth b98ab723ee travis: remove unnecessary export PATH
This is done automatically by travis as part of the go setup.
2018-11-16 11:04:51 +01:00
Johan T. Halseth 23c858576f make: define itest-only
'itest-only' can be used to only run integration tests, not build any of
the dependencies. This is useful on travis, where manually executing
each command makes it easier to track the time spent on each.
2018-11-16 11:04:50 +01:00
Johan T. Halseth c6ee2803ad make: define goveralls 2018-11-16 11:04:50 +01:00