Commit Graph
76 Commits
Author SHA1 Message Date
András Bánki-Horváth 595bb7c1ff Merge pull request #4318 from bhandras/etcd_fixes
lnd+kvdb+etcd: remove retry goroutine from the STM and integrate abort context
2020-06-03 19:21:01 +02:00
Andras Banki-Horvath 3e5e60cac4 lnd+kvdb: pass LND abort context to etcd 2020-06-03 17:44:44 +02:00
Andras Banki-Horvath d3545830c9 kvdb+etcd: integrate the abort context to the STM retry loop
This commit extends the etcd.BackendConfig to also provide an abort
context and integrates it with the STM retry loop in order to be able
stop LND when conflicting transactions keep the loop running.
2020-06-03 17:44:44 +02:00
Andras Banki-Horvath 415de2f0c7 kvdb+etcd: remove retry goroutine from the STM
This commit removes the retry goroutine from the STM as the retry loop
is only running when the STM transaction is encapsulated in Update/View
whereas for self-standing transactions we use a different approach.
By removing the goroutine we won't catch panics thrown that are supposed
to be catched outside of the STM.
2020-06-03 17:44:44 +02:00
Andras Banki-Horvath 1dd43f3669 test: add error to itest whitelist 2020-06-02 15:40:20 +02:00
Andras Banki-Horvath 2a591ee2f1 build: bump etcd to v3.3.22 2020-06-02 15:40:20 +02:00
András Bánki-Horváth 703e8acb36 Merge pull request #4326 from bhandras/testclock_improvement
clock: add module support
2020-05-28 19:47:28 +02:00
Andras Banki-Horvath 1ae27b4f44 itest: add errors to the whitelist 2020-05-28 13:32:51 +02:00
Andras Banki-Horvath fc87e8358e clock: add go.mod 2020-05-28 10:37:04 +02:00
Andras Banki-Horvath 337d282a3f clock+test: convert checks to testify asserts 2020-05-26 23:55:28 +02:00
Andras Banki-Horvath e81061bda4 clock: optionally wait until tickers are registered in TestClock 2020-05-26 23:55:26 +02:00
András Bánki-Horváth 7f1a450a7f Merge pull request #4015 from bhandras/kv-etcd
kvdb wrapper for etcd
2020-05-22 14:00:06 +02:00
Andras Banki-Horvath b53475da14 lncfg+lnd: prefix etcd db with only network name instead of full path 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath c3fcfd1530 etcd: add namespace support to separate key spaces
This commit extends etcd db with namespaces without additional storage
space requirements. This is simply done by instead of using an all zero
root bucket id, we use the sha256 hash of the name space as our root
bucket id.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath bce0597643 channeldb: silence linter 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 85aee9b064 kvdb+lncfg: fully move etcd behind build tag
This commit separates all etcd related sources (sans a few stubs and
config) from the rest of the source tree and makes compilation conditional
depending on whether the kvdb_etcd build tag is specified.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 3ef331e016 lncfg+etcd: add TLS config
This commit extends lncfg with etcd TLS config and passes these
parameters to the etcd client upon construction.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath b54f8a1b97 kvdb+channeldb: create ExtendedBackend interface to allow rich tx logic
This commit adds the ExtendedBackend interface which is an extension to
the walletdb.DB interface. This paves the way to using etcd.db.View and
etcd.db.Update in the global View and Update functions without much code
rewrite.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 23fcb59a2b channeldb: convert payment tests to use test backend 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 02f9d91e49 kvdb+etcd: only lock on bucket path mod revision for reads and writes
This commit reduces the compare set size the STM will submit in
transactions by adding only the bucket keys along the bucket path to a
specific lock set. This lock set then used to filter the read set,
effectively removing all read only keys from the transaction predicate
that are not bucket keys.
By tracking if a read-write tx actually changes something, we can also
"bump" the mod revision of the bucket keys.
With this trick we essentially implement a read-write lock for our
bucket structure greatly reducing transaction processing time.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 0e3629e2c7 channeldb+lnd: make channeldb backend configurable
This commit adds support for user configured channeldb backend.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 9d57c1a6b4 lncfg: add configuration for user specified db backend
This commit extends lncfg to support user specified database backend.
This supports configuration for both bolt and etcd (while only allowing
one or the other).
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 859a457e48 kvdb+etcd: extend kvdb and STM with stats
The commit itslef adds stats to the transactions such that we can see
how LND behaves with and etcd backend.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 248a00f211 kvdb+etcd: enable channeldb testing with both bdb/etcd 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 3b7525659c channeldb+kvdb: add walletdb interface test
This commit adds walletdb interface test suite to the existing test set.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 6a24a03cec channeldb+kvdb: walletdb/kvdb interface etcd implementation
This commit adds a full interface implementation of the walletdb/kvdb
interface with detailed tests.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath 137dee04e8 channeldb+kvdb: an extended STM on top of etcd clientv3
This commit adds an extended STM, similar to what available in etcd's
clientv3 module. This incarnation of said STM supports additional
features, like positioning in key intervals while taking into account
deletes and writes as well. This is a preliminary work to support all
features of the kvdb interface.
2020-05-22 11:26:24 +02:00
Andras Banki-Horvath 556e3525ea misc: fix error formatting in multiple files 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 2b729a78f3 lntest: fix ticker leak 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 633ea71ad1 lnd: time.Since instead of time.Now().Sub(...) 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath fe59890a4b lnd+test: remove dead test code 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath ed595adf59 lnd: one append instead of appends in a loop 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 63419c1870 lnd: remove dead code from nursery_store 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 62e6c392a5 lnd: remove dead code from fundingmanager 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 2d6c73f0fd netann: remove unused logClosure 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath f39387db3b monitoring: removing unused logClosure 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 0679d1dd4b autopilot: append instead of appends in a loop 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 28c2ef12b2 netann: one append instead of appends in a loop 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath f3f482d9f9 watchtower: one append instead of appends in a loop 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath ebf058a2a5 routing: static check fix, time.Since instead of Now().Sub() 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 4773ae17d9 routing: remove unused code 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath e9d7ba635c lnwallet: remove redundant return 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath b93140cd3c switch: remove unused code 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath ccf6e2a393 contractcourt: remove redundant return 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 1519a1bdfe channelnotifier: remove dead logging code 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath e1385af5a6 watchtower: remove dead logging code 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 72ed30e3e5 tlv: simplify byte slice comparison as recommenteded by static analysis 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath da9d7dd363 zpay32: invoice.MilliSat is uint64 which cannot be negative 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 1a13f822c6 invoices: remove dead code from logger 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath e636364ccf channeldb: remove dead code from payments test 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 3628e65378 channeldb: (nit) remove size argument from map make 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath c7fa5ea863 queue: remove unsued waitgroup 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath 70fc9c1901 invoices: elminitate sleep from expiry watcher tests
This commit removes sleeps from invoice expiry watcher tests in favor of
a timeout guard.
2020-04-09 09:58:22 +02:00
Andras Banki-Horvath 919710467e autopilot: tidying up source code to fit to 80 cols 2020-03-30 16:36:13 +02:00
Andras Banki-Horvath 93cb05142a lnrpc: rename FloatValue to FloatMetric
This commit renames lnrpc.FloatValue to lnrpc.FloatMetric as requested
in the final review of centrality PR.
2020-03-30 16:36:11 +02:00
Andras Banki-Horvath 5a4d595e53 lncli: group graph queries under Graph category
This commit moves DescribeGraph, GetNodeInfo and GetChanInfo under the
Graph category.
2020-03-27 13:56:10 +01:00
Andras Banki-Horvath 608354032c autopilot: parallelize betweenness centrality
This commit parallelizes betweenness centrality calculation, by
distributing the algo to N workers and creating partial results.
2020-03-27 13:56:10 +01:00
Andras Banki-Horvath 7e50997bb4 lnrpc: add betweenness centrality to GetNodeMetrics (new RPC call)
This commit extends the RPC interface with GetNodeMetrics will contain
all graph node metrics in the future. Currently only holds betweennes
centrality per node.
2020-03-27 13:56:08 +01:00
Andras Banki-Horvath 3fe9c70722 autopilot: betweenness centrality using Brandes algo on simplifed graph
This commit adds betweenness centrality to the available node metrics.
Betweenness centrality is a per node centrality measure which for an
arbitrary node v equals to the sum of shortest paths going trough v
divided by the number of all shortest paths for for each vertex pair
k, s where k != s != v.
2020-03-27 10:39:50 +01:00
Andras Banki-Horvath be83d504f8 autopilot: introduce NodeMetric interface for arbitrary graph metrics
This commit adds the NodeMetric interface which will be used for all
graph metrics not directly part of the autopilot but are useful in
composite heuristics to drive autopilot decisions and improve node
scores.
2020-03-27 10:36:28 +01:00
Andras Banki-Horvath e6561ca86d invoices: fix log messages
This commit fixes incorrect log messages.
2020-02-03 15:12:55 +01:00
Andras Banki-Horvath fabcdf754a invoices: fix slow startup with many expired invoices
This commit intends to fix slow first startup time when there are many
invoices that need to be canceled. The slowdown is caused by a combination
of adding invoices to the expiry watcher one-by-one and slow
cancellation. Due to slow cancellation and the unbuffered channel which
we use to pass invoices to the expiry watcher blocks the registry.
With this fix we'll instead batch add invoices to the expiry watcher and
thereby won't block the registry startup.
2020-01-24 21:01:10 +01:00
Andras Banki-Horvath 8af5d8bc73 channeldb: remove unused, test only FetchAllInvoices function
This commit removes channeldb.FetchAllInvoices and changes tests such
that expectation sets are prepared in the test case instead of selected
from the DB.
2020-01-08 16:51:51 +01:00
Andras Banki-Horvath 4136b18e3d channeldb: remove time.Now() from tests 2020-01-08 16:51:51 +01:00
Andras Banki-Horvath ec6c0689ef channeldb: fix channeldb.InvoiceHTLC deep copy
This commit fixes deep copy of chaneldb.InvoiceHTLC, where previously
the map holding the custom record set wasn't properly copied.
2020-01-08 16:51:51 +01:00
Andras Banki-Horvath 4e0ada9ea3 routing: clean up locking on topology change 2020-01-02 18:03:48 +01:00
Andras Banki-Horvath 2c3041d8bc autopilot: fix memChannelGraph channel edge addition
This PR fixes an issue that happens when adding a new channel edge
between two nodes in a memChannelGraph. Originally a channel edge held a
node value which made the graph different when iterating from the two
endpoints of an edge. This is simply fixed by holding pointers instead.
2019-12-19 22:42:37 +01:00
Andras Banki-Horvath fa14ff67ad channeldb+invoices: fix what pending invoice means in ChannelDB
This commits builds on top of PR #3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
2019-12-19 17:44:32 +01:00
Andras Banki-Horvath 9b60685961 channeldb: remove math/rand from tests
This commit removes random invoice state selection when testing
FetchAllInvoicesWithPaymentHash.
2019-12-19 17:44:31 +01:00
Andras Banki-Horvath 0758b9310e channeldb: adding a unit test for FetchAllInvoicesWithPaymentHash
This commit changes how FetchAllInvoicesWithPaymentHash behaves
when the DB is empty and also adds a unit test to test that
case as well as normal expected behavior.
2019-12-13 17:03:19 +01:00
Andras Banki-Horvath 44f13d1d60 invoices: adding InvoiceExpryWatcher to cancel expired invoices
This commit adds InvoiceExpryWatcher which is a separate class that
receives new invoices (and existing ones upon restart) from InvoiceRegistry
and actively watches their expiry. When an invoice is expired
InvoiceExpiryWatcher will call into InvoiceRegistry to cancel the
invoice and by that notify all subscribers about the state change.
2019-12-13 17:03:08 +01:00
Andras Banki-Horvath 7024f36a76 general: adding the Clock interface to aid testing
This commit adds Clock and DefaultClock and moves the private
invoices.testClock under the clock package while adding basic
unit tests for it.
Clock is an interface currently encapsulating Now() and TickAfter().
It can be added as an external dependency to any class. This way
tests can stub out time.Now() or time.After().

The DefaultClock class simply returns the real time.Now() and
time.After().
2019-12-13 16:52:22 +01:00
Andras Banki-Horvath ff3063daea queue: fixing doc after cherry pick
This commit just fixes a godoc and adds a new one to the already
merged queue.PriorityQueue.
2019-12-13 16:52:22 +01:00
Andras Banki-Horvath 88e01fa1fa invoices: refactor test helpers
This commit moves test helpers to their on file, while also adding helper to
create an invoice with a valid payment request.
2019-12-13 16:52:18 +01:00
Andras Banki-Horvath 56282db30a queue: Introducing a general purpose priority queue.
This commit introduces PriorityQueue, which is a general, heap
based priority queue, and PriorityQueueItem which is an interface
that concrete priority queue items must implement.
This implementation is encapsulated, users do not need to use any
other package for full functionality.
PriorityQueue exports the usual public methids: Push, Pop, Top,
Empty and Len. For full documentaton consult the priority_queue.go,
for usage: priority_queue_test.go
2019-12-11 16:08:05 +01:00
Andras Banki-Horvath b77bb73d4c lnwire: fixing buffer size and cleaning up uint16/32 conversion
This commit removes an unnecessarely large 32 byte buffer in favor of
a small 2 byte buffer and cleans up type conversion between uint16
and uint32 values.
2019-12-05 16:28:25 +01:00