This commit builds on the ideas of @cfromknecht in lnd/5153. The
addition is that the design is now simpler and more robust by queueing
up everything, but allowing maximal parallelism where txns don't block.
Furthermore the commit makes CommitQueue.Done() private essentially
removing the need to understand the queue externally.
This commit fixes an issue where subsequent transaction retries may have
changed the read/write sets inside the STM which in turn left junk
references to these keys in the CommitQueue. The left keys potentially
conflicted with subsequent transactions, queueing them up causing
througput degradation.
This commit changes missioncontrol's store update from per payment to
every second. Updating the missioncontrol store on every payment caused
gradual slowdown when using etcd.
We also completely eliminate the use of the cursor, further reducing
the performance bottleneck.
The bucket sequence we use as payment sequence makes the DB update in
InitPayment conflict and therefore queue up when many concurrent
payments happen. This change allocates payment sequences in blocks
of 1000 to avoid these conflicts.
This commit removes an assertion which is not needed because with etcd
we can safely create keys and values with the same key since they are
stored under different keys in the DB. This saves us one unnecessary Get
on every Put.
The etcd cursor Delete stepped to the next item in the range before
Delete to not invalidate the iteation. This is unnecessary and not
compatible with bbolt, resulting in an extra fetch too.
This commit adds a new "waiting to start" state which may be used to
query if we're still waiting to become the cluster leader. Once leader
we advance the state to "wallet not exist" or "wallet locked" given
wallet availablity.
This commit also changes the order of DB init to be run after the RPC
server is up. This will allow us to later add an RPC endpoint to be used
to query leadership status.
This commit relaxes DeleteInvoice failure cases by removing the
requirement that the invoice needs to be indexed by the payment address.
Since payment address index was introduced with an empty migration it
is possible that users have old invoices which were never added to
this index causing invoice garbage collection to get stuck.
This commit moves the contract breach event dispatch after the channel
close summary has been added to the database. This is important
otherwise it may occur that we attempt to mark the channel fully closed
while the channel close summary is not yet serialized.
This commit adds support for etcd namespaces. This is useful when
using a shared etcd database where separate users have access to
separate namespaces.
In some tests we moved channeld.db to a temp location in order to
"time travel". This commit extends the existing semantics by moving all
files, including embedded etcd db too besides the channeld.db file.
This commit adds the icase name to the log filename, to make it simpler
to find problematic tests. Additionally after this commit we'll restart
Alice and Bob (the base harness nodes) before each icase to start with a
clean state.
Similarly as with kvdb.View this commits adds a reset closure to the
kvdb.Update call in order to be able to reset external state if the
underlying db backend needs to retry the transaction.
This commit adds a reset() closure to the kvdb.View function which will
be called before each retry (including the first) of the view
transaction. The reset() closure can be used to reset external state
(eg slices or maps) where the view closure puts intermediate results.
This commit adds commitQueue which is a lightweight contention manager
for STM transactions. The queue attempts to queue up transactions that
conflict for sequential execution, while leaving all "unblocked"
transactons to run freely in parallel.
This commit extends invoice garbage collection to also remove invoices
which are canceled when LND is already up and running. When the option
GcCanceledInvoicesOnTheFly is false (default) then invoices are kept and
the behavior is unchanged.
This commit extends channeldb with the DeleteInvoices call which is when
passed a slice of delete references will attempt to delete the invoices
pointed to by the references and also clean up all our invoice indexes.
This commit adds channeldb.ScanInvoices to scan through all invoices in
the database. The new call will also replace the already existing
channeldb.FetchAllInvoicesWithPaymentHash call in preparation to collect
invoices we'd like to delete and watch for expiry in one scan in later
commits.
This commit extends compatibility with the bbolt kvdb implementation,
which returns ErrIncompatibleValue in case of a bucket/value key
collision. Furthermore the commit also adds an extra precondition to the
transaction when a key doesn't exist. This is needed as we fix reads to
a snapshot revision and other writers may commit the key otherwise.
This commit removes the lock set which was used to only add bucket keys
to the tx predicate while also bumping their mod version.
This was useful to reduce the size of the compare set but wasn't useful
to reduce contention as top level buckets were always in the lock set.
