Commit Graph
100 Commits
Author SHA1 Message Date
András Bánki-Horváth 9eb405e9c6 fn: add goroutine manager
The package provides type GoroutineManager which is used to launch goroutines
until context expires or the manager is stopped. Stop method blocks until all
started goroutines stop.

Original code by Andras https://go.dev/play/p/HhRpE-K2lA0

Adjustments and tests by Boris.
2024-10-02 20:29:59 -03:00
Andras Banki-Horvath 1f7ac8e853 docs: update docs/release-notes/release-notes-0.18.3.md 2024-09-03 19:42:52 +02:00
Andras Banki-Horvath 06d4267a76 sqldb: fix end date filter when querying invoices
Previously, the SQL implementation of the invoice query simply
converted the start and end timestamps to time and used them
in SQL queries to check for inclusivity. However, this logic
failed when the start and end timestamps were equal.

This commit addresses and corrects this issue.
2024-09-03 19:40:47 +02:00
Andras Banki-Horvath b57910ee3a sqldb+invoices: synchronize SQL invoice updater behavior with KV version
Previously SQL invoice updater ignored the set ID hint when updating an
AMP invoice resulting in update subscriptions returning all of the AMP
state as well as all AMP HTLCs. This commit synchornizes behavior with
the KV implementation such that we now only return relevant AMP state
and HTLCs when updating an AMP invoice.
2024-09-03 19:40:46 +02:00
Andras Banki-Horvath c8de7a1699 channeldb: filter AMP state to relevant set IDs
When fetching an AMP invoice we sometimes filter HTLCs to selected set
IDs, however we always kept the full AMP state which is irrelevant as it
contains state for all AMP payments. This was a side effect of
UpdateInvoice needing to serialize the whole invoice when storing after
an update but it is an unwanted "feature" as users will need to filter
to relevant set when listing an AMP payment or subsribing to an update.
2024-09-03 19:40:46 +02:00
Andras Banki-Horvath 037161ee6c docs: add release notes for 0.18.3 2024-08-01 19:04:11 +02:00
Andras Banki-Horvath f63bccb575 config: update sample-lnd.conf 2024-08-01 19:04:11 +02:00
Andras Banki-Horvath 91d9fb4764 lncfg: increase default leader session TTL to 90 seconds 2024-08-01 19:04:11 +02:00
Andras Banki-Horvath 8e49eb652d itest: add itest covering the leader healthcheck 2024-08-01 19:04:10 +02:00
Andras Banki-Horvath 8e0534f756 multi: add leader check to the healthcheck monitor
This commit extends our healtcheck with an optional leader check. This
is to ensure that given network partition or other cluster wide failure
we act as soon as possible to avoid a split-brain situation where a new
leader is elected but we still hold onto our etcd client.
2024-08-01 19:04:10 +02:00
Andras Banki-Horvath 7784d6abf6 build: pin healthcheck and kvdb modules temporarily
This is to ensure that the added functionality works correctly and
should be removed once these changes are merged and the packages are
tagged.
2024-08-01 19:04:10 +02:00
Andras Banki-Horvath 2c8d1c878e kvdb: make etcd calls timeout to ensure liveness
Previously our RPC calls to etcd would hang even in the case of properly
set dial timeouts and even if there was a network partition. To ensure
liveness we need to make sure that calls fail correctly in case of
system failure. To fix this we add a default timeout of 30 seconds to
each etcd RPC call.
2024-08-01 19:04:10 +02:00
Andras Banki-Horvath 0fd4c7d5f9 healthcheck: improve logging of observers 2024-08-01 19:04:09 +02:00
Andras Banki-Horvath 053faa6229 docs: update release notes for 0.18.3-beta 2024-07-09 08:39:58 +02:00
Andras Banki-Horvath d0c1cec8c1 sqldb: switch away from pq to pgx for Postgres
Completely switch to the better maintained pgx driver.
2024-07-09 08:39:58 +02:00
Andras Banki-Horvath 95b99420fa sqldb: add unit test for the invoice expiry migration 2024-07-09 08:39:57 +02:00
Andras Banki-Horvath ed36598504 sqldb: add helpers to create test DBs migrated up to a select version 2024-07-09 08:39:57 +02:00
Andras Banki-Horvath 5292c76e10 sqldb: extract migration into method
Based on: https://github.com/lightninglabs/taproot-assets/pull/707
2024-07-09 08:39:57 +02:00
Andras Banki-Horvath 323af946e0 sqldb+invoices: add migration to fix incorrectly stored invoice expiries
Previously, when using the native schema, invoice expiries were incorrectly
stored as 64-bit values (expiry in nanoseconds instead of seconds), causing
overflow issues. Since we cannot determine the original values, we will set
the expiries for existing invoices to 1 hour with this migration.
2024-07-09 08:39:57 +02:00
Andras Banki-Horvath b35f0606ba docs: update release notes for 0.18.2-beta 2024-07-04 17:32:12 +02:00
Andras Banki-Horvath 892561f8f0 sqldb: bump modernc.org/sqlite to 1.29.10 which fixes init data race
Tracking issue: https://gitlab.com/cznic/sqlite/-/issues/180
2024-07-04 17:32:12 +02:00
Andras Banki-Horvath e45ed86263 invoices: fix and correctly cover paginated queries
Previously paginated queries offseted the add_index_get, add_index_let,
settle_index_get and settle_index_let parameters with the paginators
current page offset, however this was incorrect as we can just use
SQL's LIMIT/OFFSET to paginate. This commit fixes this issue and adds an
optional parameter to the constructor of the invoice SQL store to set
page size. This is useful when testing as we can now cover pagination
correctly with our existing unit tests.
2024-07-04 17:32:11 +02:00
Andras Banki-Horvath 59c8bafda7 lncli: fix parsing of --amp when sending a payment 2024-06-10 17:33:28 +02:00
Andras Banki-Horvath 024598266d docs: update release notes for 0.18 2024-04-11 15:04:04 +02:00
Andras Banki-Horvath 043e4aff01 sqldb+invoices: fix ordering bug in FilterInvoices
Previously if the `reverse` named arg was unset (value of NULL), then
SQL would order by NULL instead of ID causing undifined ordering of the
returned rows. To fix that we check for NULL and also make sure to set
the `reverse` arg in the code explicitly as it in the generated code it
is an `interface{}` instead of `bool`.
2024-04-11 15:04:04 +02:00
Andras Banki-Horvath 478ae1e9b0 sqldb: cleanup scope state reset by adding reset closure to ExecTx
For SQL transactions, we often accumulate results in variables declared
outside the closure's scope. To eliminate the need for manually clearing
these containers, we introduce a reset function to ExecTx, mirroring the
approach already adopted in kvdb.
2024-04-11 15:04:04 +02:00
Andras Banki-Horvath c6073a14ca sqldb: ensure that we're using serializable isolation 2024-04-11 15:04:04 +02:00
Andras Banki-Horvath 9a28a4c105 kvdb: remove the unused SQL error mappers 2024-04-11 15:04:03 +02:00
Andras Banki-Horvath 43f4b14c28 kvdb+sqldb: use the same tx retry helper 2024-04-11 15:04:02 +02:00
Andras Banki-Horvath c804894849 docs: update release notes for 0.18 2024-04-09 20:46:12 +02:00
Andras Banki-Horvath a7bd9701b1 github: update gRPC pin to v1.59.0 2024-04-09 20:46:11 +02:00
Andras Banki-Horvath 8f729241d9 sqldb: turn sqldb into a go package 2024-04-09 20:46:11 +02:00
Andras Banki-Horvath 7f5c8219ef sqldb+invoices: move SQL invoice store impl to invoices package 2024-04-09 20:46:11 +02:00
Andras Banki-Horvath 6d316ef56f sqldb: export sql null type helpers 2024-04-09 20:46:11 +02:00
András Bánki-Horváth 16c8339034 Merge pull request #8595 from bhandras/sqldb-invoices-reset-on-retry
sqldb: reset out of scope containers on potential ExecTx retry
2024-04-02 09:43:59 +02:00
Andras Banki-Horvath a1a7982646 sqldb: reset out of scope containers on potential ExecTx retry
As SQL serializaton errors can lead to transaction retries we need to
ensure that we reset any out of scope containers where we accumulate
results. Otherwise partial data from a previously executed transacton
retry may be returned along with the latest result.
2024-03-27 17:59:54 +01:00
Andras Banki-Horvath 483f5f3bca docs: update release notes for 0.18 2024-03-20 08:46:48 +01:00
Andras Banki-Horvath 2fbffab421 itest: ensure that native SQL LND won't start if it has any KV invoices 2024-03-20 08:46:48 +01:00
Andras Banki-Horvath 2d3d11487c lnd: ensure that native SQL can only be used with a clean KV invoice DB
This commit adds a check to ensure that we don't start LND with native
SQL invoice DB if we already have any invoices in our KV channeldb. This
is needed as native SQL invoices is an experimental feature and we do
not yet support migration.
2024-03-20 08:46:48 +01:00
Andras Banki-Horvath 65c1f5483b lncfg: make sure to only use native SQL with SQL backends 2024-03-20 08:46:47 +01:00
Andras Banki-Horvath 6c09617568 docs: update release notes for v0.18.0-beta 2024-03-01 10:08:10 +01:00
Andras Banki-Horvath 29ae17ba71 github: add nativesql itests to the matrix 2024-03-01 10:08:10 +01:00
Andras Banki-Horvath ba8e7550d5 itest: add the -nativesql flag to run SQL itests with native SQL tables 2024-03-01 10:08:10 +01:00
Andras Banki-Horvath 7a45bbbeab lnd: call into the correct InvoiceStore from RPC calls 2024-03-01 10:08:10 +01:00
Andras Banki-Horvath 42508d44e5 lnd: update sample-lnd.conf 2024-03-01 10:08:10 +01:00
Andras Banki-Horvath ce1b57da2d sqldb: exclude sqlite from the JS and unsupported platform builds 2024-03-01 10:08:09 +01:00
Andras Banki-Horvath 88370de1ab lnd: use native SQL invoice store if configured 2024-03-01 10:08:09 +01:00
Andras Banki-Horvath e31879ea0a lncfg: instantiate the native SQL backend if configured
This commit adds an optional separate "native" SQL backend that will
host all tables supporting native SQL. For now since we don't have many
such tables we'll keep it in one file for SQLite, but it may be split up
if desired.
2024-03-01 10:08:09 +01:00
Andras Banki-Horvath debfaf41b9 lncfg: use the sqldb config types instead of the kvdb counterparts 2024-03-01 10:08:09 +01:00
Andras Banki-Horvath aba45018a8 sqldb: merge SQLite and Postgres configs with the kvdb counterparts
This commit is part of a refactor that unifies configuration of the
sqldb and kvdb packages for SQL backends.
In order to unify the SQLite and Postgres configuration under sqldb we
first need to ensure that the final config types are compatible with
the alreay deployed versions.
2024-03-01 10:08:09 +01:00
Andras Banki-Horvath aee1f7f563 invoices: run InvoiceStore and InvoiceRegistry tests on PostgreSQL too 2024-03-01 10:08:09 +01:00
Andras Banki-Horvath 06730824a2 sqldb: fixup PostgreSQL fixture to allow creating separate DBs per test
This change will enable us to use a single PostgreSQL container instead of
spawning new a one for each (parallel) unit test reducing overall test
runtime.
2024-03-01 10:08:08 +01:00
Andras Banki-Horvath b10ce17cd4 invoices: run InvoiceStore and InvoiceRegistry tests on SQLite too 2024-03-01 10:08:08 +01:00
Andras Banki-Horvath 734d3a9d66 invoices: small test fixes 2024-03-01 10:08:08 +01:00
Andras Banki-Horvath 39211d25ed invoices: make test HTLC circuit keys unique 2024-03-01 10:08:08 +01:00
Andras Banki-Horvath 3d56d2f1ee sqldb: add full SQL InvoiceStore implementation 2024-03-01 10:08:08 +01:00
Andras Banki-Horvath 5cf67c4440 sqldb: add utility functions for SQL NULL types 2024-03-01 10:08:07 +01:00
Andras Banki-Horvath 74e7a50f4a sqldb: convert primary key violation to ErrSQLUniqueConstraintViolation
Besides the usual unique constraint violation that we already support we
also want to return the same error if the primary key constraint is
violated.
2024-03-01 10:08:07 +01:00
Andras Banki-Horvath 6a360fb2e2 sqldb: simplify and fixup the existing invoice store schema and queries
This commit attempts to fix some issues with the invoice store's schema that we
couldn't foresee before the implementation was finished. This is safe as the
schema has not been instantiated yet outside of unit tests. Furthermore the
commit updates invoice store SQL queries according to fixes in the schema as
well as to prepare the higher level implementation in the upcoming commits.
2024-03-01 10:08:07 +01:00
Andras Banki-Horvath 3e0f98a75a sqldb: introduce replace for the BIGINT type alias 2024-03-01 10:08:07 +01:00
Andras Banki-Horvath bcc6a3f517 docs: update release notes for 0.18.0 2024-02-19 20:47:25 +01:00
Andras Banki-Horvath f5abc94bdf mod: bump kvdb to v1.4.5 2024-02-19 20:47:25 +01:00
Andras Banki-Horvath 7d56d5385d invoices: parametrize invoice(registry) tests with InvoiceDB constructor
This commit extracts the InvoiceDB construction from all invoice and
registry tests such that we can later on run subtests with multiple
backends without needing to use tags.
2024-02-19 20:47:25 +01:00
Andras Banki-Horvath 0e2b39ed44 invoices: fix linter issues after the move 2024-02-19 20:47:24 +01:00
Andras Banki-Horvath 5e746b4d2c invoices: move InvoiceDB tests unrelated to kvdb to invoices package 2024-02-19 20:47:24 +01:00
Andras Banki-Horvath 6b0931af82 invoices: move UpdateInvoice implementation to the invoices package
With the introducation of the `InvoiceUpdater` interface we are now
able to move the non-kv parts of `UpdateInvoice` completely under
the invoices package. This is a preprequisite for being able to use
the same code-base for the sql InvoiceDB implementation of
UpdateInvoice.
2024-02-19 20:47:24 +01:00
Andras Banki-Horvath ecbfc46312 invoices+channeldb: add InvoiceUpdater interface and the KV impl
This commit introduces the InvoiceUpdater interface which is meant
to abstract and assist the in-memory invoice update procedure with
the accompanying database updates. These abstract updater steps will
enable further refactoring later while also ensuring that a full
SQL implementation of the InvoiceDB interface will be possible.
2024-02-19 20:47:24 +01:00
Andras Banki-Horvath 998156930f invoices: refactor updateInvoiceAmpState to return new state
With this commit updateInvoiceAmpState becomes getUpdatedInvoiceAmpState
which will only return the new AMP state but that needs to be applied at
the call site. This is a part of a larger refactor to gather all
mutations of an invoice update to be later applied by the invoice
updater.
2024-02-19 20:47:24 +01:00
Andras Banki-Horvath 08df7f4175 invoices: change cancelSingleHtlc to be purely used for validation
This change moves the HTLC state change out of the cancelSingleHtlc
function. This is part of the larger refactor of collecting all changes
to be later applied by the invoice updater.
2024-02-19 20:47:23 +01:00
Andras Banki-Horvath 87044b815c invoices: rename updateInvoiceState to getUpdatedInvoiceState
This commit turns updateInvoiceState "const" by moving preimage update
out of the function while also removing it to getUpdatedInvoiceState.
2024-02-19 20:47:23 +01:00
Andras Banki-Horvath 342eb4eac2 invoices: refactor updateHtlc to return new state
With this refactor updateHtlc is renamed to getUpdatedHtlcState and
changed such that it won't change the HTLC's state and resolve time but
instead returns whether the change is needed. This change is part of a
multi-commit refactor to ensure that all changes to the invoice will be
tracked individually.
2024-02-19 20:47:23 +01:00
Andras Banki-Horvath ef5a31733e channeldb: do not change the update descriptor when cancelling htlcs 2024-02-19 20:47:23 +01:00
Andras Banki-Horvath 7298b2d190 channeldb: extract AMP state updates to updateInvoiceAmpState() 2024-02-19 20:47:23 +01:00
Andras Banki-Horvath 4bf6b52158 channeldb: fetch the invoice before calling into updateInvoice 2024-02-19 20:47:22 +01:00
Andras Banki-Horvath eb4198b970 channeldb: extract store update methods when updaing an invoice
This commit is a small refactor to move all actual DB updates after
an invoice state is update to separate methods. This is a small
preliminary change before we completely decouple DB updates from
in-memory invocie update.
2024-02-19 20:47:22 +01:00
Andras Banki-Horvath c4472254ef docs: update release notes for 0.18.0 2024-02-01 21:02:25 +01:00
Andras Banki-Horvath 187e61da23 itests: add itest for ListSweeps's start_height 2024-02-01 21:02:25 +01:00
Andras Banki-Horvath 7725a57055 walletkit: add startheight parameter to ListSweeps
By default ListSweeps will list all sweeps known to the wallet. Over
time this may become expensive to call and callers may not be interested
in the full history. This commit adds a `startheight` parameter to the
`ListSweeps` RPC call. This parameter can be used to fetch sweeps only
from a specified start block height or only unconfirmed ones if it is
set to -1.
2024-02-01 21:02:24 +01:00
Andras Banki-Horvath 59ce7d9845 github: bump pull-resquest-stats to v2.11.0 2024-01-25 18:08:34 +01:00
Andras Banki-Horvath 560ea95a59 docs: update release notes for 0.18 2024-01-24 21:38:54 +01:00
Andras Banki-Horvath 25af657ad5 kvdb: set postgres max_connections on the started instance 2024-01-24 21:38:54 +01:00
Andras Banki-Horvath 524fa66abf kvdb: bump embedded postgres to v1.25.0 2024-01-24 21:38:53 +01:00
Andras Banki-Horvath bedafca49d docs: update release notes for 0.18.0 2023-10-18 16:15:59 +02:00
Andras Banki-Horvath 8bf7503aa4 invoices: remove the now unused ScanInvoices method 2023-10-18 16:15:58 +02:00
Andras Banki-Horvath d7d385f78f invoices: refactor initial invoice scan when the registry starts
In this commit we change how we select invoices to follow or delete when
starting the InvoiceRegistry to instead of using the deperecated scan
func from channeldb, use specific functions to gather pending and delete
canceled invoices.
2023-10-18 16:15:58 +02:00
Andras Banki-Horvath 8ab267ad90 channeldb: add k/v implementation for InvoiceDB.DeleteCanceledInvoices method 2023-10-18 16:15:58 +02:00
Andras Banki-Horvath 1f8065de35 channeldb: add k/v implementation for InvoiceDB.FetchPendingInvoices 2023-10-18 16:15:57 +02:00
András Bánki-Horváth ad5cd9c8bb multi: extend InvoiceDB methods with a context argument (#8066)
* multi: extend InvoiceDB methods with a context argument

This commit adds a context to InvoiceDB's methods. Along this refactor
we also extend InvoiceRegistry methods with contexts where it makes
sense. This change is essential to be able to provide kvdb and sqldb
implementations for InvoiceDB.

* channeldb: restrict invoice tests to only use an InvoiceDB instance

* docs: update release notes for 0.18.0
2023-10-11 13:42:59 +02:00
Andras Banki-Horvath 91af62a031 lnd: bump lnd/cert version
This commit bumps the lnd/cert version to 1.2.1 and fixes all call sites
where cert.GenCertPair was called.
2023-02-07 09:50:56 +01:00
Andras Banki-Horvath c3445e79d9 cert: removed unused args from cert.GenCertPair
Since the cert pair is written with a subsequent call to
cert.WriteCertPair we can safely remove the two file paths from the
argument list of cert.GenCertPair.
2023-02-06 17:11:56 +01:00
Andras Banki-Horvath c09a81624c keys: update key for bhandras 2022-06-09 09:28:46 +02:00
Andras Banki-Horvath 699f9c0d23 docs: update release notes for 0.15.0 2022-05-31 17:36:44 +02:00
Andras Banki-Horvath 98f39deb38 invoices: fix deadlock in hodl subscription handling 2022-05-31 17:36:43 +02:00
Andras Banki-Horvath cc8a1f0d2b invoices: fix deadlock in notification handling 2022-05-31 17:36:40 +02:00
Andras Banki-Horvath bbe953d70d docs: update release notes 0.15.0 2022-05-06 12:29:38 +02:00
Andras Banki-Horvath c4c188f6d9 signer: fix crash when musig combine doesn't have final sig 2022-05-06 12:29:38 +02:00
Andras Banki-Horvath ddf0571031 itest: cover musig2 combine without all signatures ready 2022-05-06 12:29:32 +02:00
Andras Banki-Horvath 70ac0070fa docs: update release notes 0.15.0 2022-04-12 11:56:34 +02:00
Andras Banki-Horvath 3d5f33a0cf routing: fail in-flight attempts cleanly on terminal payment failure
In case of a multi shard payment with more than one in-flight shards,
one shard quitting with a terminal failure will stop the payment
lifecycle and close the `shardHandler`'s `quit` channel. In the
`collectResult` function we're waiting for the `Switch` to
asynchronously return a result for each shard. This may have been
interrupted by the aformentioned `quit` channel's closing skipping
attempt failure (or success) notification towards the control tower
and therefore skipping proper settle/fail info fill in the channel db.
Since payments have a composite state of a global failure reason and
settle/fail info for all attempts, any attempt with an unfilled
settle/fail info keeps a payment in-flight even if the payment itself
isn't in-flight anymore.
2022-04-12 11:56:33 +02:00
Andras Banki-Horvath 64ea883105 switch+test: fix race condition in unit test 2022-03-23 13:26:15 +01:00