The docker image have been updated so we are using another protobuf
version to generate the files. The generate files include the version of
the compiler used to creating them, so we need this commit to pass the
`rpc-check` step in our CI.
Bump all build go versions to v1.21.0
Bump the minimum build package version to v1.19.0
Debian "buster" is not longer supported. Security updates have been
discontinued since June 30th 2022. We will build using the latest
version, "bookworm".
This commit provides the scaffolding for using the new sql stores.
The new interfaces, structs and methods are in sync with other projects
like Taproot Assets.
- Transactional Queries: the sqldb package defines the interfaces required
to execute transactional queries to our storage interface.
- Migration Files Embedded: the migration files are embedded into the binary.
- Database Migrations: I kept the use of 'golang-migrate' to ensure our
codebase remains in sync with the other projects, but can be changed.
- Build Flags for Conditional DB Target: flexibility to specify our database
target at compile-time based on the build flags in the same way we do
with our kv stores.
- Update modules: ran `go mod tidy`.
Schema for AMP invocies.
AMP invoices can be paid multiple times and each payment to an AMP invoice
is identified by a `set_id`.
The A in AMP stands for `Atomic`. All the htlcs belonging to the same
AMP payment (share the same set_id) will be resolved at the same time
with the same result: settled/canceled.
AMP invoices do not have an "invoice preimage". Instead, each htcl has
its own hash/preimage. When a new htlc is added the hash for that htlc
is attached to it. When all the htlcs of a set_id have been received we
are able to compute the preimage for each one of them.
Set of queries to deal with invoices. A couple of things to take into
account:
- Because the queries are not rewritten at runtime, we cannot have a
generic `INSERT` with different tuples.
- Because the queries are not rewritten at runtime, we cannot build
one query with only the filters that matter for that queries. The
two options are a combinatorial approach (a new query for every
permutation) or a generic query using the pattern
```
SELECT *
FROM table
WHERE (
-- Can be read as:
-- Match the filter 1 value if filter_1 != nil
column_1 >= sqlc.narg('filter_1') OR
sqlc.narg('filter_1') IS NULL
) AND (
column_2 >= sqlc.narg('filter_2') OR
sqlc.narg('filter_2') IS NULL
) ...
```
This is the schema for "ordinal" BOLT11 invoices.
The invoices table aims to keep an entry for each invoice, BOLT11 or not,
that will be supported.
Invoice related HTLCs will be stored in a separete table than forwarded
htlcs.
SQLite does not support `SEQUENCE`. We achieve atomic autoincrementals
using primary keys with autoincrement/serial. An invoice `AddIndex`
translates to `invoices(id)` while `SettleIndex` is `invoice_payments(id)`.
sqlc is a tool that generates fully type-safe idiomatic code from SQL.
The result is Go code can then used execute the queries in the database.
The noraml flow looks like:
- The developer write some sql that will update the schema in the
database: new tables, indices, etc
- The developer updates the set of queries that will use the new schema.
- `sqlc` generates type-safe interfaces to those queries.
- The developer can then write application code that calls the methods
generated by sqlc.
The tool configuration needs to live in the repo's root and its name is
`sqlc.yaml`.
LND will support out of the box sqlite and postgres. The sql code needs to
be (almost) the same for both engines, so we cannot use custom functions
like `ANY` in postgres.
The SQLC config file needs to define what is the target engine, we will
set postgres but the generated code can be executed by sqlite too.
In some specific cases, we will `match and replace` some sql lines to be
sure the table definitions are valid for the targeted engine.
All the updates type are now catched in a switch statement, we can
delete the rest of the body of this function + add a default path for
not matched flows.
Previous to this commit we were able to call `UpdateInvoice` with an
updated that added and cancelled htlcs at the same time. The function
returned an error if there was overlapping between the two htlc set.
However, that behavior was not used in the LND code itself.
Eventually we want to split this method in multiple ones, among them one
for canceling htlcs and another one for adding them. For that reason,
this behavior is not supported anymore.
The only way to know if an invoice is AMP, Keysend, etc is to look at
its shape/characteristics. This commit adds a couple of helper functions
to encapsulate the logic of these checks.
If all these types cannot intersect: an invoice cannot be AMP and
Keysend or Keysend and Bolt12, etc it could be useful to add an extra
field to store this information instead of relying on checking how the
invoice looks like.
The test cases in `TestUpdatePaymentState` run in parallel. One of the
parameters is a pointer and the value to the struct it points to gets
modified during the test.
The race condition was introduced in 8d49dfb07e
To test the fix run from the main folder `go test ./routing/. -race`
before this fix and after.
Now that we have the new package `lnd/channeldb/models` we can invert the
depenency between `channeldb` and `invoices`.
- Move all the invoice related types and errors to the
`invoices` package.
- Ensure that all the packages dealing with invoices use the types and
interfaces defined in the `invoices` package.
- Implement the InvoiceDB interface (defined in `lnd/invoices`) in
channeldb.
- Add new mock for InterfaceDB.
- `InvoiceRegistery` tests are now in its own subpacakge (they need to
import both invoices & channeldb). This is temporary until we can
decouple them.
Add a new subpackage to `lnd/channeldb` to hold some of the types that
are used in the package itself and in other packages that should not
depend on `channeldb`.
This commit moves the `HeightHintCache` implementation to the
`channeldb` package and inverts the dependency relation between
`chainntnfs` and `channeldb`.
Many packages depend on channeldb for type definitions,
interfaces, etc. `chainntnfs` is an example of that. `chainntnfs`
defines the `SpendHintCache` and `ConfirmHintCache` interfaces but
it also implments them (`HeightHintCache` struct). The implementation
uses logic that should not leak from channeldb (ex: bucket paths).
This makes our code highly coupled + it would not allow us to use any
of these interfaces in a package that is imported by `channeldb`
(circular dependency).
Our OpenChannelRPC was accepting invalid values for the closing address
field. If we were able to decode the address we would use it in the
script even if the address is for another bitcoin net.
Since `go1.18` the runtime has a package that provides information about module
versions, version control information, and build flags embedded in executable
files built by the go command.
The new packages allows us to get information needed by the `version` command
without having to rely on `ldflags` set at build time.
This can be really helpful while debugging errors from people using custom
binaries. For example a build from master.
In order to reduce the number of calls to the db we try to process as
few channels as we can + try to not do extra work for each of them.
- First fetch all the channels. Then, filter all the public ones and
sort the potential candidates by remote balance.
- Filter out each potential candidate as soon as possible.
- Only check the alias if the channel supports scid aliases.
- Because we sort the channels by remote balance, we will hit the
target amount, if possible, as soon as we can.
We do not want to leak information about our remote balances, so we
shuffle the hop hints (the forced ones go always first) so the invoice
receiver does not know which channels have more balance than others.
The `log.go` file of each subserver is a copy and paste of the others.
All of them have the same typo, an extra space in a comment, and the
the definition of a function not used anywhere.
`rpc-check` is used in our CI pipeline to detect any difference
between the committed and the automatically generated rpc files.
Unfortunately, the current method only detects changes in the
already existing files and won't fail if `make rpc` generates
a totally new file.
`git status --porcelain` makes the trick, it returns a line for
each file that has been modified, created or deleted.
A user complained about getting a misleading error after a typo in the
mnemonic. The word was `hear` and it passed the check even when it is
not in the list of valid words.
The reason is that we where checking if the word is in the variable
`englishWordList` (which includes all the words) instead of checking if the
variable is in the `defaultWordList` (which is basically `englishWoldList`
split by spaces). That means that `hear` passed the check because `heart`
appears in the list.
Related issue [4733](https://github.com/lightningnetwork/lnd/issues/4733)
positiveblue