In this commit, we update the project and relevant sub-modules to sqldb
v1.0.2. The next step is to tag a new version of kvdb, then update the
main module to use that.
In this commit, we fix a race in the `TestPackager` series on channeldb.
A few tests were sharing the same global variable of the set of log
updates, which includes a pointer to an HTLC. The `ExtraData` value of
the HTLC would then be mutated once we go to encode the message on disk.
To fix this, we the global with a function that returns a new instance
of all the test data.
```
==================
WARNING: DATA RACE
Write at 0x0000021b0a48 by goroutine 2896:
github.com/lightningnetwork/lnd/lnwire.(*ExtraOpaqueData).PackRecords()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:74 +0x546
github.com/lightningnetwork/lnd/lnwire.EncodeMessageExtraData()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:121 +0x4d
github.com/lightningnetwork/lnd/lnwire.(*UpdateAddHTLC).Encode()
/home/runner/work/lnd/lnd/lnwire/update_add_htlc.go:164 +0x5af
github.com/lightningnetwork/lnd/lnwire.WriteMessage()
/home/runner/work/lnd/lnd/lnwire/message.go:330 +0x351
github.com/lightningnetwork/lnd/channeldb.WriteElement()
/home/runner/work/lnd/lnd/channeldb/codec.go:186 +0x1975
github.com/lightningnetwork/lnd/channeldb.WriteElements()
/home/runner/work/lnd/lnd/channeldb/codec.go:247 +0x14f
github.com/lightningnetwork/lnd/channeldb.serializeLogUpdate()
/home/runner/work/lnd/lnd/channeldb/channel.go:2529 +0x3c
github.com/lightningnetwork/lnd/channeldb.putLogUpdate()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:525 +0xae
github.com/lightningnetwork/lnd/channeldb.(*ChannelPackager).AddFwdPkg()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:489 +0x684
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerOnlyAdds.func1()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:283 +0x4c
github.com/btcsuite/btcwallet/walletdb/bdb.(*db).Update()
/home/runner/go/pkg/mod/github.com/btcsuite/btcwallet/walletdb@v1.4.2/bdb/db.go:429 +0xe5
github.com/lightningnetwork/lnd/kvdb.Update()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/interface.go:16 +0x258
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerOnlyAdds()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:282 +0x17b
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.(*T).Run.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x44
Previous write at 0x0000021b0a48 by goroutine 2898:
github.com/lightningnetwork/lnd/lnwire.(*ExtraOpaqueData).PackRecords()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:74 +0x546
github.com/lightningnetwork/lnd/lnwire.EncodeMessageExtraData()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:121 +0x4d
github.com/lightningnetwork/lnd/lnwire.(*UpdateAddHTLC).Encode()
/home/runner/work/lnd/lnd/lnwire/update_add_htlc.go:164 +0x5af
github.com/lightningnetwork/lnd/lnwire.WriteMessage()
/home/runner/work/lnd/lnd/lnwire/message.go:330 +0x351
github.com/lightningnetwork/lnd/channeldb.WriteElement()
/home/runner/work/lnd/lnd/channeldb/codec.go:186 +0x1975
github.com/lightningnetwork/lnd/channeldb.WriteElements()
/home/runner/work/lnd/lnd/channeldb/codec.go:247 +0x14f
github.com/lightningnetwork/lnd/channeldb.serializeLogUpdate()
/home/runner/work/lnd/lnd/channeldb/channel.go:2529 +0x3c
github.com/lightningnetwork/lnd/channeldb.putLogUpdate()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:525 +0xae
github.com/lightningnetwork/lnd/channeldb.(*ChannelPackager).AddFwdPkg()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:489 +0x684
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerAddsThenSettleFails.func1()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:490 +0x4c
github.com/btcsuite/btcwallet/walletdb/bdb.(*db).Update()
/home/runner/go/pkg/mod/github.com/btcsuite/btcwallet/walletdb@v1.4.2/bdb/db.go:429 +0xe5
github.com/lightningnetwork/lnd/kvdb.Update()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/interface.go:16 +0x2cd
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerAddsThenSettleFails()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:489 +0x1e7
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.(*T).Run.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x44
Goroutine 2896 (running) created at:
testing.(*T).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x82a
testing.runTests.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2054 +0x84
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.runTests()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2052 +0x896
testing.(*M).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1925 +0xb57
github.com/lightningnetwork/lnd/kvdb.RunTests()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/test_utils.go:23 +0x26
github.com/lightningnetwork/lnd/channeldb.TestMain()
/home/runner/work/lnd/lnd/channeldb/setup_test.go:10 +0x308
main.main()
_testmain.go:321 +0x303
Goroutine 2898 (running) created at:
testing.(*T).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x82a
testing.runTests.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2054 +0x84
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.runTests()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2052 +0x896
testing.(*M).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1925 +0xb57
github.com/lightningnetwork/lnd/kvdb.RunTests()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/test_utils.go:23 +0x26
github.com/lightningnetwork/lnd/channeldb.TestMain()
/home/runner/work/lnd/lnd/channeldb/setup_test.go:10 +0x308
main.main()
_testmain.go:321 +0x303
==================
```
In this commit, we fix an inconsistent in the API related to AMP
payments. When a payment request isn't specified, we require the `--amp`
flag on the CLI to make an AMP payment. However, for payment requests,
we don't require this flag. To fix this inconsistency, we now require
the `--amp` flag to _also_ be set for payment requests.
This type is useful when one wants to encode an integer as an underlying BigSize record. It wraps any integer, then handles the transformation into and out of the BigSize encoding on disk.
In this commit, we add a new type alias for a blob type. This type can be used in areas where a byte slice is used to store a TLV value, which may be a fully opaque nested TLV.
In this commit, we fix a bug that would cause the entire db to shutdown
if hit a panic (since db operations in the main buckets exit with a
panic) while executing a txn call back. This might be a postgres error
we need to check, so we don't want to bail out, and instead want to pass
up the error to the caller so we can retry if needed.
With the new postgres concurrency control, an error may come from a
bucket function that's actually a postgres error. In this case, we need
to return early so we can retry the txn. Otherwise, we'll be working
with an aborted tx, and never actually return the error so we don't auto
retry.
Some sub-systems like btcwallet will return an error from the database,
but they won't properly wrap it. As a result, we were unable to actually
catch the serialization errors in the first place. To work around this,
we'll now attempt to parse the error string directly.
In this commit, rather than attempt to reduce the scope of the current
`CODEOWNERS` file, we instead rename it as a way to stop Github from
automatically assigning reviewers based on it. Instead, we put things in
a "hint" mode as a way for PR OPs to figure out who they should manually
assign PRs today.
With the CODEOWNERS file reduced to a code review assignment hint, we
aim to instead promote usage of the `@code-review` team which will use a
load balancing algorithm for automatic code review assignment.
In this commit, we update new Taproot related TLVs (nonces, partial sig,
sig with nonce, etc). Along the way we were able to get rid of some
boiler plate, but most importantly, we're able to better protect against
API misuse (using a nonce that isn't initialized, etc) with the new
options API. In some areas this introduces a bit of extra boiler plate,
and where applicable I used some new helper functions to help cut down
on the noise.
Note to reviewers: this is done as a single commit, as changing the API
breaks all callers, so if we want things to compile it needs to be in a
wumbo commit.
In this commit, we add two new methods that simplify usage of an
handling a value wrapped in an Option. Thes methods allow a caller to
force the nil/None check up front, and either obtain the value, or fail
the current execution or test.
This should replace most usages of `UnsafeFromSome` where ever used
today.
These two messages will be used to implement the new and improved co-op
closing protocol. This PR also show cases how to use the new
`tlv.OptionalRecord` type to define and handle TLV level parsing.
I think we can make one additional helper function to clean up some of
the boiler plate for the encode/decode.
In this commit, we add a new `Zero` method for the `RecordT` type. This
method allows a caller to create the zero record for a type without
needing to reference the actual TLV type.
With this we go from this:
```go
sig1 := tlv.ZeroRecordT[tlv.TlvType1, Sig]()
```
To this:
```
sig1 := c.CloserNoClosee.Zero()
```
This builds on the concurrent queue to create a generic way to allow
goroutines to pub/sub information. An example includes being notified
each time a state machine is able to carry out a new state transition.
The error was never used as the init couldn't return an error, so we do
away with that. We also modify the main event loop dispatch to more
closely match other areas of the codebase.
In this commit, we make all calls to disconnect after a ping/pong
violation is detected in the `PingManager` async. We do this to avoid
circular waiting that may occur if the disconnect call back ends up
waiting on the peer goroutine to be torn down. If this happens, then the
peer goroutine will be blocked on the ping manager fully tearing down,
which is blocked on the peer disconnect succeeding.
This is a similar class of issue we've delt with recently as pertains to
the peer and the server: sync all back execution must not lead to
a circular waiting loop.
Fixes#8379
We add a WhenSome that'll pass in the actual underlying value, and not
just the record.
We also add `SomeRecord` to make it easier to create TLV records w/
`Some` set.
This will be useful when decoding optional TLV records, as we can use
this to look up in the `typeMap` for a given field to see if we decoded
it or not.
With this change, callers can now examine a struct instance of the
`TlvType` instance to figure out programmatically which integer type is
maps to. This'll be useful when decoding optional TLV types into a wire
struct (knowing when to set it to Some vs None).
In this commit, we modify the RecordT type to allow callers to re-use
the Record definition of a different type, but use the new type param to
override the integer type used on the wire.
This will let use do things like encode a signature using the same
RecordProducer instance, but with a diff type in another context.
The upcoming use for this is allowing our `lnwire.Sig` type to be
encoded in the same message using distinct TLV integer types (new co-op
close protocol).
In this commit, we add a new type, `RecordT[T, V]` to reduce some of the
common boiler plate for TLV types. This type lets you take either a
primitive type, or an existing Record, and gain common methods used to
create tlv streams.
It also serves as extra type annotation as well, since wire structs can
use this to wrap any existing type and gain the relevant record methods.
This implementation ensures that the very definition of the field also
binds the TLV type value. It does this by using the generated code to
map a struct like TlvType1 to an actually Type like Type(1).
Olaoluwa Osuntokun