Update AddEdgeProof to handle both v1 (four separate signatures) and
v2 (single aggregated signature) channel authentication proofs using
the appropriate SQL queries.
Thread the gossip version through DeleteChannelEdges to enable
version-aware channel deletion. The KVStore rejects non-v1 versions
while SQLStore properly passes the version to the underlying queries.
This prepares for v2 channel zombie handling (strict zombie pruning
for v2 is marked as TODO).
Extends the SQL store to support v2 (taproot) channel announcements:
- Add version validation in AddChannelEdge
- Store v2-specific fields: FundingPkScript, MerkleRootHash, Signature
- Update buildEdgeInfoWithBatchData to reconstruct v2 channels from DB
with optional bitcoin keys and funding script handling
- Add WithMerkleRootHash edge modifier for ChannelV2Fields
Add FundingPKScript() method that returns the funding output's
pkScript for the channel. The implementation is version-aware:
- V1: generates a 2-of-2 multisig P2WSH script from the two bitcoin
keys
- V2: will use taproot script (to be implemented)
This encapsulates the script generation logic and makes it clear
which bitcoin keys are being used. Replaces direct calls to
genMultiSigP2WSH with the cleaner method call.
Since not all will be required for V2 channels.
Wrap BitcoinKey1Bytes and BitcoinKey2Bytes in fn.Option since these
fields are only required for v1 channel announcements. V2 channels may
or may not have bitcoin keys present in their announcement.
NewV1Channel constructor wraps the bitcoin keys with fn.Some().
All access sites updated to unwrap the options, using UnwrapOr for
non-critical paths and UnwrapOrErr where the keys must be present
(e.g., KV serialization, ToChannelAnnouncement).
This makes it clear what fields must/can be set for a V1 channel.
Introduce NewV1Channel constructor to create v1 channel edges with
proper initialization and validation. The constructor:
- Takes required fields (chanID, chainHash, node keys) as parameters
- Takes v1-specific fields (bitcoin keys, extra opaque data) via
ChannelV1Fields struct
- Accepts optional fields (capacity, channel point, features, proof)
via functional options (WithCapacity, WithChannelPoint, etc.)
- Validates that if an AuthProof is provided, its version matches the
channel version
This makes it clear which fields are required vs optional for v1
channels and prevents incorrectly initialized channel edges. All
tests and production code updated to use the constructor.
Also update it to more closely match the persisted version which has the
v1 and v2 only fields as optional.
Refactor ChannelAuthProof to support both v1 and v2 channel
announcements:
- Add Version field to distinguish v1 from v2 proofs
- Wrap v1-specific fields (NodeSig1/2, BitcoinSig1/2) in fn.Option
since v2 doesn't use them
- Add optional Signature field for v2's single schnorr signature
- Add constructor functions NewV1ChannelAuthProof and
NewV2ChannelAuthProof to enforce correct initialization
- Add getter methods (NodeSig1(), BitcoinSig1(), etc.) that safely
unwrap options, returning empty slices when not present
The IsEmpty() check is updated to handle both versions correctly.
Both stores validate v1-only for now.
And set it to V1 version everywhere.
Add a Version field to ChannelEdgeInfo to distinguish between v1 and
v2 channel announcements. Set it to GossipVersion1 for all existing
channels.
Both KV and SQL stores now validate that only v1 channels are
currently supported, returning an error for v2 channels. The KV store
automatically sets version to v1 when deserializing (since all
persisted channels in KV format are v1).
This versioning is essential for handling the different field
requirements and validation logic between v1 and v2 channels.
Add a version parameter to maybeCreateShellNode to allow callers to
specify which gossip protocol version should be used when creating
shell nodes. Currently all callers pass GossipVersion1, but this
change sets up the foundation for v2 support.
Shell nodes are lightweight node entries containing only a protocol
version and public key, created before full node information is
available.
Update some of the node related graph CRUD methods to take a version
rather than hardcoding them in the SQLStore layer. Move the version up
one layer instead. This will make it easier to make it configurable
later on.
HasNode currently is very v1 specific since it returns a time.Time
timestamp which is specific to V1 node announcements. However, it is
mostly only used for the "exists" return value. So here we split it up
into HasNode which just checks existence and HasV1Node which retains the
same behavaiour as before.
Update the SQLStore node writer and readers to handle V2 ndoes.
Currently no logic will actually add such nodes. The following commits
will update what is needed so that CRUD for v2 nodes can be tested.
The underling store will store gossip messages across gossip versions
and we will instead expose version parameters on many of the methods. So
this interface really just abstracts the underlying store/schema type.
We should avoid taking the lock of a mutex inside transaction.
Currently we also take this lock in other places and there is a
chance that in case the application lock aquires the lock but
all transactions are already blocked waiting for the mutex to
unlock, we end up in a deadlock.
The DisconnectBlockAtHeight method was modifying the rejectCache and
chanCache without holding the cacheMu lock. This caused races with
other operations that properly held the lock, such as AddChannelEdge
which modifies the caches in its OnCommit callback while the batch
scheduler holds cacheMu.
Fix by acquiring cacheMu before removing channels from the caches.
This commit fixes a race condition where multiple goroutines call
SetSourceNode concurrently during startup, causing sql.ErrNoRows
errors. The race occurs when multiple code paths (setSelfNode,
createNewHiddenService, RPC updates) read the same old timestamp,
independently increment it to the same new value (T+1), and race to
write.
The fix uses the new UpsertSourceNode SQL query (without strict
timestamp constraint) instead of UpsertNode. This allows
last-write-wins semantics for our own node, ensuring all parameter
changes persist even when timestamps collide.
Refactored sql_store.go for reusability:
- upsertNodeAncillaryData: common logic for features/addresses/extras
- populateNodeParams: common parameter building with callback pattern
- buildNodeUpsertParams: builds params for strict UpsertNode
- buildSourceNodeUpsertParams: builds params for lenient UpsertSourceNode
- upsertSourceNode: new function using lenient query
Updated TestSetSourceNodeSameTimestamp to verify that concurrent
updates with the same timestamp now succeed and parameter changes
persist.
Fixes the itest error:
"unable to upsert source node: upserting node(...): sql: no rows in
result set"
It can happen that we are handling 2 of the same node announcements in
the same batch transaction. In that case, our `UpsertNode` conflict
assertion may fail. We need to handle this gracefully.
In this commit, we update the SQL store implementation to support the
new iterator-based API for ChanUpdatesInHorizon. This includes adding
SQL query pagination support and helper functions for efficient batch
processing.
The SQL implementation uses cursor-based pagination with configurable
batch sizes, allowing efficient iteration over large result sets without
loading everything into memory. The query is optimized to use indexes
effectively and minimize database round trips.
New SQL query GetChannelsByPolicyLastUpdateRange is updated to support:
- Cursor-based pagination using (max_update_time, id) compound cursor
- Configurable batch sizes via MaxResults parameter
- Efficient batch caching with updateChanCacheBatch helper
In this commit, we update the SQL store implementation to support the
new iterator-based API for NodeUpdatesInHorizon. This includes adding a
new SQL query that supports efficient pagination through result sets.
The SQL implementation uses cursor-based pagination with configurable
batch sizes, allowing efficient iteration over large result sets without
loading everything into memory. The query is optimized to use indexes
effectively and minimize database round trips.
New SQL query GetNodesByLastUpdateRange is updated to support:
* Cursor-based pagination using (last_update, pub_key) compound cursor
* Optional filtering for public nodes only
* Configurable batch sizes via MaxResults parameter
In this commit, we refactor the ChanUpdatesInHorizon method to return
an iterator instead of a slice. This change significantly reduces
memory usage when dealing with large result sets by allowing callers to
process items incrementally rather than loading everything into memory
at once.
In this commit, we refactor the NodeUpdatesInHorizon method to return
an iterator instead of a slice. This change significantly reduces
memory usage when dealing with large result sets by allowing callers to
process items incrementally rather than loading everything into memory
at once.
The new implementation uses Go 1.23's iter.Seq type to provide a
standard iterator interface. The method now supports configurable batch
sizes through functional options, allowing fine-tuned control over
memory usage and performance characteristics.
Rather than reading all the entries from disk into memory (before this
commit, we did consult the cache for most entries, skipping the disk
hits), we now expose a chunked iterator instead.
We also make the process of filtering out public nodes first class. This
saves many newly created db transactions later.
The SQL* helpers are meant to always set the `Valid` field of the
sql.Null* type to true. Otherwise they cannot be used to set a valid,
empty field. However, we dont want to break the behaviour of the
existing SQLStr helper and so this commit adds a new helper with the
desired functionality.
This commit simplifies insertChanEdgePolicyMig. Much of the logic can be
removed given that this method is only used in the context of the graph
SQL migration.
This should improve the performance of the migration quite a lot since
it removes the extra GetChannelAndNodesBySCID call.
Finally, we make the channel-policy part of the SQL migration idempotent
by adding a migration-only policy insert query which will not error out
if the policy already exists and does not have a timestamp that is newer
than the existing records timestamp. To keep the commit simple, a
insertChanEdgePolicyMig function is added which is basically identical
to the updateChanEdgePolicy function except for the fact that it uses
the newly added query. In the next commit, it will be simplified even
more.
In this commit, we make the channel part of the graph SQL migration
idempotent (retry-safe!). We do this by adding a migration-only channel
insert query that will not error out if a the query is called and a
chanenl with the given scid&version already exists. We also ensure that
errors are not thrown if existing channel features & extra types are
re-added.
There is no need to use the "collect-then-update" pattern for node
insertion during the SQL migration since if we do have any previously
persisted data for the node and happen to re-run the insertion for that
node, the data will be exactly the same. So we can make use of "On
conflict, no nothing" here too.
In this commit, the graph SQL migration is updated so that the node
migration step is retry-safe. This is done by using migration specific
logic & queries that do not use the same node-update-constraint as the
normal node upsert logic. For normal "run-time" logic, we always expect
a node update to have a newer timestamp than any previously stored one.
But for the migration, we will only ever be dealing with a single
announcement for a given node & to make things retry-safe, we dont want
the query to error if we re-insert the exact same node.
Finally, we update the migrateZombieIndex function to use batch
validation just like was done in the previous commits. Here, we
additionally make sure to validate the entire zombie index entry and not
just the SCID.
As was done in the previous commits for nodes & channels, we update the
migrateClosedSCIDIndex function here so that it validates migrated
entries in batches rather than one-by-one.
As was done in the previous commits for nodes & channels, we update the
migratePruneLog function here so that it validates migrated entries in
batches rather than one-by-one.
Restructue the `migrateChannelsAndPolicies` function so that it does the
validation of migrated channels and policies in batches. So instead of
fetching channel and its policies individually after migrating it, we
wait for a minimum batch size to be reached and then validate a batch of
them together. This lets us make way fewer DB round trips.
Restructue the `migrateNodes` function so that it does the validation of
migrated nodes in batches. So instead of fetching each node individually
after migrating it, we wait for a minimum batch size to be reached and
then validate a batch of nodes together. This lets us make way fewer DB
round trips.
We do this so that this lookup is only done in the situation it is
actually needed. During a migration, we dont need to special case this
AlreadyExists error since we will terminate the transaction either way.
So there is no need for the extra lookup during the migration.
A timing analysis showed that this query was significantly impacting the
performance of the migration when run with a postgres backend.
Elle Mouton