This commit fixes a race condition where multiple goroutines call
SetSourceNode concurrently during startup, causing sql.ErrNoRows
errors. The race occurs when multiple code paths (setSelfNode,
createNewHiddenService, RPC updates) read the same old timestamp,
independently increment it to the same new value (T+1), and race to
write.
The fix uses the new UpsertSourceNode SQL query (without strict
timestamp constraint) instead of UpsertNode. This allows
last-write-wins semantics for our own node, ensuring all parameter
changes persist even when timestamps collide.
Refactored sql_store.go for reusability:
- upsertNodeAncillaryData: common logic for features/addresses/extras
- populateNodeParams: common parameter building with callback pattern
- buildNodeUpsertParams: builds params for strict UpsertNode
- buildSourceNodeUpsertParams: builds params for lenient UpsertSourceNode
- upsertSourceNode: new function using lenient query
Updated TestSetSourceNodeSameTimestamp to verify that concurrent
updates with the same timestamp now succeed and parameter changes
persist.
Fixes the itest error:
"unable to upsert source node: upserting node(...): sql: no rows in
result set"
This commit adds TestSetSourceNodeSameTimestamp to demonstrate the
current behavior when SetSourceNode is called with the same last update
timestamp. The test reveals a difference between the SQL and bbolt
implementations:
- SQL store returns sql.ErrNoRows when attempting to update with the
same timestamp, as the upsert query's UPDATE clause requires the new
timestamp to be strictly greater than the existing one
- bbolt store silently ignores stale updates and returns no error
This behavior is important to document because our own node
announcements may change quickly with the same timestamp, unlike
announcements from other nodes where same timestamp typically means
identical parameters.
It can happen that we are handling 2 of the same node announcements in
the same batch transaction. In that case, our `UpsertNode` conflict
assertion may fail. We need to handle this gracefully.
In preparation for adding a NodeAnnouncement2 struct along with a
NodeAnnouncement interface, this commit renames the existing
NodeAnnouncment struct to NodeAnnouncement1.
In this commit, we refactor the ChanUpdatesInHorizon method to return
an iterator instead of a slice. This change significantly reduces
memory usage when dealing with large result sets by allowing callers to
process items incrementally rather than loading everything into memory
at once.
In this commit, we refactor the NodeUpdatesInHorizon method to return
an iterator instead of a slice. This change significantly reduces
memory usage when dealing with large result sets by allowing callers to
process items incrementally rather than loading everything into memory
at once.
The new implementation uses Go 1.23's iter.Seq type to provide a
standard iterator interface. The method now supports configurable batch
sizes through functional options, allowing fine-tuned control over
memory usage and performance characteristics.
Rather than reading all the entries from disk into memory (before this
commit, we did consult the cache for most entries, skipping the disk
hits), we now expose a chunked iterator instead.
We also make the process of filtering out public nodes first class. This
saves many newly created db transactions later.
The first byte of an opaque addr must be one that we dont understand
yet. We do this update in preparation for doing an on-the-fly parse of
persisted opaque addrs to see if they contain addrs that we now support.
For this to work, the first byte cant be 0x01 since this maps to a known
address.
Use the new feature of Go 1.24, fix linter warnings.
This change was produced by:
- running golangci-lint run --fix
- sed 's/context.Background/t.Context/' -i `git grep -l context.Background | grep test.go`
- manually fixing broken tests
- itest, lntest: use ht.Context() where ht or hn is available
- in HarnessNode.Stop() we keep using context.Background(), because it is
called from a cleanup handler in which t.Context() is canceled already.
Here we adjust the ForEachNodeCached graph DB method to pass in a node's
addresses into the provided call-back if requested. This will allow us
to improve the performance of node/channel iteration in the autopilot
subserver.
In this commit, we add a makefile helper that can be used to add the
"-short" flag to the "go test" command when running unit tests via "make
unit ...".
Tests can then be expanded with a `testing.Short()` check to potentially
skip the test in short mode. This is useful for if a dev wants to
quickly run most of the tests in a package but would like to opt out of
running the longer form tests such as stress tests or rapid generation
tests.
This commit adds this check to two graph/db tests.
In this commit, we move the serialisation details of a channel's
features to the DB layer and change the `models` field to instead use a
more useful `*lnwire.FeatureVector` type.
This makes the features easier to work with and moves the serialisation
to where it is actually used.
Here we start using the newly added message_flags and channel_flags
columns of the channel_policies table. The test added previoulsy to
demonstrate the bug is now updated to show that the bug has been fixed.
This commit adds a new graph store unit test that demonstrates that
there is currently a bug in the SQL version of the graph store where
unknown message or channel flags in a channel_update are not persisted
correctly. This will be fixed in an upcoming commit.
This commit demonstrates that currently the Addresses field of a
LightningNode is not populated correctly if the node has no addresses if
the SQL graph store is being used. This will be fixed in an upcoming
commit.
The comment of FetchChannelEdgesByID says that if the ErrZombieEdge
error is returned, then the ChannelEdgeInfo return parameter will also
still be populated and returned (ie, wont be nil). This commit updates
the SQLStore implementation of FetchChannelEdgesByID to do this. This is
needed to prevent nil dereference panics at any call-sites that depend
on the method working as it describes.
And delete the old MakeTestGraph function.
Here we need to update some of our unit tests to ensure that any updates
to existing nodes or channel policies have newer timestamps. This is so
that we don't violate our SQL DB constraints that prevent updates to a
node or channel policy record if the new update is not newer (has a
newer LastUpdate time) than the currently presisted record).
Elle Mouton