Commit Graph
100 Commits
Author SHA1 Message Date
Yong 65a18c5b35 Merge pull request #9515 from ellemouton/graphFixFlake
graph: ensure topology subscriber handling and network msg handling is synchronous
2025-02-17 20:48:05 +08:00
yyforyongyu 2b0c25e31d workflows: skip coverage error in job finish
Also updates the coverage action used.
2025-02-13 22:05:38 +08:00
Yong 9c2c95d46f Merge pull request #9478 from ellemouton/graph3
discovery+graph:  move funding tx validation to the gossiper
2025-02-13 15:00:14 +08:00
yyforyongyu 759dc2066e docs: update release notes 2025-02-12 09:48:03 +08:00
yyforyongyu 59759f861f rpcserver: check blockbeatDispatcher when deciding isSynced
This commit changes `GetInfo` to include `blockbeatDispatcher`'s current
state when deciding whether the system is synced to chain. Previously we
check the best height against the wallet and the channel graph, we
should also do this to the blockbeat dispatcher to make sure the
internal consumers are also synced to the best block.
2025-02-12 09:48:02 +08:00
yyforyongyu 89c4a8dfd7 chainio: add method CurrentHeight
Add a new method `CurrentHeight` to query the current best height of the
dispatcher.
2025-02-12 09:48:02 +08:00
yyforyongyu e45e1f2b0e itest: fix flake in testForwardInterceptorRestart
We need to make sure the links are recreated before calling the
interceptor, otherwise we'd get the following error, causing the test to
fail.
```
2025-02-07 15:01:38.991 [ERR] RPCS interceptor.go:624: [/routerrpc.Router/HtlcInterceptor]: fwd (Chan ID=487:3:0, HTLC ID=0) not found
```
2025-02-10 12:24:23 +08:00
yyforyongyu 58e76b726e routing: add docs and monior refactor decideNextStep
Add verbose docs and refactor the method to exit early when `allow` is
true.
2025-02-10 11:36:18 +08:00
yyforyongyu ca10707b26 itest+lntest: assert payment is failed once the htlc times out 2025-02-10 11:31:51 +08:00
yyforyongyu faa3110127 docs: update release notes 2025-02-10 11:31:50 +08:00
yyforyongyu 5a62528fd7 routing: improve loggings for attempt result handling 2025-02-10 11:31:50 +08:00
yyforyongyu 1acf4d7d4d routing: always update payment in the same goroutine
This commit refactors `collectResultAsync` such that this method is now
only responsible for collecting results from the switch. The method
`decideNextStep` is expanded to process these results in the same
goroutine where we fetch the payment from db, to make sure the lifecycle
loop always have a consistent view of a given payment.
2025-02-10 11:29:29 +08:00
yyforyongyu e1279aab20 routing: add new method reloadPayment
To further shorten the lifecycle loop.
2025-02-10 11:28:23 +08:00
yyforyongyu 1fe2cdb765 routing: move amendFirstHopData into requestRoute 2025-02-10 11:28:23 +08:00
yyforyongyu 966cfccb94 routing: add new method reloadInflightAttempts
To shorten the method `resumePayment` and make each step more clear.
2025-02-10 11:28:23 +08:00
yyforyongyu f96eb50ca8 channeldb+routing: cache circuit and onion blob for htlc attempt
This commit caches the creation of sphinx circuit and onion blob to
avoid re-creating them again.
2025-02-10 11:28:23 +08:00
yyforyongyu 46eb811543 routing: remove redundant GetHash 2025-02-10 11:28:18 +08:00
Yong 5b1eaf9978 Merge pull request #9474 from ellemouton/nodeAnnConversion
graph/db: correctly handle de(ser)ialisation of `models.LightningNode` opaque addresses
2025-02-07 18:06:10 +08:00
Yong 457a245a4e Merge pull request #9482 from yyforyongyu/itest-log-ts
lntest: log timestamp when printing errors
2025-02-06 16:16:54 +08:00
yyforyongyu 5f4716c699 lntest: log timestamp when printing errors 2025-02-05 22:51:46 +08:00
Yong bac699df8f Merge pull request #9446 from yyforyongyu/yy-prepare-fee-replace
sweeper: rename `Failed` to `Fatal` and minor refactor
2025-02-05 22:48:41 +08:00
yyforyongyu b98542bd96 docs: update release notes 2025-02-05 19:49:09 +08:00
yyforyongyu e5f39dd644 sweep: refactor storeRecord to updateRecord
To make it clear we are only updating fields, which will be handy for
the following commit where we start tracking for spending notifications.
2025-02-05 19:49:09 +08:00
yyforyongyu 7eea7a7e9a sweep: add requestID to monitorRecord
This way we can greatly simplify the method signatures, also paving the
upcoming changes where we wanna make it clear when updating the
monitorRecord, we only touch a portion of it.
2025-02-05 19:49:04 +08:00
yyforyongyu bde5124e1b sweep: shorten storeRecord method signature
This commit shortens the function signature of `storeRecord`, also makes
sure we don't call `t.records.Store` directly but always using
`storeRecord` instead so it's easier to trace the record creation.
2025-02-05 19:48:18 +08:00
yyforyongyu c68b8e8c1e sweep: rename Failed to Fatal
This commit renames `Failed` to `Fatal` as it sounds too close to
`PublishFailed`. We also wanna emphasize that inputs in this state won't
be retried.
2025-02-05 19:48:18 +08:00
yyforyongyu bc8c1643c6 itest: move channel force closes into one sub test var 2025-01-26 09:16:02 +08:00
yyforyongyu e4b205cd90 itest: fix flake in testAnchorThirdPartySpend
When mining lots of blocks in the itest, the subsystems can be out of
sync in terms of the best block height. We now assert the num of pending
sweeps on Alice's node to give her more time to sync the blocks,
essentially behaving like a `time.Sleep` as in reality, the blocks would
never be generated this fast.
2025-01-26 09:15:36 +08:00
yyforyongyu 29603954bd .github: update actions versions 2025-01-20 21:55:20 +08:00
yyforyongyu c24f839fbe itest: fix flake in testSweepHTLCs
We need to make sure Caarol finishes settling her invoice with Bob
before shutting down, so we make sure `AssertHTLCNotActive` on Bob
happens before shutting node Carol.
2025-01-17 22:58:37 +08:00
Yong fb91b04906 Merge pull request #9405 from yyforyongyu/fix-unused-params
discovery+lnd: make param `ProofMatureDelta` configurable
2025-01-17 22:57:26 +08:00
yyforyongyu ae2bcfe3d8 docs: add release notes 2025-01-17 21:44:23 +08:00
yyforyongyu 27a05694cb multi: make ProofMatureDelta configurable
We add a new config option to set the `ProofMatureDelta` so the users
can tune their graphs based on their own preference over the num of
confs found in the announcement signatures.
2025-01-17 21:44:23 +08:00
yyforyongyu 56ff6d1fe0 docs: update release notes 2025-01-17 18:58:20 +08:00
yyforyongyu 772a9d5f42 discovery: fix mocked peer in unit tests
The mocked peer used here blocks on `sendToPeer`, which is not the
behavior of the `SendMessageLazy` of `lnpeer.Peer`. To reflect the
reality, we now make sure the `sendToPeer` is non-blocking in the tests.
2025-01-17 17:59:06 +08:00
yyforyongyu 9fecfed3b5 discovery: fix race access to syncer's state
This commit fixes the following race,
1. syncer(state=syncingChans) sends QueryChannelRange
2. remote peer replies ReplyChannelRange
3. ProcessQueryMsg fails to process the remote peer's msg as its state
   is neither waitingQueryChanReply nor waitingQueryRangeReply.
4. syncer marks its new state waitingQueryChanReply, but too late.

The historical sync will now fail, and the syncer will be stuck at this
state. What's worse is it cannot forward channel announcements to other
connected peers now as it will skip the broadcasting during initial
graph sync.

This is now fixed to make sure the following two steps are atomic,
1. syncer(state=syncingChans) sends QueryChannelRange
2. syncer marks its new state waitingQueryChanReply.
2025-01-17 02:39:07 +08:00
yyforyongyu 4b30b09d1c discovery: add new method handleSyncingChans
This is a pure refactor to add a dedicated handler when the gossiper is
in state syncingChans.
2025-01-17 00:22:22 +08:00
yyforyongyu eb2b0c783f graph: fix staticcheck suggestion
From staticcheck: QF1002 - Convert untagged switch to tagged switch.
2025-01-17 00:21:45 +08:00
yyforyongyu 001e5599b6 multi: add debug logs for edge policy flow
This commit adds more logs around the ChannelUpdate->edge policy process
flow.
2025-01-17 00:17:23 +08:00
Yong e0a920af44 Merge pull request #9420 from yyforyongyu/assert-channel-graph
itest+lntest: make sure to assert edge in both graph db and cache
2025-01-17 00:13:02 +08:00
yyforyongyu faa1f67480 lntest: assert channel edge in both graph db and cache
We need to make sure the channel edge has been updated in both the graph
DB and cache.
2025-01-16 23:00:22 +08:00
yyforyongyu e576d661ef itest: fix docs 2025-01-16 23:00:21 +08:00
yyforyongyu 848f42ea1d itest: fix flake in graph_topology_notifications
We need to make sure to wait for the `ListPeers` to give us the latest
response.
2025-01-16 16:19:37 +08:00
Yong 4b16c2902e Merge pull request #9383 from ziggie1984/bugfix-createmissingedge
bugfix createmissingedge
2025-01-14 16:20:15 +08:00
yyforyongyu bafe5d009f chainntnfs: fix test testSingleConfirmationNotification 2024-12-20 22:08:32 +08:00
yyforyongyu 2913f6e4c9 itest: fix flake in testCoopCloseWithExternalDeliveryImpl
The response from `ClosedChannels` may not be up-to-date, so we wrap it
inside a wait closure.
2024-12-20 19:38:15 +08:00
yyforyongyu 76eeae32d6 itest: document and fix wallet UTXO flake 2024-12-20 19:38:14 +08:00
yyforyongyu 7ab4081ffd lntest: make sure chain backend is synced to miner
We sometimes see `timeout waiting for UTXOs` error from bitcoind-related
itests due to the chain backend not synced to the miner. We now assert
it's synced before continue.
2024-12-20 19:38:14 +08:00
yyforyongyu 1dec926165 workflows: increase num of tranches to 16
Keep the SQL, etcd, bitcoin rpcpolling builds and non-ubuntu builds at 8
since they are less stable.
2024-12-20 19:38:14 +08:00
yyforyongyu 31b66962d8 lntest: properly handle shutdown error
This commit removes the panic used in checking the shutdown log.
Instead, the error is returned and asserted in `shutdownAllNodes` so
it's easier to check which node failed in which test. We also catch all
the errors returned from `StopDaemon` call to properly access the
shutdown behavior.
2024-12-20 19:38:14 +08:00
yyforyongyu 73574d919d lntest: add timeouts for windows
For Windows the tests run much slower so we create customized timeouts
for them.
2024-12-20 19:38:14 +08:00
yyforyongyu d7f8fa6ab6 lntest: increase port timeout 2024-12-20 19:38:14 +08:00
yyforyongyu 33b07be8c3 itest: even out num of tests per tranche
Previous splitting logic simply put all the remainder in the last
tranche, which could make the last tranche run significantly more test
cases. We now change it so the remainder is evened out across tranches.
2024-12-20 19:38:14 +08:00
yyforyongyu c536bc220f itest: add a prefix before appending a subtest case 2024-12-20 19:38:13 +08:00
yyforyongyu 686a7dd31c docs: update release notes 2024-12-20 19:38:13 +08:00
yyforyongyu becbdce64c lntest: limit the num of blocks mined in each test 2024-12-20 19:38:13 +08:00
yyforyongyu 5236c05dc6 itest+lntest: add new method FundNumCoins
Most of the time we only need to fund the node with given number of
UTXOs without concerning the amount, so we add the more efficient
funding method as it mines a single block in the end.
2024-12-20 19:38:13 +08:00
yyforyongyu 691a6267be workflows: use btcd for macOS
To increase the speed from 40m per run to roughly 20m per run.
2024-12-20 19:38:13 +08:00
yyforyongyu 77b2fa0271 lntest: make sure policies are populated in AssertChannelInGraph 2024-12-20 19:38:13 +08:00
yyforyongyu c97c31a70b lntest: increase node start timeout and payment benchmark timeout 2024-12-20 19:38:12 +08:00
yyforyongyu efe81f2d3c itest: track and skip flaky tests for windows
To make the CI indicative, we now starting tracking the flaky tests
found when running on Windows. As a starting point, rather than ignore
the windows CI entirely, we now identify there are cases where lnd can
be buggy when running in windows.

We should fix the tests in the future, otherwise the windows build
should be deleted.
2024-12-20 19:38:12 +08:00
yyforyongyu e79ad6e5aa itest: further reduce block mined in tests 2024-12-20 19:38:12 +08:00
yyforyongyu 6f2e7feb94 itest: breakdown testSendDirectPayment
Also fixes a wrong usage of `ht.Subtest`.
2024-12-20 19:38:12 +08:00
yyforyongyu c7b8379602 itest: break down channel fundmax tests 2024-12-20 19:38:12 +08:00
yyforyongyu 37b8210f37 itest: break down taproot tests 2024-12-20 19:38:12 +08:00
yyforyongyu efae8ea56f itest: break down single hop send to route
Also removed the duplicate test cases.
2024-12-20 19:38:11 +08:00
yyforyongyu c029f0a84f itest: break down basic funding flow tests 2024-12-20 19:38:11 +08:00
yyforyongyu c58fa01a66 itest: break down wallet import account tests 2024-12-20 19:38:11 +08:00
yyforyongyu 31aada65a4 itest: break down channel backup restore tests 2024-12-20 19:38:11 +08:00
yyforyongyu 7b1427a565 itest: break down payment failed tests 2024-12-20 19:38:11 +08:00
yyforyongyu 3319d0d983 itest: break down open channel fee policy 2024-12-20 19:38:11 +08:00
yyforyongyu 21c5d36812 itest: break down scid alias channel update tests 2024-12-20 19:38:10 +08:00
yyforyongyu 04a15039d7 itest: break all multihop test cases 2024-12-20 19:38:10 +08:00
yyforyongyu a76ff79adc itest: break down utxo selection funding tests 2024-12-20 19:38:10 +08:00
yyforyongyu b1cb819f07 itest: break down channel restore commit types cases 2024-12-20 19:38:10 +08:00
yyforyongyu 5663edfc2f itest: break remote signer into independent cases
So the test can run faster.
2024-12-20 19:38:10 +08:00
yyforyongyu cca2364097 itest: optimize blocks mined in testGarbageCollectLinkNodes
There's no need to mine 80ish blocks here.
2024-12-20 19:38:10 +08:00
yyforyongyu 1950d89925 itest: document a flake found in SendToRoute 2024-12-20 19:38:10 +08:00
yyforyongyu 7e80b77535 itest: document a rare flake found in macOS 2024-12-20 19:38:09 +08:00
yyforyongyu 39104c53d4 lntest: fix flakeness in openChannelsForNodes
We now make sure the channel participants have heard their private
channel when opening channels.
2024-12-20 19:38:09 +08:00
yyforyongyu cfb5713cda itest+lntest: fix flake in MPP-related tests 2024-12-20 19:38:09 +08:00
yyforyongyu fb59669ae8 itest: document details about MPP-related tests
This is needed so we can have one place to fix the flakes found in the
MPP-related tests, which is fixed in the following commit.
2024-12-20 19:38:09 +08:00
yyforyongyu f912f407c8 lntest: increase rpcmaxwebsockets for btcd
This has been seen in the itest which can lead to the node startup
failure,
```
2024-11-20 18:55:15.727 [INF] RPCS: Max websocket clients exceeded [25] - disconnecting client 127.0.0.1:57224
```
2024-12-20 19:38:09 +08:00
yyforyongyu 4e85d86a2e itest: fix flake in update_pending_open_channels 2024-12-20 19:38:09 +08:00
yyforyongyu 23edca8d19 itest: fix flake in testPrivateUpdateAlias 2024-12-20 19:38:08 +08:00
yyforyongyu 7c3564eeb6 itest: fix flake in runPsbtChanFundingWithNodes 2024-12-20 19:38:08 +08:00
yyforyongyu 7ceb9a4af5 lntest+itest: remove AssertNumActiveEdges
This is no longer needed since we don't have standby nodes, plus it's
causing panic in windows build due to `edge.Policy` being nil.
2024-12-20 19:38:08 +08:00
yyforyongyu 8f3100c984 itest: put mpp tests in one file 2024-12-20 19:38:08 +08:00
yyforyongyu fee6b70519 itest: use ht.CreateSimpleNetwork whenever applicable
So we won't forget to assert the topology after opening a chain of
channels.
2024-12-20 19:38:08 +08:00
yyforyongyu 4eea2078fb itest+routing: fix flake in runFeeEstimationTestCase
The test used 10s as the timeout value, which can easily cause a timeout
in a slow build so we increase it to 60s.
2024-12-20 19:38:08 +08:00
yyforyongyu 8b8f0c4eb4 itest: fix flake in testSwitchOfflineDelivery
The reconnection will happen automatically when the nodes have a
channel, so we just ensure the connection instead of reconnecting
directly.
2024-12-20 19:38:07 +08:00
yyforyongyu 66b35018b8 itest: fix flake in testRevokedCloseRetributionZeroValueRemoteOutput
We need to mine an empty block as the tx may already have entered the
mempool. This should be fixed once we start using the sweeper to handle
the justice tx.
2024-12-20 19:38:07 +08:00
yyforyongyu c07162603d itest: remove loop in wsTestCaseBiDirectionalSubscription
So we know which open channel operation failed.
2024-12-20 19:38:07 +08:00
yyforyongyu 782edde213 itest: fix and document flake in sweeping tests
We previously didn't see this issue because we always have nodes being
over-funded.
2024-12-20 19:38:07 +08:00
yyforyongyu 9f764c25f9 itest: flatten PSBT funding test cases
So it's easier to get the logs and debug.
2024-12-20 19:38:07 +08:00
yyforyongyu 762e59d78c itest: fix flake for neutrino backend 2024-12-20 19:38:07 +08:00
yyforyongyu 3a45492398 itest: fix spawning temp miner 2024-12-20 19:38:07 +08:00
yyforyongyu 2a9b7ec536 itest: fix flake in testSendDirectPayment
This bug was hidden because we used standby nodes before, which always
have more-than-necessary wallet utxos.
2024-12-20 19:38:06 +08:00
yyforyongyu e7310ff1b6 itest: fix testOpenChannelUpdateFeePolicy
This commit fixes a misuse of `ht.Subtest`, where the nodes should
always be created inside the subtest.
2024-12-20 19:38:06 +08:00
yyforyongyu 010a4f1571 lntest: add human-readble names and check num of nodes 2024-12-20 19:38:06 +08:00