Commit Graph
100 Commits
Author SHA1 Message Date
Yong 5d921723b1 Merge pull request #9609 from yyforyongyu/fix-listunspent
Fix inaccurate `listunspent` result
2025-03-21 09:22:11 +08:00
yyforyongyu bdcd980868 docs: update release notes 2025-03-21 08:06:19 +08:00
yyforyongyu ea66dacc07 itest: skip force close restart in windows 2025-03-21 08:06:19 +08:00
yyforyongyu feeb9acaa7 itest: document flakeRaceInBitcoinClientNotifications 2025-03-21 08:06:19 +08:00
yyforyongyu f0300762c0 lnwallet: move btcwallet log under BTWL
So we can focus on debugging `BTWL` without concerning `lnwallet`, which
has a lot of channel-specific loggings.
2025-03-21 08:06:18 +08:00
yyforyongyu 07f60bd910 lnd: enable logging for sqldb 2025-03-21 08:06:18 +08:00
yyforyongyu 9a25d7172e sweep: improve loggings 2025-03-21 08:06:18 +08:00
yyforyongyu 533aeb8a25 itest: remove flakeFundExtraUTXO 2025-03-21 08:06:18 +08:00
yyforyongyu 07c60c85f9 lnwallet: remove direct walletdb reference
To make sure the db layer is not exposed and is only managed by the
`btcwallet` package.
2025-03-21 08:06:18 +08:00
yyforyongyu 6e1c098923 gomod: update btcwallet 2025-03-21 08:06:18 +08:00
yyforyongyu 16348e61db sqldb: add trace logs in ExecuteSQLTransactionWithRetry 2025-03-21 08:06:17 +08:00
yyforyongyu 3b7f9e1975 docs: update release notes 2025-03-21 03:25:24 +08:00
yyforyongyu 652d39dcc7 itest: fix flakeSkipPendingSweepsCheckDarwin
Now that we have the new RPC to assert the HTLC state, this flake should
be fixed.
2025-03-21 03:25:24 +08:00
yyforyongyu 5a72d5258f htlcswitch+itest: catch link quit signal when processing hodlQueue
This commit makes sure when processing resolutions, e.g, settling
invoices, when the link is already broken, the process would exit with
an error. This fixes the issue we found in the itest, where an
unexpected empty remote pending commitment was created although the
remote peer is already offline.
2025-03-21 03:25:24 +08:00
yyforyongyu 0892b595dd lnrpc+rpcserver: add new field LockedIn for HTLCs
In this commit, we add a new field `LockedIn` on HTLCs so it can be used
to decide whether an HTLC found on the local commitment has been
committed on the remote commitment.
2025-03-21 03:25:23 +08:00
yyforyongyu be4c4cc8ff multi: improve logging 2025-03-21 03:24:26 +08:00
yyforyongyu 66c94cf51f contractcourt: add String to CommitSet for logging 2025-03-21 03:24:26 +08:00
yyforyongyu c5f17180bc netann: remove unused param in newMockGraph 2025-03-21 03:24:26 +08:00
Yong 09b6745086 Merge pull request #9600 from lightningnetwork/yy-more-flakes
lntest+itest: document and fix more flakes
2025-03-21 03:23:02 +08:00
yyforyongyu 8b2ade7ec9 lnwallet+protofsm: fix logging 2025-03-18 11:44:20 -05:00
yyforyongyu 49382a8f67 gomod: update tor to v1.1.6 2025-03-18 20:24:34 +08:00
yyforyongyu 1db110f4cb itest: fix flake in testFundingExpiryBlocksOnPending
We need to make sure to wait for the RPC to show the updated results.
2025-03-18 20:24:34 +08:00
yyforyongyu d122a2ecc6 itest+lntest: add flakePaymentStreamReturnEarly
Fix the flake found in the `testRelayingBlindedError` and documents the
flake found in other tests in one place.
2025-03-18 20:24:34 +08:00
yyforyongyu e0296083ca itest+lntest: fix flake in coopCloseWithHTLCsWithRestart
Previously we'd restart Alice and then restart Bob, which means once
Alice is shut down and started again before we shut down Bob, Bob will
attempt to connect Alice since the connection is permanent, which could
put the node in a weird state. We now make sure both nodes are shut down
first, then bring them back online to avoid the above case. We may,
however, create another test in the future to check the above case if needed.
2025-03-18 20:24:34 +08:00
yyforyongyu a067aa45ef itest: add flakeInconsistentHTLCView 2025-03-18 20:24:33 +08:00
yyforyongyu e62aa7d43a docs: update release notes 2025-03-17 22:59:56 +08:00
yyforyongyu 2d6148291f cmd: fix error parsed from status
The `codes.Unimplemented` is only returned when the RPCs are not built.
When the wallet is in an unexpected state, `codes.Unknown` is returned
instead, so we need to catch it properly to make sure we return the
right error msg.
2025-03-17 22:59:56 +08:00
yyforyongyu faf8ce161a docs: update release notes 2025-03-10 17:03:00 +08:00
yyforyongyu d0abfbbaff itest: remove redundant resume action
Removed a redundant resume action found in
`testForwardInterceptorRestart`, which was put there likely due to a
mistake.
2025-03-10 16:58:16 +08:00
yyforyongyu 51daa13cd7 routerrpc: improve logging in forwardInterceptor 2025-03-10 16:58:16 +08:00
yyforyongyu 37799b95b7 discovery: fix state transition in GossipSyncer
Previously, we would set the state of the syncer after sending the msg,
which has the following flow,

1. In state `queryNewChannels`, we send the msg `QueryShortChanIDs`.
2. Once the msg is sent, we change to state `waitingQueryChanReply`.

But there's no guarantee the remote won't reply back inbetween the two
step. When that happens, our syncer would still be in state
`queryNewChannels`, causing the following error,
```
[ERR] DISC gossiper.go:873: Process query msg from peer [Alice] got unexpected msg *lnwire.ReplyShortChanIDsEnd received in state queryNewChannels
```

To fix it, we now make sure the state is updated before sending the msg.
2025-03-10 16:58:16 +08:00
yyforyongyu 4d05730c79 tor: fix msg order in TestReconnectSucceed 2025-03-10 16:58:16 +08:00
yyforyongyu 76ade177af chanbackup: fix test flake in TestUpdateAndSwap
To make it easier to debug, we break the old test into smaller ones and
fix a flake caused by uncleaned test dir.
2025-03-09 13:41:22 +08:00
yyforyongyu 95569df92e itest: document a flake found on macOS 2025-03-06 22:46:31 +08:00
yyforyongyu 0f8f092ddd itest: document a flake from TxNotifier 2025-03-06 09:11:30 +08:00
yyforyongyu 2f545717c9 itest: remove old TODOs
As they are fixed now.
2025-03-06 09:11:30 +08:00
yyforyongyu f349323923 itest: move test testDisconnectingTargetPeer
Hence finish an old TODO.
2025-03-06 09:11:30 +08:00
yyforyongyu 4755eff0a8 itest: remove time.Sleep before closing channels
Remove an old TODO, which has been fixed with the `NoWait` flag for coop
close.
2025-03-06 09:11:13 +08:00
yyforyongyu a116eef5eb tor: fix TestReconnectSucceed
We now make sure the proxy server is running on a unique port. In
addition, we close the old conn before making a new conn.
2025-03-05 19:09:15 +08:00
yyforyongyu ec7c36fd6a itest: document missing wallet UTXO 2025-03-05 18:54:13 +08:00
yyforyongyu 061b7abf76 itest: document the flake found in preimage extraction
We now use dedicated methods to properly document the flakes in *one*
place.
2025-03-05 18:54:13 +08:00
yyforyongyu 2d5a2ce78a lntest+itest: change the method signature of AssertPaymentStatus
So this can be used in other tests when we only care about the payment
hash.
2025-02-28 19:07:38 +08:00
yyforyongyu cfa4341740 routing: fix flake in TestFilteredChainView/bitcoind_polling 2025-02-26 19:51:53 +08:00
yyforyongyu 56fa3aae34 routing/chainview: refactor TestFilteredChainView
So each test has its own miner and chainView.
2025-02-26 19:51:53 +08:00
yyforyongyu 4bfcfea2ee lntest+chainntnfs: make sure bitcoind node is synced
This commit makes sure the bitcoind node is synced to the miner when
initialized and returned from `NewBitcoindBackend`.
2025-02-26 19:51:52 +08:00
yyforyongyu 99d49dec6a multi: change NewBitcoindBackend to take a miner as its param
Prepares for the following commit where we assert the chain backend is
synced to the miner.
2025-02-26 19:51:52 +08:00
yyforyongyu c725ba9f25 docs: update release notes re flake fix 2025-02-26 19:51:52 +08:00
yyforyongyu 1618d2c789 chainntnfs+lntest: fix sync to miner block flake
In this commit we document an unexpected behavior found when connecting
a bitcoind node to a btcd node. We mitigate this in our test by
reconnecting the nodes when the connection is broken. We also limit the
connection made from `bitcoind` to be v1 only.
2025-02-26 19:51:52 +08:00
yyforyongyu 175301628f lntest/unittest: make sure miner is connected to bitcoind
We change how the `bitcoind` node connects to the miner by creating a
temp RPC client and use it to connect to the miner. In addition we also
assert the connection is made.
2025-02-25 21:11:26 +08:00
yyforyongyu 245ea85894 lntest/unittest: assert bitcoind is shut down
Make sure the shutdown of `bitcoind` is finished without any errors.
2025-02-25 21:11:25 +08:00
yyforyongyu 7666d62a43 lntest/unittest: update config for miner and bitcoind
The config used for the miner is updated to skip banning and debug. For
bitcoind, the config is updated to use a unique port for P2P conn.
2025-02-25 21:10:40 +08:00
yyforyongyu fa8527af09 Makefile+scripts: add unit test flake hunter
This commit adds a script to hunt flakes for a specific unit test with
trace logs. Also rename the make commands to make them more clear on
whether it's a unit test, itest, or paralleled itest.
2025-02-25 21:10:40 +08:00
Yong fca9fae2d8 Merge pull request #9433 from hieblmi/estimate-route-fee-fix
routerrpc: fix estimateroutefee for public route hints
2025-02-25 13:58:43 +08:00
yyforyongyu 9f7e2bfd96 contractcourt: fix errorlint 2025-02-20 23:14:12 +08:00
yyforyongyu 7ab0e15937 sweep: fix error logging 2025-02-20 14:41:52 +08:00
yyforyongyu 353f208031 sweep: refactor IsOurTx to not return an error
Before this commit, the only error returned from `IsOurTx` is when the
root bucket was not created. In that case, we should consider the tx to
be not found in our db, since technically our db is empty.

A future PR may consider treating our wallet as the single source of
truth and query the wallet instead to check for past sweeping txns.
2025-02-20 14:41:52 +08:00
yyforyongyu 8d49246a54 docs: add release notes 2025-02-20 14:41:50 +08:00
yyforyongyu c61f781be7 itest: split up force close tests
So we can focus on testing normal flow vs persistence flow.
2025-02-20 14:40:54 +08:00
yyforyongyu 74161f0d57 sweep: make sure recovered inputs are retried
Previously, when a given input is found spent in the mempool, we'd mark
it as Published and never offer it to the fee bumper. This is dangerous
as the input will never be fee bumped. We now fix it by always
initializing the input with state Init, and only use mempool to check
for fee and fee rate.

This changes the current restart behavior - as previously when a
sweeping tx is broadcast, the node shuts down, when it starts again, the
input will be offered to the sweeper again, but not to the fee bumper,
which means the sweeping tx will stay in the mempool with the last-tried
fee rate. After this change, after a restart, the input will be swept
again, and the fee bumper will monitor its status. The restart will also
behave like a fee bump if there's already an existing sweeping tx in the
mempool.
2025-02-20 14:40:54 +08:00
yyforyongyu 4bd1a344b9 sweep: signal tx in markInputFatal
This commit adds the failed tx to the result when marking the input as
fatal, which is used in the commit resolver when handling revoked
outputs.
2025-02-20 14:40:54 +08:00
yyforyongyu b184afe227 sweep: handle missing inputs during fee bumping
This commit handles the case when the input is missing during the RBF
process, which could happen when the bumped tx has inputs being spent by
a third party. Normally we should be able to catch the spend early via
the spending notification and never attempt to fee bump the record.
However, due to the possible race between block notification and spend
notification, this cannot be guaranteed. Thus, we need to handle the
case during the RBF when seeing a `ErrMissingInputs`, which can only
happen when the inputs are spent by others.
2025-02-20 14:40:53 +08:00
yyforyongyu 4f469de18e sweep: refactor handleInitialTxError and createAndCheckTx
This commit refactors `handleInitialTxError` and `createAndCheckTx` to
take a `monitorRecord` param, which prepares for the following commit
where we start handling missing inputs.
2025-02-20 14:40:53 +08:00
yyforyongyu f614e7aed9 sweep: add createUnknownSpentBumpResult
A minor refactor to break the method `handleUnknownSpent` into two
steps, which prepares the following commit where we start handling
missing inputs.
2025-02-20 14:40:53 +08:00
yyforyongyu db8319d70b sweep: add method handleReplacementTxError
This is a minor refactor so the `createAndPublishTx` flow becomes more
clear, also prepares for the following commit where we start to handle
missing inputs.
2025-02-20 14:40:53 +08:00
yyforyongyu 42818949dc sweep: retry sweeping inputs upon TxUnknownSpend
We now start handling `TxUnknownSpend` in our sweeper to make sure the
failed inputs are retried when possible.
2025-02-20 14:40:53 +08:00
yyforyongyu 2f1205a394 sweep: start tracking inputs spent by unknown tx
This commit adds a new field `InputsSpent` to the `BumpResult` so they
can be used to track inputs spent by txns not recoginized by the fee
bumper.
2025-02-20 14:40:53 +08:00
yyforyongyu 388183e173 itest: add fee replacement test 2025-02-20 14:40:52 +08:00
yyforyongyu db351e1908 sweep: rename methods for clarity
We now rename "third party" to "unknown" as the inputs can be spent via
an older sweeping tx, a third party (anchor), or a remote party (pin).
In fee bumper we don't have the info to distinguish the above cases, and
leave them to be further handled by the sweeper as it has more context.
2025-02-20 14:40:52 +08:00
yyforyongyu 121116cff7 sweep: remove dead code and add better logging 2025-02-20 14:40:52 +08:00
yyforyongyu 50bc191feb sweep: handle unknown spent in processRecords
This commit refactors the `processRecords` to always handle the inputs
spent when processing the records. We now make sure to handle unknown
spends for all backends (previously only neutrino), and rely solely on
the spending notification to give us the onchain status of inputs.
2025-02-20 14:40:52 +08:00
yyforyongyu 61cec43951 sweep: add a new event TxUnknownSpend 2025-02-20 14:40:52 +08:00
yyforyongyu 8c9ba327cc sweep: add method getSpentInputs
To track the input and its spending tx, which will be used later to
detect unknown spends.
2025-02-20 14:40:52 +08:00
Yong 65a18c5b35 Merge pull request #9515 from ellemouton/graphFixFlake
graph: ensure topology subscriber handling and network msg handling is synchronous
2025-02-17 20:48:05 +08:00
yyforyongyu 2b0c25e31d workflows: skip coverage error in job finish
Also updates the coverage action used.
2025-02-13 22:05:38 +08:00
Yong 9c2c95d46f Merge pull request #9478 from ellemouton/graph3
discovery+graph:  move funding tx validation to the gossiper
2025-02-13 15:00:14 +08:00
yyforyongyu 759dc2066e docs: update release notes 2025-02-12 09:48:03 +08:00
yyforyongyu 59759f861f rpcserver: check blockbeatDispatcher when deciding isSynced
This commit changes `GetInfo` to include `blockbeatDispatcher`'s current
state when deciding whether the system is synced to chain. Previously we
check the best height against the wallet and the channel graph, we
should also do this to the blockbeat dispatcher to make sure the
internal consumers are also synced to the best block.
2025-02-12 09:48:02 +08:00
yyforyongyu 89c4a8dfd7 chainio: add method CurrentHeight
Add a new method `CurrentHeight` to query the current best height of the
dispatcher.
2025-02-12 09:48:02 +08:00
yyforyongyu e45e1f2b0e itest: fix flake in testForwardInterceptorRestart
We need to make sure the links are recreated before calling the
interceptor, otherwise we'd get the following error, causing the test to
fail.
```
2025-02-07 15:01:38.991 [ERR] RPCS interceptor.go:624: [/routerrpc.Router/HtlcInterceptor]: fwd (Chan ID=487:3:0, HTLC ID=0) not found
```
2025-02-10 12:24:23 +08:00
yyforyongyu 58e76b726e routing: add docs and monior refactor decideNextStep
Add verbose docs and refactor the method to exit early when `allow` is
true.
2025-02-10 11:36:18 +08:00
yyforyongyu ca10707b26 itest+lntest: assert payment is failed once the htlc times out 2025-02-10 11:31:51 +08:00
yyforyongyu faa3110127 docs: update release notes 2025-02-10 11:31:50 +08:00
yyforyongyu 5a62528fd7 routing: improve loggings for attempt result handling 2025-02-10 11:31:50 +08:00
yyforyongyu 1acf4d7d4d routing: always update payment in the same goroutine
This commit refactors `collectResultAsync` such that this method is now
only responsible for collecting results from the switch. The method
`decideNextStep` is expanded to process these results in the same
goroutine where we fetch the payment from db, to make sure the lifecycle
loop always have a consistent view of a given payment.
2025-02-10 11:29:29 +08:00
yyforyongyu e1279aab20 routing: add new method reloadPayment
To further shorten the lifecycle loop.
2025-02-10 11:28:23 +08:00
yyforyongyu 1fe2cdb765 routing: move amendFirstHopData into requestRoute 2025-02-10 11:28:23 +08:00
yyforyongyu 966cfccb94 routing: add new method reloadInflightAttempts
To shorten the method `resumePayment` and make each step more clear.
2025-02-10 11:28:23 +08:00
yyforyongyu f96eb50ca8 channeldb+routing: cache circuit and onion blob for htlc attempt
This commit caches the creation of sphinx circuit and onion blob to
avoid re-creating them again.
2025-02-10 11:28:23 +08:00
yyforyongyu 46eb811543 routing: remove redundant GetHash 2025-02-10 11:28:18 +08:00
Yong 5b1eaf9978 Merge pull request #9474 from ellemouton/nodeAnnConversion
graph/db: correctly handle de(ser)ialisation of `models.LightningNode` opaque addresses
2025-02-07 18:06:10 +08:00
Yong 457a245a4e Merge pull request #9482 from yyforyongyu/itest-log-ts
lntest: log timestamp when printing errors
2025-02-06 16:16:54 +08:00
yyforyongyu 5f4716c699 lntest: log timestamp when printing errors 2025-02-05 22:51:46 +08:00
Yong bac699df8f Merge pull request #9446 from yyforyongyu/yy-prepare-fee-replace
sweeper: rename `Failed` to `Fatal` and minor refactor
2025-02-05 22:48:41 +08:00
yyforyongyu b98542bd96 docs: update release notes 2025-02-05 19:49:09 +08:00
yyforyongyu e5f39dd644 sweep: refactor storeRecord to updateRecord
To make it clear we are only updating fields, which will be handy for
the following commit where we start tracking for spending notifications.
2025-02-05 19:49:09 +08:00
yyforyongyu 7eea7a7e9a sweep: add requestID to monitorRecord
This way we can greatly simplify the method signatures, also paving the
upcoming changes where we wanna make it clear when updating the
monitorRecord, we only touch a portion of it.
2025-02-05 19:49:04 +08:00
yyforyongyu bde5124e1b sweep: shorten storeRecord method signature
This commit shortens the function signature of `storeRecord`, also makes
sure we don't call `t.records.Store` directly but always using
`storeRecord` instead so it's easier to trace the record creation.
2025-02-05 19:48:18 +08:00
yyforyongyu c68b8e8c1e sweep: rename Failed to Fatal
This commit renames `Failed` to `Fatal` as it sounds too close to
`PublishFailed`. We also wanna emphasize that inputs in this state won't
be retried.
2025-02-05 19:48:18 +08:00
yyforyongyu bc8c1643c6 itest: move channel force closes into one sub test var 2025-01-26 09:16:02 +08:00
yyforyongyu e4b205cd90 itest: fix flake in testAnchorThirdPartySpend
When mining lots of blocks in the itest, the subsystems can be out of
sync in terms of the best block height. We now assert the num of pending
sweeps on Alice's node to give her more time to sync the blocks,
essentially behaving like a `time.Sleep` as in reality, the blocks would
never be generated this fast.
2025-01-26 09:15:36 +08:00