mirror/bitcoin
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge bitcoin/bitcoin#33629: Cluster mempool

Browse Source 
17cf9ff7ef Use cluster size limit for -maxmempool bound, and allow -maxmempool=0 in general (Suhas Daftuar)
315e43e5d8 Sanity check `GetFeerateDiagram()` in CTxMemPool::check() (Suhas Daftuar)
de2e9a24c4 test: extend package rbf functional test to larger clusters (Suhas Daftuar)
4ef4ddb504 doc: update policy/packages.md for new package acceptance logic (Suhas Daftuar)
79f73ad713 Add check that GetSortedScoreWithTopology() agrees with CompareMiningScoreWithTopology() (Suhas Daftuar)
a86ac11768 Update comments for CTxMemPool class (Suhas Daftuar)
9567eaa66d Invoke TxGraph::DoWork() at appropriate times (Suhas Daftuar)
6c5c44f774 test: add functional test for new cluster mempool RPCs (Suhas Daftuar)
72f60c877e doc: Update mempool_replacements.md to reflect feerate diagram checks (Suhas Daftuar)
21693f031a Expose cluster information via rpc (Suhas Daftuar)
72e74e0d42 fuzz: try to add more code coverage for mempool fuzzing (Suhas Daftuar)
f107417490 bench: add more mempool benchmarks (Suhas Daftuar)
7976eb1ae7 Avoid violating mempool policy limits in tests (Suhas Daftuar)
84de685cf7 Stop tracking parents/children outside of txgraph (Suhas Daftuar)
88672e205b Rewrite GatherClusters to use the txgraph implementation (Suhas Daftuar)
1ca4f01090 Fix miniminer_tests to work with cluster limits (Suhas Daftuar)
1902111e0f Eliminate CheckPackageLimits, which no longer does anything (Suhas Daftuar)
3a646ec462 Rework RBF and TRUC validation (Suhas Daftuar)
19b8479868 Make getting parents/children a function of the mempool, not a mempool entry (Suhas Daftuar)
5560913e51 Rework truc_policy to use descendants, not children (Suhas Daftuar)
a4458d6c40 Use txgraph to calculate descendants (Suhas Daftuar)
c8b6f70d64 Use txgraph to calculate ancestors (Suhas Daftuar)
241a3e666b Simplify ancestor calculation functions (Suhas Daftuar)
b9cec7f0a1 Make removeConflicts private (Suhas Daftuar)
0402e6c780 Remove unused limits from CalculateMemPoolAncestors (Suhas Daftuar)
08be765ac2 Remove mempool logic designed to maintain ancestor/descendant state (Suhas Daftuar)
fc4e3e6bc1 Remove unused members from CTxMemPoolEntry (Suhas Daftuar)
ff3b398d12 mempool: eliminate accessors to mempool entry ancestor/descendant cached state (Suhas Daftuar)
b9a2039f51 Eliminate use of cached ancestor data in miniminer_tests and truc_policy (Suhas Daftuar)
ba09fc9774 mempool: Remove unused function CalculateDescendantMaximum (Suhas Daftuar)
8e49477e86 wallet: Replace max descendant count with cluster_count (Suhas Daftuar)
e031085fd4 Eliminate Single-Conflict RBF Carve Out (Suhas Daftuar)
cf3ab8e1d0 Stop enforcing descendant size/count limits (Suhas Daftuar)
89ae38f489 test: remove rbf carveout test from mempool_limit.py (Suhas Daftuar)
c0bd04d18f Calculate descendant information for mempool RPC output on-the-fly (Suhas Daftuar)
bdcefb8a8b Use mempool/txgraph to determine if a tx has descendants (Suhas Daftuar)
69e1eaa6ed Add test case for cluster size limits to TRUC logic (Suhas Daftuar)
9cda64b86c Stop enforcing ancestor size/count limits (Suhas Daftuar)
1f93227a84 Remove dependency on cached ancestor data in mini-miner (Suhas Daftuar)
9fbe0a4ac2 rpc: Calculate ancestor data from scratch for mempool rpc calls (Suhas Daftuar)
7961496dda Reimplement GetTransactionAncestry() to not rely on cached data (Suhas Daftuar)
feceaa42e8 Remove CTxMemPool::GetSortedDepthAndScore (Suhas Daftuar)
21b5cea588 Use cluster linearization for transaction relay sort order (Suhas Daftuar)
6445aa7d97 Remove the ancestor and descendant indices from the mempool (Suhas Daftuar)
216e693729 Implement new RBF logic for cluster mempool (Suhas Daftuar)
ff8f115dec policy: Remove CPFP carveout rule (Suhas Daftuar)
c3f1afc934 test: rewrite PopulateMempool to not violate mempool policy (cluster size) limits (Suhas Daftuar)
47ab32fdb1 Select transactions for blocks based on chunk feerate (Suhas Daftuar)
dec138d1dd fuzz: remove comparison between mini_miner block construction and miner (Suhas Daftuar)
6c2bceb200 bench: rewrite ComplexMemPool to not create oversized clusters (Suhas Daftuar)
1ad4590f63 Limit mempool size based on chunk feerate (Suhas Daftuar)
b11c89cab2 Rework miner_tests to not require large cluster limit (Suhas Daftuar)
95a8297d48 Check cluster limits when using -walletrejectlongchains (Suhas Daftuar)
95762e6759 Do not allow mempool clusters to exceed configured limits (Suhas Daftuar)
edb3e7cdf6 [test] rework/delete feature_rbf tests requiring large clusters (glozow)
435fd56711 test: update feature_rbf.py replacement test (Suhas Daftuar)
34e32985e8 Add new (unused) limits for cluster size/count (Suhas Daftuar)
838d7e3553 Add transactions to txgraph, but without cluster dependencies (Suhas Daftuar)
d5ed9cb3eb Add accessor for sigops-adjusted weight (Suhas Daftuar)
1bf3b51396 Add sigops adjusted weight calculator (Suhas Daftuar)
c18c68a950 Create a txgraph inside CTxMemPool (Suhas Daftuar)
29a94d5b2f Make CTxMemPoolEntry derive from TxGraph::Ref (Suhas Daftuar)
92b0079fe3 Allow moving CTxMemPoolEntry objects, disallow copying (Suhas Daftuar)
6c73e47448 mempool: Store iterators into mapTx in mapNextTx (Suhas Daftuar)
51430680ec Allow moving an Epoch::Marker (Suhas Daftuar)

Pull request description:

  [Reopening #28676 here as a new PR, because GitHub is slow to load the page making it hard to scroll through and see comments.  Also, that PR was originally opened with a prototype implementation which has changed significantly with the introduction of `TxGraph`.]

  This is an implementation of the [cluster mempool proposal](https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393).

  This branch implements the following observable behavior changes:

   - Maintains a partitioning of the mempool into connected clusters (via the `txgraph` class), which are limited in vsize to 101 kvB by default, and limited in count to 64 by default.
   - Each cluster is sorted ("linearized") to try to optimize for selecting highest-feerate-subsets of a cluster first
   - Transaction selection for mining is updated to use the cluster linearizations, selecting highest feerate "chunks" first for inclusion in a block template.
   - Mempool eviction is updated to use the cluster linearizations, selecting lowest feerate "chunks" first for removal.
   - The RBF rules are updated to: (a) drop the requirement that no new inputs are introduced; (b) change the feerate requirement to instead check that the feerate diagram of the mempool will strictly improve; (c) replace the direct conflicts limit with a directly-conflicting-clusters limit.
   - The CPFP carveout rule is eliminated (it doesn't make sense in a cluster-limited mempool)
   - The ancestor and descendant limits are no longer enforced.
   - New cluster count/cluster vsize limits are now enforced instead.
   - Transaction relay now uses chunk feerate comparisons to determine the order that newly received transactions are announced to peers.

  Additionally, the cached ancestor and descendant data are dropped from the mempool, along with the multi_index indices that were maintained to sort the mempool by ancestor and descendant feerates. For compatibility (eg with wallet behavior or RPCs exposing this), this information is now calculated dynamically instead.

ACKs for top commit:
  instagibbs:
    reACK 17cf9ff7ef
  glozow:
    reACK 17cf9ff7ef
  sipa:
    ACK 17cf9ff7ef

Tree-SHA512: bbde46d913d56f8d9c0426cb0a6c4fa80b01b0a4c2299500769921f886082fb4f51f1694e0ee1bc318c52e1976d7ebed8134a64eda0b8044f3a708c04938eee7
This commit is contained in:
merge-script committed 2025-11-25 10:35:11 +00:00
2 parents 238c1c8933 17cf9ff7ef
commit fa283d28e2
61 files changed
+1864 -3220
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

test/functional/feature_dbcrash.py
-1
View File
@@ -53,7 +53,6 @@ class ChainstateWriteCrashTest(BitcoinTestFramework):
# Set -maxmempool=0 to turn off mempool memory sharing with dbcache
self.base_args = [
"-limitdescendantsize=0",
"-maxmempool=0",
"-dbbatchsize=200000",
]
test/functional/feature_rbf.py
+40 -134
View File
@@ -13,32 +13,25 @@ from test_framework.messages import (
from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import (
assert_equal,
assert_greater_than,
assert_greater_than_or_equal,
assert_raises_rpc_error,
get_fee,
)
from test_framework.wallet import MiniWallet
from test_framework.address import ADDRESS_BCRT1_UNSPENDABLE
from test_framework.mempool_util import DEFAULT_CLUSTER_LIMIT
MAX_REPLACEMENT_LIMIT = 100
class ReplaceByFeeTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 2
self.extra_args = [
[
"-limitancestorcount=50",
"-limitancestorsize=101",
"-limitdescendantcount=200",
"-limitdescendantsize=101",
],
# second node has default mempool parameters
[
],
]
self.uses_wallet = None
def run_test(self):
self.wallet = MiniWallet(self.nodes[0])
self.generate(self.nodes[0], 100)
self.log.info("Running test simple doublespend...")
self.test_simple_doublespend()
@@ -58,12 +51,12 @@ class ReplaceByFeeTest(BitcoinTestFramework):
self.log.info("Running test new unconfirmed inputs...")
self.test_new_unconfirmed_inputs()
self.log.info("Running test new unconfirmed input from low feerate tx...")
self.test_new_unconfirmed_input_with_low_feerate()
self.log.info("Running test too many replacements...")
self.test_too_many_replacements()
self.log.info("Running test too many replacements using default mempool params...")
self.test_too_many_replacements_with_default_mempool_params()
self.log.info("Running test RPC...")
self.test_rpc()
@@ -114,7 +107,7 @@ class ReplaceByFeeTest(BitcoinTestFramework):
# This will raise an exception due to insufficient fee
reject_reason = "insufficient fee"
reject_details = f"{reject_reason}, rejecting replacement {tx.txid_hex}; new feerate 0.00300000 BTC/kvB <= old feerate 0.00300000 BTC/kvB"
reject_details = f"{reject_reason}, rejecting replacement {tx.txid_hex}, not enough additional fees to relay; 0.00 < 0.00000011"
res = self.nodes[0].testmempoolaccept(rawtxs=[tx_hex])[0]
assert_equal(res["reject-reason"], reject_reason)
assert_equal(res["reject-details"], reject_details)
@@ -143,7 +136,9 @@ class ReplaceByFeeTest(BitcoinTestFramework):
prevout = tx0_outpoint
remaining_value = initial_nValue
chain_txids = []
while remaining_value > 1 * COIN:
for _ in range(DEFAULT_CLUSTER_LIMIT):
if remaining_value <= 1 * COIN:
break
remaining_value -= int(0.1 * COIN)
prevout = self.wallet.send_self_transfer(
from_node=self.nodes[0],
@@ -216,7 +211,7 @@ class ReplaceByFeeTest(BitcoinTestFramework):
yield x
fee = int(0.00001 * COIN)
n = MAX_REPLACEMENT_LIMIT
n = DEFAULT_CLUSTER_LIMIT
tree_txs = list(branch(tx0_outpoint, initial_nValue, n, fee=fee))
assert_equal(len(tree_txs), n)
@@ -242,25 +237,6 @@ class ReplaceByFeeTest(BitcoinTestFramework):
for txid in tree_txs:
assert txid not in mempool
# Try again, but with more total transactions than the "max txs
# double-spent at once" anti-DoS limit.
for n in (MAX_REPLACEMENT_LIMIT + 1, MAX_REPLACEMENT_LIMIT * 2):
fee = int(0.00001 * COIN)
tx0_outpoint = self.make_utxo(self.nodes[0], initial_nValue)
tree_txs = list(branch(tx0_outpoint, initial_nValue, n, fee=fee))
assert_equal(len(tree_txs), n)
dbl_tx_hex = self.wallet.create_self_transfer(
utxo_to_spend=tx0_outpoint,
sequence=0,
fee=2 * (Decimal(fee) / COIN) * n,
)["hex"]
# This will raise an exception
assert_raises_rpc_error(-26, "too many potential replacements", self.nodes[0].sendrawtransaction, dbl_tx_hex, 0)
for txid in tree_txs:
self.nodes[0].getrawtransaction(txid)
def test_replacement_feeperkb(self):
"""Replacement requires fee-per-KB to be higher"""
tx0_outpoint = self.make_utxo(self.nodes[0], int(1.1 * COIN))
@@ -282,7 +258,7 @@ class ReplaceByFeeTest(BitcoinTestFramework):
)["hex"]
# This will raise an exception due to insufficient fee
assert_raises_rpc_error(-26, "insufficient fee", self.nodes[0].sendrawtransaction, tx1b_hex, 0)
assert_raises_rpc_error(-26, "does not improve feerate diagram", self.nodes[0].sendrawtransaction, tx1b_hex, 0)
def test_spends_of_conflicting_outputs(self):
"""Replacements that spend conflicting tx outputs are rejected"""
@@ -332,9 +308,9 @@ class ReplaceByFeeTest(BitcoinTestFramework):
assert_raises_rpc_error(-26, "bad-txns-spends-conflicting-tx", self.nodes[0].sendrawtransaction, tx2_hex, 0)
def test_new_unconfirmed_inputs(self):
"""Replacements that add new unconfirmed inputs are rejected"""
"""Replacements that add new unconfirmed inputs may be accepted"""
confirmed_utxo = self.make_utxo(self.nodes[0], int(1.1 * COIN))
unconfirmed_utxo = self.make_utxo(self.nodes[0], int(0.1 * COIN), confirmed=False)
unconfirmed_utxo = self.make_utxo(self.nodes[0], int(0.2 * COIN), confirmed=False)
self.wallet.send_self_transfer(
from_node=self.nodes[0],
@@ -350,19 +326,30 @@ class ReplaceByFeeTest(BitcoinTestFramework):
)["tx"]
tx2_hex = tx2.serialize().hex()
# This will raise an exception
reject_reason = "replacement-adds-unconfirmed"
reject_details = f"{reject_reason}, replacement {tx2.txid_hex} adds unconfirmed input, idx 1"
res = self.nodes[0].testmempoolaccept(rawtxs=[tx2_hex])[0]
assert_equal(res["reject-reason"], reject_reason)
assert_equal(res["reject-details"], reject_details)
assert_raises_rpc_error(-26, f"{reject_details}", self.nodes[0].sendrawtransaction, tx2_hex, 0)
# This will not raise an exception
tx2_id = self.nodes[0].sendrawtransaction(tx2_hex, 0)
assert tx2_id in self.nodes[0].getrawmempool()
def test_new_unconfirmed_input_with_low_feerate(self):
"""Replacements that add new unconfirmed inputs are allowed, but must pass the feerate diagram check"""
confirmed_utxos = [self.make_utxo(self.nodes[0], int(1.1 * COIN)) for _ in range(3)]
large_low_feerate = self.wallet.create_self_transfer(utxo_to_spend=confirmed_utxos[0], target_vsize=10000, fee=Decimal("0.00001000"))
self.nodes[0].sendrawtransaction(large_low_feerate['hex'])
unconfirmed_utxo = large_low_feerate['new_utxo']
# These two transactions are approximately the same size. The replacement tx pays twice the fee.
tx_to_replace = self.wallet.create_self_transfer_multi(utxos_to_spend=[confirmed_utxos[1], confirmed_utxos[2]], fee_per_output=2000)
tx_replacement = self.wallet.create_self_transfer_multi(utxos_to_spend=[confirmed_utxos[1], unconfirmed_utxo], fee_per_output=4000)
assert_greater_than(tx_replacement['fee']*tx_to_replace['tx'].get_vsize(), tx_to_replace['fee']*tx_replacement['tx'].get_vsize())
self.nodes[0].sendrawtransaction(tx_to_replace['hex'])
assert_raises_rpc_error(-26, "insufficient feerate: does not improve feerate diagram", self.nodes[0].sendrawtransaction, tx_replacement['hex'])
def test_too_many_replacements(self):
"""Replacements that evict too many transactions are rejected"""
# Try directly replacing more than MAX_REPLACEMENT_LIMIT
# transactions
"""Replacements that conflict with too many clusters are rejected"""
# Try directly replacing transactions in more than MAX_REPLACEMENT_LIMIT
# distinct clusters
# Start by creating a single transaction with many outputs
initial_nValue = 10 * COIN
@@ -378,6 +365,8 @@ class ReplaceByFeeTest(BitcoinTestFramework):
amount_per_output=split_value,
)["new_utxos"]
self.generate(self.nodes[0], 1)
# Now spend each of those outputs individually
for utxo in splitting_tx_utxos:
self.wallet.send_self_transfer(
@@ -400,7 +389,7 @@ class ReplaceByFeeTest(BitcoinTestFramework):
# This will raise an exception
reject_reason = "too many potential replacements"
reject_details = f"{reject_reason}, rejecting replacement {double_tx.txid_hex}; too many potential replacements ({MAX_REPLACEMENT_LIMIT + 1} > {MAX_REPLACEMENT_LIMIT})"
reject_details = f"{reject_reason}, rejecting replacement {double_tx.txid_hex}; too many conflicting clusters ({MAX_REPLACEMENT_LIMIT + 1} > {MAX_REPLACEMENT_LIMIT})"
res = self.nodes[0].testmempoolaccept(rawtxs=[double_tx_hex])[0]
assert_equal(res["reject-reason"], reject_reason)
assert_equal(res["reject-details"], reject_details)
@@ -412,89 +401,6 @@ class ReplaceByFeeTest(BitcoinTestFramework):
double_tx_hex = double_tx.serialize().hex()
self.nodes[0].sendrawtransaction(double_tx_hex, 0)
def test_too_many_replacements_with_default_mempool_params(self):
"""
Test rule 5 (do not allow replacements that cause more than 100
evictions) without having to rely on non-default mempool parameters.
In order to do this, create a number of "root" UTXOs, and then hang
enough transactions off of each root UTXO to exceed the MAX_REPLACEMENT_LIMIT.
Then create a conflicting RBF replacement transaction.
"""
# Clear mempools to avoid cross-node sync failure.
for node in self.nodes:
self.generate(node, 1)
normal_node = self.nodes[1]
wallet = MiniWallet(normal_node)
# This has to be chosen so that the total number of transactions can exceed
# MAX_REPLACEMENT_LIMIT without having any one tx graph run into the descendant
# limit; 10 works.
num_tx_graphs = 10
# (Number of transactions per graph, rule 5 failure expected)
cases = [
# Test the base case of evicting fewer than MAX_REPLACEMENT_LIMIT
# transactions.
((MAX_REPLACEMENT_LIMIT // num_tx_graphs) - 1, False),
# Test hitting the rule 5 eviction limit.
(MAX_REPLACEMENT_LIMIT // num_tx_graphs, True),
]
for (txs_per_graph, failure_expected) in cases:
self.log.debug(f"txs_per_graph: {txs_per_graph}, failure: {failure_expected}")
# "Root" utxos of each txn graph that we will attempt to double-spend with
# an RBF replacement.
root_utxos = []
# For each root UTXO, create a package that contains the spend of that
# UTXO and `txs_per_graph` children tx.
for graph_num in range(num_tx_graphs):
root_utxos.append(wallet.get_utxo())
parent_tx = wallet.send_self_transfer_multi(
from_node=normal_node,
utxos_to_spend=[root_utxos[graph_num]],
num_outputs=txs_per_graph,
)
new_utxos = parent_tx['new_utxos']
for utxo in new_utxos:
# Create spends for each output from the "root" of this graph.
child_tx = wallet.send_self_transfer(
from_node=normal_node,
utxo_to_spend=utxo,
)
assert normal_node.getmempoolentry(child_tx['txid'])
num_txs_invalidated = len(root_utxos) + (num_tx_graphs * txs_per_graph)
if failure_expected:
assert num_txs_invalidated > MAX_REPLACEMENT_LIMIT
else:
assert num_txs_invalidated <= MAX_REPLACEMENT_LIMIT
# Now attempt to submit a tx that double-spends all the root tx inputs, which
# would invalidate `num_txs_invalidated` transactions.
tx_hex = wallet.create_self_transfer_multi(
utxos_to_spend=root_utxos,
fee_per_output=10_000_000, # absurdly high feerate
)["hex"]
if failure_expected:
assert_raises_rpc_error(
-26, "too many potential replacements", normal_node.sendrawtransaction, tx_hex, 0)
else:
txid = normal_node.sendrawtransaction(tx_hex, 0)
assert normal_node.getmempoolentry(txid)
# Clear the mempool once finished, and rescan the other nodes' wallet
# to account for the spends we've made on `normal_node`.
self.generate(normal_node, 1)
self.wallet.rescan_utxos()
def test_prioritised_transactions(self):
# Ensure that fee deltas used via prioritisetransaction are
# correctly used by replacement logic
@@ -518,7 +424,7 @@ class ReplaceByFeeTest(BitcoinTestFramework):
)["hex"]
# Verify tx1b cannot replace tx1a.
assert_raises_rpc_error(-26, "insufficient fee", self.nodes[0].sendrawtransaction, tx1b_hex, 0)
assert_raises_rpc_error(-26, "does not improve feerate diagram", self.nodes[0].sendrawtransaction, tx1b_hex, 0)
# Use prioritisetransaction to set tx1a's fee to 0.
self.nodes[0].prioritisetransaction(txid=tx1a_txid, fee_delta=int(-0.1 * COIN))
test/functional/mempool_cluster.py
Executable
+327
View File
@@ -0,0 +1,327 @@
#!/usr/bin/env python3
# Copyright (c) 2024 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test cluster mempool accessors and limits"""
from decimal import Decimal
from test_framework.mempool_util import (
DEFAULT_CLUSTER_LIMIT,
DEFAULT_CLUSTER_SIZE_LIMIT_KVB,
)
from test_framework.messages import (
COIN,
)
from test_framework.test_framework import BitcoinTestFramework
from test_framework.wallet import (
MiniWallet,
)
from test_framework.util import (
assert_equal,
assert_greater_than,
assert_greater_than_or_equal,
assert_raises_rpc_error,
)
def weight_to_vsize(weight):
# Divide by 4, round up
return (weight + 3) // 4
def cleanup(func):
def wrapper(self, *args, **kwargs):
try:
func(self, *args, **kwargs)
finally:
# Mine blocks to clear the mempool and replenish the wallet's confirmed UTXOs.
while (len(self.nodes[0].getrawmempool()) > 0):
self.generate(self.nodes[0], 1)
self.wallet.rescan_utxos(include_mempool=True)
return wrapper
class MempoolClusterTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 1
def add_chain_cluster(self, node, cluster_count, target_vsize=None):
"""Create a cluster of transactions, with the count specified.
The topology is a chain: the i'th transaction depends on the (i-1)'th transaction.
Optionally provide a target_vsize for each transaction.
"""
parent_tx = self.wallet.send_self_transfer(from_node=node, confirmed_only=True, target_vsize=target_vsize)
utxo_to_spend = parent_tx["new_utxo"]
all_txids = [parent_tx["txid"]]
all_results = [parent_tx]
while len(all_results) < cluster_count:
next_tx = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=utxo_to_spend, target_vsize=target_vsize)
assert next_tx["txid"] in node.getrawmempool()
# Confirm that each transaction is in the same cluster as the first.
assert_equal(node.getmempoolcluster(next_tx['txid']), node.getmempoolcluster(parent_tx['txid']))
# Confirm that the ancestors are what we expect
mempool_ancestors = node.getmempoolancestors(next_tx['txid'])
assert_equal(sorted(mempool_ancestors), sorted(all_txids))
# Confirm that each successive transaction is added as a descendant.
assert all([ next_tx["txid"] in node.getmempooldescendants(x) for x in all_txids ])
# Update for next iteration
all_results.append(next_tx)
all_txids.append(next_tx["txid"])
utxo_to_spend = next_tx["new_utxo"]
assert node.getmempoolcluster(parent_tx['txid'])['txcount'] == cluster_count
return all_results
def check_feerate_diagram(self, node):
"""Sanity check the feerate diagram."""
feeratediagram = node.getmempoolfeeratediagram()
last_val = {"weight": 0, "fee": 0}
for x in feeratediagram:
# The weight is always positive, except for the first iteration
assert x['weight'] > 0 or x['fee'] == 0
# Monotonically decreasing fee per weight
assert_greater_than_or_equal(last_val['fee'] * x['weight'], x['fee'] * last_val['weight'])
last_val = x
def test_limit_enforcement(self, cluster_submitted, target_vsize_per_tx=None):
"""
the cluster may change as a result of these transactions, so cluster_submitted is mutated accordingly
"""
# Cluster has already been submitted and has at least 3 transactions, otherwise this test won't work.
assert_greater_than_or_equal(len(cluster_submitted), 3)
node = self.nodes[0]
last_result = cluster_submitted[-1]
# Test that adding one more transaction to the cluster will fail.
bad_tx = self.wallet.create_self_transfer(utxo_to_spend=last_result["new_utxo"], target_vsize=target_vsize_per_tx)
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, bad_tx["hex"])
# It should also limit cluster sizes during replacement
utxo_to_double_spend = self.wallet.get_utxo(confirmed_only=True)
fee = Decimal("0.000001")
tx_to_replace = self.wallet.create_self_transfer(utxo_to_spend=utxo_to_double_spend, fee=fee)
node.sendrawtransaction(tx_to_replace["hex"])
# Multiply fee by 5, which should easily cover the cost to replace (but
# is still too large a cluster). Otherwise, use the target vsize at
# 10sat/vB
fee_to_use = target_vsize_per_tx * 10 if target_vsize_per_tx is not None else int(fee * COIN * 5)
bad_tx_also_replacement = self.wallet.create_self_transfer_multi(
utxos_to_spend=[last_result["new_utxo"], utxo_to_double_spend],
target_vsize=target_vsize_per_tx,
fee_per_output=fee_to_use,
)
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, bad_tx_also_replacement["hex"])
# Replace the last transaction. We are extending the cluster by one, but also removing one: 64 + 1 - 1 = 64
# In the case of vsize, it should similarly cancel out.
second_to_last_utxo = cluster_submitted[-2]["new_utxo"]
fee_to_beat = cluster_submitted[-1]["fee"]
vsize_to_use = cluster_submitted[-1]["tx"].get_vsize() if target_vsize_per_tx is not None else None
good_tx_replacement = self.wallet.create_self_transfer(utxo_to_spend=second_to_last_utxo, fee=fee_to_beat * 5, target_vsize=vsize_to_use)
node.sendrawtransaction(good_tx_replacement["hex"], maxfeerate=0)
cluster_submitted[-1] = good_tx_replacement
def test_limit_enforcement_package(self, cluster_submitted):
node = self.nodes[0]
# Create a package from the second to last transaction. This shouldn't work because the effect is 64 + 2 - 1 = 65
last_utxo = cluster_submitted[-2]["new_utxo"]
fee_to_beat = cluster_submitted[-1]["fee"]
# We do not use package RBF here because it has additional restrictions on mempool ancestors.
parent_tx_bad = self.wallet.create_self_transfer(utxo_to_spend=last_utxo, fee=fee_to_beat * 5)
child_tx_bad = self.wallet.create_self_transfer(utxo_to_spend=parent_tx_bad["new_utxo"])
# The parent should be submitted, but the child rejected.
result_parent_only = node.submitpackage([parent_tx_bad["hex"], child_tx_bad["hex"]])
assert parent_tx_bad["txid"] in node.getrawmempool()
assert child_tx_bad["txid"] not in node.getrawmempool()
assert_equal(result_parent_only["package_msg"], "transaction failed")
assert_equal(result_parent_only["tx-results"][child_tx_bad["wtxid"]]["error"], "too-large-cluster")
# Now, create a package from the second to last transaction. This should work because the effect is 64 + 2 - 2 = 64
third_to_last_utxo = cluster_submitted[-3]["new_utxo"]
parent_tx_good = self.wallet.create_self_transfer(utxo_to_spend=third_to_last_utxo)
child_tx_good = self.wallet.create_self_transfer(utxo_to_spend=parent_tx_good["new_utxo"], fee=fee_to_beat * 5)
result_both_good = node.submitpackage([parent_tx_good["hex"], child_tx_good["hex"]], maxfeerate=0)
assert_equal(result_both_good["package_msg"], "success")
assert parent_tx_good["txid"] in node.getrawmempool()
assert child_tx_good["txid"] in node.getrawmempool()
@cleanup
def test_cluster_count_limit(self, max_cluster_count):
node = self.nodes[0]
cluster_submitted = self.add_chain_cluster(node, max_cluster_count)
self.check_feerate_diagram(node)
for result in cluster_submitted:
assert_equal(node.getmempoolcluster(result["txid"])['txcount'], max_cluster_count)
self.log.info("Test that cluster count limit is enforced")
self.test_limit_enforcement(cluster_submitted)
self.log.info("Test that the resulting cluster count is correctly calculated in a package")
self.test_limit_enforcement_package(cluster_submitted)
@cleanup
def test_cluster_size_limit(self, max_cluster_size_vbytes):
node = self.nodes[0]
# This number should be smaller than the cluster count limit.
num_txns = 10
# Leave some buffer so it is possible to add a reasonably-sized transaction.
target_vsize_per_tx = int((max_cluster_size_vbytes - 500) / num_txns)
cluster_submitted = self.add_chain_cluster(node, num_txns, target_vsize_per_tx)
vsize_remaining = max_cluster_size_vbytes - weight_to_vsize(node.getmempoolcluster(cluster_submitted[0]["txid"])['weight'])
self.log.info("Test that cluster size limit is enforced")
self.test_limit_enforcement(cluster_submitted, target_vsize_per_tx=vsize_remaining + 4)
# Try another cluster and add a small transaction: it should succeed
last_result = cluster_submitted[-1]
small_tx = self.wallet.create_self_transfer(utxo_to_spend=last_result["new_utxo"], target_vsize=vsize_remaining)
node.sendrawtransaction(small_tx["hex"])
@cleanup
def test_cluster_merging(self, max_cluster_count):
node = self.nodes[0]
self.log.info(f"Test merging 2 clusters with transaction counts totaling {max_cluster_count}")
for num_txns_cluster1 in [1, 5, 10]:
# Create a chain of transactions
cluster1 = self.add_chain_cluster(node, num_txns_cluster1)
for result in cluster1:
node.sendrawtransaction(result["hex"])
utxo_from_cluster1 = cluster1[-1]["new_utxo"]
# Make the next cluster, which contains the remaining transactions
assert_greater_than(max_cluster_count, num_txns_cluster1)
num_txns_cluster2 = max_cluster_count - num_txns_cluster1
cluster2 = self.add_chain_cluster(node, num_txns_cluster2)
for result in cluster2:
node.sendrawtransaction(result["hex"])
utxo_from_cluster2 = cluster2[-1]["new_utxo"]
# Now create a transaction that spends from both clusters, which would merge them.
tx_merger = self.wallet.create_self_transfer_multi(utxos_to_spend=[utxo_from_cluster1, utxo_from_cluster2])
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, tx_merger["hex"])
# Spending from the clusters independently should work
tx_spending_cluster1 = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=utxo_from_cluster1)
tx_spending_cluster2 = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=utxo_from_cluster2)
assert tx_spending_cluster1["txid"] in node.getrawmempool()
assert tx_spending_cluster2["txid"] in node.getrawmempool()
self.log.info(f"Test merging {max_cluster_count} clusters with 1 transaction spending from all of them")
utxos_to_merge = []
for _ in range(max_cluster_count):
# Use a confirmed utxo to ensure distinct clusters
confirmed_utxo = self.wallet.get_utxo(confirmed_only=True)
singleton = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=confirmed_utxo)
assert singleton["txid"] in node.getrawmempool()
utxos_to_merge.append(singleton["new_utxo"])
assert_equal(len(utxos_to_merge), max_cluster_count)
tx_merger = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_to_merge)
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, tx_merger["hex"])
# Spending from 1 fewer cluster should work
tx_merger_all_but_one = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_to_merge[:-1])
node.sendrawtransaction(tx_merger_all_but_one["hex"])
assert tx_merger_all_but_one["txid"] in node.getrawmempool()
@cleanup
def test_cluster_merging_size(self, max_cluster_size_vbytes):
node = self.nodes[0]
self.log.info(f"Test merging clusters with sizes totaling {max_cluster_size_vbytes} vB")
num_txns = 10
# Leave some buffer so it is possible to add a reasonably-sized transaction.
utxos_to_merge = []
vsize_remaining = max_cluster_size_vbytes
for _ in range(num_txns):
confirmed_utxo = self.wallet.get_utxo(confirmed_only=True)
singleton = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=confirmed_utxo)
assert singleton["txid"] in node.getrawmempool()
utxos_to_merge.append(singleton["new_utxo"])
vsize_remaining -= singleton["tx"].get_vsize()
assert_greater_than_or_equal(vsize_remaining, 500)
# Create a transaction spending from all clusters that exceeds the cluster size limit.
tx_merger_too_big = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_to_merge, target_vsize=vsize_remaining + 4, fee_per_output=10000)
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, tx_merger_too_big["hex"])
# A transaction that is slightly smaller should work.
tx_merger_small = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_to_merge[:-1], target_vsize=vsize_remaining - 4, fee_per_output=10000)
node.sendrawtransaction(tx_merger_small["hex"])
assert tx_merger_small["txid"] in node.getrawmempool()
@cleanup
def test_cluster_limit_rbf(self, max_cluster_count):
node = self.nodes[0]
# Use min feerate for the to-be-replaced transactions. There are many, so replacement cost can be expensive.
min_feerate = node.getmempoolinfo()["mempoolminfee"]
self.log.info("Test that cluster size calculation takes RBF into account")
utxos_created_by_parents = []
fees_rbf_sats = 0
for _ in range(max_cluster_count - 1):
parent_tx = self.wallet.send_self_transfer(from_node=node, confirmed_only=True)
utxo_to_replace = parent_tx["new_utxo"]
child_tx = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=utxo_to_replace, fee_rate=min_feerate)
fees_rbf_sats += int(child_tx["fee"] * COIN)
utxos_created_by_parents.append(utxo_to_replace)
# This transaction would create a cluster of size max_cluster_count
# Importantly, the node should account for the fact that half of the transactions will be replaced.
tx_merger_replacer = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_created_by_parents, fee_per_output=fees_rbf_sats * 2)
node.sendrawtransaction(tx_merger_replacer["hex"])
assert tx_merger_replacer["txid"] in node.getrawmempool()
assert_equal(node.getmempoolcluster(tx_merger_replacer["txid"])['txcount'], max_cluster_count)
self.log.info("Test that cluster size calculation takes package RBF into account")
utxos_to_replace = []
fee_rbf_decimal = 0
for _ in range(max_cluster_count):
confirmed_utxo = self.wallet.get_utxo(confirmed_only=True)
tx_to_replace = self.wallet.send_self_transfer(from_node=node, utxo_to_spend=confirmed_utxo, fee_rate=min_feerate)
fee_rbf_decimal += tx_to_replace["fee"]
utxos_to_replace.append(confirmed_utxo)
tx_replacer = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_to_replace)
tx_replacer_sponsor = self.wallet.create_self_transfer(utxo_to_spend=tx_replacer["new_utxos"][0], fee=fee_rbf_decimal * 2)
node.submitpackage([tx_replacer["hex"], tx_replacer_sponsor["hex"]], maxfeerate=0)
assert tx_replacer["txid"] in node.getrawmempool()
assert tx_replacer_sponsor["txid"] in node.getrawmempool()
assert_equal(node.getmempoolcluster(tx_replacer["txid"])['txcount'], 2)
def run_test(self):
node = self.nodes[0]
self.wallet = MiniWallet(node)
self.generate(self.wallet, 400)
self.test_cluster_limit_rbf(DEFAULT_CLUSTER_LIMIT)
for cluster_size_limit_kvb in [10, 20, 33, 100, DEFAULT_CLUSTER_SIZE_LIMIT_KVB]:
self.log.info(f"-> Resetting node with -limitclustersize={cluster_size_limit_kvb}")
self.restart_node(0, extra_args=[f"-limitclustersize={cluster_size_limit_kvb}"])
cluster_size_limit = cluster_size_limit_kvb * 1000
self.test_cluster_size_limit(cluster_size_limit)
self.test_cluster_merging_size(cluster_size_limit)
for cluster_count_limit in [4, 10, 16, 32, DEFAULT_CLUSTER_LIMIT]:
self.log.info(f"-> Resetting node with -limitclustercount={cluster_count_limit}")
self.restart_node(0, extra_args=[f"-limitclustercount={cluster_count_limit}"])
self.test_cluster_count_limit(cluster_count_limit)
if cluster_count_limit > 10:
self.test_cluster_merging(cluster_count_limit)
if __name__ == '__main__':
MempoolClusterTest(__file__).main()
test/functional/mempool_limit.py
-52
View File
@@ -32,57 +32,6 @@ class MempoolLimitTest(BitcoinTestFramework):
"-maxmempool=5",
]]
def test_rbf_carveout_disallowed(self):
node = self.nodes[0]
self.log.info("Check that individually-evaluated transactions in a package don't increase package limits for other subpackage parts")
# We set chain limits to 2 ancestors, 1 descendant, then try to get a parents-and-child chain of 2 in mempool
#
# A: Solo transaction to be RBF'd (to bump descendant limit for package later)
# B: First transaction in package, RBFs A by itself under individual evaluation, which would give it +1 descendant limit
# C: Second transaction in package, spends B. If the +1 descendant limit persisted, would make it into mempool
self.restart_node(0, extra_args=self.extra_args[0] + ["-limitancestorcount=2", "-limitdescendantcount=1"])
# Generate a confirmed utxo we will double-spend
rbf_utxo = self.wallet.send_self_transfer(
from_node=node,
confirmed_only=True
)["new_utxo"]
self.generate(node, 1)
# tx_A needs to be RBF'd, set minfee at set size
A_vsize = 250
mempoolmin_feerate = node.getmempoolinfo()["mempoolminfee"]
tx_A = self.wallet.send_self_transfer(
from_node=node,
fee_rate=mempoolmin_feerate,
target_vsize=A_vsize,
utxo_to_spend=rbf_utxo,
confirmed_only=True
)
# RBF's tx_A, is not yet submitted
tx_B = self.wallet.create_self_transfer(
fee=tx_A["fee"] * 4,
target_vsize=A_vsize,
utxo_to_spend=rbf_utxo,
confirmed_only=True
)
# Spends tx_B's output, too big for cpfp carveout (because that would also increase the descendant limit by 1)
non_cpfp_carveout_vsize = 10001 # EXTRA_DESCENDANT_TX_SIZE_LIMIT + 1
tx_C = self.wallet.create_self_transfer(
target_vsize=non_cpfp_carveout_vsize,
fee_rate=mempoolmin_feerate,
utxo_to_spend=tx_B["new_utxo"],
confirmed_only=True
)
res = node.submitpackage([tx_B["hex"], tx_C["hex"]])
assert_equal(res["package_msg"], "transaction failed")
assert "too-long-mempool-chain" in res["tx-results"][tx_C["wtxid"]]["error"]
def test_mid_package_eviction_success(self):
node = self.nodes[0]
self.log.info("Check a package where each parent passes the current mempoolminfee but a parent could be evicted before getting child's descendant feerate")
@@ -336,7 +285,6 @@ class MempoolLimitTest(BitcoinTestFramework):
self.test_mid_package_eviction_success()
self.test_mid_package_replacement()
self.test_rbf_carveout_disallowed()
if __name__ == '__main__':
test/functional/mempool_package_limits.py
+3 -144
View File
@@ -15,7 +15,7 @@ from test_framework.wallet import MiniWallet
# 2) run the subtest, which may submit some transaction(s) to the mempool and
# create a list of hex transactions
# 3) testmempoolaccept the package hex and check that it fails with the error
# "package-mempool-limits" for each tx
# "too-large-cluster" for each tx
# 4) after mining a block, clearing the pre-submitted transactions from mempool,
# check that submitting the created package succeeds
def check_package_limits(func):
@@ -26,7 +26,7 @@ def check_package_limits(func):
testres_error_expected = node.testmempoolaccept(rawtxs=package_hex)
assert_equal(len(testres_error_expected), len(package_hex))
for txres in testres_error_expected:
assert "package-mempool-limits" in txres["package-error"]
assert "too-large-cluster" in txres["package-error"]
# Clear mempool and check that the package passes now
self.generate(node, 1)
@@ -39,6 +39,7 @@ class MempoolPackageLimitsTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 1
self.setup_clean_chain = True
self.extra_args = [["-limitclustercount=25"]]
def run_test(self):
self.wallet = MiniWallet(self.nodes[0])
@@ -48,10 +49,6 @@ class MempoolPackageLimitsTest(BitcoinTestFramework):
self.test_chain_limits()
self.test_desc_count_limits()
self.test_desc_count_limits_2()
self.test_anc_count_limits()
self.test_anc_count_limits_2()
self.test_anc_count_limits_bushy()
self.test_anc_size_limits()
self.test_desc_size_limits()
@check_package_limits
@@ -162,144 +159,6 @@ class MempoolPackageLimitsTest(BitcoinTestFramework):
assert_equal(2, len(package_hex))
return package_hex
@check_package_limits
def test_anc_count_limits(self):
"""Create a 'V' shaped chain with 24 transactions in the mempool and 3 in the package:
M1a M1b
^ ^
M2a M2b
. .
. .
. .
M12a M12b
^ ^
Pa Pb
^ ^
Pc
The lowest descendant, Pc, exceeds ancestor limits, but only if the in-mempool
and in-package ancestors are all considered together.
"""
node = self.nodes[0]
package_hex = []
pc_parent_utxos = []
self.log.info("Check that in-mempool and in-package ancestors are calculated properly in packages")
# Two chains of 13 transactions each
for _ in range(2):
chain_tip_utxo = self.wallet.send_self_transfer_chain(from_node=node, chain_length=12)[-1]["new_utxo"]
# Save the 13th transaction for the package
tx = self.wallet.create_self_transfer(utxo_to_spend=chain_tip_utxo)
package_hex.append(tx["hex"])
pc_parent_utxos.append(tx["new_utxo"])
# Child Pc
pc_hex = self.wallet.create_self_transfer_multi(utxos_to_spend=pc_parent_utxos)["hex"]
package_hex.append(pc_hex)
assert_equal(24, node.getmempoolinfo()["size"])
assert_equal(3, len(package_hex))
return package_hex
@check_package_limits
def test_anc_count_limits_2(self):
"""Create a 'Y' shaped chain with 24 transactions in the mempool and 2 in the package:
M1a M1b
^ ^
M2a M2b
. .
. .
. .
M12a M12b
^ ^
Pc
^
Pd
The lowest descendant, Pd, exceeds ancestor limits, but only if the in-mempool
and in-package ancestors are all considered together.
"""
node = self.nodes[0]
pc_parent_utxos = []
self.log.info("Check that in-mempool and in-package ancestors are calculated properly in packages")
# Two chains of 12 transactions each
for _ in range(2):
chaintip_utxo = self.wallet.send_self_transfer_chain(from_node=node, chain_length=12)[-1]["new_utxo"]
# last 2 transactions will be the parents of Pc
pc_parent_utxos.append(chaintip_utxo)
# Child Pc
pc_tx = self.wallet.create_self_transfer_multi(utxos_to_spend=pc_parent_utxos)
# Child Pd
pd_tx = self.wallet.create_self_transfer(utxo_to_spend=pc_tx["new_utxos"][0])
assert_equal(24, node.getmempoolinfo()["size"])
return [pc_tx["hex"], pd_tx["hex"]]
@check_package_limits
def test_anc_count_limits_bushy(self):
"""Create a tree with 20 transactions in the mempool and 6 in the package:
M1...M4 M5...M8 M9...M12 M13...M16 M17...M20
^ ^ ^ ^ ^ (each with 4 parents)
P0 P1 P2 P3 P4
^ ^ ^ ^ ^ (5 parents)
PC
Where M(4i+1)...M+(4i+4) are the parents of Pi and P0, P1, P2, P3, and P4 are the parents of PC.
P0... P4 individually only have 4 parents each, and PC has no in-mempool parents. But
combined, PC has 25 in-mempool and in-package parents.
"""
node = self.nodes[0]
package_hex = []
pc_parent_utxos = []
for _ in range(5): # Make package transactions P0 ... P4
pc_grandparent_utxos = []
for _ in range(4): # Make mempool transactions M(4i+1)...M(4i+4)
pc_grandparent_utxos.append(self.wallet.send_self_transfer(from_node=node)["new_utxo"])
# Package transaction Pi
pi_tx = self.wallet.create_self_transfer_multi(utxos_to_spend=pc_grandparent_utxos)
package_hex.append(pi_tx["hex"])
pc_parent_utxos.append(pi_tx["new_utxos"][0])
# Package transaction PC
pc_hex = self.wallet.create_self_transfer_multi(utxos_to_spend=pc_parent_utxos)["hex"]
package_hex.append(pc_hex)
assert_equal(20, node.getmempoolinfo()["size"])
assert_equal(6, len(package_hex))
return package_hex
@check_package_limits
def test_anc_size_limits(self):
"""Test Case with 2 independent transactions in the mempool and a parent + child in the
package, where the package parent is the child of both mempool transactions (30KvB each):
A B
^ ^
C
^
D
The lowest descendant, D, exceeds ancestor size limits, but only if the in-mempool
and in-package ancestors are all considered together.
"""
node = self.nodes[0]
parent_utxos = []
target_vsize = 30_000
high_fee = 10 * target_vsize # 10 sats/vB
self.log.info("Check that in-mempool and in-package ancestor size limits are calculated properly in packages")
# Mempool transactions A and B
for _ in range(2):
bulked_tx = self.wallet.create_self_transfer(target_vsize=target_vsize)
self.wallet.sendrawtransaction(from_node=node, tx_hex=bulked_tx["hex"])
parent_utxos.append(bulked_tx["new_utxo"])
# Package transaction C
pc_tx = self.wallet.create_self_transfer_multi(utxos_to_spend=parent_utxos, fee_per_output=high_fee, target_vsize=target_vsize)
# Package transaction D
pd_tx = self.wallet.create_self_transfer(utxo_to_spend=pc_tx["new_utxos"][0], target_vsize=target_vsize)
assert_equal(2, node.getmempoolinfo()["size"])
return [pc_tx["hex"], pd_tx["hex"]]
@check_package_limits
def test_desc_size_limits(self):
"""Create 3 mempool transactions and 2 package transactions (21KvB each):
test/functional/mempool_package_onemore.py
-80
View File
@@ -1,80 +0,0 @@
#!/usr/bin/env python3
# Copyright (c) 2014-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test descendant package tracking carve-out allowing one final transaction in
an otherwise-full package as long as it has only one parent and is <= 10k in
size.
"""
from test_framework.messages import (
DEFAULT_ANCESTOR_LIMIT,
)
from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import (
assert_equal,
assert_raises_rpc_error,
)
from test_framework.wallet import MiniWallet
class MempoolPackagesTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 1
def chain_tx(self, utxos_to_spend, *, num_outputs=1):
return self.wallet.send_self_transfer_multi(
from_node=self.nodes[0],
utxos_to_spend=utxos_to_spend,
num_outputs=num_outputs)['new_utxos']
def run_test(self):
self.wallet = MiniWallet(self.nodes[0])
# DEFAULT_ANCESTOR_LIMIT transactions off a confirmed tx should be fine
chain = []
utxo = self.wallet.get_utxo()
for _ in range(4):
utxo, utxo2 = self.chain_tx([utxo], num_outputs=2)
chain.append(utxo2)
for _ in range(DEFAULT_ANCESTOR_LIMIT - 4):
utxo, = self.chain_tx([utxo])
chain.append(utxo)
second_chain, = self.chain_tx([self.wallet.get_utxo(confirmed_only=True)])
# Check mempool has DEFAULT_ANCESTOR_LIMIT + 1 transactions in it
assert_equal(len(self.nodes[0].getrawmempool()), DEFAULT_ANCESTOR_LIMIT + 1)
# Adding one more transaction on to the chain should fail.
assert_raises_rpc_error(-26, "too-long-mempool-chain, too many unconfirmed ancestors [limit: 25]", self.chain_tx, [utxo])
# ... or if it chains on from some point in the middle of the chain.
assert_raises_rpc_error(-26, "too-long-mempool-chain, too many descendants", self.chain_tx, [chain[2]])
assert_raises_rpc_error(-26, "too-long-mempool-chain, too many descendants", self.chain_tx, [chain[1]])
# ...even if it chains on to two parent transactions with one in the chain.
assert_raises_rpc_error(-26, "too-long-mempool-chain, too many descendants", self.chain_tx, [chain[0], second_chain])
# ...especially if its > 40k weight
assert_raises_rpc_error(-26, "too-long-mempool-chain, too many descendants", self.chain_tx, [chain[0]], num_outputs=350)
# ...even if it's submitted with other transactions
replaceable_tx = self.wallet.create_self_transfer_multi(utxos_to_spend=[chain[0]])
txns = [replaceable_tx["tx"], self.wallet.create_self_transfer_multi(utxos_to_spend=replaceable_tx["new_utxos"])["tx"]]
txns_hex = [tx.serialize().hex() for tx in txns]
assert_equal(self.nodes[0].testmempoolaccept(txns_hex)[0]["reject-reason"], "too-long-mempool-chain")
pkg_result = self.nodes[0].submitpackage(txns_hex)
assert "too-long-mempool-chain" in pkg_result["tx-results"][txns[0].wtxid_hex]["error"]
assert_equal(pkg_result["tx-results"][txns[1].wtxid_hex]["error"], "bad-txns-inputs-missingorspent")
# But not if it chains directly off the first transaction
self.nodes[0].sendrawtransaction(replaceable_tx["hex"])
# and the second chain should work just fine
self.chain_tx([second_chain])
# Ensure an individual transaction with single direct conflict can RBF the chain which used our carve-out rule
replacement_tx = replaceable_tx["tx"]
replacement_tx.vout[0].nValue -= 1000000
self.nodes[0].sendrawtransaction(replacement_tx.serialize().hex())
# Finally, check that we added two transactions
assert_equal(len(self.nodes[0].getrawmempool()), DEFAULT_ANCESTOR_LIMIT + 3)
if __name__ == '__main__':
MempoolPackagesTest(__file__).main()
test/functional/mempool_package_rbf.py
+16 -178
View File
@@ -89,14 +89,11 @@ class PackageRBFTest(BitcoinTestFramework):
self.test_too_numerous_ancestors()
self.test_package_rbf_with_wrong_pkg_size()
self.test_insufficient_feerate()
self.test_wrong_conflict_cluster_size_linear()
self.test_wrong_conflict_cluster_size_parents_child()
self.test_wrong_conflict_cluster_size_parent_children()
self.test_0fee_package_rbf()
self.test_child_conflicts_parent_mempool_ancestor()
def test_package_rbf_basic(self):
self.log.info("Test that a child can pay to replace its parents' conflicts of cluster size 2")
self.log.info("Test that a child can pay to replace its parents' conflicts")
node = self.nodes[0]
# Reuse the same coins so that the transactions conflict with one another.
parent_coin = self.coins.pop()
@@ -193,17 +190,17 @@ class PackageRBFTest(BitcoinTestFramework):
def test_package_rbf_max_conflicts(self):
node = self.nodes[0]
self.log.info("Check Package RBF cannot replace more than MAX_REPLACEMENT_CANDIDATES transactions")
num_coins = 51
self.log.info("Check Package RBF cannot conflict with more than MAX_REPLACEMENT_CANDIDATES clusters")
num_coins = 101
parent_coins = self.coins[:num_coins]
del self.coins[:num_coins]
# Original transactions: 51 transactions with 1 descendants each -> 102 total transactions
size_two_clusters = []
# Original transactions: 101 transactions with 2 descendants each -> 303 total transactions, 101 clusters
size_three_clusters = []
for coin in parent_coins:
size_two_clusters.append(self.wallet.send_self_transfer_chain(from_node=node, chain_length=2, utxo_to_spend=coin))
expected_txns = [txn["tx"] for parent_child_txns in size_two_clusters for txn in parent_child_txns]
assert_equal(len(expected_txns), num_coins * 2)
size_three_clusters.append(self.wallet.send_self_transfer_chain(from_node=node, chain_length=3, utxo_to_spend=coin))
expected_txns = [txn["tx"] for parent_child_txns in size_three_clusters for txn in parent_child_txns]
assert_equal(len(expected_txns), num_coins * 3)
self.assert_mempool_contents(expected=expected_txns)
# parent feeerate needs to be high enough for minrelay
@@ -213,12 +210,13 @@ class PackageRBFTest(BitcoinTestFramework):
parent_fee_per_conflict = 10000
child_feerate = 10000 * DEFAULT_FEE
# Conflict against all transactions by double-spending each parent, causing 102 evictions
# Conflict against all transactions by double-spending each parent, causing 101 cluster conflicts
package_parent = self.wallet.create_self_transfer_multi(utxos_to_spend=parent_coins, fee_per_output=parent_fee_per_conflict)
package_child = self.wallet.create_self_transfer(fee_rate=child_feerate, utxo_to_spend=package_parent["new_utxos"][0])
pkg_results = node.submitpackage([package_parent["hex"], package_child["hex"]], maxfeerate=0)
assert_equal(f"package RBF failed: too many potential replacements, rejecting replacement {package_child['tx'].txid_hex}; too many potential replacements (102 > 100)", pkg_results["package_msg"])
assert_equal("transaction failed", pkg_results["package_msg"])
assert_equal(f"too many potential replacements, rejecting replacement {package_parent['txid']}; too many conflicting clusters (101 > 100)", pkg_results["tx-results"][package_parent["wtxid"]]["error"])
self.assert_mempool_contents(expected=expected_txns)
# Make singleton tx to conflict with in next batch
@@ -227,21 +225,22 @@ class PackageRBFTest(BitcoinTestFramework):
node.sendrawtransaction(singleton_tx["hex"])
expected_txns.append(singleton_tx["tx"])
# Double-spend same set minus last, and double-spend singleton. This hits 101 evictions; should still fail.
# Double-spend same set minus last, and double-spend singleton. This is still too many conflicted clusters.
# N.B. we can't RBF just a child tx in the clusters, as that would make resulting cluster of size 3.
double_spending_coins = parent_coins[:-1] + [singleton_coin]
package_parent = self.wallet.create_self_transfer_multi(utxos_to_spend=double_spending_coins, fee_per_output=parent_fee_per_conflict)
package_child = self.wallet.create_self_transfer(fee_rate=child_feerate, utxo_to_spend=package_parent["new_utxos"][0])
pkg_results = node.submitpackage([package_parent["hex"], package_child["hex"]], maxfeerate=0)
assert_equal(f"package RBF failed: too many potential replacements, rejecting replacement {package_child['tx'].txid_hex}; too many potential replacements (101 > 100)", pkg_results["package_msg"])
assert_equal("transaction failed", pkg_results["package_msg"])
assert_equal(f"too many potential replacements, rejecting replacement {package_parent['txid']}; too many conflicting clusters (101 > 100)", pkg_results["tx-results"][package_parent["wtxid"]]["error"])
self.assert_mempool_contents(expected=expected_txns)
# Finally, evict MAX_REPLACEMENT_CANDIDATES
# Finally, conflict with MAX_REPLACEMENT_CANDIDATES clusters
package_parent = self.wallet.create_self_transfer_multi(utxos_to_spend=parent_coins[:-1], fee_per_output=parent_fee_per_conflict)
package_child = self.wallet.create_self_transfer(fee_rate=child_feerate, utxo_to_spend=package_parent["new_utxos"][0])
pkg_results = node.submitpackage([package_parent["hex"], package_child["hex"]], maxfeerate=0)
assert_equal(pkg_results["package_msg"], "success")
self.assert_mempool_contents(expected=[singleton_tx["tx"], size_two_clusters[-1][0]["tx"], size_two_clusters[-1][1]["tx"], package_parent["tx"], package_child["tx"]] )
self.assert_mempool_contents(expected=[singleton_tx["tx"], size_three_clusters[-1][0]["tx"], size_three_clusters[-1][1]["tx"], size_three_clusters[-1][2]["tx"], package_parent["tx"], package_child["tx"]] )
self.generate(node, 1)
@@ -290,167 +289,6 @@ class PackageRBFTest(BitcoinTestFramework):
self.assert_mempool_contents(expected=package_txns1 + package_txns2_succeed)
self.generate(node, 1)
def test_wrong_conflict_cluster_size_linear(self):
self.log.info("Test that conflicting with a cluster not sized two is rejected: linear chain")
node = self.nodes[0]
# Coins we will conflict with
coin1 = self.coins.pop()
coin2 = self.coins.pop()
coin3 = self.coins.pop()
# Three transactions chained; package RBF against any of these
# should be rejected
self.ctr += 1
parent_result = self.wallet.create_self_transfer(
fee=DEFAULT_FEE,
utxo_to_spend=coin1,
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
self.ctr += 1
child_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
utxos_to_spend=[parent_result["new_utxo"], coin2],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
self.ctr += 1
grandchild_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
utxos_to_spend=[child_result["new_utxos"][0], coin3],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
expected_txns = [parent_result["tx"], child_result["tx"], grandchild_result["tx"]]
for tx in expected_txns:
node.sendrawtransaction(tx.serialize().hex())
self.assert_mempool_contents(expected=expected_txns)
# Now make conflicting packages for each coin
package_hex1, _package_txns1 = self.create_simple_package(coin1, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex1)
assert_equal(f"package RBF failed: {parent_result['tx'].txid_hex} has 2 descendants, max 1 allowed", package_result["package_msg"])
package_hex2, _package_txns2 = self.create_simple_package(coin2, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex2)
assert_equal(f"package RBF failed: {child_result['tx'].txid_hex} has both ancestor and descendant, exceeding cluster limit of 2", package_result["package_msg"])
package_hex3, _package_txns3 = self.create_simple_package(coin3, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex3)
assert_equal(f"package RBF failed: {grandchild_result['tx'].txid_hex} has 2 ancestors, max 1 allowed", package_result["package_msg"])
# Check that replacements were actually rejected
self.assert_mempool_contents(expected=expected_txns)
self.generate(node, 1)
def test_wrong_conflict_cluster_size_parents_child(self):
self.log.info("Test that conflicting with a cluster not sized two is rejected: two parents one child")
node = self.nodes[0]
# Coins we will conflict with
coin1 = self.coins.pop()
coin2 = self.coins.pop()
coin3 = self.coins.pop()
self.ctr += 1
parent1_result = self.wallet.create_self_transfer(
fee=DEFAULT_FEE,
utxo_to_spend=coin1,
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
self.ctr += 1
parent2_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
utxos_to_spend=[coin2],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
self.ctr += 1
child_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
utxos_to_spend=[parent1_result["new_utxo"], parent2_result["new_utxos"][0], coin3],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
expected_txns = [parent1_result["tx"], parent2_result["tx"], child_result["tx"]]
for tx in expected_txns:
node.sendrawtransaction(tx.serialize().hex())
self.assert_mempool_contents(expected=expected_txns)
# Now make conflicting packages for each coin
package_hex1, _package_txns1 = self.create_simple_package(coin1, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex1)
assert_equal(f"package RBF failed: {parent1_result['tx'].txid_hex} is not the only parent of child {child_result['tx'].txid_hex}", package_result["package_msg"])
package_hex2, _package_txns2 = self.create_simple_package(coin2, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex2)
assert_equal(f"package RBF failed: {child_result['tx'].txid_hex} has 2 ancestors, max 1 allowed", package_result["package_msg"])
package_hex3, _package_txns3 = self.create_simple_package(coin3, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex3)
assert_equal(f"package RBF failed: {child_result['tx'].txid_hex} has 2 ancestors, max 1 allowed", package_result["package_msg"])
# Check that replacements were actually rejected
self.assert_mempool_contents(expected=expected_txns)
self.generate(node, 1)
def test_wrong_conflict_cluster_size_parent_children(self):
self.log.info("Test that conflicting with a cluster not sized two is rejected: one parent two children")
node = self.nodes[0]
# Coins we will conflict with
coin1 = self.coins.pop()
coin2 = self.coins.pop()
coin3 = self.coins.pop()
self.ctr += 1
parent_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
num_outputs=2,
utxos_to_spend=[coin1],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
self.ctr += 1
child1_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
utxos_to_spend=[parent_result["new_utxos"][0], coin2],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
self.ctr += 1
child2_result = self.wallet.create_self_transfer_multi(
fee_per_output=int(DEFAULT_FEE * COIN),
utxos_to_spend=[parent_result["new_utxos"][1], coin3],
sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr,
)
# Submit them to mempool
expected_txns = [parent_result["tx"], child1_result["tx"], child2_result["tx"]]
for tx in expected_txns:
node.sendrawtransaction(tx.serialize().hex())
self.assert_mempool_contents(expected=expected_txns)
# Now make conflicting packages for each coin
package_hex1, _package_txns1 = self.create_simple_package(coin1, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex1)
assert_equal(f"package RBF failed: {child1_result['tx'].txid_hex} is not the only child of parent {parent_result['tx'].txid_hex}", package_result["package_msg"])
package_hex2, _package_txns2 = self.create_simple_package(coin2, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex2)
assert_equal(f"package RBF failed: {child1_result['tx'].txid_hex} is not the only child of parent {parent_result['tx'].txid_hex}", package_result["package_msg"])
package_hex3, _package_txns3 = self.create_simple_package(coin3, DEFAULT_FEE, DEFAULT_CHILD_FEE)
package_result = node.submitpackage(package_hex3)
assert_equal(f"package RBF failed: {child2_result['tx'].txid_hex} is not the only child of parent {parent_result['tx'].txid_hex}", package_result["package_msg"])
# Check that replacements were actually rejected
self.assert_mempool_contents(expected=expected_txns)
self.generate(node, 1)
def test_package_rbf_with_wrong_pkg_size(self):
self.log.info("Test that package RBF doesn't work with packages larger than 2 due to pkg size")
node = self.nodes[0]
test/functional/mempool_packages.py
+23 -55
View File
@@ -7,22 +7,18 @@
from decimal import Decimal
from test_framework.messages import (
DEFAULT_ANCESTOR_LIMIT,
DEFAULT_DESCENDANT_LIMIT,
DEFAULT_CLUSTER_LIMIT,
)
from test_framework.p2p import P2PTxInvStore
from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import (
assert_equal,
assert_raises_rpc_error,
)
from test_framework.wallet import MiniWallet
# custom limits for node1
CUSTOM_ANCESTOR_LIMIT = 5
CUSTOM_DESCENDANT_LIMIT = 10
assert CUSTOM_DESCENDANT_LIMIT >= CUSTOM_ANCESTOR_LIMIT
CUSTOM_CLUSTER_LIMIT = 10
assert CUSTOM_CLUSTER_LIMIT < DEFAULT_CLUSTER_LIMIT
class MempoolPackagesTest(BitcoinTestFramework):
def set_test_params(self):
@@ -33,8 +29,7 @@ class MempoolPackagesTest(BitcoinTestFramework):
[
],
[
"-limitancestorcount={}".format(CUSTOM_ANCESTOR_LIMIT),
"-limitdescendantcount={}".format(CUSTOM_DESCENDANT_LIMIT),
"-limitclustercount={}".format(CUSTOM_CLUSTER_LIMIT),
],
]
@@ -44,8 +39,8 @@ class MempoolPackagesTest(BitcoinTestFramework):
peer_inv_store = self.nodes[0].add_p2p_connection(P2PTxInvStore()) # keep track of invs
# DEFAULT_ANCESTOR_LIMIT transactions off a confirmed tx should be fine
chain = self.wallet.create_self_transfer_chain(chain_length=DEFAULT_ANCESTOR_LIMIT)
# DEFAULT_CLUSTER_LIMIT transactions off a confirmed tx should be fine for default node
chain = self.wallet.create_self_transfer_chain(chain_length=DEFAULT_CLUSTER_LIMIT)
witness_chain = [t["wtxid"] for t in chain]
ancestor_vsize = 0
ancestor_fees = Decimal(0)
@@ -59,22 +54,18 @@ class MempoolPackagesTest(BitcoinTestFramework):
# Otherwise, getrawmempool may be inconsistent with getmempoolentry if unbroadcast changes in between
peer_inv_store.wait_for_broadcast(witness_chain)
# Check mempool has DEFAULT_ANCESTOR_LIMIT transactions in it, and descendant and ancestor
# Check mempool has DEFAULT_CLUSTER_LIMIT transactions in it, and descendant and ancestor
# count and fees should look correct
mempool = self.nodes[0].getrawmempool(True)
assert_equal(len(mempool), DEFAULT_ANCESTOR_LIMIT)
assert_equal(len(mempool), DEFAULT_CLUSTER_LIMIT)
descendant_count = 1
descendant_fees = 0
descendant_vsize = 0
assert_equal(ancestor_vsize, sum([mempool[tx]['vsize'] for tx in mempool]))
ancestor_count = DEFAULT_ANCESTOR_LIMIT
ancestor_count = DEFAULT_CLUSTER_LIMIT
assert_equal(ancestor_fees, sum([mempool[tx]['fees']['base'] for tx in mempool]))
# Adding one more transaction on to the chain should fail.
next_hop = self.wallet.create_self_transfer(utxo_to_spend=chain[-1]["new_utxo"])["hex"]
assert_raises_rpc_error(-26, "too-long-mempool-chain", lambda: self.nodes[0].sendrawtransaction(next_hop))
descendants = []
ancestors = [t["txid"] for t in chain]
chain = [t["txid"] for t in chain]
@@ -190,23 +181,7 @@ class MempoolPackagesTest(BitcoinTestFramework):
assert_equal(entry['fees']['modified'], entry['fees']['base'] + Decimal("0.00002"))
assert_equal(entry['fees']['descendant'], descendant_fees + Decimal("0.00002"))
# Check that node1's mempool is as expected (-> custom ancestor limit)
mempool0 = self.nodes[0].getrawmempool(False)
mempool1 = self.nodes[1].getrawmempool(False)
assert_equal(len(mempool1), CUSTOM_ANCESTOR_LIMIT)
assert set(mempool1).issubset(set(mempool0))
for tx in chain[:CUSTOM_ANCESTOR_LIMIT]:
assert tx in mempool1
entry0 = self.nodes[0].getmempoolentry(tx)
entry1 = self.nodes[1].getmempoolentry(tx)
assert not entry0['unbroadcast']
assert not entry1['unbroadcast']
assert_equal(entry1['fees']['base'], entry0['fees']['base'])
assert_equal(entry1['vsize'], entry0['vsize'])
assert_equal(entry1['depends'], entry0['depends'])
# Now test descendant chain limits
tx_children = []
# First create one parent tx with 10 children
tx_with_children = self.wallet.send_self_transfer_multi(from_node=self.nodes[0], num_outputs=10)
@@ -215,7 +190,7 @@ class MempoolPackagesTest(BitcoinTestFramework):
# Sign and send up to MAX_DESCENDANT transactions chained off the parent tx
chain = [] # save sent txs for the purpose of checking node1's mempool later (see below)
for _ in range(DEFAULT_DESCENDANT_LIMIT - 1):
for _ in range(DEFAULT_CLUSTER_LIMIT - 1):
utxo = transaction_package.pop(0)
new_tx = self.wallet.send_self_transfer_multi(from_node=self.nodes[0], num_outputs=10, utxos_to_spend=[utxo])
txid = new_tx["txid"]
@@ -225,38 +200,31 @@ class MempoolPackagesTest(BitcoinTestFramework):
transaction_package.extend(new_tx["new_utxos"])
mempool = self.nodes[0].getrawmempool(True)
assert_equal(mempool[parent_transaction]['descendantcount'], DEFAULT_DESCENDANT_LIMIT)
assert_equal(mempool[parent_transaction]['descendantcount'], DEFAULT_CLUSTER_LIMIT)
assert_equal(sorted(mempool[parent_transaction]['spentby']), sorted(tx_children))
for child in tx_children:
assert_equal(mempool[child]['depends'], [parent_transaction])
# Sending one more chained transaction will fail
next_hop = self.wallet.create_self_transfer(utxo_to_spend=transaction_package.pop(0))["hex"]
assert_raises_rpc_error(-26, "too-long-mempool-chain", lambda: self.nodes[0].sendrawtransaction(next_hop))
# Check that node1's mempool is as expected, containing:
# - txs from previous ancestor test (-> custom ancestor limit)
# - parent tx for descendant test
# - txs chained off parent tx (-> custom descendant limit)
self.wait_until(lambda: len(self.nodes[1].getrawmempool()) ==
CUSTOM_ANCESTOR_LIMIT + 1 + CUSTOM_DESCENDANT_LIMIT, timeout=10)
self.wait_until(lambda: len(self.nodes[1].getrawmempool()) == 2*CUSTOM_CLUSTER_LIMIT, timeout=10)
mempool0 = self.nodes[0].getrawmempool(False)
mempool1 = self.nodes[1].getrawmempool(False)
assert set(mempool1).issubset(set(mempool0))
assert parent_transaction in mempool1
for tx in chain[:CUSTOM_DESCENDANT_LIMIT]:
assert tx in mempool1
for tx in chain[CUSTOM_DESCENDANT_LIMIT:]:
assert tx not in mempool1
for tx in mempool1:
entry0 = self.nodes[0].getmempoolentry(tx)
entry1 = self.nodes[1].getmempoolentry(tx)
assert not entry0['unbroadcast']
assert not entry1['unbroadcast']
assert_equal(entry1['fees']['base'], entry0['fees']['base'])
assert_equal(entry1['vsize'], entry0['vsize'])
assert_equal(entry1['depends'], entry0['depends'])
for tx in chain:
if tx in mempool1:
entry0 = self.nodes[0].getmempoolentry(tx)
entry1 = self.nodes[1].getmempoolentry(tx)
assert not entry0['unbroadcast']
assert not entry1['unbroadcast']
assert entry1["descendantcount"] <= CUSTOM_CLUSTER_LIMIT
assert_equal(entry1['fees']['base'], entry0['fees']['base'])
assert_equal(entry1['vsize'], entry0['vsize'])
assert_equal(entry1['depends'], entry0['depends'])
# Test reorg handling
# First, the basics:
self.generate(self.nodes[0], 1)
test/functional/mempool_persist.py
+2 -1
View File
@@ -128,7 +128,8 @@ class MempoolPersistTest(BitcoinTestFramework):
assert_equal(fees['base'] + Decimal('0.00001000'), fees['modified'])
self.log.debug('Verify all fields are loaded correctly')
assert_equal(last_entry, self.nodes[0].getmempoolentry(txid=last_txid))
new_entry = self.nodes[0].getmempoolentry(txid=last_txid)
assert_equal({**last_entry, "clusterid": None}, {**new_entry, "clusterid": None})
self.nodes[0].sendrawtransaction(tx_prioritised_not_submitted['hex'])
entry_prioritised_before_restart = self.nodes[0].getmempoolentry(txid=tx_prioritised_not_submitted['txid'])
assert_equal(entry_prioritised_before_restart['fees']['base'] + Decimal('0.00009999'), entry_prioritised_before_restart['fees']['modified'])
test/functional/mempool_sigoplimit.py
+4 -4
View File
@@ -169,14 +169,14 @@ class BytesPerSigOpTest(BitcoinTestFramework):
assert_equal(parent_individual_testres["vsize"], max_multisig_vsize)
# But together, it's exceeding limits in the *package* context. If sigops adjusted vsize wasn't being checked
# here, it would get further in validation and give too-long-mempool-chain error instead.
# here, it would get further in validation and give too-large-cluster error instead.
packet_test = self.nodes[0].testmempoolaccept([tx_parent.serialize().hex(), tx_child.serialize().hex()])
expected_package_error = f"package-mempool-limits, package size {2*max_multisig_vsize} exceeds ancestor size limit [limit: 101000]"
expected_package_error = "too-large-cluster"
assert_equal([x["package-error"] for x in packet_test], [expected_package_error] * 2)
# When we actually try to submit, the parent makes it into the mempool, but the child would exceed ancestor vsize limits
# When we actually try to submit, the parent makes it into the mempool, but the child would exceed cluster vsize limits
res = self.nodes[0].submitpackage([tx_parent.serialize().hex(), tx_child.serialize().hex()])
assert "too-long-mempool-chain" in res["tx-results"][tx_child.wtxid_hex]["error"]
assert "too-large-cluster" in res["tx-results"][tx_child.wtxid_hex]["error"]
assert tx_parent.txid_hex in self.nodes[0].getrawmempool()
# Transactions are tiny in weight
test/functional/mempool_truc.py
+16 -30
View File
@@ -196,15 +196,15 @@ class MempoolTRUC(BitcoinTestFramework):
node.invalidateblock(block["hash"])
self.check_mempool([tx_v3_block["txid"], tx_v2_block["txid"], tx_v3_block2["txid"], tx_v2_from_v3["txid"], tx_v3_from_v2["txid"], tx_v3_child_large["txid"], tx_chain_1["txid"], tx_chain_2["txid"], tx_chain_3["txid"], tx_chain_4["txid"]])
@cleanup(extra_args=["-limitdescendantsize=10"])
@cleanup(extra_args=["-limitclustercount=1"])
def test_nondefault_package_limits(self):
"""
Max standard tx size + TRUC rules imply the ancestor/descendant rules (at their default
Max standard tx size + TRUC rules imply the cluster rules (at their default
values), but those checks must not be skipped. Ensure both sets of checks are done by
changing the ancestor/descendant limit configurations.
changing the cluster limit configurations.
"""
node = self.nodes[0]
self.log.info("Test that a decreased limitdescendantsize also applies to TRUC child")
self.log.info("Test that a decreased cluster count limit also applies to TRUC child")
parent_target_vsize = 9990
child_target_vsize = 500
tx_v3_parent_large1 = self.wallet.send_self_transfer(
@@ -218,35 +218,26 @@ class MempoolTRUC(BitcoinTestFramework):
version=3
)
# Parent and child are within v3 limits, but parent's 10kvB descendant limit is exceeded
# Parent and child are within v3 limits, but cluster count limit is exceeded.
assert_greater_than_or_equal(TRUC_MAX_VSIZE, tx_v3_parent_large1["tx"].get_vsize())
assert_greater_than_or_equal(TRUC_CHILD_MAX_VSIZE, tx_v3_child_large1["tx"].get_vsize())
assert_greater_than(tx_v3_parent_large1["tx"].get_vsize() + tx_v3_child_large1["tx"].get_vsize(), 10000)
assert_raises_rpc_error(-26, f"too-long-mempool-chain, exceeds descendant size limit for tx {tx_v3_parent_large1['txid']}", node.sendrawtransaction, tx_v3_child_large1["hex"])
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, tx_v3_child_large1["hex"])
self.check_mempool([tx_v3_parent_large1["txid"]])
assert_equal(node.getmempoolentry(tx_v3_parent_large1["txid"])["descendantcount"], 1)
self.generate(node, 1)
self.log.info("Test that a decreased limitancestorsize also applies to v3 parent")
self.restart_node(0, extra_args=["-limitancestorsize=10"])
tx_v3_parent_large2 = self.wallet.send_self_transfer(
from_node=node,
target_vsize=parent_target_vsize,
version=3
)
tx_v3_child_large2 = self.wallet.create_self_transfer(
utxo_to_spend=tx_v3_parent_large2["new_utxo"],
target_vsize=child_target_vsize,
version=3
)
self.log.info("Test that a decreased limitclustersize also applies to TRUC child")
self.restart_node(0, extra_args=["-limitclustersize=10", "-acceptnonstdtxn=1"])
tx_v3_parent_large2 = self.wallet.send_self_transfer(from_node=node, target_vsize=parent_target_vsize, version=3)
tx_v3_child_large2 = self.wallet.create_self_transfer(utxo_to_spend=tx_v3_parent_large2["new_utxo"], target_vsize=child_target_vsize, version=3)
# Parent and child are within TRUC limits
assert_greater_than_or_equal(TRUC_MAX_VSIZE, tx_v3_parent_large2["tx"].get_vsize())
assert_greater_than_or_equal(TRUC_CHILD_MAX_VSIZE, tx_v3_child_large2["tx"].get_vsize())
assert_greater_than(tx_v3_parent_large2["tx"].get_vsize() + tx_v3_child_large2["tx"].get_vsize(), 10000)
assert_raises_rpc_error(-26, "too-long-mempool-chain, exceeds ancestor size limit", node.sendrawtransaction, tx_v3_child_large2["hex"])
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, tx_v3_child_large2["hex"])
self.log.info("Test that a decreased limitclustercount also applies to TRUC transactions")
self.restart_node(0, extra_args=["-limitclustercount=1", "-acceptnonstdtxn=1"])
assert_raises_rpc_error(-26, "too-large-cluster", node.sendrawtransaction, tx_v3_child_large2["hex"])
self.check_mempool([tx_v3_parent_large2["txid"]])
@cleanup()
@@ -513,7 +504,7 @@ class MempoolTRUC(BitcoinTestFramework):
tx_v3_child_2_rule6 = self.wallet.create_self_transfer(
utxo_to_spend=tx_v3_parent["new_utxos"][1], fee_rate=DEFAULT_FEE, version=3
)
rule6_str = f"insufficient fee (including sibling eviction), rejecting replacement {tx_v3_child_2_rule6['txid']}; new feerate"
rule6_str = f"insufficient fee (including sibling eviction), rejecting replacement {tx_v3_child_2_rule6['txid']}"
assert_raises_rpc_error(-26, rule6_str, node.sendrawtransaction, tx_v3_child_2_rule6["hex"])
self.check_mempool([tx_v3_parent['txid'], tx_v3_child_1['txid']])
@@ -544,11 +535,6 @@ class MempoolTRUC(BitcoinTestFramework):
tx_v3_replacement_only = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_for_conflict, fee_per_output=4000000)
# Override maxfeerate - it costs a lot to replace these 100 transactions.
assert node.testmempoolaccept([tx_v3_replacement_only["hex"]], maxfeerate=0)[0]["allowed"]
# Adding another one exceeds the limit.
utxos_for_conflict.append(tx_v3_parent["new_utxos"][1])
tx_v3_child_2_rule5 = self.wallet.create_self_transfer_multi(utxos_to_spend=utxos_for_conflict, fee_per_output=4000000, version=3)
rule5_str = f"too many potential replacements (including sibling eviction), rejecting replacement {tx_v3_child_2_rule5['txid']}; too many potential replacements (101 > 100)"
assert_raises_rpc_error(-26, rule5_str, node.sendrawtransaction, tx_v3_child_2_rule5["hex"])
self.check_mempool(txids_v2_100 + [tx_v3_parent["txid"], tx_v3_child_1["txid"]])
self.log.info("Test sibling eviction is successful if it meets all RBF rules")
@@ -566,7 +552,7 @@ class MempoolTRUC(BitcoinTestFramework):
fee_to_beat = max(int(tx_v3_child_2["fee"] * COIN), int(tx_unrelated_replacee["fee"]*COIN))
tx_v3_child_3 = self.wallet.create_self_transfer_multi(
utxos_to_spend=[tx_v3_parent["new_utxos"][0], utxo_unrelated_conflict], fee_per_output=fee_to_beat*2, version=3
utxos_to_spend=[tx_v3_parent["new_utxos"][0], utxo_unrelated_conflict], fee_per_output=fee_to_beat*4, version=3
)
node.sendrawtransaction(tx_v3_child_3["hex"])
self.check_mempool(txids_v2_100 + [tx_v3_parent["txid"], tx_v3_child_3["txid"]])
test/functional/mempool_updatefromblock.py
+5 -5
View File
@@ -18,6 +18,7 @@ from test_framework.blocktools import (
from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import assert_equal, assert_raises_rpc_error
from test_framework.wallet import MiniWallet
from test_framework.mempool_util import DEFAULT_CLUSTER_LIMIT
MAX_DISCONNECTED_TX_POOL_BYTES = 20_000_000
@@ -27,8 +28,7 @@ CUSTOM_DESCENDANT_COUNT = CUSTOM_ANCESTOR_COUNT
class MempoolUpdateFromBlockTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 1
# Ancestor and descendant limits depend on transaction_graph_test requirements
self.extra_args = [['-limitdescendantsize=1000', '-limitancestorsize=1000', f'-limitancestorcount={CUSTOM_ANCESTOR_COUNT}', f'-limitdescendantcount={CUSTOM_DESCENDANT_COUNT}']]
self.extra_args = [['-limitclustersize=1000']]
def create_empty_fork(self, fork_length):
'''
@@ -198,11 +198,11 @@ class MempoolUpdateFromBlockTest(BitcoinTestFramework):
fork_blocks = self.create_empty_fork(fork_length=10)
# Two higher than descendant count
chain = wallet.create_self_transfer_chain(chain_length=CUSTOM_DESCENDANT_COUNT + 2)
chain = wallet.create_self_transfer_chain(chain_length=DEFAULT_CLUSTER_LIMIT + 2)
for tx in chain[:-2]:
self.nodes[0].sendrawtransaction(tx["hex"])
assert_raises_rpc_error(-26, "too-long-mempool-chain, too many unconfirmed ancestors [limit: 100]", self.nodes[0].sendrawtransaction, chain[-2]["hex"])
assert_raises_rpc_error(-26, "too-large-cluster", self.nodes[0].sendrawtransaction, chain[-2]["hex"])
# Mine a block with all but last transaction, non-standardly long chain
self.generateblock(self.nodes[0], output="raw(42)", transactions=[tx["hex"] for tx in chain[:-1]])
@@ -220,7 +220,7 @@ class MempoolUpdateFromBlockTest(BitcoinTestFramework):
def run_test(self):
# Mine in batches of 25 to test multi-block reorg under chain limits
self.transaction_graph_test(size=CUSTOM_ANCESTOR_COUNT, n_tx_to_mine=[25, 50, 75])
self.transaction_graph_test(size=DEFAULT_CLUSTER_LIMIT, n_tx_to_mine=[25, 50, 75])
self.test_max_disconnect_pool_bytes()
test/functional/mining_prioritisetransaction.py
+7
View File
@@ -101,14 +101,21 @@ class PrioritiseTransactionTest(BitcoinTestFramework):
self.nodes[0].prioritisetransaction(txid=txid_c, fee_delta=int(fee_delta_c_1 * COIN))
self.nodes[0].prioritisetransaction(txid=txid_c, fee_delta=int(fee_delta_c_2 * COIN))
raw_before[txid_a]["fees"]["descendant"] += fee_delta_b + fee_delta_c_1 + fee_delta_c_2
# We expect tx_a to have a chunk fee that includes tx_b and tx_c.
raw_before[txid_a]["fees"]["chunk"] += fee_delta_b + fee_delta_c_1 + fee_delta_c_2
raw_before[txid_b]["fees"]["modified"] += fee_delta_b
raw_before[txid_b]["fees"]["ancestor"] += fee_delta_b
raw_before[txid_b]["fees"]["descendant"] += fee_delta_b
# We also expect tx_b and tx_c to have their chunk fees modified too,
# since they chunk together.
raw_before[txid_b]["fees"]["chunk"] += fee_delta_b + fee_delta_c_1 + fee_delta_c_2
raw_before[txid_c]["fees"]["modified"] += fee_delta_c_1 + fee_delta_c_2
raw_before[txid_c]["fees"]["ancestor"] += fee_delta_c_1 + fee_delta_c_2
raw_before[txid_c]["fees"]["descendant"] += fee_delta_c_1 + fee_delta_c_2
raw_before[txid_c]["fees"]["chunk"] += fee_delta_b + fee_delta_c_1 + fee_delta_c_2
raw_before[txid_d]["fees"]["ancestor"] += fee_delta_b + fee_delta_c_1 + fee_delta_c_2
raw_after = self.nodes[0].getrawmempool(verbose=True)
# Don't bother comparing cluster ids, which are not meant to be stable.
assert_equal(raw_before[txid_a], raw_after[txid_a])
assert_equal(raw_before, raw_after)
assert_equal(self.nodes[0].getprioritisedtransactions(), {txid_b: {"fee_delta" : fee_delta_b*COIN, "in_mempool" : True, "modified_fee": int(fee_delta_b*COIN + COIN * tx_o_b["fee"])}, txid_c: {"fee_delta" : (fee_delta_c_1 + fee_delta_c_2)*COIN, "in_mempool" : True, "modified_fee": int((fee_delta_c_1 + fee_delta_c_2 ) * COIN + COIN * tx_o_c["fee"])}})
test/functional/test_framework/mempool_util.py
+2
View File
@@ -33,6 +33,8 @@ from .wallet import (
DEFAULT_MIN_RELAY_TX_FEE = 100
# Default for -incrementalrelayfee in sat/kvB
DEFAULT_INCREMENTAL_RELAY_FEE = 100
DEFAULT_CLUSTER_LIMIT = 64
DEFAULT_CLUSTER_SIZE_LIMIT_KVB = 101
TRUC_MAX_VSIZE = 10000
TRUC_CHILD_MAX_VSIZE = 1000
test/functional/test_framework/messages.py
+1
View File
@@ -75,6 +75,7 @@ WITNESS_SCALE_FACTOR = 4
DEFAULT_ANCESTOR_LIMIT = 25 # default max number of in-mempool ancestors
DEFAULT_DESCENDANT_LIMIT = 25 # default max number of in-mempool descendants
DEFAULT_CLUSTER_LIMIT = 64 # default max number of transactions in a cluster
# Default setting for -datacarriersize.
test/functional/test_runner.py
+1 -1
View File
@@ -254,7 +254,6 @@ BASE_SCRIPTS = [
'feature_utxo_set_hash.py',
'feature_rbf.py',
'mempool_packages.py',
'mempool_package_onemore.py',
'mempool_package_limits.py',
'mempool_package_rbf.py',
'tool_utxo_to_sqlite.py',
@@ -342,6 +341,7 @@ BASE_SCRIPTS = [
'p2p_sendtxrcncl.py',
'rpc_scantxoutset.py',
'feature_unsupported_utxo_db.py',
'mempool_cluster.py',
'feature_logging.py',
'interface_ipc.py',
'feature_anchors.py',
test/functional/wallet_balance.py
+2 -2
View File
@@ -55,9 +55,9 @@ class WalletTest(BitcoinTestFramework):
# whitelist peers to speed up tx relay / mempool sync
self.noban_tx_relay = True
self.extra_args = [
# Limit mempool descendants as a hack to have wallet txs rejected from the mempool.
# Limit mempool clusters as a hack to have wallet txs rejected from the mempool.
# Set walletrejectlongchains=0 so the wallet still creates the transactions.
['-limitdescendantcount=3', '-walletrejectlongchains=0'],
['-limitclustercount=3', '-walletrejectlongchains=0'],
[],
]
test/functional/wallet_basic.py
+5 -5
View File
@@ -467,9 +467,9 @@ class WalletTest(BitcoinTestFramework):
self.log.info("Test -reindex")
self.stop_nodes()
# set lower ancestor limit for later
self.start_node(0, ['-reindex', "-walletrejectlongchains=0", "-limitancestorcount=" + str(chainlimit)])
self.start_node(1, ['-reindex', "-limitancestorcount=" + str(chainlimit)])
self.start_node(2, ['-reindex', "-limitancestorcount=" + str(chainlimit)])
self.start_node(0, ['-reindex', "-walletrejectlongchains=0", "-limitancestorcount=" + str(chainlimit), "-limitclustercount=" + str(chainlimit)])
self.start_node(1, ['-reindex', "-limitclustercount=" + str(chainlimit)])
self.start_node(2, ['-reindex', "-limitclustercount=" + str(chainlimit)])
# reindex will leave rpc warm up "early"; Wait for it to finish
self.wait_until(lambda: [block_count] * 3 == [self.nodes[i].getblockcount() for i in range(3)])
assert_equal(balance_nodes, [self.nodes[i].getbalance() for i in range(3)])
@@ -510,7 +510,7 @@ class WalletTest(BitcoinTestFramework):
# Try with walletrejectlongchains
# Double chain limit but require combining inputs, so we pass AttemptSelection
self.stop_node(0)
extra_args = ["-walletrejectlongchains", "-limitancestorcount=" + str(2 * chainlimit)]
extra_args = ["-walletrejectlongchains", "-limitclustercount=" + str(2 * chainlimit), "-limitancestorcount=" + str(2*chainlimit)]
self.start_node(0, extra_args=extra_args)
# wait until the wallet has submitted all transactions to the mempool
@@ -521,7 +521,7 @@ class WalletTest(BitcoinTestFramework):
node0_balance = self.nodes[0].getbalance()
# With walletrejectlongchains we will not create the tx and store it in our wallet.
assert_raises_rpc_error(-6, f"too many unconfirmed ancestors [limit: {chainlimit * 2}]", self.nodes[0].sendtoaddress, sending_addr, node0_balance - Decimal('0.01'))
assert_raises_rpc_error(-6, "too many unconfirmed transactions in cluster", self.nodes[0].sendtoaddress, sending_addr, node0_balance - Decimal('0.01'))
# Verify nothing new in wallet
assert_equal(total_txs, len(self.nodes[0].listtransactions("*", 99999)))