mirror/bitcoin
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

clusterlin: pool SetInfos (preparation)

Browse Source 
This significantly changes the data structures used in SFL, based on the
observation that the DepData::top_setinfo fields are quite wasteful:
there is one per dependency (up to n^2/4), but we only ever need one per
active dependency (of which there at most n-1). In total, the number of
chunks plus the number of active dependencies is always exactly equal to
the number of transactions, so it makes sense to have a shared pool of
SetInfos, which are used for both chunks and top sets.

To that effect, introduce a separate m_set_info variable, which stores a
SetInfo per transaction. Some of these are used for chunk sets, and some
for active dependencies' top sets. Every activation transforms the
parent's chunk into the top set for the new dependency. Every
deactivation transforms the top set into the new parent chunk.

With indexes into m_set_data (SetIdx) becoming bounded by the number of
transactions, we can use a SetType to represent sets of SetIdxs.
Specifically, an m_chunk_idxs is added which contains all SetIdx
referring to chunks. This leads to a much more natural way of iterating
over chunks.

Also use this opportunity to normalize many variable names.
This commit is contained in:
Pieter Wuille committed 2026-02-17 09:04:36 -05:00
1 parent 20e2f3e96d
commit 7c6f63a8a9
2 files changed
+253 -258
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/cluster_linearize.h
+247 -252
View File
@@ -649,9 +649,13 @@ private:
using TxIdx = DepGraphIndex; using TxIdx = DepGraphIndex;
/** Data type to represent indexing into m_dep_data. */ /** Data type to represent indexing into m_dep_data. */
using DepIdx = uint32_t; using DepIdx = uint32_t;
/** Data type to represent indexing into m_set_info. */
using SetIdx = uint32_t;
/** Structure with information about a single transaction. For transactions that are the /** An invalid SetIdx. */
* representative for the chunk they are in, this also stores chunk information. */ static constexpr SetIdx INVALID_SET_IDX = SetIdx(-1);
/** Structure with information about a single transaction. */
struct TxData { struct TxData {
/** The dependencies to children of this transaction. Immutable after construction. */ /** The dependencies to children of this transaction. Immutable after construction. */
std::vector<DepIdx> child_deps; std::vector<DepIdx> child_deps;
@@ -659,12 +663,8 @@ private:
SetType parents; SetType parents;
/** The set of child transactions of this transaction. Immutable after construction. */ /** The set of child transactions of this transaction. Immutable after construction. */
SetType children; SetType children;
/** Which transaction holds the chunk_setinfo for the chunk this transaction is in /** Which chunk this transaction belongs to. */
* (the representative for the chunk). */ SetIdx chunk_idx;
TxIdx chunk_idx;
/** (Only if this transaction is the representative for the chunk it is in) the total
* chunk set and feerate. */
SetInfo<SetType> chunk_setinfo;
}; };
/** Structure with information about a single dependency. */ /** Structure with information about a single dependency. */
@@ -675,25 +675,30 @@ private:
TxIdx parent, child; TxIdx parent, child;
/** (Only if this dependency is active) the would-be top chunk and its feerate that would /** (Only if this dependency is active) the would-be top chunk and its feerate that would
* be formed if this dependency were to be deactivated. */ * be formed if this dependency were to be deactivated. */
SetInfo<SetType> top_setinfo; SetIdx dep_top_idx;
}; };
/** The set of all TxIdx's of transactions in the cluster indexing into m_tx_data. */ /** The set of all TxIdx's of transactions in the cluster indexing into m_tx_data. */
SetType m_transaction_idxs; SetType m_transaction_idxs;
/** The set of all chunk SetIdx's. This excludes the SetIdxs that refer to active
* dependencies' tops. */
SetType m_chunk_idxs;
/** Information about each transaction (and chunks). Keeps the "holes" from DepGraph during /** Information about each transaction (and chunks). Keeps the "holes" from DepGraph during
* construction. Indexed by TxIdx. */ * construction. Indexed by TxIdx. */
std::vector<TxData> m_tx_data; std::vector<TxData> m_tx_data;
/** Information about each set (chunk, or active dependency top set). Indexed by SetIdx. */
std::vector<SetInfo<SetType>> m_set_info;
/** Information about each dependency. Indexed by DepIdx. */ /** Information about each dependency. Indexed by DepIdx. */
std::vector<DepData> m_dep_data; std::vector<DepData> m_dep_data;
/** A FIFO of chunk representatives of chunks that may be improved still. */ /** A FIFO of chunk SetIdxs for chunks that may be improved still. */
VecDeque<TxIdx> m_suboptimal_chunks; VecDeque<SetIdx> m_suboptimal_chunks;
/** A FIFO of chunk representatives with a pivot transaction in them, and a flag to indicate /** A FIFO of chunk indexes with a pivot transaction in them, and a flag to indicate their
* their status: * status:
* - bit 1: currently attempting to move the pivot down, rather than up. * - bit 1: currently attempting to move the pivot down, rather than up.
* - bit 2: this is the second stage, so we have already tried moving the pivot in the other * - bit 2: this is the second stage, so we have already tried moving the pivot in the other
* direction. * direction.
*/ */
VecDeque<std::tuple<TxIdx, TxIdx, unsigned>> m_nonminimal_chunks; VecDeque<std::tuple<SetIdx, TxIdx, unsigned>> m_nonminimal_chunks;
/** The number of updated transactions in activations/deactivations. */ /** The number of updated transactions in activations/deactivations. */
uint64_t m_cost{0}; uint64_t m_cost{0};
@@ -715,17 +720,17 @@ private:
} }
/** Update a chunk: /** Update a chunk:
* - All transactions have their chunk representative set to `chunk_idx`. * - All transactions have their chunk index set to `chunk_idx`.
* - All dependencies which have `query` in their top_setinfo get `dep_change` added to it * - All dependencies which have `query` in their top set get `dep_change` added to it
* (if `!Subtract`) or removed from it (if `Subtract`). * (if `!Subtract`) or removed from it (if `Subtract`).
*/ */
template<bool Subtract> template<bool Subtract>
void UpdateChunk(const SetType& chunk, TxIdx query, TxIdx chunk_idx, const SetInfo<SetType>& dep_change) noexcept void UpdateChunk(const SetType& tx_idxs, TxIdx query, SetIdx chunk_idx, const SetInfo<SetType>& dep_change) noexcept
{ {
// Iterate over all the chunk's transactions. // Iterate over all the chunk's transactions.
for (auto tx_idx : chunk) { for (auto tx_idx : tx_idxs) {
auto& tx_data = m_tx_data[tx_idx]; auto& tx_data = m_tx_data[tx_idx];
// Update the chunk representative. // Update the chunk index for this transaction.
tx_data.chunk_idx = chunk_idx; tx_data.chunk_idx = chunk_idx;
// Iterate over all active dependencies with tx_idx as parent. Combined with the outer // Iterate over all active dependencies with tx_idx as parent. Combined with the outer
// loop this iterates over all internal active dependencies of the chunk. // loop this iterates over all internal active dependencies of the chunk.
@@ -735,37 +740,40 @@ private:
Assume(dep_entry.parent == tx_idx); Assume(dep_entry.parent == tx_idx);
// Skip inactive dependencies. // Skip inactive dependencies.
if (!dep_entry.active) continue; if (!dep_entry.active) continue;
// If this dependency's top_setinfo contains query, update it to add/remove auto& top_set_info = m_set_info[dep_entry.dep_top_idx];
// If this dependency's top set contains query, update it to add/remove
// dep_change. // dep_change.
if (dep_entry.top_setinfo.transactions[query]) { if (top_set_info.transactions[query]) {
if constexpr (Subtract) { if constexpr (Subtract) {
dep_entry.top_setinfo -= dep_change; top_set_info -= dep_change;
} else { } else {
dep_entry.top_setinfo |= dep_change; top_set_info |= dep_change;
} }
} }
} }
} }
} }
/** Make a specified inactive dependency active. Returns the merged chunk representative. */ /** Make a specified inactive dependency active. Returns the merged chunk index. */
TxIdx Activate(DepIdx dep_idx) noexcept TxIdx Activate(DepIdx dep_idx) noexcept
{ {
auto& dep_data = m_dep_data[dep_idx]; auto& dep_data = m_dep_data[dep_idx];
Assume(!dep_data.active); Assume(!dep_data.active);
auto& child_tx_data = m_tx_data[dep_data.child];
auto& parent_tx_data = m_tx_data[dep_data.parent];
// Gather information about the parent and child chunks. // Gather and check information about the parent and child transactions.
Assume(parent_tx_data.chunk_idx != child_tx_data.chunk_idx); auto& parent_data = m_tx_data[dep_data.parent];
auto& par_chunk_data = m_tx_data[parent_tx_data.chunk_idx]; auto& child_data = m_tx_data[dep_data.child];
auto& chl_chunk_data = m_tx_data[child_tx_data.chunk_idx]; Assume(parent_data.children[dep_data.child]);
TxIdx top_idx = parent_tx_data.chunk_idx; // Get the set index of the chunks the parent and child are currently in. The parent chunk
auto top_part = par_chunk_data.chunk_setinfo; // will become the top set of the newly activated dependency, while the child chunk will be
auto bottom_part = chl_chunk_data.chunk_setinfo; // grown to become the merged chunk.
// Update the parent chunk to also contain the child. auto parent_chunk_idx = parent_data.chunk_idx;
par_chunk_data.chunk_setinfo |= bottom_part; auto child_chunk_idx = child_data.chunk_idx;
m_cost += par_chunk_data.chunk_setinfo.transactions.Count(); Assume(parent_chunk_idx != child_chunk_idx);
Assume(m_chunk_idxs[parent_chunk_idx]);
Assume(m_chunk_idxs[child_chunk_idx]);
auto& top_info = m_set_info[parent_chunk_idx];
auto& bottom_info = m_set_info[child_chunk_idx];
// Consider the following example: // Consider the following example:
// //
@@ -782,22 +790,27 @@ private:
// dependency being activated (E->C here) in its top set, will have the opposite part added // dependency being activated (E->C here) in its top set, will have the opposite part added
// to it. This is true for B->A and F->E, but not for C->A and F->D. // to it. This is true for B->A and F->E, but not for C->A and F->D.
// //
// Let UpdateChunk traverse the old parent chunk top_part (ABC in example), and add // Let UpdateChunk traverse the old parent chunk top_info (ABC in example), and add
// bottom_part (DEF) to every dependency's top_set which has the parent (C) in it. The // bottom_info (DEF) to every dependency's top set which has the parent (C) in it. At the
// representative of each of these transactions was already top_idx, so that is not being // same time, change the chunk_idx for each to be child_chunk_idx, which becomes the set for
// changed here. // the merged chunk.
UpdateChunk<false>(/*chunk=*/top_part.transactions, /*query=*/dep_data.parent, UpdateChunk<false>(/*tx_idxs=*/top_info.transactions, /*query=*/dep_data.parent,
/*chunk_idx=*/top_idx, /*dep_change=*/bottom_part); /*chunk_idx=*/child_chunk_idx, /*dep_change=*/bottom_info);
// Let UpdateChunk traverse the old child chunk bottom_part (DEF in example), and add // Let UpdateChunk traverse the old child chunk bottom_info (DEF in example), and add
// top_part (ABC) to every dependency's top_set which has the child (E) in it. At the same // top_info (ABC) to every dependency's top set which has the child (E) in it. The chunk
// time, change the representative of each of these transactions to be top_idx, which // these are part of isn't being changed here (already child_chunk_idx for each).
// becomes the representative for the merged chunk. UpdateChunk<false>(/*tx_idxs=*/bottom_info.transactions, /*query=*/dep_data.child,
UpdateChunk<false>(/*chunk=*/bottom_part.transactions, /*query=*/dep_data.child, /*chunk_idx=*/child_chunk_idx, /*dep_change=*/top_info);
/*chunk_idx=*/top_idx, /*dep_change=*/top_part); // Merge top_info into bottom_info, which becomes the merged chunk.
bottom_info |= top_info;
m_cost += bottom_info.transactions.Count();
// Make parent chunk the set for the new active dependency.
dep_data.dep_top_idx = parent_chunk_idx;
m_chunk_idxs.Reset(parent_chunk_idx);
// Make active. // Make active.
dep_data.active = true; dep_data.active = true;
dep_data.top_setinfo = top_part; // Return the newly merged chunk.
return top_idx; return child_chunk_idx;
} }
/** Make a specified active dependency inactive. */ /** Make a specified active dependency inactive. */
@@ -805,62 +818,60 @@ private:
{ {
auto& dep_data = m_dep_data[dep_idx]; auto& dep_data = m_dep_data[dep_idx];
Assume(dep_data.active); Assume(dep_data.active);
auto& parent_tx_data = m_tx_data[dep_data.parent];
// Make inactive. // Make inactive.
dep_data.active = false; dep_data.active = false;
// Update representatives.
auto& chunk_data = m_tx_data[parent_tx_data.chunk_idx];
m_cost += chunk_data.chunk_setinfo.transactions.Count();
auto top_part = dep_data.top_setinfo;
auto bottom_part = chunk_data.chunk_setinfo - top_part;
TxIdx bottom_idx = dep_data.child;
auto& bottom_chunk_data = m_tx_data[bottom_idx];
bottom_chunk_data.chunk_setinfo = bottom_part;
TxIdx top_idx = dep_data.parent;
auto& top_chunk_data = m_tx_data[top_idx];
top_chunk_data.chunk_setinfo = top_part;
// See the comment above in Activate(). We perform the opposite operations here, // Gather and check information about the parent transactions.
// removing instead of adding. auto& parent_data = m_tx_data[dep_data.parent];
// Assume(parent_data.children[dep_data.child]);
// Let UpdateChunk traverse the old parent chunk top_part, and remove bottom_part from // Get the top set of the active dependency (which will become the parent chunk) and the
// every dependency's top_set which has the parent in it. At the same time, change the // chunk set the transactions are currently in (which will become the bottom chunk).
// representative of each of these transactions to be top_idx. auto parent_chunk_idx = dep_data.dep_top_idx;
UpdateChunk<true>(/*chunk=*/top_part.transactions, /*query=*/dep_data.parent, auto child_chunk_idx = parent_data.chunk_idx;
/*chunk_idx=*/top_idx, /*dep_change=*/bottom_part); Assume(parent_chunk_idx != child_chunk_idx);
// Let UpdateChunk traverse the old child chunk bottom_part, and remove top_part from every Assume(m_chunk_idxs[child_chunk_idx]);
// dependency's top_set which has the child in it. At the same time, change the Assume(!m_chunk_idxs[parent_chunk_idx]); // top set, not a chunk
// representative of each of these transactions to be bottom_idx. auto& top_info = m_set_info[parent_chunk_idx];
UpdateChunk<true>(/*chunk=*/bottom_part.transactions, /*query=*/dep_data.child, auto& bottom_info = m_set_info[child_chunk_idx];
/*chunk_idx=*/bottom_idx, /*dep_change=*/top_part); // Remove the active dependency.
dep_data.dep_top_idx = INVALID_SET_IDX;
m_chunk_idxs.Set(parent_chunk_idx);
m_cost += bottom_info.transactions.Count();
// Subtract the top_info from the bottom_info, as it will become the child chunk.
bottom_info -= top_info;
// See the comment above in Activate(). We perform the opposite operations here, removing
// instead of adding.
UpdateChunk<true>(/*tx_idxs=*/top_info.transactions, /*query=*/dep_data.parent,
/*chunk_idx=*/parent_chunk_idx, /*dep_change=*/bottom_info);
UpdateChunk<true>(/*tx_idxs=*/bottom_info.transactions, /*query=*/dep_data.child,
/*chunk_idx=*/child_chunk_idx, /*dep_change=*/top_info);
} }
/** Activate a dependency from the chunk represented by bottom_idx to the chunk represented by /** Activate a dependency from the bottom set to the top set. Return the index of the merged
* top_idx. Return the representative of the merged chunk, or TxIdx(-1) if no merge is * chunk, or INVALID_SET_IDX if no merge is possible. */
* possible. */ SetIdx MergeChunks(SetIdx top_idx, SetIdx bottom_idx) noexcept
TxIdx MergeChunks(TxIdx top_idx, TxIdx bottom_idx) noexcept
{ {
auto& top_chunk = m_tx_data[top_idx]; Assume(m_chunk_idxs[top_idx]);
Assume(top_chunk.chunk_idx == top_idx); Assume(m_chunk_idxs[bottom_idx]);
auto& bottom_chunk = m_tx_data[bottom_idx]; auto& top_chunk_info = m_set_info[top_idx];
Assume(bottom_chunk.chunk_idx == bottom_idx); auto& bottom_chunk_info = m_set_info[bottom_idx];
// Count the number of dependencies between bottom_chunk and top_chunk. // Count the number of dependencies between bottom_chunk and top_chunk.
unsigned num_deps{0}; unsigned num_deps{0};
for (auto tx : top_chunk.chunk_setinfo.transactions) { for (auto tx_idx : top_chunk_info.transactions) {
auto& tx_data = m_tx_data[tx]; auto& tx_data = m_tx_data[tx_idx];
num_deps += (tx_data.children & bottom_chunk.chunk_setinfo.transactions).Count(); num_deps += (tx_data.children & bottom_chunk_info.transactions).Count();
} }
if (num_deps == 0) return TxIdx(-1); if (num_deps == 0) return INVALID_SET_IDX;
// Uniformly randomly pick one of them and activate it. // Uniformly randomly pick one of them and activate it.
unsigned pick = m_rng.randrange(num_deps); unsigned pick = m_rng.randrange(num_deps);
for (auto tx : top_chunk.chunk_setinfo.transactions) { for (auto tx_idx : top_chunk_info.transactions) {
auto& tx_data = m_tx_data[tx]; auto& tx_data = m_tx_data[tx_idx];
auto intersect = tx_data.children & bottom_chunk.chunk_setinfo.transactions; auto intersect = tx_data.children & bottom_chunk_info.transactions;
auto count = intersect.Count(); auto count = intersect.Count();
if (pick < count) { if (pick < count) {
for (auto dep : tx_data.child_deps) { for (auto dep : tx_data.child_deps) {
auto& dep_data = m_dep_data[dep]; auto& dep_data = m_dep_data[dep];
if (bottom_chunk.chunk_setinfo.transactions[dep_data.child]) { if (bottom_chunk_info.transactions[dep_data.child]) {
if (pick == 0) return Activate(dep); if (pick == 0) return Activate(dep);
--pick; --pick;
} }
@@ -871,17 +882,17 @@ private:
pick -= count; pick -= count;
} }
Assume(false); Assume(false);
return TxIdx(-1); return INVALID_SET_IDX;
} }
/** Perform an upward or downward merge step, on the specified chunk representative. Returns /** Perform an upward or downward merge step, on the specified chunk. Returns the merged chunk,
* the representative of the merged chunk, or TxIdx(-1) if no merge took place. */ * or INVALID_SET_IDX if no merge took place. */
template<bool DownWard> template<bool DownWard>
TxIdx MergeStep(TxIdx chunk_idx) noexcept SetIdx MergeStep(SetIdx chunk_idx) noexcept
{ {
/** Information about the chunk that tx_idx is currently in. */ /** Information about the chunk. */
auto& chunk_data = m_tx_data[chunk_idx]; auto& chunk_info = m_set_info[chunk_idx];
SetType chunk_txn = chunk_data.chunk_setinfo.transactions; SetType chunk_txn = chunk_info.transactions;
// Iterate over all transactions in the chunk, figuring out which other chunk each // Iterate over all transactions in the chunk, figuring out which other chunk each
// depends on, but only testing each other chunk once. For those depended-on chunks, // depends on, but only testing each other chunk once. For those depended-on chunks,
// remember the highest-feerate (if DownWard) or lowest-feerate (if !DownWard) one. // remember the highest-feerate (if DownWard) or lowest-feerate (if !DownWard) one.
@@ -894,14 +905,14 @@ private:
/** The minimum feerate (if downward) or maximum feerate (if upward) to consider when /** The minimum feerate (if downward) or maximum feerate (if upward) to consider when
* looking for candidate chunks to merge with. Initially, this is the original chunk's * looking for candidate chunks to merge with. Initially, this is the original chunk's
* feerate, but is updated to be the current best candidate whenever one is found. */ * feerate, but is updated to be the current best candidate whenever one is found. */
FeeFrac best_other_chunk_feerate = chunk_data.chunk_setinfo.feerate; FeeFrac best_other_chunk_feerate = chunk_info.feerate;
/** The representative for the best candidate chunk to merge with. -1 if none. */ /** The chunk index for the best candidate chunk to merge with. INVALID_SET_IDX if none. */
TxIdx best_other_chunk_idx = TxIdx(-1); SetIdx best_other_chunk_idx = INVALID_SET_IDX;
/** We generate random tiebreak values to pick between equal-feerate candidate chunks. /** We generate random tiebreak values to pick between equal-feerate candidate chunks.
* This variable stores the tiebreak of the current best candidate. */ * This variable stores the tiebreak of the current best candidate. */
uint64_t best_other_chunk_tiebreak{0}; uint64_t best_other_chunk_tiebreak{0};
for (auto tx : chunk_txn) { for (auto tx_idx : chunk_txn) {
auto& tx_data = m_tx_data[tx]; auto& tx_data = m_tx_data[tx_idx];
/** The transactions reached by following dependencies from tx that have not been /** The transactions reached by following dependencies from tx that have not been
* explored before. */ * explored before. */
auto newly_reached = (DownWard ? tx_data.children : tx_data.parents) - explored; auto newly_reached = (DownWard ? tx_data.children : tx_data.parents) - explored;
@@ -909,28 +920,28 @@ private:
while (newly_reached.Any()) { while (newly_reached.Any()) {
// Find a chunk inside newly_reached, and remove it from newly_reached. // Find a chunk inside newly_reached, and remove it from newly_reached.
auto reached_chunk_idx = m_tx_data[newly_reached.First()].chunk_idx; auto reached_chunk_idx = m_tx_data[newly_reached.First()].chunk_idx;
auto& reached_chunk = m_tx_data[reached_chunk_idx].chunk_setinfo; auto& reached_chunk_info = m_set_info[reached_chunk_idx];
newly_reached -= reached_chunk.transactions; newly_reached -= reached_chunk_info.transactions;
// See if it has an acceptable feerate. // See if it has an acceptable feerate.
auto cmp = DownWard ? FeeRateCompare(best_other_chunk_feerate, reached_chunk.feerate) auto cmp = DownWard ? FeeRateCompare(best_other_chunk_feerate, reached_chunk_info.feerate)
: FeeRateCompare(reached_chunk.feerate, best_other_chunk_feerate); : FeeRateCompare(reached_chunk_info.feerate, best_other_chunk_feerate);
if (cmp > 0) continue; if (cmp > 0) continue;
uint64_t tiebreak = m_rng.rand64(); uint64_t tiebreak = m_rng.rand64();
if (cmp < 0 || tiebreak >= best_other_chunk_tiebreak) { if (cmp < 0 || tiebreak >= best_other_chunk_tiebreak) {
best_other_chunk_feerate = reached_chunk.feerate; best_other_chunk_feerate = reached_chunk_info.feerate;
best_other_chunk_idx = reached_chunk_idx; best_other_chunk_idx = reached_chunk_idx;
best_other_chunk_tiebreak = tiebreak; best_other_chunk_tiebreak = tiebreak;
} }
} }
} }
// Stop if there are no candidate chunks to merge with. // Stop if there are no candidate chunks to merge with.
if (best_other_chunk_idx == TxIdx(-1)) return TxIdx(-1); if (best_other_chunk_idx == INVALID_SET_IDX) return INVALID_SET_IDX;
if constexpr (DownWard) { if constexpr (DownWard) {
chunk_idx = MergeChunks(chunk_idx, best_other_chunk_idx); chunk_idx = MergeChunks(chunk_idx, best_other_chunk_idx);
} else { } else {
chunk_idx = MergeChunks(best_other_chunk_idx, chunk_idx); chunk_idx = MergeChunks(best_other_chunk_idx, chunk_idx);
} }
Assume(chunk_idx != TxIdx(-1)); Assume(chunk_idx != INVALID_SET_IDX);
return chunk_idx; return chunk_idx;
} }
@@ -941,9 +952,9 @@ private:
{ {
auto chunk_idx = m_tx_data[tx_idx].chunk_idx; auto chunk_idx = m_tx_data[tx_idx].chunk_idx;
while (true) { while (true) {
auto merged_idx = MergeStep<DownWard>(chunk_idx); auto merged_chunk_idx = MergeStep<DownWard>(chunk_idx);
if (merged_idx == TxIdx(-1)) break; if (merged_chunk_idx == INVALID_SET_IDX) break;
chunk_idx = merged_idx; chunk_idx = merged_chunk_idx;
} }
// Add the chunk to the queue of improvable chunks. // Add the chunk to the queue of improvable chunks.
m_suboptimal_chunks.push_back(chunk_idx); m_suboptimal_chunks.push_back(chunk_idx);
@@ -980,6 +991,9 @@ public:
m_transaction_idxs = depgraph.Positions(); m_transaction_idxs = depgraph.Positions();
auto num_transactions = m_transaction_idxs.Count(); auto num_transactions = m_transaction_idxs.Count();
m_tx_data.resize(depgraph.PositionRange()); m_tx_data.resize(depgraph.PositionRange());
m_set_info.resize(num_transactions);
size_t num_chunks = 0;
// Reserve the maximum number of (reserved) dependencies the cluster can have, so // Reserve the maximum number of (reserved) dependencies the cluster can have, so
// m_dep_data won't need any reallocations during construction. For a cluster with N // m_dep_data won't need any reallocations during construction. For a cluster with N
// transactions, the worst case consists of two sets of transactions, the parents and the // transactions, the worst case consists of two sets of transactions, the parents and the
@@ -988,29 +1002,30 @@ public:
// and the other can be (N - 1)/2, meaning (N^2 - 1)/4 dependencies. Because N^2 is odd in // and the other can be (N - 1)/2, meaning (N^2 - 1)/4 dependencies. Because N^2 is odd in
// this case, N^2/4 (with rounding-down division) is the correct value in both cases. // this case, N^2/4 (with rounding-down division) is the correct value in both cases.
m_dep_data.reserve((num_transactions * num_transactions) / 4); m_dep_data.reserve((num_transactions * num_transactions) / 4);
for (auto tx : m_transaction_idxs) { for (auto tx_idx : m_transaction_idxs) {
// Fill in transaction data. // Fill in transaction data.
auto& tx_data = m_tx_data[tx]; auto& tx_data = m_tx_data[tx_idx];
tx_data.chunk_idx = tx; tx_data.parents = depgraph.GetReducedParents(tx_idx);
tx_data.chunk_setinfo.transactions = SetType::Singleton(tx); // Create a singleton chunk for it.
tx_data.chunk_setinfo.feerate = depgraph.FeeRate(tx); tx_data.chunk_idx = num_chunks;
m_set_info[num_chunks++] = SetInfo(depgraph, tx_idx);
// Add its dependencies. // Add its dependencies.
SetType parents = depgraph.GetReducedParents(tx); for (auto parent_idx : tx_data.parents) {
for (auto par : parents) { auto& par_tx_data = m_tx_data[parent_idx];
auto& par_tx_data = m_tx_data[par];
auto dep_idx = m_dep_data.size(); auto dep_idx = m_dep_data.size();
// Construct new dependency. // Construct new dependency.
auto& dep = m_dep_data.emplace_back(); auto& dep = m_dep_data.emplace_back();
dep.active = false; dep.active = false;
dep.parent = par; dep.parent = parent_idx;
dep.child = tx; dep.child = tx_idx;
// Add it as parent of the child.
tx_data.parents.Set(par);
// Add it as child of the parent. // Add it as child of the parent.
par_tx_data.child_deps.push_back(dep_idx); par_tx_data.child_deps.push_back(dep_idx);
par_tx_data.children.Set(tx); par_tx_data.children.Set(tx_idx);
} }
} }
Assume(num_chunks == num_transactions);
// Mark all chunk sets as chunks.
m_chunk_idxs = SetType::Fill(num_chunks);
} }
/** Load an existing linearization. Must be called immediately after constructor. The result is /** Load an existing linearization. Must be called immediately after constructor. The result is
@@ -1019,12 +1034,12 @@ public:
void LoadLinearization(std::span<const DepGraphIndex> old_linearization) noexcept void LoadLinearization(std::span<const DepGraphIndex> old_linearization) noexcept
{ {
// Add transactions one by one, in order of existing linearization. // Add transactions one by one, in order of existing linearization.
for (DepGraphIndex tx : old_linearization) { for (DepGraphIndex tx_idx : old_linearization) {
auto chunk_idx = m_tx_data[tx].chunk_idx; auto chunk_idx = m_tx_data[tx_idx].chunk_idx;
// Merge the chunk upwards, as long as merging succeeds. // Merge the chunk upwards, as long as merging succeeds.
while (true) { while (true) {
chunk_idx = MergeStep<false>(chunk_idx); chunk_idx = MergeStep<false>(chunk_idx);
if (chunk_idx == TxIdx(-1)) break; if (chunk_idx == INVALID_SET_IDX) break;
} }
} }
} }
@@ -1033,38 +1048,34 @@ public:
void MakeTopological() noexcept void MakeTopological() noexcept
{ {
Assume(m_suboptimal_chunks.empty()); Assume(m_suboptimal_chunks.empty());
for (auto tx : m_transaction_idxs) { for (auto chunk_idx : m_chunk_idxs) {
auto& tx_data = m_tx_data[tx]; m_suboptimal_chunks.emplace_back(chunk_idx);
if (tx_data.chunk_idx == tx) { // Randomize the initial order of suboptimal chunks in the queue.
m_suboptimal_chunks.emplace_back(tx); SetIdx j = m_rng.randrange<SetIdx>(m_suboptimal_chunks.size());
// Randomize the initial order of suboptimal chunks in the queue. if (j != m_suboptimal_chunks.size() - 1) {
TxIdx j = m_rng.randrange<TxIdx>(m_suboptimal_chunks.size()); std::swap(m_suboptimal_chunks.back(), m_suboptimal_chunks[j]);
if (j != m_suboptimal_chunks.size() - 1) {
std::swap(m_suboptimal_chunks.back(), m_suboptimal_chunks[j]);
}
} }
} }
while (!m_suboptimal_chunks.empty()) { while (!m_suboptimal_chunks.empty()) {
// Pop an entry from the potentially-suboptimal chunk queue. // Pop an entry from the potentially-suboptimal chunk queue.
TxIdx chunk = m_suboptimal_chunks.front(); SetIdx chunk_idx = m_suboptimal_chunks.front();
m_suboptimal_chunks.pop_front(); m_suboptimal_chunks.pop_front();
auto& chunk_data = m_tx_data[chunk]; // If what was popped is not currently a chunk, continue. This may
// If what was popped is not currently a chunk representative, continue. This may
// happen when it was merged with something else since being added. // happen when it was merged with something else since being added.
if (chunk_data.chunk_idx != chunk) continue; if (!m_chunk_idxs[chunk_idx]) continue;
int flip = m_rng.randbool(); int flip = m_rng.randbool();
for (int i = 0; i < 2; ++i) { for (int i = 0; i < 2; ++i) {
if (i ^ flip) { if (i ^ flip) {
// Attempt to merge the chunk upwards. // Attempt to merge the chunk upwards.
auto result_up = MergeStep<false>(chunk); auto result_up = MergeStep<false>(chunk_idx);
if (result_up != TxIdx(-1)) { if (result_up != INVALID_SET_IDX) {
m_suboptimal_chunks.push_back(result_up); m_suboptimal_chunks.push_back(result_up);
break; break;
} }
} else { } else {
// Attempt to merge the chunk downwards. // Attempt to merge the chunk downwards.
auto result_down = MergeStep<true>(chunk); auto result_down = MergeStep<true>(chunk_idx);
if (result_down != TxIdx(-1)) { if (result_down != INVALID_SET_IDX) {
m_suboptimal_chunks.push_back(result_down); m_suboptimal_chunks.push_back(result_down);
break; break;
} }
@@ -1078,15 +1089,12 @@ public:
{ {
Assume(m_suboptimal_chunks.empty()); Assume(m_suboptimal_chunks.empty());
// Mark chunks suboptimal. // Mark chunks suboptimal.
for (auto tx : m_transaction_idxs) { for (auto chunk_idx : m_chunk_idxs) {
auto& tx_data = m_tx_data[tx]; m_suboptimal_chunks.push_back(chunk_idx);
if (tx_data.chunk_idx == tx) { // Randomize the initial order of suboptimal chunks in the queue.
m_suboptimal_chunks.push_back(tx); SetIdx j = m_rng.randrange<SetIdx>(m_suboptimal_chunks.size());
// Randomize the initial order of suboptimal chunks in the queue. if (j != m_suboptimal_chunks.size() - 1) {
TxIdx j = m_rng.randrange<TxIdx>(m_suboptimal_chunks.size()); std::swap(m_suboptimal_chunks.back(), m_suboptimal_chunks[j]);
if (j != m_suboptimal_chunks.size() - 1) {
std::swap(m_suboptimal_chunks.back(), m_suboptimal_chunks[j]);
}
} }
} }
} }
@@ -1096,27 +1104,28 @@ public:
{ {
while (!m_suboptimal_chunks.empty()) { while (!m_suboptimal_chunks.empty()) {
// Pop an entry from the potentially-suboptimal chunk queue. // Pop an entry from the potentially-suboptimal chunk queue.
TxIdx chunk = m_suboptimal_chunks.front(); SetIdx chunk_idx = m_suboptimal_chunks.front();
m_suboptimal_chunks.pop_front(); m_suboptimal_chunks.pop_front();
auto& chunk_data = m_tx_data[chunk]; auto& chunk_info = m_set_info[chunk_idx];
// If what was popped is not currently a chunk representative, continue. This may // If what was popped is not currently a chunk, continue. This may
// happen when a split chunk merges in Improve() with one or more existing chunks that // happen when a split chunk merges in Improve() with one or more existing chunks that
// are themselves on the suboptimal queue already. // are themselves on the suboptimal queue already.
if (chunk_data.chunk_idx != chunk) continue; if (!m_chunk_idxs[chunk_idx]) continue;
// Remember the best dependency seen so far. // Remember the best dependency seen so far.
DepIdx candidate_dep = DepIdx(-1); DepIdx candidate_dep = DepIdx(-1);
uint64_t candidate_tiebreak = 0; uint64_t candidate_tiebreak = 0;
// Iterate over all transactions. // Iterate over all transactions.
for (auto tx : chunk_data.chunk_setinfo.transactions) { for (auto tx_idx : chunk_info.transactions) {
const auto& tx_data = m_tx_data[tx]; const auto& tx_data = m_tx_data[tx_idx];
// Iterate over all active child dependencies of the transaction. // Iterate over all active child dependencies of the transaction.
const auto children = std::span{tx_data.child_deps}; const auto children = std::span{tx_data.child_deps};
for (DepIdx dep_idx : children) { for (DepIdx dep_idx : children) {
const auto& dep_data = m_dep_data[dep_idx]; const auto& dep_data = m_dep_data[dep_idx];
if (!dep_data.active) continue; if (!dep_data.active) continue;
auto& dep_top_info = m_set_info[dep_data.dep_top_idx];
// Skip if this dependency is ineligible (the top chunk that would be created // Skip if this dependency is ineligible (the top chunk that would be created
// does not have higher feerate than the chunk it is currently part of). // does not have higher feerate than the chunk it is currently part of).
auto cmp = FeeRateCompare(dep_data.top_setinfo.feerate, chunk_data.chunk_setinfo.feerate); auto cmp = FeeRateCompare(dep_top_info.feerate, chunk_info.feerate);
if (cmp <= 0) continue; if (cmp <= 0) continue;
// Generate a random tiebreak for this dependency, and reject it if its tiebreak // Generate a random tiebreak for this dependency, and reject it if its tiebreak
// is worse than the best so far. This means that among all eligible // is worse than the best so far. This means that among all eligible
@@ -1147,16 +1156,13 @@ public:
m_nonminimal_chunks.reserve(m_transaction_idxs.Count()); m_nonminimal_chunks.reserve(m_transaction_idxs.Count());
// Gather all chunks, and for each, add it with a random pivot in it, and a random initial // Gather all chunks, and for each, add it with a random pivot in it, and a random initial
// direction, to m_nonminimal_chunks. // direction, to m_nonminimal_chunks.
for (auto tx : m_transaction_idxs) { for (auto chunk_idx : m_chunk_idxs) {
auto& tx_data = m_tx_data[tx]; TxIdx pivot_idx = PickRandomTx(m_set_info[chunk_idx].transactions);
if (tx_data.chunk_idx == tx) { m_nonminimal_chunks.emplace_back(chunk_idx, pivot_idx, m_rng.randbits<1>());
TxIdx pivot_idx = PickRandomTx(tx_data.chunk_setinfo.transactions); // Randomize the initial order of nonminimal chunks in the queue.
m_nonminimal_chunks.emplace_back(tx, pivot_idx, m_rng.randbits<1>()); SetIdx j = m_rng.randrange<SetIdx>(m_nonminimal_chunks.size());
// Randomize the initial order of nonminimal chunks in the queue. if (j != m_nonminimal_chunks.size() - 1) {
TxIdx j = m_rng.randrange<TxIdx>(m_nonminimal_chunks.size()); std::swap(m_nonminimal_chunks.back(), m_nonminimal_chunks[j]);
if (j != m_nonminimal_chunks.size() - 1) {
std::swap(m_nonminimal_chunks.back(), m_nonminimal_chunks[j]);
}
} }
} }
} }
@@ -1169,8 +1175,7 @@ public:
// Pop an entry from the potentially-non-minimal chunk queue. // Pop an entry from the potentially-non-minimal chunk queue.
auto [chunk_idx, pivot_idx, flags] = m_nonminimal_chunks.front(); auto [chunk_idx, pivot_idx, flags] = m_nonminimal_chunks.front();
m_nonminimal_chunks.pop_front(); m_nonminimal_chunks.pop_front();
auto& chunk_data = m_tx_data[chunk_idx]; auto& chunk_info = m_set_info[chunk_idx];
Assume(chunk_data.chunk_idx == chunk_idx);
/** Whether to move the pivot down rather than up. */ /** Whether to move the pivot down rather than up. */
bool move_pivot_down = flags & 1; bool move_pivot_down = flags & 1;
/** Whether this is already the second stage. */ /** Whether this is already the second stage. */
@@ -1182,20 +1187,21 @@ public:
uint64_t candidate_tiebreak{0}; uint64_t candidate_tiebreak{0};
bool have_any = false; bool have_any = false;
// Iterate over all transactions. // Iterate over all transactions.
for (auto tx_idx : chunk_data.chunk_setinfo.transactions) { for (auto tx_idx : chunk_info.transactions) {
const auto& tx_data = m_tx_data[tx_idx]; const auto& tx_data = m_tx_data[tx_idx];
// Iterate over all active child dependencies of the transaction. // Iterate over all active child dependencies of the transaction.
for (auto dep_idx : tx_data.child_deps) { for (auto dep_idx : tx_data.child_deps) {
auto& dep_data = m_dep_data[dep_idx]; auto& dep_data = m_dep_data[dep_idx];
// Skip inactive child dependencies. // Skip inactive child dependencies.
if (!dep_data.active) continue; if (!dep_data.active) continue;
const auto& dep_top_info = m_set_info[dep_data.dep_top_idx];
// Skip if this dependency does not have equal top and bottom set feerates. Note // Skip if this dependency does not have equal top and bottom set feerates. Note
// that the top cannot have higher feerate than the bottom, or OptimizeSteps would // that the top cannot have higher feerate than the bottom, or OptimizeSteps would
// have dealt with it. // have dealt with it.
if (dep_data.top_setinfo.feerate << chunk_data.chunk_setinfo.feerate) continue; if (dep_top_info.feerate << chunk_info.feerate) continue;
have_any = true; have_any = true;
// Skip if this dependency does not have pivot in the right place. // Skip if this dependency does not have pivot in the right place.
if (move_pivot_down == dep_data.top_setinfo.transactions[pivot_idx]) continue; if (move_pivot_down == dep_top_info.transactions[pivot_idx]) continue;
// Remember this as our chosen dependency if it has a better tiebreak. // Remember this as our chosen dependency if it has a better tiebreak.
uint64_t tiebreak = m_rng.rand64() | 1; uint64_t tiebreak = m_rng.rand64() | 1;
if (tiebreak > candidate_tiebreak) { if (tiebreak > candidate_tiebreak) {
@@ -1223,7 +1229,7 @@ public:
// Try to activate a dependency between the new bottom and the new top (opposite from the // Try to activate a dependency between the new bottom and the new top (opposite from the
// dependency that was just deactivated). // dependency that was just deactivated).
auto merged_chunk_idx = MergeChunks(child_chunk_idx, parent_chunk_idx); auto merged_chunk_idx = MergeChunks(child_chunk_idx, parent_chunk_idx);
if (merged_chunk_idx != TxIdx(-1)) { if (merged_chunk_idx != INVALID_SET_IDX) {
// A self-merge happened. // A self-merge happened.
// Re-insert the chunk into the queue, in the same direction. Note that the chunk_idx // Re-insert the chunk into the queue, in the same direction. Note that the chunk_idx
// will have changed. // will have changed.
@@ -1237,11 +1243,11 @@ public:
// possible already. The new chunk without the current pivot gets a new randomly-chosen // possible already. The new chunk without the current pivot gets a new randomly-chosen
// one. // one.
if (move_pivot_down) { if (move_pivot_down) {
auto parent_pivot_idx = PickRandomTx(m_tx_data[parent_chunk_idx].chunk_setinfo.transactions); auto parent_pivot_idx = PickRandomTx(m_set_info[parent_chunk_idx].transactions);
m_nonminimal_chunks.emplace_back(parent_chunk_idx, parent_pivot_idx, m_rng.randbits<1>()); m_nonminimal_chunks.emplace_back(parent_chunk_idx, parent_pivot_idx, m_rng.randbits<1>());
m_nonminimal_chunks.emplace_back(child_chunk_idx, pivot_idx, flags); m_nonminimal_chunks.emplace_back(child_chunk_idx, pivot_idx, flags);
} else { } else {
auto child_pivot_idx = PickRandomTx(m_tx_data[child_chunk_idx].chunk_setinfo.transactions); auto child_pivot_idx = PickRandomTx(m_set_info[child_chunk_idx].transactions);
m_nonminimal_chunks.emplace_back(parent_chunk_idx, pivot_idx, flags); m_nonminimal_chunks.emplace_back(parent_chunk_idx, pivot_idx, flags);
m_nonminimal_chunks.emplace_back(child_chunk_idx, child_pivot_idx, m_rng.randbits<1>()); m_nonminimal_chunks.emplace_back(child_chunk_idx, child_pivot_idx, m_rng.randbits<1>());
} }
@@ -1272,19 +1278,17 @@ public:
{ {
/** The output linearization. */ /** The output linearization. */
std::vector<DepGraphIndex> ret; std::vector<DepGraphIndex> ret;
ret.reserve(m_transaction_idxs.Count()); ret.reserve(m_set_info.size());
/** A heap with all chunks (by representative) that can currently be included, sorted by /** A heap with all chunks (by set index) that can currently be included, sorted by
* chunk feerate (high to low), chunk size (small to large), and by least maximum element * chunk feerate (high to low), chunk size (small to large), and by least maximum element
* according to the fallback order (which is the second pair element). */ * according to the fallback order (which is the second pair element). */
std::vector<std::pair<TxIdx, TxIdx>> ready_chunks; std::vector<std::pair<SetIdx, TxIdx>> ready_chunks;
/** For every chunk, indexed by representative, the number of unmet dependencies the chunk has on /** For every chunk, indexed by SetIdx, the number of unmet dependencies the chunk has on
* other chunks (not including dependencies within the chunk itself). */ * other chunks (not including dependencies within the chunk itself). */
std::vector<TxIdx> chunk_deps(m_tx_data.size(), 0); std::vector<TxIdx> chunk_deps(m_set_info.size(), 0);
/** For every transaction, indexed by TxIdx, the number of unmet dependencies the /** For every transaction, indexed by TxIdx, the number of unmet dependencies the
* transaction has. */ * transaction has. */
std::vector<TxIdx> tx_deps(m_tx_data.size(), 0); std::vector<TxIdx> tx_deps(m_tx_data.size(), 0);
/** The set of all chunk representatives. */
SetType chunk_idxs;
/** A heap with all transactions within the current chunk that can be included, sorted by /** A heap with all transactions within the current chunk that can be included, sorted by
* tx feerate (high to low), tx size (small to large), and fallback order. */ * tx feerate (high to low), tx size (small to large), and fallback order. */
std::vector<TxIdx> ready_tx; std::vector<TxIdx> ready_tx;
@@ -1293,13 +1297,12 @@ public:
const auto& chl_data = m_tx_data[chl_idx]; const auto& chl_data = m_tx_data[chl_idx];
tx_deps[chl_idx] = chl_data.parents.Count(); tx_deps[chl_idx] = chl_data.parents.Count();
auto chl_chunk_idx = chl_data.chunk_idx; auto chl_chunk_idx = chl_data.chunk_idx;
chunk_idxs.Set(chl_chunk_idx); auto& chl_chunk_info = m_set_info[chl_chunk_idx];
const auto& chl_chunk_txn = m_tx_data[chl_chunk_idx].chunk_setinfo.transactions; chunk_deps[chl_chunk_idx] += (chl_data.parents - chl_chunk_info.transactions).Count();
chunk_deps[chl_chunk_idx] += (chl_data.parents - chl_chunk_txn).Count();
} }
/** Function to compute the highest element of a chunk, by fallback_order. */ /** Function to compute the highest element of a chunk, by fallback_order. */
auto max_fallback_fn = [&](TxIdx chunk_idx) noexcept { auto max_fallback_fn = [&](SetIdx chunk_idx) noexcept {
auto& chunk = m_tx_data[chunk_idx].chunk_setinfo.transactions; auto& chunk = m_set_info[chunk_idx].transactions;
auto it = chunk.begin(); auto it = chunk.begin();
DepGraphIndex ret = *it; DepGraphIndex ret = *it;
++it; ++it;
@@ -1337,8 +1340,8 @@ public:
// Bail out for identical chunks. // Bail out for identical chunks.
if (a.first == b.first) return false; if (a.first == b.first) return false;
// First sort by increasing chunk feerate. // First sort by increasing chunk feerate.
auto& chunk_feerate_a = m_tx_data[a.first].chunk_setinfo.feerate; auto& chunk_feerate_a = m_set_info[a.first].feerate;
auto& chunk_feerate_b = m_tx_data[b.first].chunk_setinfo.feerate; auto& chunk_feerate_b = m_set_info[b.first].feerate;
auto feerate_cmp = FeeRateCompare(chunk_feerate_a, chunk_feerate_b); auto feerate_cmp = FeeRateCompare(chunk_feerate_a, chunk_feerate_b);
if (feerate_cmp != 0) return feerate_cmp < 0; if (feerate_cmp != 0) return feerate_cmp < 0;
// Then by decreasing chunk size. // Then by decreasing chunk size.
@@ -1353,7 +1356,7 @@ public:
return a.second < b.second; return a.second < b.second;
}; };
// Construct a heap with all chunks that have no out-of-chunk dependencies. // Construct a heap with all chunks that have no out-of-chunk dependencies.
for (TxIdx chunk_idx : chunk_idxs) { for (SetIdx chunk_idx : m_chunk_idxs) {
if (chunk_deps[chunk_idx] == 0) { if (chunk_deps[chunk_idx] == 0) {
ready_chunks.emplace_back(chunk_idx, max_fallback_fn(chunk_idx)); ready_chunks.emplace_back(chunk_idx, max_fallback_fn(chunk_idx));
} }
@@ -1364,9 +1367,8 @@ public:
auto [chunk_idx, _rnd] = ready_chunks.front(); auto [chunk_idx, _rnd] = ready_chunks.front();
std::pop_heap(ready_chunks.begin(), ready_chunks.end(), chunk_cmp_fn); std::pop_heap(ready_chunks.begin(), ready_chunks.end(), chunk_cmp_fn);
ready_chunks.pop_back(); ready_chunks.pop_back();
Assume(m_tx_data[chunk_idx].chunk_idx == chunk_idx);
Assume(chunk_deps[chunk_idx] == 0); Assume(chunk_deps[chunk_idx] == 0);
const auto& chunk_txn = m_tx_data[chunk_idx].chunk_setinfo.transactions; const auto& chunk_txn = m_set_info[chunk_idx].transactions;
// Build heap of all includable transactions in chunk. // Build heap of all includable transactions in chunk.
Assume(ready_tx.empty()); Assume(ready_tx.empty());
for (TxIdx tx_idx : chunk_txn) { for (TxIdx tx_idx : chunk_txn) {
@@ -1406,7 +1408,7 @@ public:
} }
} }
} }
Assume(ret.size() == m_transaction_idxs.Count()); Assume(ret.size() == m_set_info.size());
return ret; return ret;
} }
@@ -1426,10 +1428,8 @@ public:
std::vector<FeeFrac> GetDiagram() const noexcept std::vector<FeeFrac> GetDiagram() const noexcept
{ {
std::vector<FeeFrac> ret; std::vector<FeeFrac> ret;
for (auto tx : m_transaction_idxs) { for (auto chunk_idx : m_chunk_idxs) {
if (m_tx_data[tx].chunk_idx == tx) { ret.push_back(m_set_info[chunk_idx].feerate);
ret.push_back(m_tx_data[tx].chunk_setinfo.feerate);
}
} }
std::sort(ret.begin(), ret.end(), std::greater{}); std::sort(ret.begin(), ret.end(), std::greater{});
return ret; return ret;
@@ -1473,56 +1473,52 @@ public:
// Verify the chunks against the list of active dependencies // Verify the chunks against the list of active dependencies
// //
SetType chunk_cover; SetType chunk_cover;
for (auto tx_idx: m_depgraph.Positions()) { for (auto chunk_idx : m_chunk_idxs) {
// Only process chunks for now. const auto& chunk_info = m_set_info[chunk_idx];
if (m_tx_data[tx_idx].chunk_idx == tx_idx) { // Verify that transactions in the chunk point back to it. This guarantees
const auto& chunk_data = m_tx_data[tx_idx]; // that chunks are non-overlapping.
// Verify that transactions in the chunk point back to it. This guarantees for (auto chunk_tx : chunk_info.transactions) {
// that chunks are non-overlapping. assert(m_tx_data[chunk_tx].chunk_idx == chunk_idx);
for (auto chunk_tx : chunk_data.chunk_setinfo.transactions) {
assert(m_tx_data[chunk_tx].chunk_idx == tx_idx);
}
assert(!chunk_cover.Overlaps(chunk_data.chunk_setinfo.transactions));
chunk_cover |= chunk_data.chunk_setinfo.transactions;
// Verify the chunk's transaction set: it must contain the representative, and for
// every active dependency, if it contains the parent or child, it must contain
// both. It must have exactly N-1 active dependencies in it, guaranteeing it is
// acyclic.
SetType expected_chunk = SetType::Singleton(tx_idx);
while (true) {
auto old = expected_chunk;
size_t active_dep_count{0};
for (const auto& [par, chl, _dep] : active_dependencies) {
if (expected_chunk[par] || expected_chunk[chl]) {
expected_chunk.Set(par);
expected_chunk.Set(chl);
++active_dep_count;
}
}
if (old == expected_chunk) {
assert(expected_chunk.Count() == active_dep_count + 1);
break;
}
}
assert(chunk_data.chunk_setinfo.transactions == expected_chunk);
// Verify the chunk's feerate.
assert(chunk_data.chunk_setinfo.feerate ==
m_depgraph.FeeRate(chunk_data.chunk_setinfo.transactions));
} }
assert(!chunk_cover.Overlaps(chunk_info.transactions));
chunk_cover |= chunk_info.transactions;
// Verify the chunk's transaction set: start from an arbitrary chunk transaction,
// and for every active dependency, if it contains the parent or child, add the
// other. It must have exactly N-1 active dependencies in it, guaranteeing it is
// acyclic.
assert(chunk_info.transactions.Any());
SetType expected_chunk = SetType::Singleton(chunk_info.transactions.First());
while (true) {
auto old = expected_chunk;
size_t active_dep_count{0};
for (const auto& [par, chl, _dep] : active_dependencies) {
if (expected_chunk[par] || expected_chunk[chl]) {
expected_chunk.Set(par);
expected_chunk.Set(chl);
++active_dep_count;
}
}
if (old == expected_chunk) {
assert(expected_chunk.Count() == active_dep_count + 1);
break;
}
}
assert(chunk_info.transactions == expected_chunk);
// Verify the chunk's feerate.
assert(chunk_info.feerate == m_depgraph.FeeRate(chunk_info.transactions));
} }
// Verify that together, the chunks cover all transactions. // Verify that together, the chunks cover all transactions.
assert(chunk_cover == m_depgraph.Positions()); assert(chunk_cover == m_depgraph.Positions());
// //
// Verify other transaction data. // Verify transaction data.
// //
assert(m_transaction_idxs == m_depgraph.Positions()); assert(m_transaction_idxs == m_depgraph.Positions());
for (auto tx_idx : m_transaction_idxs) { for (auto tx_idx : m_transaction_idxs) {
const auto& tx_data = m_tx_data[tx_idx]; const auto& tx_data = m_tx_data[tx_idx];
// Verify it has a valid chunk representative, and that chunk includes this // Verify it has a valid chunk index, and that chunk includes this transaction.
// transaction. assert(m_chunk_idxs[tx_data.chunk_idx]);
assert(m_tx_data[tx_data.chunk_idx].chunk_idx == tx_data.chunk_idx); assert(m_set_info[tx_data.chunk_idx].transactions[tx_idx]);
assert(m_tx_data[tx_data.chunk_idx].chunk_setinfo.transactions[tx_idx]);
// Verify parents/children. // Verify parents/children.
assert(tx_data.parents == m_depgraph.GetReducedParents(tx_idx)); assert(tx_data.parents == m_depgraph.GetReducedParents(tx_idx));
assert(tx_data.children == m_depgraph.GetReducedChildren(tx_idx)); assert(tx_data.children == m_depgraph.GetReducedChildren(tx_idx));
@@ -1541,11 +1537,11 @@ public:
} }
// //
// Verify active dependencies' top_setinfo. // Verify active dependencies' top sets.
// //
for (const auto& [par_idx, chl_idx, dep_idx] : active_dependencies) { for (const auto& [par_idx, chl_idx, dep_idx] : active_dependencies) {
const auto& dep_data = m_dep_data[dep_idx]; const auto& dep_data = m_dep_data[dep_idx];
// Verify the top_info's transactions: it must contain the parent, and for every // Verify the top set's transactions: it must contain the parent, and for every
// active dependency, except dep_idx itself, if it contains the parent or child, it // active dependency, except dep_idx itself, if it contains the parent or child, it
// must contain both. It must have exactly N-1 active dependencies in it, guaranteeing // must contain both. It must have exactly N-1 active dependencies in it, guaranteeing
// it is acyclic. // it is acyclic.
@@ -1566,18 +1562,18 @@ public:
} }
} }
assert(!expected_top[chl_idx]); assert(!expected_top[chl_idx]);
assert(dep_data.top_setinfo.transactions == expected_top); auto& dep_top_info = m_set_info[dep_data.dep_top_idx];
// Verify the top_info's feerate. assert(dep_top_info.transactions == expected_top);
assert(dep_data.top_setinfo.feerate == // Verify the top set's feerate.
m_depgraph.FeeRate(dep_data.top_setinfo.transactions)); assert(dep_top_info.feerate == m_depgraph.FeeRate(dep_top_info.transactions));
} }
// //
// Verify m_suboptimal_chunks. // Verify m_suboptimal_chunks.
// //
for (size_t i = 0; i < m_suboptimal_chunks.size(); ++i) { for (size_t i = 0; i < m_suboptimal_chunks.size(); ++i) {
auto tx_idx = m_suboptimal_chunks[i]; auto chunk_idx = m_suboptimal_chunks[i];
assert(m_transaction_idxs[tx_idx]); assert(chunk_idx < m_set_info.size());
} }
// //
@@ -1586,12 +1582,11 @@ public:
SetType nonminimal_idxs; SetType nonminimal_idxs;
for (size_t i = 0; i < m_nonminimal_chunks.size(); ++i) { for (size_t i = 0; i < m_nonminimal_chunks.size(); ++i) {
auto [chunk_idx, pivot, flags] = m_nonminimal_chunks[i]; auto [chunk_idx, pivot, flags] = m_nonminimal_chunks[i];
assert(m_tx_data[chunk_idx].chunk_idx == chunk_idx);
assert(m_tx_data[pivot].chunk_idx == chunk_idx); assert(m_tx_data[pivot].chunk_idx == chunk_idx);
assert(!nonminimal_idxs[chunk_idx]); assert(!nonminimal_idxs[chunk_idx]);
nonminimal_idxs.Set(chunk_idx); nonminimal_idxs.Set(chunk_idx);
} }
assert(nonminimal_idxs.IsSubsetOf(m_transaction_idxs)); assert(nonminimal_idxs.IsSubsetOf(m_chunk_idxs));
} }
}; };
src/test/util/cluster_linearize.h
+6 -6
View File
@@ -404,12 +404,12 @@ inline uint64_t MaxOptimalLinearizationIters(DepGraphIndex cluster_count)
0, 0,
0, 4, 10, 34, 76, 156, 229, 380, 0, 4, 10, 34, 76, 156, 229, 380,
432, 607, 738, 896, 1037, 1366, 1464, 1711, 432, 607, 738, 896, 1037, 1366, 1464, 1711,
2060, 2542, 3068, 3116, 4029, 3949, 5324, 5402, 2060, 2542, 3068, 3116, 4029, 3467, 5324, 5402,
6481, 7161, 7441, 8329, 9307, 9353, 11104, 11269, 6481, 7161, 7441, 8329, 8843, 9353, 11104, 11269,
11791, 11981, 12413, 14513, 15331, 12397, 13581, 19665, 11791, 11981, 12413, 14259, 15331, 12397, 13581, 18569,
18737, 16581, 23217, 25542, 27123, 28913, 32969, 33951, 18737, 16581, 23217, 23271, 27350, 28591, 33636, 34486,
34414, 26227, 38792, 38045, 40814, 29622, 38732, 32122, 34414, 26227, 35570, 38045, 40814, 29622, 37793, 32122,
35915, 49823, 39722, 43765, 44002, 49716, 59417, 67035 35915, 49823, 39722, 43765, 42365, 53620, 59417, 67035
}; };
assert(cluster_count < std::size(ITERS)); assert(cluster_count < std::size(ITERS));
// Multiply the table number by two, to account for the fact that they are not absolutes. // Multiply the table number by two, to account for the fact that they are not absolutes.