Merge bitcoin/bitcoin#34023: Optimized SFL cluster linearization

c2fcf25069 clusterlin: inline GetReachable into Deactivate (optimization) (Pieter Wuille)
d90f98ab4a clusterlin: inline UpdateChunk into (De)Activate (optimization) (Pieter Wuille)
b684f954bb clusterlin: unidirectional MakeTopological initially (optimization) (Pieter Wuille)
1daa600c1c clusterlin: track suboptimal chunks (optimization) (Pieter Wuille)
63b06d5523 clusterlin: keep track of active children (optimization) (Pieter Wuille)
ae16485aa9 clusterlin: special-case self-merges (optimization) (Pieter Wuille)
3221f1a074 clusterlin: make MergeSequence take SetIdx (simplification) (Pieter Wuille)
7194de3f7c clusterlin: precompute reachable sets (optimization) (Pieter Wuille)
6f898dbb8b clusterlin: simplify PickMergeCandidate (optimization) (Pieter Wuille)
dcf458ffb9 clusterlin: split up OptimizeStep (refactor) (Pieter Wuille)
cbd684a471 clusterlin: abstract out functions from MergeStep (refactor) (Pieter Wuille)
b75574a653 clusterlin: improve TxData::dep_top_idx type (optimization) (Pieter Wuille)
73cbd15d45 clusterlin: get rid of DepData (optimization) (Pieter Wuille)
7c6f63a8a9 clusterlin: pool SetInfos (preparation) (Pieter Wuille)
20e2f3e96d scripted-diff: rename _rep -> _idx in SFL (Pieter Wuille)
268fcb6a53 clusterlin: add more Assumes and sanity checks (tests) (Pieter Wuille)
d69c9f56ea clusterlin: count chunk deps without loop (optimization) (Pieter Wuille)
f66fa69ce0 clusterlin: split tx/chunk dep counting (preparation) (Pieter Wuille)
900e459778 clusterlin: avoid depgraph argument in SanityCheck (cleanup) (Pieter Wuille)
666b37970f clusterlin: fix type to count dependencies (Pieter Wuille)

Pull request description:

  Follow-up to #32545, part of #30289.

  This contains many of the optimizations that were originally part of #32545 itself.

  Here is a list of commits and the geometric average of the benchmark timings. Note that these aren't worst cases, but because of the nature of the optimizations here, I do expect them to apply roughly equally to all kinds of clusters. In other words, the relative improvement shown by these numbers should be representative:

  | commit title | ns per optimal linearization |
  |:--|--:|
  | clusterlin: split tx/chunk dep counting (preparation) | 24,760.30 |
  | clusterlin: count chunk deps without loop (optimization) | 24,677.64 |
  | scripted-diff: rename _rep -> _idx in SFL | 24,640.08 |
  | clusterlin: get rid of DepData, reuse sets (optimization) | 24,389.01 |
  | clusterlin: improve TxData::dep_top_idx type (optimization) | 22,578.58 |
  | clusterlin: abstract out functions from MergeStep (refactor) | 22,577.15 |
  | clusterlin: split up OptimizeStep (refactor) | 22,981.11 |
  | clusterlin: simplify PickMergeCandidate (optimization) | 22,018.63 |
  | clusterlin: precompute reachable sets (optimization) | 21,194.91 |
  | clusterlin: make MergeSequence take SetIdx (simplification) | 21,135.60 |
  | clusterlin: special-case self-merges (optimization) | 20,588.13 |
  | clusterlin: keep track of active children (optimization) | 13,911.22 |
  | clusterlin: track suboptimal chunks (optimization) | 13,629.42 |
  | clusterlin: unidirectional MakeTopological initially (optimization) | 12,796.57 |
  | clusterlin: inline UpdateChunk into (De)Activate (optimization) | 12,706.35 |
  | clusterlin: inline GetReachable into Deactivate (optimization) | 12,525.66 |

  And to show that they apply to all clusters roughly similarly:

  <img width="1239" height="640" alt="output(1)" src="https://github.com/user-attachments/assets/edd73937-3f87-4582-b2b9-eaed7e6ff352" />

ACKs for top commit:
  instagibbs:
    reACK  c2fcf25069
  ajtowns:
    reACK c2fcf25069

Tree-SHA512: e8920f7952a6681b4c1d70c864bd0e9784127ae4fd7c740be3e24a473f72e83544d2293066ed9b3e685b755febd6bbbc6c7da0c9b6ef3699b05eaa8d5bc073a0

src/test/fuzz/cluster_linearize.cpp

@@ -919,7 +919,7 @@ FUZZ_TARGET(clusterlin_sfl)
         if (rng.randbits(4) == 0) {
             // Perform sanity checks from time to time (too computationally expensive to do after
             // every step).
-            sfl.SanityCheck(depgraph);
+            sfl.SanityCheck();
         }
         auto diagram = sfl.GetDiagram();
         if (rng.randbits(4) == 0) {

src/test/util/cluster_linearize.h

@@ -402,14 +402,14 @@ inline uint64_t MaxOptimalLinearizationIters(DepGraphIndex cluster_count)
     // *some* reasonable cost bound, optimal linearizations are always found.
     static constexpr uint64_t ITERS[65] = {
         0,
-        0, 4, 10, 34, 76, 118, 184, 225,
+        0, 4, 10, 34, 76, 156, 229, 380,
-        320, 376, 464, 573, 830, 868, 1019, 1468,
+        441, 517, 678, 933, 1037, 1366, 1464, 1711,
-        1375, 1785, 1880, 1854, 2551, 2559, 4336, 4784,
+        2111, 2542, 3068, 3116, 4029, 3467, 5324, 5402,
-        5547, 5807, 6157, 6075, 6961, 7403, 7756, 8001,
+        6481, 7161, 7441, 8183, 8843, 9353, 11104, 11455,
-        8041, 7579, 8483, 10077, 9015, 9388, 9626, 12371,
+        11791, 12570, 13480, 14259, 14525, 12426, 14477, 20201,
-        12847, 12102, 15173, 15800, 20319, 22190, 23183, 24361,
+        18737, 16581, 23622, 28486, 30652, 33021, 32942, 32745,
-        24909, 19225, 27419, 23789, 25909, 21993, 25596, 24130,
+        34046, 26227, 34662, 38019, 40814, 31113, 41448, 33968,
-        26349, 31823, 31855, 31250, 32688, 34825, 41710, 45478
+        35024, 59207, 42872, 41277, 42365, 51833, 63410, 67035
     };
     assert(cluster_count < std::size(ITERS));
     // Multiply the table number by two, to account for the fact that they are not absolutes.