Merge bitcoin/bitcoin#32579: p2p: Correct unrealistic headerssync unit test behavior

cc5dda1de3 headerssync: Make HeadersSyncState more flexible and move constants (Hodlinator)
8fd1c2893e test(headerssync): Test returning of pow_validated_headers behavior (Hodlinator)
7b00643ef5 test(headerssync): headers_sync_chainwork test improvements (Hodlinator)
04eeb9578c doc(test): Improve comments (Hodlinator)
fe896f8faa refactor(test): Store HeadersSyncState on the stack (Hodlinator)
f03686892a refactor(test): Break up headers_sync_state (Hodlinator)
e984618d0b refactor(headerssync): Process spans of headers (Hodlinator)
a4ac9915a9 refactor(headerssync): Extract test constants ahead of breakup into functions (Hodlinator)

Pull request description:

  ### Background

  As part of the release process we often run *contrib/devtools/headerssync-params.py* and increase the values of the constants `HEADER_COMMITMENT_PERIOD` and `REDOWNLOAD_BUFFER_SIZE` in *src/headerssync.cpp* as per *doc/release-process.md* (example: 11a2d3a63e). This helps fine tune the memory consumption per `HeadersSyncState`-instance in the face of malicious peers.

  (The `REDOWNLOAD_BUFFER_SIZE`/`HEADER_COMMITMENT_PERIOD` ratio determines how many Headers Sync commitment bits must match between PRESYNC & REDOWNLOAD phases before we start permanently storing headers from a peer. For more details see comments in *src/headerssync.h* and *contrib/devtools/headerssync-params.py*).

  ### Problem: Not feeding back headers until completing sync

  During v30 release process #33274 made `REDOWNLOAD_BUFFER_SIZE` exceed the `target_blocks` constant used to control the length of chains generated for testing Headers Sync (`15000`, *headers_sync_chainwork_tests.cpp*).

  The `HeadersSyncState::m_redownloaded_headers`-buffer now does not reach the `REDOWNLOAD_BUFFER_SIZE`-threshold during those unit tests. As a consequence `HeadersSyncState::PopHeadersReadyForAcceptance()` will not start feeding back headers until the PoW threshold has been met. While this will not cause the unit test to start failing on master, it means we have gone from testing behavior that resembles mainnet (way more than `REDOWNLOAD_BUFFER_SIZE` headers to reach the PoW limit), to behavior that is not possible/expected there.

  ### Solution

  Avoid testing this unrealistic condition of completing Headers Sync before reaching `REDOWNLOAD_BUFFER_SIZE` by making tests able to define their own values through the new `HeadersSyncParams` instead of having them hard-coded for all chains & tests.

  ### Commits

  * First 6 commits refactor and improve the unit tests in order to clarify latter changes.
  * We then add checks for the behavior around the `REDOWNLOAD_BUFFER_SIZE` threshold.
  * The main change: we extract the section from *headerssync.cpp* containing the constants to *kernel/chainparams.cpp*, making `HeadersSyncState` no longer hard-coded to mainnet.

  ### Notes

  This PR used to be called "headerssync: Preempt unrealistic unit test behavior".

ACKs for top commit:
  l0rinc:
    reACK cc5dda1de3
  marcofleon:
    code review ACK cc5dda1de3
  danielabrozzoni:
    reACK cc5dda1de3

Tree-SHA512: ccc824dcbbb8ad5ae98c3bf5808b38467aac0230739898a758c9b939eecd74f982df088fa0ba81cc1c1732f19a607b135a6e9577bb9fcf7f8570567ce92f66e6

contrib/devtools/README.md

@@ -137,7 +137,7 @@ BUILDDIR=$PWD/my-build-dir contrib/devtools/gen-manpages.py
 headerssync-params.py
 =====================
 
-A script to generate optimal parameters for the headerssync module (src/headerssync.cpp). It takes no command-line
+A script to generate optimal parameters for the headerssync module (stored in src/kernel/chainparams.cpp). It takes no command-line
 options, as all its configuration is set at the top of the file. It runs many times faster inside PyPy. Invocation:
 
 ```bash

contrib/devtools/headerssync-params.py

@@ -5,8 +5,8 @@
 
 """Script to find the optimal parameters for the headerssync module through simulation."""
 
-from math import log, exp, sqrt
 from datetime import datetime, timedelta
+from math import log, exp, sqrt
 import random
 
 # Parameters:
@@ -337,15 +337,15 @@ def analyze(when):
     attack_volume = NET_HEADER_SIZE * MINCHAINWORK_HEADERS
     # And report them.
     print()
-    print("Optimal configuration:")
+    print(f"Given current min chainwork headers of {MINCHAINWORK_HEADERS}, the optimal parameters for low")
+    print(f"memory usage on mainchain for release until {TIME:%Y-%m-%d} is:")
     print()
-    print("//! Store one header commitment per HEADER_COMMITMENT_PERIOD blocks.")
-    print(f"constexpr size_t HEADER_COMMITMENT_PERIOD{{{period}}};")
-    print()
-    print("//! Only feed headers to validation once this many headers on top have been")
-    print("//! received and validated against commitments.")
-    print(f"constexpr size_t REDOWNLOAD_BUFFER_SIZE{{{bufsize}}};"
+    print(f"        // Generated by headerssync-params.py on {datetime.today():%Y-%m-%d}.")
+    print( "        m_headers_sync_params = HeadersSyncParams{")
+    print(f"            .commitment_period = {period},")
+    print(f"            .redownload_buffer_size = {bufsize},"
           f" // {bufsize}/{period} = ~{bufsize/period:.1f} commitments")
+    print( "        };")
     print()
     print("Properties:")
     print(f"- Per-peer memory for mainchain sync: {mem_mainchain / 8192:.3f} KiB")