The backwards compatibility test was failing intermittently due to two
related timing issues in the test setup.
The issue was that Dave's `wait_graph_sync dave 3` was hanging
for up to 60 minutes. Dave's initial gossip sync with Charlie could
complete before Charlie had forwarded the alice-bob channel
announcement, leaving Dave stuck at 2 channels until lnd's historical
syncer fired at its default interval of 1 hour. After this 1-hour idle,
some routing state had become stale, causing the subsequent payment from
alice to dave to fail with FAILURE_REASON_NO_ROUTE.
This issues is now addressed by setting `--historicalsyncinterval=10s` on
all nodes. This causes nodes to periodically re-sync the full gossip
state from their peers every 10 seconds instead of every hour. Dave
therefore picks up any missed channel announcements and routing policies
within seconds, and alice's routing graph stays up-to-date throughout
the test.
Additionally, lnd debug logs from all containers are now collected
before teardown on failure and uploaded as a CI artifact, making future
failures easier to diagnose.
In this commit, we revamp the way our skip-labels worked to instead mark
certain stages as green, instead of skipping them. Skipped changes
didn't count towards the set of required checks, which slowed down PR
velocity.
Previously, when PRs were labeled with 'no-itest' or 'no-changelog',
the corresponding CI jobs would be completely skipped. This caused the
GitHub checks to show as skipped rather than successful, which could
be confusing and prevented certain merge rules from working properly.
This commit changes the behavior so that these jobs still run but
immediately report success when the special labels are detected. Each
affected job now starts with a label check step that sets a skip
flag, and all subsequent steps are conditionally executed based on
this flag. When skipped, the jobs add a notice to the GitHub step
summary explaining that tests were auto-passed due to the label.
The change affects five jobs: basic-integration-test, integration-test,
windows-integration-test, macos-integration-test, and milestone-check.
This commit adds a more descriptive docs about the caching process. In
addition, the `github.job` is now removed from the cache key
construction to allow jobs sharing the same cache. In addition, given we
now use `inputs.key-prefix`, the `github.job` is no longer relevant as
the `key-prefix` already creates a unique name for the cache.
We also update the lint and check commits job to use the same cache from
the unit test, given they are running in the same environment.
In this commit, a new `test_native_sql` build flag is defined. If this
build flag is used along with the `--use-native-sql` config option, then
the SQLStore implementation of the graphdb.V1Store will be initialised.
This is then used to run our itest suite against the new SQLStore graph
implementation.
NOTE that this only works for new nodes currently - the migration from
kv-to-sql is yet to be implemeneted.
In this commit, we implement the postgres and sqlite versions of the
NewTestDB function. We add the various build flags so that only one of
the three versions of this function can be active at a time.
We also introduce the SQLStore struct which is the SQL implementation of
the V1Store interface.
NOTE: it currently temporarily embeds the KVStore struct so that we can
implement the V1Store interface incrementally. For any method not
implemented, things will fall back to the KVStore. This is ONLY the
case for the time being while this struct is purely used in unit tests
only. Once all the methods have been implemented, the KVStore field will
be removed from the SQLStore struct.
Sometimes only the "Send coverage" step of a CI job will fail. This
commit turns this step into a "best effort" step instead so that it
does not block a CI job from passing.
It can, for example, often happen that a single job is re-run from the
GH UI, it then passes but the "Send coverage" step fails due to the
"Can't add a job to a build that is already closed." error meaning that
the only way to get the CI step to pass is to re-push and retrigger a
full CI run.
This is a tradeoff of disk space (and with that cache size) and
compilation speed. Because we're still running into disk full errors
with the full build cache, we remove it for the cross compile step.
Which means we'll do more work each time.
Since we now have a scripted check that makes sure the Golang version is
synced throughout all files, it is important that the main version in
the Makefile is changed, then all other versions can be detected by that
script.
ziggie