Commit Graph
70 Commits
Author SHA1 Message Date
dependabot[bot] ded9fb87d2 build(deps): bump golang.org/x/net from 0.4.0 to 0.7.0 in /kvdb
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.4.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-17 16:03:17 +00:00
Oliver Gugger 97f139b617 Merge pull request #7353 from bottlepay/bump-kvdb-etcd
kvdb: bump etcd to v3.5.7
2023-01-24 15:58:15 +01:00
Joost Jager 1725e5bcbe kvdb: bump etcd to v3.5.7 2023-01-24 13:52:20 +01:00
Elle Mouton 2b4de46c72 kvdb+docs: remove etcd test fixture cleanup calls
Remove the cleanup calls which no longer exist since `t.Cleanup` is now
called from within `NewEtcdTestFixture`
2023-01-24 09:23:50 +02:00
Elle Mouton 74b9c9ce9a kvdb: add sqlite 2023-01-23 09:14:08 +02:00
Chris Geihsler 3d91bb65f7 kvdb: unify how the db backend for tests is set 2023-01-23 09:05:11 +02:00
Elle Mouton 170160f28a kvdb+refactor: move all general sqlite code to seprate dir
In this commit, all the sql, non-postgres-specific, code is moved out of
the postgres package and into a new sqlbase package. This will make it
more easily reusable for future sql integrations.
2023-01-23 09:05:11 +02:00
Elle Mouton 30ba8cbae9 kvdb/postgres: separate general sql code from postgres
In this commit, changes are made to the `kvdb/postgres` package so that
all all the non-postgres-specific code is generalised to be applicable
for all sql code. A follow up commit will move all the general sql code
into its own package.
2023-01-23 09:05:11 +02:00
Elle Mouton 3dfaa88bb4 kvdb/postgres: remove old +build directive and cleanup
Due to the update of the kvdb package to go1.18, the `// +build`
directives are no longer required. They are removed in this commit in
order to simplify upcoming commits. Along the way, a few typos are fixed
and an unused struct is removed.
2023-01-23 09:05:11 +02:00
Elle Mouton c6abf585ee kvdb/postgres: make global dbConns thread safe
In this commit, a mutex is added to guard access to the global dbConns
variable in the kvdb/postgres package.
2023-01-23 09:05:11 +02:00
Elle Mouton 72dbc3dbb4 kvdb: bump go version to 1.18
Bump the go version to go1.18 so that we can get rid of the '// +build'
comments in the upcoming commits where some of the '//go:build'
comments will become too complex to write as '// +build' comments.
2023-01-23 09:05:10 +02:00
yyforyongyu 84fd911b47 multi: fix make lint
Fixes new lint errors caught by the latest version.
2022-11-18 20:48:23 +08:00
yyforyongyu 4db6617d45 multi: skip checking long lines for config files 2022-10-20 23:59:09 +08:00
Eng Zer Jun 889cb0af47 kvdb/etcd: replace defer cleanup with t.Cleanup
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-10-13 17:47:03 +08:00
Oliver Gugger ee9d6f21bd etcd: add SequenceKey function 2022-10-13 09:37:52 +02:00
Oliver Gugger d576184fdb etcd: extract NewEtcdClient into exported function
In order for us to be able to re-use the code for initializing an etcd
client connection, we extract and export the NewEtcdClient function.
2022-10-13 09:37:52 +02:00
Oliver Gugger 98f359c5b7 etcd: fix typos 2022-10-13 09:37:52 +02:00
Eng Zer Jun 0edc16aa01 kvdb: use T.TempDir to create temporary test directory
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-08-24 09:03:04 +08:00
Andras Banki-Horvath 1093683b47 etcd: set embedded etcd default max msg size and up txn limit 2022-02-28 12:03:05 +01:00
Andras Banki-Horvath 5e90193180 etcd: fix config clone 2022-02-28 12:03:05 +01:00
Andras Banki-Horvath 40af029413 etcd: extend kvdb_etcd with a real ForAll implementation 2022-02-28 12:03:04 +01:00
Joost Jager b8408a1484 kvdb: add ForAll
A new method to allow efficient range queries for backends that support
it.
2022-01-12 19:11:53 +01:00
Joost Jager 99566b768e kvdb/postgres: use readonly db transaction if possible 2022-01-12 13:38:56 +01:00
Joost Jager 7cd7cef6c6 htlcswitch: fix duplicate close
The decayed log database opening and closing is managed at a higher
level in config_builder.go.
2022-01-10 08:42:32 +01:00
Joost Jager 0cae55c162 kvdb/postgres: fix context cancellation 2021-12-22 14:54:43 +01:00
Joost Jager 62a2f3c809 kvdb/postgres: fix tests 2021-12-22 14:54:41 +01:00
Andras Banki-Horvath 1ccc5e8014 etcd: make max message size configurable 2021-12-02 21:07:49 +01:00
Jamie Turley fb43843253 kvdb/bolt_compact: have log message take account space multiplier
[skip ci]
2021-11-21 21:52:30 -08:00
Joost Jager 274faff980 postgres: add connection limit 2021-11-17 11:08:43 +01:00
Oliver Gugger 41c579fd09 multi: run go fmt with go 1.17 2021-11-08 22:39:41 +01:00
Oliver Gugger 97786c72fc Merge pull request #5878 from halseth/filter-ascii-bbolt-name
kvdb/bolt_compact: filter out non-ASCII from bucket name before printing
2021-11-02 10:22:46 +01:00
Johan T. Halseth 6be6585643 kvdb/bolt_compact: filter out non-ASCII from bucket name before printing
Printing the raw bucket name lead to weird characters and even sounds
being printed.
2021-11-02 08:54:35 +01:00
Johan T. HalsethandOliver Gugger 10aaa35db5 kvdb: add loggableKeyName method
Safely logs the bucket key name as hex if it includes special
characters.

Co-authored-by: Oliver Gugger <gugger@gmail.com>
2021-11-02 08:54:35 +01:00
Joost Jager d997bbf6b3 channeldb/test: test with postgres 2021-10-29 10:54:44 +02:00
Joost Jager 7c048efa21 kvdb/postgres/test: single instance embedded postgres database
Prepare for parallel tests that use a postgres backend. We don't want a
high number of embedded postgres instances running simultaneously.
2021-10-29 10:54:42 +02:00
Joost Jager bd291286f7 kvdb/postgres: convert all types of panic data to error
Previously the assumption existed that panic data was already an error.
If that wasn't the case for example because panic("string") was used,
the transaction wasn't unlocked.
2021-10-29 10:54:40 +02:00
naveen 06433d351e Upgrade to go 1.16 for the left over packages
Upgraded the go to 1.16 for the rest of the packages.
2021-10-19 03:55:13 +00:00
naveen d5e3302265 mod : replace gogo/protobuf for GO-2021-0053
The gogo/protobuf has OSV GO-2021-0053
https://deps.dev/advisory/OSV/GO-2021-0053?from=%2Fgo%2Fgithub.com%252Flightningnetwork%252Flnd

This is used indirectly and has to be replaced in go.mod to avoid the
direct reference to that version.

The OSV has been addressed in v1.3.2
2021-10-01 13:07:52 +00:00
Olaoluwa Osuntokun 8ba68ca59f Merge pull request #5789 from naveensrinivasan/naveen/feat/fix-CVE-2021-29482
mod: Upgraded xz library to FIX the CVE-2021-29482
2021-09-30 20:16:13 -07:00
Oliver Gugger 0b4e03f5fc multi: add golang 1.17 compatible build tags
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md

The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
2021-09-29 17:31:37 -07:00
naveen 04f57c6f57 mod: Upgraded xz library to FIX the CVE-2021-29482
Included a replace directive to avoid using an high severity CVE
https://github.com/advisories/GHSA-25xm-hr59-7c27

This library is indirectly referenced and cannot be upgraded directly.

The https://github.com/fergusstrange/embedded-postgres/pull/42 was
merged to fix the CVE issue.
2021-09-29 15:16:26 +00:00
ErikEk d5463818a8 bbolt sync-freelist ignored 2021-09-27 06:49:56 +01:00
Joost Jager 3eb80cac97 kvdb: add postgres 2021-09-21 10:44:18 +02:00
Oliver Gugger 29a8661517 Merge pull request #5640 from bhandras/kvdb-prefetch
kvdb+channeld: extend `kvdb` with `Prefetch` for prefetching buckets in one go and speed up payment control by prefetching payments on hot paths
2021-09-20 09:42:18 +02:00
naveen 8b4bbfbf61 mod : Replace for OSV GO-2020-0017 JWT
The github.com/dgrijalva/jwt-go is no longer maintained
has this CVE https://github.com/advisories/GHSA-w73w-5m7g-f7qc

But it is being used by https://github.com/etcd-io/etcd/issues/13254
which `lnd` uses.

The https://github.com/golang-jwt/jwt/releases/tag/v3.2.1 is a fork of
the same version which was 3.2.0 with the security fix.
2021-09-16 11:23:28 +00:00
Andras Banki-Horvath 152d1dc36a kvdb: extra test coverage for kvdb.Prefetch 2021-09-14 13:12:23 +02:00
Andras Banki-Horvath 256b62e0d5 etcd: add kvdb.Prefetch
This commit extends the kvdb interface in a backwards compatible way
such that we'll be able to prefetch all keys in a bucket in one go reducing the
number of roundtrips.
2021-09-14 13:12:22 +02:00
Andras Banki-Horvath daec66a4e4 etcd: STM to allow prefetching of keys and full ranges
This commit adds more full range caching to the STM such that we'll
be able to prefetch keys and ranges on the kvdb level. This will allow
us to reduce roundtrips to etcd do fast iteration of small ranges.
2021-09-14 13:12:19 +02:00
Andras Banki-Horvath 6c2d8bb176 etcd: enable optional log file for embedded etcd log output
In this commit we add an extra config for enabling logging to an
external file when using embedded etcd. This can be useful when running
integration tests to see more details about etcd related issues.
2021-09-10 14:40:54 +02:00
Andras Banki-Horvath b29ae94e10 etcd: redesign commit queue to make it more robust and scalable
This commit builds on the ideas of @cfromknecht in lnd/5153. The
addition is that the design is now simpler and more robust by queueing
up everything, but allowing maximal parallelism where txns don't block.
Furthermore the commit makes CommitQueue.Done() private essentially
removing the need to understand the queue externally.
2021-08-06 07:53:13 +02:00