docker/pgcat
1
0
Fork 0
mirror of https://github.com/postgresml/pgcat.git synced 2026-03-23 01:16:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
404 Commits 84 Branches 20 Tags
58ce76d9b9f3706c753c57513c8cd76f459ebcac
Commit Graph

49 Commits

Author SHA1 Message Date
Jose Fernández
58ce76d9b9 Refactor stats to use atomics (#375) 
* Refactor stats to use atomics

When we are dealing with a high number of connections, generated
stats cannot be consumed fast enough by the stats collector loop.
This makes the stats subsystem inconsistent and a log of
warning messages are thrown due to unregistered server/clients.

This change refactors the stats subsystem so it uses atomics:

- Now counters are handled using U64 atomics
- Event system is dropped and averages are calculated using a loop
  every 15 seconds.
- Now, instead of snapshots being generated ever second we keep track of servers/clients
  that have registered. Each pool/server/client has its own instance of the counter and
  makes changes directly, instead of adding an event that gets processed later.

* Manually mplement Hash/Eq in `config::Address` ignoring stats

* Add tests for client connection counters

* Allow connecting to dockerized dev pgcat from the host

* stats: Decrease cl_idle when idle socket disconnects
 2023-03-28 17:19:37 +02:00
Lev Kokotov
90aba9c011 V1 (#383) 2023-03-24 17:10:12 -07:00
dependabot[bot]
b765581975 chore(deps): bump sqlparser from 0.31.0 to 0.32.0 (#343) 
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.31.0 to 0.32.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.31.0...v0.32.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-07 01:18:10 -08:00
dependabot[bot]
bd286d9128 chore(deps): bump sqlparser from 0.30.0 to 0.31.0 (#335) 
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.30.0 to 0.31.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.30.0...v0.31.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-02 00:40:52 -05:00
zainkabani
eb8cfdb1f1 Adds SHUTDOWN command as alternate option to sending SIGINT (#331) 
* Adds SHUTDOWN command to PgCat as alternate option to sending SIGINT

* Check if we're already in SHUTDOWN sequence

* Send signal directly from shutdown instead of using channel

* Add tests

* trigger build

* Lowercase response and boolean change

* Update tests

* Fix tests

* typo
 2023-02-26 22:16:30 -08:00
Mostafa Abdelraouf
f1265a5570 Introduce tcp_keepalives to PgCat (#315) 
We have encountered a case where PgCat pools were stuck following a database incident. Our best understanding at this point is that the PgCat -> Postgres connections died silently and because Tokio defaults to disabling keepalives, connections in the pool were marked as busy forever. Only when we deployed PgCat did we see recovery.

This PR introduces tcp_keepalives to PgCat. This sets the defaults to be

keepalives_idle: 5        # seconds
keepalives_interval: 5 # seconds
keepalives_count: 5    # a count
These settings can detect the death of an idle connection within 30 seconds of its death. Please note that the connection can remain idle forever (from an application perspective) as long as the keepalive packets are flowing so disconnection will only occur if the other end is not acknowledging keepalive packets (keepalive packet acks are handled by the OS, the application does not need to do anything). I plan to add tcp_user_timeout in a follow-up PR.
 2023-02-08 11:35:38 -06:00
dependabot[bot]
bb0b64e089 chore(deps): bump toml from 0.6.0 to 0.7.0 (#305) 
Bumps [toml](https://github.com/toml-rs/toml) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.6.0...toml-v0.7.0)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-30 08:59:37 -08:00
Lev Kokotov
24e79dcf05 Startup improvements & PAUSE/RESUME (#300) 
* Dont require servers to be online to start pooler

* PAUSE/RESUME

* fix

* Refresh pool

* Fixes

* lint
 2023-01-28 15:36:35 -08:00
dependabot[bot]
fbe256cc4e chore(deps): bump toml from 0.5.11 to 0.6.0 (#292) 
Bumps [toml](https://github.com/toml-rs/toml) from 0.5.11 to 0.6.0.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.5.11...toml-v0.6.0)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-24 11:48:21 -08:00
dependabot[bot]
4977489b89 chore(deps): bump base64 from 0.20.0 to 0.21.0 (#279) 
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases)
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-08 20:53:39 -08:00
dependabot[bot]
880bc3e0a8 chore(deps): bump sqlparser from 0.28.0 to 0.30.0 (#275) 
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.28.0 to 0.30.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.28.0...v0.30.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-02 23:57:29 -08:00
dependabot[bot]
077528b2ac chore(deps): bump base64 from 0.13.1 to 0.20.0 (#250) 
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.13.1 to 0.20.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases)
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.13.1...v0.20.0)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-12 09:58:50 -08:00
dependabot[bot]
3e39a07626 chore(deps): bump sqlparser from 0.27.0 to 0.28.0 (#248) 
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.27.0 to 0.28.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.27.0...v0.28.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-05 22:56:29 -08:00
dependabot[bot]
51b4439697 chore(deps): bump env_logger from 0.9.3 to 0.10.0 (#241) 
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.9.3 to 0.10.0.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/env_logger/compare/v0.9.3...v0.10.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-25 01:27:06 -08:00
dependabot[bot]
ab85000ad4 chore(deps): bump sqlparser from 0.26.0 to 0.27.0 (#229) 
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-15 16:26:22 -07:00
dependabot[bot]
09451a469e chore(deps): bump jemallocator from 0.3.2 to 0.5.0 (#218) 
Bumps [jemallocator](https://github.com/tikv/jemallocator) from 0.3.2 to 0.5.0.
- [Release notes](https://github.com/tikv/jemallocator/releases)
- [Changelog](https://github.com/tikv/jemallocator/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tikv/jemallocator/commits/0.5.0)

---
updated-dependencies:
- dependency-name: jemallocator
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-02 12:10:41 -07:00
Lev Kokotov
dea952e4ca Re-enable query parser and parse multiple statements (#191) 
* Re-enable query parser and parse multiple statements

* no diff
 2022-10-23 16:59:51 -07:00
Mostafa Abdelraouf
eceb7f092e Use Jemalloc (#189) 
Jemalloc performs better than the standard allocator in various metrics (http://ithare.com/testing-memory-allocators-ptmalloc2-tcmalloc-hoard-jemalloc-while-trying-to-simulate-real-world-loads/).

This PR makes changes to use Jemalloc as the global allocator for Pgcat. Windows is not officially supported by Pgcat but it should still compile but without Jemalloc as the allocator.
 2022-10-13 11:13:45 -05:00
zainkabani
85cc2f4147 Update to latest library versions (#170) 2022-09-21 13:48:33 -07:00
Mostafa Abdelraouf
3bc4f9351c Exit with failure codes if configs are bad (#146) 
* Exit with failure codes if configs are bad

* fmt
 2022-08-25 18:56:18 -07:00
Nicholas Dujay
1b166b462d create a prometheus exporter on a standard http port (#107) 
* create a hyper server and add option to enable it in config

* move prometheus stuff to its own file; update format

* create metric type and help lookup table

* finish the metric help type map

* switch to a boolean and a standard port

* dont emit unimplemented metrics

* fail if curl returns a non 200

* resolve conflicts

* move log out of config.show and into main

* terminating new line

* upgrade curl

* include unimplemented stats
 2022-08-09 12:19:11 -07:00
Mostafa Abdelraouf
2ae4b438e3 Add support for multi-database / multi-user pools (#96) 
* Add support for multi-database / multi-user pools

* Nothing

* cargo fmt

* CI

* remove test users

* rename pool

* Update tests to use admin user/pass

* more fixes

* Revert bad change

* Use PGDATABASE env var

* send server info in case of admin
 2022-07-27 19:47:55 -07:00
Lev
21bf07258c lock em up 2022-06-27 17:05:45 -07:00
Lev Kokotov
b974aacd71 check 2022-06-27 09:46:33 -07:00
Lev Kokotov
b93303eb83 Live reloading entire config and bug fixes (#84) 
* Support reloading the entire config (including sharding logic) without restart.

* Fix bug incorrectly handing error reporting when the shard is set incorrectly via SET SHARD TO command.
selected wrong shard and the connection keep reporting fatal #80.

* Fix total_received and avg_recv admin database statistics.

* Enabling the query parser by default.

* More tests.
 2022-06-24 14:52:38 -07:00
Lev Kokotov
d3310a62c2 Client md5 auth and clean up scram (#77) 
* client md5 auth and clean up scram

* add pw

* add user

* add user

* log
 2022-06-20 06:15:54 -07:00
Lev Kokotov
d412238f47 Implement SCRAM-SHA-256 for server authentication (PG14) (#76) 
* Implement SCRAM-SHA-256

* test it

* trace

* move to community for auth

* hmm
 2022-06-18 18:36:00 -07:00
Lev Kokotov
df85139281 Update README. Comments. Version bump. (#60) 
* update readme

* comments

* just a version bump
 2022-03-10 01:33:29 -08:00
Lev Kokotov
35828a0a8c Per-shard statistics (#57) 
* per shard stats

* aight

* cleaner

* fix show lists

* comments

* more friendly

* case-insensitive

* test all shards

* ok

* HUH?
 2022-03-04 17:04:27 -08:00
Lev Kokotov
5972b6fa52 Switch to parking_lot RwLock & Mutex. Use trace! for protocol instead of debug! (#42) 
* RwLock & parking_lot::Mutex

* upgrade to trace
 2022-02-24 08:44:41 -08:00
Lev Kokotov
44b5e7eeee use logger lib; minor refactor; sv_* stats (#29) 2022-02-20 22:47:08 -08:00
Lev Kokotov
d4c1fc87ee Reloadable config (#26) 
* Reloadable config

* readme

* live config reload

* test matrix
 2022-02-19 13:57:35 -08:00
Lev Kokotov
aa796289bf Query parser 3.0 (#23) 
* Starting query parsing

* Query parser

* working config

* disable by default

* fix tsets

* introducing log crate; test for query router; comments

* typo

* fixes for banning

* added test for prepared stmt
 2022-02-18 07:10:18 -08:00
Lev Kokotov
659b1e00b8 bump statsd 2022-02-14 10:27:32 -08:00
Lev Kokotov
8e5e28a139 Some stats (#19) 2022-02-14 10:00:55 -08:00
Lev Kokotov
06575eae7b once_cell is way faster 2022-02-10 17:05:20 -08:00
Lev Kokotov
1798065b76 max_workers = 4 = so much faster 2022-02-10 13:48:56 -08:00
Lev Kokotov
9fe50c48e8 rebuild 2022-02-08 18:02:26 -08:00
Lev Kokotov
edf6a69ca4 warnings 2022-02-08 17:08:17 -08:00
Lev Kokotov
af6f770271 sharded query routing 2022-02-08 13:11:50 -08:00
Lev Kokotov
c27a7d30dc config support; started more sharding 2022-02-08 09:25:59 -08:00
Lev Kokotov
f8d6bd8d89 rename 2022-02-06 11:22:09 -08:00
Lev Kokotov
b943ff3fa6 sharding 2022-02-05 19:43:48 -08:00
Lev Kokotov
e13795d762 hm 2022-02-05 14:11:49 -08:00
Lev Kokotov
a6574acbc3 replica pool & banlist 2022-02-05 10:02:13 -08:00
Lev Kokotov
6deb7b1162 name servers & dont leave open servers with bad state 2022-02-03 17:06:19 -08:00
Lev Kokotov
e3ec5036d7 connection pool 2022-02-03 16:25:05 -08:00
Lev Kokotov
f4d647ce2f it works! 2022-02-03 15:17:04 -08:00
Lev Kokotov
c0b747ba34 working startup 2022-02-03 13:35:40 -08:00