Commit Graph

74 Commits

Author SHA1 Message Date
Jose Fernández
99247f7c88 Allow setting idle_timeout for server connections. (#257)
In postgres, you can specify an `idle_session_timeout` which will close
sessions idling for that amount of time. If a session is closed because
of a timeout, PgCat will erroneously mark the server as unhealthy as the next
health check will return an error because the connection was drop, if no
health check is to be executed, it will simply fail trying to send the query
to the server for the same reason, the conn was drop.

Given that bb8 allows configuring an idle_timeout for pools, it would be
nice to allow setting this parameter in the config file, this way you can
set it to something shorter than the server one. Also, server pool will be kept
smaller in moments of less traffic. Actually, currently this value is set as its
default in bb8, which is 10 minutes.

This changes allows setting the parameter using the config file. It can be set both
globally and per pool. When creating the pool, if the pool don't have it defined, global
value is used.
2022-12-16 08:01:00 -08:00
zainkabani
0c96156dae Adds health check setting to pool and avoids get_config in hotpath (#235)
* Adds healthcheck settings to pool

* fmt

* Fix test
2022-11-16 18:51:15 -08:00
Cluas
dfa26ec6f8 chore: make clippy lint happy (#225)
* chore: make clippy happy

* chore: cargo fmt

* chore: cargo fmt
2022-11-09 10:04:31 -08:00
Lev Kokotov
0524787d31 Automatic sharding: part one of many (#194)
Starting automatic sharding
2022-10-25 11:47:41 -07:00
Mostafa Abdelraouf
83fd639918 A bit faster get_pool (#187)
* A bit faster get_pool

* fmt
2022-10-08 08:16:04 -07:00
Mostafa Abdelraouf
3d33ccf4b0 Fix maxwait metric (#183)
Max wait was being reported as 0 after #159

This PR fixes that and adds test
2022-10-05 21:41:09 -05:00
Lev Kokotov
7987c5ffad Replace a few types with more developer-friendly names (#182)
* Replace a few types with more developer-friendly names

* UserPool -> PoolIdentifier
2022-10-01 10:25:59 -07:00
zainkabani
24f5eec3ea Change sharding config to enum and move validation of configs into public functions (#178)
Moves config validation to own functions to enable tools to use them
Moves sharding config to enum
Makes defaults public
Make connect_timeout on pool and option which is overwritten by general connect_timeout
2022-09-28 08:50:14 -05:00
Lev Kokotov
19fd677891 Fix the pool fix (#176)
* Always listen to the compiler

* Its fine
2022-09-23 12:06:07 -07:00
Lev Kokotov
964a5e1708 Don't drop connections if DB hasn't changed (#175)
* Don't drop connections if DB hasn't changed

* Incoporate connect_timeout into the pool config

* use the field
2022-09-23 11:32:05 -07:00
zainkabani
f72dac420b Add defaults for configs (#174)
* add statement timeout to readme

* Add defaults to various configs

* primary read enabled default to false
2022-09-22 23:00:46 -07:00
zainkabani
3a729bb75b Minor refactor for configs (#172)
* Changes shard struct to use vector of ServerConfig

* Adds to query router

* Change client disconnect with error message to warn instead of debug

* Add warning logs for clean up actions
2022-09-22 10:07:02 -07:00
zainkabani
85cc2f4147 Update to latest library versions (#170) 2022-09-21 13:48:33 -07:00
Mostafa Abdelraouf
4ae1bc8d32 Add SHOW CLIENTS / SHOW SERVERS + Stats refactor and tests (#159)
* wip

* Main Thread Panic when swarmed with clients

* fix

* fix

* 1024

* fix

* remove test

* Add SHOW CLIENTS

* revert

* fmt

* Refactor + tests

* fmt

* add test

* Add SHOW SERVERS + Make PR unreviewable

* prometheus

* add state to clients and servers

* fmt

* Add application_name to server stats

* Add tests for waiting clients

* Docs

* remove comment

* comments

* typo

* cleanup

* CI
2022-09-14 11:20:41 -04:00
Mostafa Abdelraouf
36339bd96f Log Address information in connection create/drop (#154)
* Log Address information in connection create/drop

* run ci
2022-09-01 11:16:22 -07:00
Lev Kokotov
9d84d6f131 Graceful shutdown and refactor (#144)
* Graceful shutdown and refactor

* ok

* _Graceful_ shutdown

* Remove hardcoded setting

* clean up

* end

* timeout

* hmm

* hmm!

* bash

* bash

* hmm

* maybe maybe

* Adds tests and move non-admin connection rejection to startup (#145)

* Move error response

* Adds tests and removes unused variable

* Adds debug log

Co-authored-by: zainkabani <77307340+zainkabani@users.noreply.github.com>
2022-08-25 06:40:56 -07:00
Lev Kokotov
069d76029f Fix incorrect routing for replicas (#139)
* Fix incorrect routing for replicas

* name
2022-08-21 22:40:49 -07:00
Mostafa Abdelraouf
5f5b5e2543 Random instance selection (#136)
* wip

* revert some'

* revert more

* poor-man's integration test

* remove test

* fmt

* --workspace

* fix build

* fix integration test

* another stab

* log

* run after integration

* cargo test after integration

* revert

* revert more

* Refactor + clean up

* more clean up
2022-08-21 22:15:20 -07:00
zainkabani
5948fef6cf Minor Refactoring of re-used code and server stat reporting (#129)
* Minor changes to stats reporting and recduce re-used code

* fmt
2022-08-18 05:12:38 -07:00
Mostafa Abdelraouf
790898c20e Add pool name and username to address object (#128)
* Add pool name and username to address object

* Fix address name

* fmt
2022-08-17 08:40:47 -07:00
Lev Kokotov
be254cedd9 Fix debug log (#120) 2022-08-11 22:47:47 -07:00
zainkabani
f963b12821 Health check delay (#118)
* initial commit of server check delay implementation

* fmt

* spelling

* Update name to last_healthcheck and some comments

* Moved server tested stat to after require_healthcheck check

* Make health check delay configurable

* Rename to last_activity

* Fix typo

* Add debug log for healthcheck

* Add address to debug log
2022-08-11 14:42:40 -07:00
Mostafa Abdelraouf
48cff1f955 Slightly more light weight health check (#100) 2022-07-29 11:58:25 -07:00
Mostafa Abdelraouf
2ae4b438e3 Add support for multi-database / multi-user pools (#96)
* Add support for multi-database / multi-user pools

* Nothing

* cargo fmt

* CI

* remove test users

* rename pool

* Update tests to use admin user/pass

* more fixes

* Revert bad change

* Use PGDATABASE env var

* send server info in case of admin
2022-07-27 19:47:55 -07:00
Lev Kokotov
b93303eb83 Live reloading entire config and bug fixes (#84)
* Support reloading the entire config (including sharding logic) without restart.

* Fix bug incorrectly handing error reporting when the shard is set incorrectly via SET SHARD TO command.
selected wrong shard and the connection keep reporting fatal #80.

* Fix total_received and avg_recv admin database statistics.

* Enabling the query parser by default.

* More tests.
2022-06-24 14:52:38 -07:00
Lev Kokotov
fe32b5ef17 Reduce traffic on the stats channel (#69) 2022-05-17 13:05:25 -07:00
Lev Kokotov
54699222f8 Possible fix for clients waiting stat leak (#68) 2022-05-14 21:35:33 -07:00
Lev Kokotov
df85139281 Update README. Comments. Version bump. (#60)
* update readme

* comments

* just a version bump
2022-03-10 01:33:29 -08:00
Lev Kokotov
341ebf4123 docs and remove Option (#58)
* docs and remove Option

* lint
2022-03-07 23:05:40 -08:00
Lev Kokotov
35828a0a8c Per-shard statistics (#57)
* per shard stats

* aight

* cleaner

* fix show lists

* comments

* more friendly

* case-insensitive

* test all shards

* ok

* HUH?
2022-03-04 17:04:27 -08:00
Lev Kokotov
d4186b7815 More admin (#53)
* more admin

* more admin

* show lists

* tests
2022-03-01 22:49:43 -08:00
Lev Kokotov
aaeef69d59 Refactor admin (#52) 2022-03-01 08:47:19 -08:00
Lev Kokotov
b21e0f4a7e admin SHOW DATABASES (#51)
* admin SHOW DATABASES

* test

* correct replica count
2022-02-28 17:22:28 -08:00
Lev Kokotov
99d65fc475 Check server versions on startup & refactor (#48)
* Refactor and check server parameters

* warnings

* fix validator
2022-02-26 11:01:52 -08:00
Lev Kokotov
5972b6fa52 Switch to parking_lot RwLock & Mutex. Use trace! for protocol instead of debug! (#42)
* RwLock & parking_lot::Mutex

* upgrade to trace
2022-02-24 08:44:41 -08:00
Lev Kokotov
f21a3d8d8c Add server login stat; refactor for better naming (#34) 2022-02-21 17:28:50 -08:00
Lev Kokotov
86941d62e4 Reset query router setting to default (#32) 2022-02-21 00:00:50 -08:00
Lev Kokotov
44b5e7eeee use logger lib; minor refactor; sv_* stats (#29) 2022-02-20 22:47:08 -08:00
Lev Kokotov
3b795464a8 Fix client states reporting (#27)
* Fix client states reporting

* clean
2022-02-20 12:40:09 -08:00
Lev Kokotov
d4c1fc87ee Reloadable config (#26)
* Reloadable config

* readme

* live config reload

* test matrix
2022-02-19 13:57:35 -08:00
Lev Kokotov
aa796289bf Query parser 3.0 (#23)
* Starting query parsing

* Query parser

* working config

* disable by default

* fix tsets

* introducing log crate; test for query router; comments

* typo

* fixes for banning

* added test for prepared stmt
2022-02-18 07:10:18 -08:00
Lev Kokotov
4c8a3987fe Refactor query routing into its own module (#22)
* Refactor query routing into its own module

* commments; tests; dead code

* error message

* safer startup

* hm

* dont have to be public

* wow

* fix ci

* ok

* nl

* no more silent errors
2022-02-16 22:52:11 -08:00
Lev Kokotov
7b0ceefb96 Constants, comments, CI fixes, dead code clean-up (#21)
* constants

* server.rs docs

* client.rs comments

* dead code; comments

* comment

* query cancellation comments

* remove unnecessary cast

* move db setup up one step

* query cancellation test

* new line; good night
2022-02-15 22:45:45 -08:00
Lev Kokotov
05b4cccb97 More statistics (#20)
* cleaner stats

* remove shard selection there
2022-02-15 08:18:01 -08:00
Lev Kokotov
8e5e28a139 Some stats (#19) 2022-02-14 10:00:55 -08:00
Lev Kokotov
20ceb729a0 print session duration; connect to all servers when validating (#11) 2022-02-12 09:24:24 -08:00
Lev Kokotov
526b9eb666 Pass real server info to the client (#10) 2022-02-11 22:19:49 -08:00
Lev Kokotov
0d369ab90a add default server role; bug fix 2022-02-11 11:19:40 -08:00
Lev Kokotov
495d6ce6c3 fmt 2022-02-10 10:38:06 -08:00
Lev Kokotov
883b6ee793 todo complete 2022-02-10 10:37:55 -08:00