Commit Graph

70 Commits

Author SHA1 Message Date
Jose Fernández
9241df18e2 Allow sending logs to stdout by using STDOUT_LOG env var (#334)
* Allow sending logs to stdout by using STDOUT_LOG env var

* Increase stats buffer size
2023-02-28 13:10:40 -08:00
zainkabani
eb8cfdb1f1 Adds SHUTDOWN command as alternate option to sending SIGINT (#331)
* Adds SHUTDOWN command to PgCat as alternate option to sending SIGINT

* Check if we're already in SHUTDOWN sequence

* Send signal directly from shutdown instead of using channel

* Add tests

* trigger build

* Lowercase response and boolean change

* Update tests

* Fix tests

* typo
2023-02-26 22:16:30 -08:00
Kurtsley
1c73889fb9 Add initial Windows support, ref #298 (#301) 2023-01-28 15:51:05 -08:00
Lev Kokotov
2e3eb2663e Fix formatting (#299) 2023-01-28 09:17:49 -08:00
Jose Fernández
9e8ef566c6 Allow setting the number of runtime workers to be used. (#258)
This change adds a new configuration parameter called `worker_threads` that
allows setting the number of workers the Tokio Runtime will use. It defaults to
4 to maintain backward compatibility.

Given that the config file parse is done asynchronously, first, a transient runtime
is created for reading config, and once it has been parsed, the actual runtime
that will be used for PgCat execution is created.
2022-12-16 11:13:13 -08:00
zainkabani
5145b20e02 Move ClientBadStartup error log to debug (#237) 2022-11-16 22:16:16 -08:00
zainkabani
fe0b012832 Adds configuration for logging connections and removes get_config from entrypoint (#236)
* Adds configuration for logging connections and removes get_config from entrypoint

* typo

* rename connection config var and add to toml files

* update config log

* fmt
2022-11-16 22:15:47 -08:00
Cluas
dfa26ec6f8 chore: make clippy lint happy (#225)
* chore: make clippy happy

* chore: cargo fmt

* chore: cargo fmt
2022-11-09 10:04:31 -08:00
Mostafa Abdelraouf
eceb7f092e Use Jemalloc (#189)
Jemalloc performs better than the standard allocator in various metrics (http://ithare.com/testing-memory-allocators-ptmalloc2-tcmalloc-hoard-jemalloc-while-trying-to-simulate-real-world-loads/).

This PR makes changes to use Jemalloc as the global allocator for Pgcat. Windows is not officially supported by Pgcat but it should still compile but without Jemalloc as the allocator.
2022-10-13 11:13:45 -05:00
zainkabani
3a729bb75b Minor refactor for configs (#172)
* Changes shard struct to use vector of ServerConfig

* Adds to query router

* Change client disconnect with error message to warn instead of debug

* Add warning logs for clean up actions
2022-09-22 10:07:02 -07:00
zainkabani
8c09ab6c20 Export pgcat objects in lib (#169)
* Export pgcat objects in lib

* fmt
2022-09-20 18:47:32 -07:00
zainkabani
6a9a4db648 Adds microsecond logging and also reformats duration to include milliseconds (#156)
* Adds microsecond logging and also reformats duration to include milliseconds

* fmt

* attempt to fix cd

* revert
2022-09-05 01:21:27 -07:00
Mostafa Abdelraouf
976b406468 Main Thread Panic when swarmed with clients (#158)
* Main Thread Panic when swarmed with clients

* fix

* fix

* 1024

* fix

* remove test

* Update src/client.rs

* Update src/main.rs

* Update src/client.rs

* Update src/main.rs

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
2022-09-05 01:21:06 -07:00
zain-kabani
417358c35d Patch graceful shutdown bug (#157)
* Fixes non-admin client counting error

* Add log when sigterm received and log number of active clients when shutdown timeout is reached
2022-09-05 01:02:49 -07:00
zainkabani
2628dec42e Move autoreloader to own tokio task (#148) 2022-08-29 00:08:44 -07:00
Mostafa Abdelraouf
3bc4f9351c Exit with failure codes if configs are bad (#146)
* Exit with failure codes if configs are bad

* fmt
2022-08-25 18:56:18 -07:00
Lev Kokotov
9d84d6f131 Graceful shutdown and refactor (#144)
* Graceful shutdown and refactor

* ok

* _Graceful_ shutdown

* Remove hardcoded setting

* clean up

* end

* timeout

* hmm

* hmm!

* bash

* bash

* hmm

* maybe maybe

* Adds tests and move non-admin connection rejection to startup (#145)

* Move error response

* Adds tests and removes unused variable

* Adds debug log

Co-authored-by: zainkabani <77307340+zainkabani@users.noreply.github.com>
2022-08-25 06:40:56 -07:00
Lev Kokotov
cea35db35c Fix lost statistics (#125)
* Lost events

* more logging
2022-08-15 23:54:49 -07:00
Pradeep Chhetri
52303cc808 Make prometheus port configurable (#121)
* Make prometheus port configurable

* Update circleci config
2022-08-13 10:25:14 -07:00
Nicholas Dujay
1b166b462d create a prometheus exporter on a standard http port (#107)
* create a hyper server and add option to enable it in config

* move prometheus stuff to its own file; update format

* create metric type and help lookup table

* finish the metric help type map

* switch to a boolean and a standard port

* dont emit unimplemented metrics

* fail if curl returns a non 200

* resolve conflicts

* move log out of config.show and into main

* terminating new line

* upgrade curl

* include unimplemented stats
2022-08-09 12:19:11 -07:00
zainkabani
3719c22322 Implementing graceful shutdown (#105)
* Initial commit for graceful shutdown

* fmt

* Add .vscode to gitignore

* Updates shutdown logic to use channels

* fmt

* fmt

* Adds shutdown timeout

* Fmt and updates tomls

* Updates readme

* fmt and updates log levels

* Update python tests to test shutdown

* merge changes

* Rename listener rx and update bash to be in line with master

* Update python test bash script ordering

* Adds error response message before shutdown

* Add details on shutdown event loop

* Fixes response length for error

* Adds handler for sigterm

* Uses ready for query function and fixes number of bytes

* fmt
2022-08-08 16:01:24 -07:00
Mostafa Abdelraouf
2ae4b438e3 Add support for multi-database / multi-user pools (#96)
* Add support for multi-database / multi-user pools

* Nothing

* cargo fmt

* CI

* remove test users

* rename pool

* Update tests to use admin user/pass

* more fixes

* Revert bad change

* Use PGDATABASE env var

* send server info in case of admin
2022-07-27 19:47:55 -07:00
Lev
186f8be5b3 lint 2022-06-27 17:01:40 -07:00
Lev
8f3202ed92 hmm 2022-06-27 16:45:41 -07:00
Lev
eb58920870 at least it compiles 2022-06-27 15:52:01 -07:00
Lev Kokotov
b974aacd71 check 2022-06-27 09:46:33 -07:00
Lev Kokotov
7dfe59a91a Fix stats dymanic reload (#87) 2022-06-25 12:22:46 -07:00
Lev Kokotov
5bcd3bf9c3 Automatically reload config every seconds (disabled by default) (#86)
* Automatically reload config every seconds (disabld by default)

* add that
2022-06-25 11:46:20 -07:00
Lev Kokotov
b93303eb83 Live reloading entire config and bug fixes (#84)
* Support reloading the entire config (including sharding logic) without restart.

* Fix bug incorrectly handing error reporting when the shard is set incorrectly via SET SHARD TO command.
selected wrong shard and the connection keep reporting fatal #80.

* Fix total_received and avg_recv admin database statistics.

* Enabling the query parser by default.

* More tests.
2022-06-24 14:52:38 -07:00
Lev Kokotov
d412238f47 Implement SCRAM-SHA-256 for server authentication (PG14) (#76)
* Implement SCRAM-SHA-256

* test it

* trace

* move to community for auth

* hmm
2022-06-18 18:36:00 -07:00
Lev Kokotov
df85139281 Update README. Comments. Version bump. (#60)
* update readme

* comments

* just a version bump
2022-03-10 01:33:29 -08:00
Lev Kokotov
b309ead58f Handle SIGTERM. Add docker-compose.yml (#59)
* docker-compsoe

* remove statsd config

* readme
2022-03-08 17:18:48 -08:00
Lev Kokotov
35828a0a8c Per-shard statistics (#57)
* per shard stats

* aight

* cleaner

* fix show lists

* comments

* more friendly

* case-insensitive

* test all shards

* ok

* HUH?
2022-03-04 17:04:27 -08:00
Lev Kokotov
f74101cdfe admin: SHOW STATS (#46)
* admin: show stats

* warning

* tests

* lint

* type mod
2022-02-25 18:20:15 -08:00
Lev Kokotov
5972b6fa52 Switch to parking_lot RwLock & Mutex. Use trace! for protocol instead of debug! (#42)
* RwLock & parking_lot::Mutex

* upgrade to trace
2022-02-24 08:44:41 -08:00
Lev Kokotov
af1716bcd7 Flush stats (#38)
* flush stats

* stats

* refactor
2022-02-22 18:10:30 -08:00
Lev Kokotov
a6fc935040 Can pass config path as argument (#36)
* show better errors for config parsing

* lint
2022-02-21 20:41:32 -08:00
Lev Kokotov
44b5e7eeee use logger lib; minor refactor; sv_* stats (#29) 2022-02-20 22:47:08 -08:00
Lev Kokotov
108f5715c0 License as MIT (#28)
* License as MIT

* license

* s
2022-02-20 13:49:30 -08:00
Lev Kokotov
d4c1fc87ee Reloadable config (#26)
* Reloadable config

* readme

* live config reload

* test matrix
2022-02-19 13:57:35 -08:00
Lev Kokotov
aa796289bf Query parser 3.0 (#23)
* Starting query parsing

* Query parser

* working config

* disable by default

* fix tsets

* introducing log crate; test for query router; comments

* typo

* fixes for banning

* added test for prepared stmt
2022-02-18 07:10:18 -08:00
Lev Kokotov
4c8a3987fe Refactor query routing into its own module (#22)
* Refactor query routing into its own module

* commments; tests; dead code

* error message

* safer startup

* hm

* dont have to be public

* wow

* fix ci

* ok

* nl

* no more silent errors
2022-02-16 22:52:11 -08:00
Lev Kokotov
7b0ceefb96 Constants, comments, CI fixes, dead code clean-up (#21)
* constants

* server.rs docs

* client.rs comments

* dead code; comments

* comment

* query cancellation comments

* remove unnecessary cast

* move db setup up one step

* query cancellation test

* new line; good night
2022-02-15 22:45:45 -08:00
Lev Kokotov
8e5e28a139 Some stats (#19) 2022-02-14 10:00:55 -08:00
Lev Kokotov
4aa9c3d3c7 Cleaner shutdown (#12)
* Cleaner shutdown

* mark as bad just in case although im pretty sure we dont need it

* server session duration

* test clean shutdown

* ah
2022-02-12 10:16:05 -08:00
Lev Kokotov
20ceb729a0 print session duration; connect to all servers when validating (#11) 2022-02-12 09:24:24 -08:00
Lev Kokotov
526b9eb666 Pass real server info to the client (#10) 2022-02-11 22:19:49 -08:00
Lev Kokotov
0d369ab90a add default server role; bug fix 2022-02-11 11:19:40 -08:00
Lev Kokotov
06575eae7b once_cell is way faster 2022-02-10 17:05:20 -08:00
Lev Kokotov
1798065b76 max_workers = 4 = so much faster 2022-02-10 13:48:56 -08:00