Compare commits

..

498 Commits

Author SHA1 Message Date
drdrsh
8e1b7d3f07 Update index.yaml
Signed-off-by: drdrsh <drdrsh@users.noreply.github.com>
2024-11-11 15:24:31 +00:00
Mostafa
52514bcd6e Verify ownership (#862) 2024-11-11 09:16:42 -06:00
Mostafa
7f94e62d06 Initial gh-pages commit (#860) 2024-11-11 08:57:18 -06:00
drdrsh
c7a8434374 Update index.yaml
Signed-off-by: drdrsh <drdrsh@users.noreply.github.com>
2024-11-08 12:07:27 +00:00
drdrsh
f21098fd99 Update index.yaml
Signed-off-by: drdrsh <drdrsh@users.noreply.github.com>
2024-11-02 23:05:55 +00:00
drdrsh
b483f37ea2 Update index.yaml
Signed-off-by: drdrsh <drdrsh@users.noreply.github.com>
2024-11-02 16:55:33 +00:00
Nicolas Vanelslande
a68071dd28 Bump bb8 from 0.8.1 to 0.8.6 (#709)
* Update bb8 to 0.8.6

To get https://github.com/djc/bb8/pull/186 and https://github.com/djc/bb8/pull/189
which fix potential deadlocks (https://github.com/djc/bb8/issues/154).

Also, this (https://github.com/djc/bb8/pull/225) was needed to prevent a connection
leak which was conveniently spotted in our integration tests.

* Ignore ./.bundle (created by dev console)

---------

Co-authored-by: Jose Fernandez (magec) <joseferper@gmail.com>
2024-10-28 06:49:36 -05:00
Mostafa
c27d801abf Rename a couple of variables (#839) 2024-10-23 06:38:07 -05:00
Javier Goday
186e72298f #829: read/write splitting on CTE mutable statements (#835) 2024-10-23 06:20:04 -05:00
Sebastian Serth
3935366d86 End Prometheus stats with a new line separator (#826)
End prometheus stats with a new line separator

According to the [OpenMetrics specification](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#overall-structure), each line MUST end with `\n`. Previously, the last line was not ending with `\n`, so that strict parsers had issues reading the Prometheus stats.
2024-09-22 17:14:04 -05:00
Sean McGivern
b575935b1d Improve documentation for connect_timeout and add min_pool_size (#822)
Currently, `connect_timeout` sounds like it should be for connections to
the Postgres server. It's actually used for obtaining a connection from
the pool.
2024-09-18 06:56:17 -05:00
Shijun Wang
efbab1c333 Helm chart improvements including allowing user password to be pulled from K8s secret (#753)
* Make user min_pool_size configurable

* Set user server_lifetime only if specified

* Increment chart version

* Use default instea of or

* Allow enabling server_tls

* statement_timeout default value

* Allow pulling password from existing secret

---------

Co-authored-by: Mostafa Abdelraouf <mostafa.mohmmed@gmail.com>
2024-09-14 09:57:17 -05:00
Mostafa Abdelraouf
9f12d7958e Fix Ruby tests (#819)
Build is failing with this error

Downloading activerecord-3.2.14 revealed dependencies not in the API or the
lockfile (activesupport (= 3.2.14), activemodel (= 3.2.14), arel (~> 3.0.2),
tzinfo (~> 0.3.29)).
Either installing with `--full-index` or running `bundle update activerecord`
should fix the problem.

After ActiveSupport was updated.

This PR fixes that
2024-09-13 20:02:38 -05:00
dependabot[bot]
e6634ef461 chore(deps): bump activesupport from 7.0.4.1 to 7.0.7.1 in /tests/ruby (#804)
Bumps [activesupport](https://github.com/rails/rails) from 7.0.4.1 to 7.0.7.1.
- [Release notes](https://github.com/rails/rails/releases)
- [Changelog](https://github.com/rails/rails/blob/v7.2.1/activesupport/CHANGELOG.md)
- [Commits](https://github.com/rails/rails/compare/v7.0.4.1...v7.0.7.1)

---
updated-dependencies:
- dependency-name: activesupport
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-13 19:43:26 -05:00
dependabot[bot]
dab2e58647 chore(deps): bump helm/chart-releaser-action from 1.5.0 to 1.6.0 (#812)
Bumps [helm/chart-releaser-action](https://github.com/helm/chart-releaser-action) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/helm/chart-releaser-action/releases)
- [Commits](be16258da8...a917fd15b2)

---
updated-dependencies:
- dependency-name: helm/chart-releaser-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-13 19:41:25 -05:00
dependabot[bot]
4aaa4378cf chore(deps): bump rexml from 3.2.8 to 3.3.6 in /tests/ruby (#803)
Bumps [rexml](https://github.com/ruby/rexml) from 3.2.8 to 3.3.6.
- [Release notes](https://github.com/ruby/rexml/releases)
- [Changelog](https://github.com/ruby/rexml/blob/master/NEWS.md)
- [Commits](https://github.com/ruby/rexml/compare/v3.2.8...v3.3.6)

---
updated-dependencies:
- dependency-name: rexml
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-13 19:19:30 -05:00
Andrew Jackson
670311daf9 Implement Trust Authentication (#805)
* Implement Trust Authentication

* Remove remaining LDAP stuff

* Reverted LDAP changes, Cleaned up tests

---------

Co-authored-by: Andrew Jackson <andrewjackson2988@gmail.com>
Co-authored-by: CommanderKeynes <andrewjackson947@gmail.coma>
2024-09-10 09:29:45 -05:00
dependabot[bot]
b9ec7f8036 chore(deps): bump actions/setup-python from 4.1.0 to 5.1.0 (#715)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.1.0 to 5.1.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.1.0...v5.1.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-07 12:21:21 -05:00
dependabot[bot]
d91d23848b chore(deps): bump helm/kind-action from 1.7.0 to 1.10.0 (#732)
Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.7.0 to 1.10.0.
- [Release notes](https://github.com/helm/kind-action/releases)
- [Commits](https://github.com/helm/kind-action/compare/v1.7.0...v1.10.0)

---
updated-dependencies:
- dependency-name: helm/kind-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-07 12:20:38 -05:00
dependabot[bot]
bbbc01a467 chore(deps): bump rexml from 3.2.5 to 3.2.8 in /tests/ruby (#743)
Bumps [rexml](https://github.com/ruby/rexml) from 3.2.5 to 3.2.8.
- [Release notes](https://github.com/ruby/rexml/releases)
- [Changelog](https://github.com/ruby/rexml/blob/master/NEWS.md)
- [Commits](https://github.com/ruby/rexml/compare/v3.2.5...v3.2.8)

---
updated-dependencies:
- dependency-name: rexml
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-07 12:20:01 -05:00
Sebastian Serth
9bb71ede9d Automatically build deb package on a new version tag (#801)
In #796, I noticed that the deb package was not build since an automation was missing.

With this PR, I add the missing automation.

I tested the workflow in my repo...

    when starting the workflow manually: https://github.com/MrSerth/pgcat/actions/runs/10737879151/job/29780286094
    when drafting a new release: https://github.com/MrSerth/pgcat/actions/runs/10737835796/job/29780146212

Obviously, both workflows failed since I cannot upload to the APT repo. However, the version substitution for the workflow is working correctly (as shown when collapsing the first line of the "Build and release package" step).
2024-09-06 09:11:52 -05:00
Sebastian Serth
88b2afb19b Automatically start systemd service if config file is present (#800)
Previously, upgrading the deb package stopped the service but didn't reenable it after a successful upgrade. This made upgrading the package more difficult and required a second step to restart the service. With this commit, the systemd service is automatically started when the default config file is present.
2024-09-06 09:07:01 -05:00
Mostafa Abdelraouf
f0865ca616 Improve Prometheus exporter output (#795)
* Prometheus metrics updates:

 * Add username label to deconflict metrics that would otherwise
   have duplicate labels across different pools.
 * Group metrics by name and only print HELP and TYPE once per
   metric name.
 * Sort labels for a deterministic output.

---------

Co-authored-by: Curtis Myzie <curtis.myzie@gmail.com>
Co-authored-by: Towhid Khan
2024-09-05 08:58:18 -05:00
Andrew Jackson
7d047c6c19 Implemented python tests with pytest (#790)
Currently the python tests act as scripts. A lot of output is generated to stdout which makes it very hard to figure out where problems were. Also if you want to run only a single test you basically need to comment out code in order to accomplish this.

This PR modifies the python tests to us the pytest python testing framework. This framework allows individual tests to be targeted via the command line, without touching the source code. It also suppressed stdout by default making the test output much easier to read. Also after the tests run it will provide a summary of what failed, what succeded, etc.


Co-authored-by: CommanderKeynes <andrewjackson947@gmail.coma>
Co-authored-by: Andrew Jackson <andrewjackson2988@gmail.com>
2024-09-05 08:16:45 -05:00
Andrew Jackson
f73d15f82c Fix CI script to allow consecutive runs locally (#793)
Co-authored-by: CommanderKeynes <andrewjackson947@gmail.coma>
2024-09-05 08:01:33 -05:00
Mostafa Abdelraouf
69af6cc5e5 Make iterating on integration tests easier (#789)
Writing and iterating on integration tests are cumbersome, having to wait 10 minutes for the test-suite to run just to see if your test works or not is unacceptable.

In this PR, I added a detailed workflow for writing tests that should shorten the feedback cycle of modifying tests to be as low as a few seconds.

It will involve opening a shell into a long-lived container that has all the setup and dependencies necessary and then running your desired tests directly there. I added a convenience script that bootstraps the environment and then opens an interactive shell into the container and you can then run tests immediately in an environment that is more or less identical to what we have running in CircleCI
2024-09-03 11:15:53 -05:00
Mostafa Abdelraouf
ca34597002 Fix broken integration test #740 (#787) 2024-08-31 17:15:13 -05:00
Mostafa Abdelraouf
2def40ea6a Add test case for issue 776 (#786)
I am adding a tiny test that uses the SQL statement that was reported to break an older version of SQL parser library

#776
2024-08-31 10:52:33 -05:00
Mostafa Abdelraouf
c05129018d Improve Prometheus stats + Add Grafana dashboard (#785)
We were missing some labels on metrics generated by the Prometheus exporter so I fixed that. There are still some gaps that I want to address with respect to the metrics we track but this seems like a good start.

I also created a Grafana Dashboard and exported it to JSON. It is designed with the same metric names the Prometheus exporter uses.
2024-08-31 08:18:57 -05:00
Mostafa Abdelraouf
4a7a6a8e7a Cut 1.2.0 release (#783) 2024-08-30 08:30:16 -05:00
Mostafa Abdelraouf
29a476e190 QueryRouter: route to primary when locks exists (select for update) (#782)
Authored-by: Javier Goday <jgoday@gmail.com>
2024-08-30 04:26:36 -05:00
KwongTN
81933b918d Add linux/arm64 docker image build support (#774) 2024-08-29 13:50:38 -05:00
Saraj Munjal
7cbc9178d8 Bump the hyper crate to v1.4.1 and rework prometheus server handling (#778)
Bump hyper to v1.4.1 and rework prometheus server handling
2024-08-29 09:47:58 -05:00
Mostafa Abdelraouf
2c8b2f0776 Fix CI image build step (#780)
The docker CI build image is failing due to this error

249.5     Finished release [optimized] target(s) in 2m 49s
249.5   Installing /home/circleci/.cargo/bin/rustfilt
249.5    Installed package `rustfilt v0.2.1` (executable `rustfilt`)
249.5 error: failed to compile `cargo-binutils v0.3.6`, intermediate artifacts can be found at `/tmp/cargo-installrWENQG`
249.5 
249.5 Caused by:
249.5   package `cargo-platform v0.1.8` cannot be built because it requires rustc 1.73 or newer, while the currently active rustc version is 1.67.1
249.5   Try re-running cargo install with `--locked`
249.5      Summary Successfully installed rustfilt! Failed to install cargo-binutils (see error(s) above).
249.5 error: some crates failed to install

So I am bumping the version up
2024-08-29 08:37:13 -05:00
Mostafa Abdelraouf
8f9a2b8e6f Fix a Panic in admin commands (#779)
We have a panic when we send SHOW or ;;;;;;;;;;;;;;;;; to admin database.

This PR fixes these panics and adds a couple of tests
2024-08-28 21:29:40 -05:00
brandonpike
cbf4d58144 Fix lint warnings for rust-1.79 (#769)
2 things that are recommended by rust-lang - implementing `std::fmt::Display` rather than ToString (1) and using clone_from (2).

[1] https://rust-lang.github.io/rust-clippy/master/index.html#/to_string_trait_impl
[2] https://rust-lang.github.io/rust-clippy/master/index.html#assigning_clones

Signed-off-by: Brandon Pike <pikebrandon@att.net>
2024-07-15 20:30:26 -07:00
Олег Дулецкий
731aa047ba Add ExecReload option to pgcat.service for configuration reloads (#760) 2024-06-24 08:57:58 -07:00
Adrian Garcia Badaracco
88dbcc21d1 update rust version in docker image (#762) 2024-06-24 08:51:38 -07:00
Adrian Garcia Badaracco
c34b15bddc Add STOPSIGNAL to Dockerfile (#758) 2024-06-20 23:23:41 -07:00
Andrey Stikheev
0b034a6831 Add TCP_NODELAY option to improve performance for large response queries (#749)
This commit adds the TCP_NODELAY option to the socket configuration in
`configure_socket` function. Without this option, we observed significant
performance issues when executing SELECT queries with large responses.

Before the fix:
postgres=> SELECT repeat('a', 1); SELECT repeat('a', 8153);
Time: 1.368 ms
Time: 41.364 ms

After the fix:
postgres=> SELECT repeat('a', 1); SELECT repeat('a', 8153);
Time: 1.332 ms
Time: 1.528 ms

By setting TCP_NODELAY, we eliminate the Nagle's algorithm delay, which
results in a substantial improvement in response times for large queries.

This problem was discussed in https://github.com/postgresml/pgcat/issues/616.
2024-05-26 14:47:21 -07:00
Mostafa Abdelraouf
966b8e093c Report checkout error when all servers are down (#736)
We shouldn't report checkout_success when we are going to return Error.
2024-05-08 12:18:27 -05:00
Horacio
c9270a47d4 Use rust:bullseye as base image (#725)
Use rust:bullseye base image

With the original rust:1.70-bullseye image, the container cannot be
built:

17.06   Installing /usr/local/cargo/bin/rustfilt
17.06    Installed package `rustfilt v0.2.1` (executable `rustfilt`)
17.06 error: failed to compile `cargo-binutils v0.3.6`, intermediate artifacts can be found at `/tmp/cargo-installrc6mPb`
17.06
17.06 Caused by:
17.06   package `cargo-platform v0.1.8` cannot be built because it requires rustc 1.73 or newer, while the currently active rustc version is 1.70.0
17.06   Try re-running cargo install with `--locked`
17.06      Summary Successfully installed rustfilt! Failed to install cargo-binutils (see error(s) above).
17.06 error: some crates failed to install

This is the same base image used on tests/docker/Dockerfile
2024-04-19 09:12:57 -07:00
Toby Hede
0d94d0b90a Update sqlparser to 0.41 (#666) 2024-04-12 22:12:37 -07:00
David ALEXANDRE
358724f7a9 feat: add helm chart (#619)
* add workflow

* feat: add pgcat helm chart

* fix: set the right include into configmap

Signed-off-by: David ALEXANDRE <david.alexandre@w6d.io>

* update values and config

* prettifying config

---------

Signed-off-by: David ALEXANDRE <david.alexandre@w6d.io>
2024-02-22 09:26:58 -08:00
Mostafa Abdelraouf
e1e4929d43 Report waiting time only for currently waiting clients (#678)
The pool maxwait metric currently operates differently from Pgbouncer.

The way it operates today is that we keep track of max_wait on each connected client, when SHOW POOLS query is made, we go over the connected clients and we get the max of max_wait times among clients. This means the pool maxwait will never reset, it will always be monotonically increasing until the client with the highest maxwait disconnects.

This PR changes this behavior, by keeping track of the wait_start time on each client, when a client goes into WAITING state, we record the time offset from connect_time. When we either successfully or unsuccessfully checkout a connection from the pool, we reset the wait_start time.

When SHOW POOLS query is made, we go over all connected clients and we only consider clients whose wait_start is non-zero, for clients that have non-zero wait times, we compare them and report the maximum waiting time as maxwait for the pool.
2024-01-18 11:57:28 -06:00
Lev Kokotov
dc4d6edf17 Revert max_wait changes (#658)
* Revert "Reset wait times when checked out successfully (#656)"

This reverts commit ec3920d60f.

* Revert "Not sure how this sneaked past CI"

This reverts commit 4c5498b915.

* Revert "only report wait times from clients currently waiting to match behavior of pgbouncer (#655)"

This reverts commit 0e8064b049.
2023-12-05 01:47:38 -08:00
Lev Kokotov
ec3920d60f Reset wait times when checked out successfully (#656) 2023-12-04 18:33:08 -08:00
Lev
4c5498b915 Not sure how this sneaked past CI 2023-12-04 18:30:03 -08:00
Daniel Babiak
0e8064b049 only report wait times from clients currently waiting to match behavior of pgbouncer (#655)
* Change maxwait to only report wait times from clients currently waiting to match behavior of pgbouncer

* Fix tests
2023-12-04 18:19:51 -08:00
Alec
4dbef49ec9 Require a reason when marking a server bad (#654)
When calling mark_bad require a reason so it can be logged rather than
the generic message
2023-12-04 16:09:41 -08:00
Lev Kokotov
bc07dc9c81 Broken blog link 2 2023-12-03 21:01:23 -08:00
Lev Kokotov
9b8166b313 Broken blog link (#652)
Update README.md
2023-12-03 20:58:39 -08:00
Lev Kokotov
e58d69f3de Fix deb build overwriting config (#651) 2023-12-03 20:27:44 -08:00
Lev Kokotov
e76d720ffb Dont cache prepared statement with errors (#647)
* Fix prepared statement not found when prepared stmt has error

* cleanup debug

* remove more debug msgs

* sure debugged this..

* version bump

* add rust tests
2023-11-28 21:13:30 -08:00
Calvin Hughes
998cc16a3c Expose clients maxwait time in SHOW CLIENTS response via admin (#639)
* Expose clients maxwait time in SHOW CLIENTS response via PgCat admin
Displays the maxwait via maxwait_seconds and maxwait_us columns for each client that can be used to track down the wait time per client in a case where the overall pool stats shows waiting time. The maxwait_us, similar to the pool stats setup, is configured to display as a remainder alongside the maxwait_seconds.

* Use maxwait instead of maxwait_seconds to match pools column name

---------

Co-authored-by: Calvin Hughes <9379992+calvinhughes@users.noreply.github.com>
2023-11-13 11:24:39 -08:00
Jakob Schultz-Falk
7c37da2fad Support unnamed prepared statements (#635)
* Add golang test suite to reproduce issue with unnamed parameterized prepared statements

* Allow caching of unnamed prepared statements

* Passthrough describe on portals

* Remove unneeded kill

* Update Dockerfile.ci with golang

* Move out update of Dockerfiles to separate PR
2023-11-08 16:36:45 -08:00
Jakob Schultz-Falk
b45c6b1d23 Update Dockerfile.ci with golang (#637) 2023-11-08 08:25:49 -08:00
Lev Kokotov
dae240d30c Add connet_timeout and idle_timeout to the user (#634)
* Add connect_timeout to the user

* Allow user to override connect timeout

* version

* lock

* Add both timeouts to the user
2023-11-06 12:18:52 -08:00
Lev Kokotov
b52ea8e7f1 bump version (#629) 2023-10-26 10:50:45 -07:00
Zain Kabani
7d3003a16a Reimplement prepared statements with LRU cache and statement deduplication (#618)
* Initial commit

* Cleanup and add stats

* Use an arc instead of full clones to store the parse packets

* Use mutex instead

* fmt

* clippy

* fmt

* fix?

* fix?

* fmt

* typo

* Update docs

* Refactor custom protocol

* fmt

* move custom protocol handling to before parsing

* Support describe

* Add LRU for server side statement cache

* rename variable

* Refactoring

* Move docs

* Fix test

* fix

* Update tests

* trigger build

* Add more tests

* Reorder handling sync

* Support when a named describe is sent along with Parse (go pgx) and expecting results

* don't talk to client if not needed when client sends Parse

* fmt :(

* refactor tests

* nit

* Reduce hashing

* Reducing work done to decode describe and parse messages

* minor refactor

* Merge branch 'main' into zain/reimplment-prepared-statements-with-global-lru-cache

* Rewrite extended and prepared protocol message handling to better support mocking response packets and close

* An attempt to better handle if there are DDL changes that might break cached plans with ideas about how to further improve it

* fix

* Minor stats fixed and cleanup

* Cosmetic fixes (#64)

* Cosmetic fixes

* fix test

* Change server drop for statement cache error to a `deallocate all`

* Updated comments and added new idea for handling DDL changes impacting cached plans

* fix test?

* Revert test change

* trigger build, flakey test

* Avoid potential race conditions by changing get_or_insert to promote for pool LRU

* remove ps enabled variable on the server in favor of using an option

* Add close to the Extended Protocol buffer

---------

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
2023-10-25 15:11:57 -07:00
Zain Kabani
d37df43a90 Reduces the amount of time the get_pool operation takes (#625)
* Reduces the amount of time the get_pool operation takes

* trigger build

* Fix admin
2023-10-19 23:49:05 -07:00
Mohammad Dashti
2c7bf52c17 Removed unnecessary clippy overrides. (#614)
Removed unnecessary clippy overrides.
2023-10-11 10:13:23 -07:00
Mohammad Dashti
de8df29ca4 Added clippy to CI and fixed all clippy warnings (#613)
* Fixed all clippy warnings.

* Added `clippy` to CI.

* Reverted an unwanted change + Applied `cargo fmt`.

* Fixed the idiom version.

* Revert "Fixed the idiom version."

This reverts commit 6f78be0d42.

* Fixed clippy issues on CI.

* Revert "Fixed clippy issues on CI."

This reverts commit a9fa6ba189.

* Revert "Reverted an unwanted change + Applied `cargo fmt`."

This reverts commit 6bd37b6479.

* Revert "Fixed all clippy warnings."

This reverts commit d1f3b847e3.

* Removed Clippy

* Removed Lint

* `admin.rs` clippy fixes.

* Applied more clippy changes.

* Even more clippy changes.

* `client.rs` clippy fixes.

* `server.rs` clippy fixes.

* Revert "Removed Lint"

This reverts commit cb5042b144.

* Revert "Removed Clippy"

This reverts commit 6dec8bffb1.

* Applied lint.

* Revert "Revert "Fixed clippy issues on CI.""

This reverts commit 49164a733c.
2023-10-10 09:18:21 -07:00
Mohammad Dashti
c4fb72b9fc Added yj to dev Dockerfile (#612) 2023-10-05 18:13:22 -07:00
Mohammad Dashti
3371c01e0e Added a Plugin trait (#536)
* Improved logging

* Improved logging for more `Address` usages

* Fixed lint issues.

* Reverted the `Address` logging changes.

* Applied the PR comment by @levkk.

* Applied the PR comment by @levkk.

* Applied the PR comment by @levkk.

* Applied the PR comment by @levkk.
2023-10-03 13:13:21 -07:00
Mohammad Dashti
c2a483f36a Automatic sharding for INSERT, UPDATE, and DELETE statements. (#610)
Added support for INSERT, UPDATE, and DELETE for auto-sharding.
2023-10-03 09:36:13 -07:00
dependabot[bot]
51cd13b8b5 chore(deps): bump webpki from 0.22.0 to 0.22.2 in /tests/rust (#609)
Bumps [webpki](https://github.com/briansmith/webpki) from 0.22.0 to 0.22.2.
- [Commits](https://github.com/briansmith/webpki/commits)

---
updated-dependencies:
- dependency-name: webpki
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-02 15:30:42 -07:00
Nicolas Vanelslande
a054b454d2 Add psql to the container image. (#607)
It could be used to implement container health checks.
Example:
  PGPASSWORD="<some-password>" psql -U pgcat -p 6432 -h 127.0.0.1 -tA -c "show version;" -d pgcat >/dev/null
2023-09-27 09:03:39 -07:00
Kevin Elliott
04e9814770 Fix incorrect data output for plugin query_logger (#601)
Update query_logger.rs

Pool and user were incorrectly swapped and needed to be fixed.
2023-09-25 18:45:51 -07:00
Lev Kokotov
037d232fcd Mark admin clients as disconnected on error (#597) 2023-09-21 15:55:22 -07:00
Lev Kokotov
b2933762e7 Report maxwait for clients that end up not getting a connection (#596) 2023-09-21 14:50:18 -07:00
Mohammad Dashti
df8aa888f9 Add a cache layer to Docker for development (#594)
* Add a cache layer to Docker.

* Created a separate `dev` Docker file.

* Fixed `Docker.dev` to build in non-release mode.
2023-09-20 10:29:30 -07:00
Mohammad Dashti
7f5639c94a Include thread_id in the logs (#592)
Include `thread_id` in the logs.
2023-09-20 09:11:16 -07:00
Lev Kokotov
c0112f6f12 Revert "User-friendly error messages" (#587)
Revert "User-friendly error messages (#586)"

This reverts commit b7ceee2ddf.
2023-09-11 16:39:31 -07:00
Lev Kokotov
b7ceee2ddf User-friendly error messages (#586) 2023-09-11 16:39:11 -07:00
Mostafa Abdelraouf
0b01d70b55 Allow configuring routing decision when no shard is selected (#578)
The TL;DR for the change is that we allow QueryRouter to set the active shard to None. This signals to the Pool::get method that we have no shard selected. The get method follows a no_shard_specified_behavior config to know how to route the query.

Original PR description
Ruby-pg library makes a startup query to SET client_encoding to ... if Encoding.default_internal value is set (Code). This query is troublesome because we cannot possibly attach a routing comment to it. PgCat, by default, will route that query to the default shard.

Everything is fine until shard 0 has issues, Clients will all be attempting to send this query to shard0 which increases the connection latency significantly for all clients, even those not interested in shard0

This PR introduces no_shard_specified_behavior that defines the behavior in case we have routing-by-comment enabled but we get a query without a comment. The allowed behaviors are

random: Picks a shard at random
random_healthy: Picks a shard at random favoring shards with the least number of recent connection/checkout errors
shard_<number>: e.g. shard_0, shard_4, etc. picks a specific shard, everytime
In order to achieve this, this PR introduces an error_count on the Address Object that tracks the number of errors since the last checkout and uses that metric to sort shards by error count before making a routing decision.
I didn't want to use address stats to avoid introducing a routing dependency on internal stats (We might do that in the future but I prefer to avoid this for the time being.

I also made changes to the test environment to replace Ruby's TOML reader library, It appears to be abandoned and does not support mixed arrays (which we use in the config toml), and it also does not play nicely with single-quoted regular expressions. I opted for using yj which is a CLI tool that can convert from toml to JSON and back. So I refactor the tests to use that library.
2023-09-11 13:47:28 -05:00
hellower
33db0dffa8 stream.peer_addr() & auth_query (#575)
* Don't unwrap stream.peer_addr()

https://github.com/postgresml/pgcat/pull/562 (same code)
(another lines changed)

* auth_query (real sample)

# single quote need
auth_query="SELECT usename, passwd FROM pg_shadow WHERE usename='$1'"
2023-08-31 14:11:38 -07:00
hi019
7994a661d9 Fix Docker image runs erroring due to glibc incompatability (#572)
Fix Docker image builds breaking due to glibc incompatability
2023-08-30 16:51:31 -07:00
Tommy Li
9937193332 Allow pause/resuming all pools (#566)
support pausing all pools
2023-08-29 10:07:36 -07:00
Mostafa Abdelraouf
baa00ff546 Add yj to CI image (#568) 2023-08-28 21:20:53 -05:00
Zain Kabani
ffe820497f Don't unwrap stream.peer_addr() (#562) 2023-08-25 10:33:39 -07:00
Zain Kabani
be549f3faa Fixes try_execute_command message parsing bug (#560)
* Fixes try_execute_command message parsing bug

* Fix initial segment logic

* Add test
2023-08-24 11:25:43 -07:00
dependabot[bot]
4301ab0606 chore(deps): bump rustls-webpki from 0.100.1 to 0.100.2 (#555)
Bumps [rustls-webpki](https://github.com/rustls/webpki) from 0.100.1 to 0.100.2.
- [Release notes](https://github.com/rustls/webpki/releases)
- [Commits](https://github.com/rustls/webpki/compare/v/0.100.1...v/0.100.2)

---
updated-dependencies:
- dependency-name: rustls-webpki
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-22 11:41:09 -07:00
Cluas
5143500c9a docs: complete the missing general items (#553)
docs: complete the missing general items.
2023-08-20 19:14:19 -07:00
Zain Kabani
3255323bff Adds option to log which parameter status is changed by the client (#550) 2023-08-16 11:01:21 -07:00
Zain Kabani
bb27586758 Reset instead of discard all (#549)
* Use reset all instead of discard all

* Move 'X' handling to before admin handle

* fix tests
2023-08-16 10:08:48 -07:00
Lev Kokotov
4f0f45b576 Add pgcat user (#546)
* Add pgcat user

* warn

* dev
2023-08-10 12:25:43 -07:00
Zain Kabani
f94ce97ebc Handle and track startup parameters (#478)
* User server parameters struct instead of server info bytesmut

* Refactor to use hashmap for all params and add server parameters to client

* Sync parameters on client server checkout

* minor refactor

* update client side parameters when changed

* Move the SET statement logic from the C packet to the S packet.

* trigger build

* revert validation changes

* remove comment

* Try fix

* Reset cleanup state after sync

* fix server version test

* Track application name through client life for stats

* Add tests

* minor refactoring

* fmt

* fix

* fmt
2023-08-10 08:18:46 -07:00
Sebastian Webber
9ab128579d parse server error messages (#543)
This commit adds a parser to the Postgres error message, providing better
error messages.

Implemented based in:
  https://www.postgresql.org/docs/12/protocol-error-fields.html

Signed-off-by: Sebastian Webber <sebastian@swebber.me>
2023-08-09 09:14:05 -07:00
Lev Kokotov
1cde74f05e Revert "Preserve existing behavior" (#542)
Revert "Preserve existing behavior (#541)"

This reverts commit a4de6c1eb6.
2023-08-08 17:45:48 -07:00
Lev Kokotov
a4de6c1eb6 Preserve existing behavior (#541) 2023-08-08 13:48:52 -07:00
Zain Kabani
e14b283f0c Make infer role configurable and fix double parse bug (#533)
* Make infer role configurable and fix double parse bug

* Fix tests

* Enable infer_role_from query in toml for tests

* Fix test

* Add max length config, add logging for which application is failing to parse, and change config name

* fmt

* Update src/config.rs

---------

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
2023-08-08 13:10:03 -07:00
Lev Kokotov
7c3c90c38e Add systemd service (#540) 2023-08-08 11:51:38 -07:00
Lev Kokotov
2ca21b2bec pgcat deb package (#539) 2023-08-08 11:08:46 -07:00
Matthias Pfeil
3986eaa4b2 Add github tag as tag to image (#537) 2023-08-04 10:20:56 -07:00
Lev Kokotov
1f2c6507f7 debug -> release 2023-08-01 17:47:34 -07:00
Lev Kokotov
aefcf4281c Fix for #534 and #535 2023-08-01 17:46:34 -07:00
Bertrand Paquet
9d1c46a3e9 Fix typo in the config documentation (#532) 2023-07-28 00:31:53 -07:00
Spindel Ljungmark
328108aeb5 Restore the ability to filter spammy log messages (#530)
* Move connection checkin log messages to their own target

Under heavy load they can happen thousands of times per second, and
should generally be considered a nuisance at best. This marks the state
discard as an info rather than a warning, and moves all the messages
into their own log-target, so they can be filtered separately from the
more relevant warnings.

Signed-off-by: D.S. Ljungmark <spider@skuggor.se>

* Remove left-over env_logger dependencies

When moving to tracing-subscriber for logging, the env_logger
dependencies were left around, this cuts them out as dead code.

Signed-off-by: D.S. Ljungmark <spider@skuggor.se>

* Restore ability to filter log messages at runtime

This restores the RUST_LOG filters from env_logger but now with the
tracing subscriber setup. The filters are chained so commandline options
mark the default in case either option is set, which should be the path
of least confusion for users.  ( RUST_LOG setting level to debug, and
commandline to warning is an odd user case, and I don't know what a user
who does that is expecting. )

It also bumps the version number as a fix to see which versions have
which behaviour.

Signed-off-by: D.S. Ljungmark <spider@skuggor.se>

---------

Signed-off-by: D.S. Ljungmark <spider@skuggor.se>
2023-07-27 08:51:23 -07:00
Lev Kokotov
4cf54a6122 Release 1.1 (#526) 2023-07-25 10:27:04 -07:00
Mostafa Abdelraouf
2a8f3653a6 Fix COPY FROM and add tests (#522)
* Fix COPY FROM and add tests

* E

* fmt
2023-07-20 23:06:01 -07:00
Sebastian Webber
19cb8a3022 add --no-color option to disable colors in the terminal (#518)
add --no-color option to disable colors

this commit adds a new option to disable colors in the terminal and also
moves the logger configuration to a different crate.

Signed-off-by: Sebastian Webber <sebastian@swebber.me>
2023-07-19 21:15:55 -07:00
Sebastian Webber
f85e5bd9e8 add support for multiple log formats (#517)
this commit adds the tracing-subscriber crate and use its formatters to
support multiple log formats.

More details in
https://github.com/postgresml/pgcat/issues/464#issuecomment-1641430299

Signed-off-by: Sebastian Webber <sebastian@swebber.me>
2023-07-18 23:07:13 -07:00
Sebastian Webber
7bdb4e5cd9 Add cmd line parser (#512)
This commit adds the clap library and configures the necessary args to
parse from the command line,  expanding the current option of a single
file and adding support for environment variables.

Signed-off-by: Sebastian Webber <sebastian@swebber.me>
2023-07-18 13:52:40 -07:00
Sebastian Webber
5d87e3781e push and build only in main and tags (#508)
this commit changes the CI behavior to only build and push when something is committed to main or is a new tag.
2023-07-14 10:30:49 -07:00
dependabot[bot]
3e08c6bd8d chore(deps): bump num_cpus from 1.15.0 to 1.16.0 (#507)
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-14 07:58:11 -07:00
Sebastian Webber
15b6db8e4e add "show help" command (#505)
This commit adds a new function to handle notify and use it
in the SHOW HELP command, which displays the available options
in the admin console.

Also, adding Fabrízio as a co-author for all the help with the
protocol and the help to structure this PR.

Signed-off-by: Sebastian Webber <sebastian@swebber.me>
Co-authored-by: Fabrízio de Royes Mello <fabriziomello@gmail.com>
2023-07-13 22:40:04 -07:00
dependabot[bot]
b2e6dfd9bb chore(deps): bump rustls-pemfile from 1.0.2 to 1.0.3 (#504)
Bumps [rustls-pemfile](https://github.com/rustls/pemfile) from 1.0.2 to 1.0.3.
- [Commits](https://github.com/rustls/pemfile/commits)

---
updated-dependencies:
- dependency-name: rustls-pemfile
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-12 21:41:48 -07:00
Mostafa Abdelraouf
3c9565d351 Add support for tcp_user_timeout (#503)
* Add support for tcp_user_timeout

* option

* duration

* Some()

* docs

* fmt, compile
2023-07-12 11:24:30 -07:00
dependabot[bot]
67579c9af4 chore(deps): bump rustls from 0.21.1 to 0.21.5 (#501)
Bumps [rustls](https://github.com/rustls/rustls) from 0.21.1 to 0.21.5.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Commits](https://github.com/rustls/rustls/compare/v/0.21.1...v/0.21.5)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-12 05:46:31 -07:00
Cluas
cf7f6f35ab docs: fix general.autoreload description (#491)
* docs: fix autoreload description

Signed-off-by: Cluas <Cluas@live.cn>

* docs: add blank line

Signed-off-by: Cluas <Cluas@live.cn>

---------

Signed-off-by: Cluas <Cluas@live.cn>
2023-07-12 05:42:44 -07:00
Voldemarich
7205537b49 [BUG] Fix binding of NULL value parameters in prepared statements (#496)
Fix binding of NULL value parameters in prepared statements

Co-authored-by: anon <anon@non.existent>
2023-07-10 10:35:43 +02:00
Zain Kabani
1ed6e925ed Fixes the default for round robing in General (#488) 2023-06-23 09:15:44 -07:00
Lev Kokotov
4b78af9676 Implement Close for prepared statements (#482)
* Partial support for Close

* Close

* respect config value

* prepared spec

* Hmm

* Print cache size
2023-06-18 23:02:34 -07:00
Lev Kokotov
73500c0c96 Fix build (#481) 2023-06-17 09:09:54 -07:00
Lev Kokotov
b167de5aa3 fmt (#480) 2023-06-17 08:57:33 -07:00
Juraj Bubniak
473bb3d17d Log not implemented messages as debug in prometheus metrics. (#477) 2023-06-16 18:48:38 -07:00
Lev Kokotov
c7d6273037 Support for prepared statements (#474)
* Start prepared statements

* parse

* Ok

* optional

* dont rewrite anonymous prepared stmts

* Dont rewrite anonymous prep statements

* hm?

* prep statements

* I see!

* comment

* Print config value

* Rewrite bind and add sqlx test

* fmt

* ok

* Fix

* Fix stats

* its late

* clean up PREPARE
2023-06-16 12:57:44 -07:00
Jeff Chen
94c781881f Report min_pool_size correctly (#471) 2023-06-12 09:23:56 -07:00
dependabot[bot]
a8c81e5df6 chore(deps): bump pin-project from 1.0.12 to 1.1.0 (#440)
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.0.12 to 1.1.0.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.0.12...v1.1.0)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-12 00:51:24 -07:00
dependabot[bot]
1d3746ec9e chore(deps): bump sqlparser from 0.33.0 to 0.34.0 (#448)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.33.0 to 0.34.0.
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.33.0...v0.34.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-12 00:51:05 -07:00
dependabot[bot]
b5489dc1e6 chore(deps): bump regex from 1.8.1 to 1.8.4 (#466)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.1 to 1.8.4.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.1...1.8.4)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-12 00:50:46 -07:00
dependabot[bot]
557b425fb1 chore(deps): bump log from 0.4.17 to 0.4.19 (#470)
Bumps [log](https://github.com/rust-lang/log) from 0.4.17 to 0.4.19.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.17...0.4.19)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-12 00:50:26 -07:00
Zain Kabani
aca9738821 Make queue strategy configurable and default to Fifo (#463)
* Change idle timeout default to 10 minutes

* Revert lifo for now while we investigate connection thrashing issues

* Make queue strategy configurable

* test revert idle time out

* Add pgcat start to python test
2023-06-09 11:35:20 -07:00
Zain Kabani
0bc453a771 Change default server lifetime and bump bb8 version to use LIFO correctly (#462)
Change default server lifetime and idle timeouts and bump bb8 version to use LIFO correctly
2023-05-31 08:25:42 -07:00
Zain Kabani
b67c33b6d0 Use latest bb8 and use Lifo as the queue strategy in the pool (#455)
* Use git bb8

* Use latest bb8 and change pool is use stack
2023-05-28 19:46:13 -07:00
Mostafa Abdelraouf
a8a30ad43b Refactor Pool Stats to be based off of Server/Client stats (#445)
What is wrong
Stats reported by SHOW POOLS seem to be leaking. We see lingering cl_idle , cl_waiting, and similarly for sv_idle , sv_active. We confirmed that these are reporting issues not actual lingering clients.

This behavior is readily reproducible by running

while true; do
    psql "postgres://sharding_user:sharding_user@localhost:6432/sharded_db" -c "SELECT 1" > /dev/null 2>&1  &
done

Why it happens
I wasn't able to get to figure our the reason for the bug but my best guess is that we have race conditions when updating pool-level stats. So even though individual update operations are atomic, we perform a check then update sequence which is not protected by a guard.
https://github.com/postgresml/pgcat/blob/main/src/stats/pool.rs#L174-L179

I am also suspecting that using Relaxed ordering might allow this behavior (I changed all operations to use Ordering::SeqCst but still got lingering clients)

How to fix
Since SHOW POOLS/SHOW SERVER/SHOW CLIENTS all show the current state of the proxy (as opposed to SHOW STATS which show aggregate values), this PR refactors SHOW POOLS to have it construct the results directly from SHOW SERVER and SHOW CLIENT datasets. This reduces the complexity of stat updates and eliminates the need for having locks when updating pool stats as we only care about updating individual client/server states.

This will change the semantics of maxwait, so instead of it holding the maxwait time ever encountered by a client (connected or disconnected), it will only consider connected clients which should be okay given PgCat tends to hold on to client connections more than Pgbouncer.
2023-05-23 08:44:49 -05:00
dependabot[bot]
d63be9b93a chore(deps): bump toml from 0.7.3 to 0.7.4 (#447)
Bumps [toml](https://github.com/toml-rs/toml) from 0.7.3 to 0.7.4.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.3...toml-v0.7.4)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-19 08:58:13 -07:00
Lev Kokotov
100778670c Ensure data makes it to the client (#446)
* Ensure data makes it to the client

* flush all buffers
2023-05-18 16:41:22 -07:00
Lev Kokotov
37e3349c24 Optionally clean up server connections (#444)
* Optionally clean up server connections

* move setting to pool

* fix test

* Print setting to screen

* fmt

* Fix pool_settings override in tests
2023-05-18 10:46:55 -07:00
Zain Kabani
7f57a89d75 Fix time based average stats (#442)
* keep track of current stats and zero them after updating averages

* Try tests

* typo

* remove commented test stuff

* Avoid dividing by zero

* Fix test

* refactor, get rid of iterator. do it manually

* trigger build

* Fix
2023-05-17 21:38:10 -07:00
Lev Kokotov
0898461c01 Allow to deploy pools without checking (#438) 2023-05-12 12:48:37 -07:00
Lev Kokotov
52b1b43850 Prewarmer (#435)
* Prewarmer

* hmm

* Tests

* default

* fix test

* Correct configuration

* Added minimal config example

* remove connect_timeout
2023-05-12 09:50:52 -07:00
Zain Kabani
0907f1b77f Improve logging for connection cleanup (#428)
* initial commit

* fix

* fmt
2023-05-11 17:40:10 -07:00
Zain Kabani
73260690b0 Fixes average stats bug (#436)
* Add test

* Fix test

* Add fix
2023-05-11 17:37:58 -07:00
Mostafa Abdelraouf
5056cbe8ed Fix docker-compose dev stack for Apple silicon (#432)
The docker-compose dev setup is broken under Apple silicon, starting the stack fails with the following error. Switching to a different docker image fixes the issue.
2023-05-10 10:24:35 -05:00
Lev Kokotov
571b02e178 Calculate averages correctly and preserve totals like before (#429)
* Reset totals after avg calculation

* like it used to be
2023-05-08 10:06:16 -07:00
Andrew Tanner
159eb89bf0 First try with role reset (#427)
* First try with role rest

* update

* extra line

* Update src/server.rs

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>

* Update tests/ruby/misc_spec.rb

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>

---------

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
2023-05-05 15:31:27 -07:00
Lev Kokotov
389993bf3e Accurate log messages (#425) 2023-05-05 08:27:19 -07:00
Lev Kokotov
ba5243b6dd Optionally validate config on boot (#423) 2023-05-03 17:07:23 -07:00
Lev Kokotov
128ef72911 lowercase config query (#422)
* lowercase config query

* remove debug
2023-05-03 16:47:20 -07:00
Lev Kokotov
811885f464 Actually plugins (#421)
* more plugins

* clean up

* fix tests

* fix flakey test
2023-05-03 16:13:45 -07:00
dependabot[bot]
d5e329fec5 chore(deps): bump regex from 1.8.0 to 1.8.1 (#413)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/commits/1.8.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-03 10:00:05 -07:00
Lev Kokotov
09e54e1175 Plugins! (#420)
* Some queries

* Plugins!!

* cleanup

* actual names

* the actual plugins

* comment

* fix tests

* Tests

* unused errors

* Increase reaper rate to actually enforce settings

* ok
2023-05-03 09:13:05 -07:00
dependabot[bot]
23819c8549 chore(deps): bump rustls from 0.21.0 to 0.21.1 (#419)
Bumps [rustls](https://github.com/rustls/rustls) from 0.21.0 to 0.21.1.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.21.0...v/0.21.1)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-02 07:32:44 -07:00
Jose Fernández
7dfbd993f2 Add dns_cache for server addresses as in pgbouncer (#249)
* Add dns_cache so server addresses are cached and invalidated when DNS changes.

Adds a module to deal with dns_cache feature. It's
main struct is CachedResolver, which is a simple thread safe
hostname <-> Ips cache with the ability to refresh resolutions
every `dns_max_ttl` seconds. This way, a client can check whether its
ip address has changed.

* Allow reloading dns cached

* Add documentation for dns_cached
2023-05-02 10:26:40 +02:00
Lev Kokotov
3601130ba1 Readme update (#418)
* Readme update

* m

* wording
2023-04-30 09:44:25 -07:00
Lev Kokotov
0d504032b2 Server TLS (#417)
* Server TLS

* Finish up TLS

* thats it

* diff

* remove dead code

* maybe?

* dirty shutdown

* skip flakey test

* remove unused error

* fetch config once
2023-04-30 09:41:46 -07:00
Lev Kokotov
4a87b4807d Add more pool settings (#416)
* Add some pool settings

* fmt
2023-04-26 16:33:26 -07:00
Shawn
cb5ff40a59 fix typo (#415)
chore: typo
2023-04-26 08:28:54 -07:00
dependabot[bot]
62b2d994c1 chore(deps): bump regex from 1.7.3 to 1.8.0 (#411)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.3 to 1.8.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/commits)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-21 06:33:52 -07:00
Lev Kokotov
66805d7e77 README updates (#409)
* Better table

* add image

* promote auth passthrough to stable

* fmt
2023-04-20 07:53:55 -07:00
Lev Kokotov
4ccc1e7fa3 Fix CONFIG (#408)
Fix readme
2023-04-19 07:45:26 -07:00
Lev Kokotov
3dae3d0777 Separate server and client passwords optionally (#407)
* Separate server and user passwords

* config
2023-04-18 09:57:17 -07:00
dependabot[bot]
a18eb42df5 chore(deps): bump serde from 1.0.159 to 1.0.160 (#404)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.159 to 1.0.160.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.159...v1.0.160)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-14 10:25:00 -07:00
dependabot[bot]
6aacf1fa19 chore(deps): bump serde_derive from 1.0.159 to 1.0.160 (#403)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.159 to 1.0.160.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.159...v1.0.160)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-14 10:24:52 -07:00
dependabot[bot]
8e99e65215 chore(deps): bump sqlparser from 0.32.0 to 0.33.0 (#399)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.32.0 to 0.33.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.32.0...v0.33.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-14 10:24:44 -07:00
dependabot[bot]
5dfbc102a9 chore(deps): bump hyper from 0.14.25 to 0.14.26 (#406)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.25 to 0.14.26.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.26/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.25...v0.14.26)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-14 10:23:52 -07:00
Cluas
bae12fca99 feat: set keepalive for pgcat server itself (#402)
* feat: set keepalive for pgcat server self

* docs: note also set for client
2023-04-12 09:29:43 -07:00
Lev Kokotov
421c5d4b64 Load config on client connect (#401) 2023-04-11 10:32:48 -07:00
Kian-Meng Ang
d568739db9 Fix typos (#398)
Found via `typos --format brief`
2023-04-10 18:37:16 -07:00
Lev Kokotov
692353c839 A couple things (#397)
* Format cleanup

* fmt

* finally
2023-04-10 14:51:01 -07:00
Lev Kokotov
a62f6b0eea Fix port; add user pool mode (#395)
* Fix port; add user pool mode

* will probably break our session/transaction mode tests
2023-04-05 15:06:19 -07:00
dependabot[bot]
89e15f09b5 chore(deps): bump tokio-rustls from 0.23.4 to 0.24.0 (#394)
Bumps [tokio-rustls](https://github.com/tokio-rs/tls) from 0.23.4 to 0.24.0.
- [Release notes](https://github.com/tokio-rs/tls/releases)
- [Commits](https://github.com/tokio-rs/tls/commits)

---
updated-dependencies:
- dependency-name: tokio-rustls
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-02 23:00:09 -07:00
Mostafa Abdelraouf
7ddd23b514 Protocol-level test helpers (#393)
I needed to have granular control over protocol message testing. For example, being able to send protocol messages one-by-one and then be able to inspect the results.

In order to do that, I created this low-level ruby client that can be used to send protocol messages in any order without blocking and also allows inspection of response messages.
2023-04-01 15:27:57 -05:00
dependabot[bot]
faa9c1f64a chore(deps): bump futures from 0.3.27 to 0.3.28 (#392)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.27 to 0.3.28.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.27...0.3.28)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-31 09:35:01 -07:00
dependabot[bot]
9094988491 chore(deps): bump postgres-protocol from 0.6.4 to 0.6.5 (#391)
Bumps [postgres-protocol](https://github.com/sfackler/rust-postgres) from 0.6.4 to 0.6.5.
- [Release notes](https://github.com/sfackler/rust-postgres/releases)
- [Commits](https://github.com/sfackler/rust-postgres/compare/postgres-protocol-v0.6.4...postgres-protocol-v0.6.5)

---
updated-dependencies:
- dependency-name: postgres-protocol
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-31 09:34:51 -07:00
Jose Fernández
6f768a84ce Auth passthrough (auth_query) (#266)
* Add a new exec_simple_query method

This adds a new `exec_simple_query` method so we can make 'out of band'
queries to servers that don't interfere with pools at all.
In order to reuse startup code for making these simple queries,
we need to set the stats (`Reporter`) optional, so using these
simple queries wont interfere with stats.

* Add auth passthough (auth_query)

Adds a feature that allows setting auth passthrough for md5 auth.

It adds 3 new (general and pool) config parameters:

- `auth_query`: An string containing a query that will be executed on boot
to obtain the hash of a given user. This query have to use a placeholder `$1`,
so pgcat can replace it with the user its trying to fetch the hash from.
- `auth_query_user`: The user to use for connecting to the server and executing the
auth_query.
- `auth_query_password`: The password to use for connecting to the server and executing the
auth_query.

The configuration can be done either on the general config (so pools share them) or in a per-pool basis.

The behavior is, at boot time, when validating server connections, a hash is fetched per server
and stored in the pool. When new server connections are created, and no cleartext password is specified,
the obtained hash is used for creating them, if the hash could not be obtained for whatever reason, it retries
it.

When client authentication is tried, it uses cleartext passwords if specified, it not, it checks whether
we have query_auth set up, if so, it tries to use the obtained hash for making client auth. If there is no
hash (we could not obtain one when validating the connection), a new fetch is tried.

Once we have a hash, we authenticate using it against whathever the client has sent us, if there is a failure
we refetch the hash and retry auth (so password changes can be done).

The idea with this 'retrial' mechanism is to make it fault tolerant, so if for whatever reason hash could not be
obtained during connection validation, or the password has change, we can still connect later.

* Add documentation for Auth passthrough
2023-03-30 13:29:23 -07:00
dependabot[bot]
0757d7f3a0 chore(deps): bump serde from 1.0.158 to 1.0.159 (#386)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.158 to 1.0.159.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.158...v1.0.159)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-28 09:54:39 -07:00
dependabot[bot]
568f04feee chore(deps): bump serde_derive from 1.0.154 to 1.0.159 (#387)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.154 to 1.0.159.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.154...v1.0.159)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-28 09:54:31 -07:00
Jose Fernández
58ce76d9b9 Refactor stats to use atomics (#375)
* Refactor stats to use atomics

When we are dealing with a high number of connections, generated
stats cannot be consumed fast enough by the stats collector loop.
This makes the stats subsystem inconsistent and a log of
warning messages are thrown due to unregistered server/clients.

This change refactors the stats subsystem so it uses atomics:

- Now counters are handled using U64 atomics
- Event system is dropped and averages are calculated using a loop
  every 15 seconds.
- Now, instead of snapshots being generated ever second we keep track of servers/clients
  that have registered. Each pool/server/client has its own instance of the counter and
  makes changes directly, instead of adding an event that gets processed later.

* Manually mplement Hash/Eq in `config::Address` ignoring stats

* Add tests for client connection counters

* Allow connecting to dockerized dev pgcat from the host

* stats: Decrease cl_idle when idle socket disconnects
2023-03-28 17:19:37 +02:00
dependabot[bot]
9a2076a9eb chore(deps): bump futures from 0.3.26 to 0.3.27 (#356)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.26 to 0.3.27.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.26...0.3.27)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 09:13:45 -07:00
dependabot[bot]
e7e7118725 chore(deps): bump hyper from 0.14.24 to 0.14.25 (#358)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.24 to 0.14.25.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.25/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.24...v0.14.25)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 09:13:36 -07:00
dependabot[bot]
99f790cacf chore(deps): bump toml from 0.7.2 to 0.7.3 (#360)
Bumps [toml](https://github.com/toml-rs/toml) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.2...toml-v0.7.3)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 09:13:25 -07:00
dependabot[bot]
434b0bb69e chore(deps): bump serde from 1.0.154 to 1.0.158 (#376)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.154 to 1.0.158.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.154...v1.0.158)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 09:13:13 -07:00
dependabot[bot]
714e043ef0 chore(deps): bump async-trait from 0.1.66 to 0.1.68 (#382)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.66 to 0.1.68.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.66...0.1.68)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 09:12:54 -07:00
dependabot[bot]
863104aadd chore(deps): bump regex from 1.7.1 to 1.7.3 (#385)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.1 to 1.7.3.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.7.1...1.7.3)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 09:12:43 -07:00
Lev Kokotov
7dd96141e3 Update README.md 2023-03-26 00:33:05 -07:00
Lev Kokotov
0d5feac4b2 Contributors (#384) 2023-03-24 17:12:12 -07:00
Lev Kokotov
90aba9c011 V1 (#383) 2023-03-24 17:10:12 -07:00
Montana Low
0f34b49503 point CI at updated repo 2023-03-24 12:59:03 -07:00
Zain Kabani
ca4431b67e Add idle client in transaction configuration (#380)
* Add idle client in transaction configuration

* fmt

* Update docs

* trigger build

* Add tests

* Make the config dynamic from reloads

* fmt

* comments

* trigger build

* fix config.md

* remove error
2023-03-24 08:20:30 -07:00
Mostafa Abdelraouf
d66b377a8e Check Slice bounds in read_message to avoid panics (#371)
When recv is called in the mirroring client, we noticed an occasional panic when reading the message.

thread 'tokio-runtime-worker' panicked at 'slice index starts at 5 but ends at 0', src/messages.rs:522:18
We are still debugging the reason why this happens but adding a check for slice bounds seems like a good idea. Instead of panicking, this will return an Err to the caller which will close the connection.
2023-03-17 12:31:43 -05:00
Fraser Isbester
ac21ce50f1 github/workflows: adds automated image building (#370)
* github/workflow: add ghcr build-push workflow

* github/workflow: add build caching and push

* github/workflows: add registry prefix

* github/workflows: add build-concurrency groups w/ termination
2023-03-16 13:07:02 -07:00
Mostafa Abdelraouf
e5df179ac9 Reduce memory and CPU footprint of mirroring (#369)
The experimental mirroring feature used a lot of memory and CPU when put under production traffic. This change attempts to reduce memory and CPU usage.

Memory footprint is reduced by making the channel smaller. CPU usage is reduced by avoiding allocations if the channel is full or is closed.

We might lose more messages this way if the mirror falls behind but that is more acceptable than crashing the entire process when it goes out-of-memory (OOM)
2023-03-15 17:58:45 -05:00
Mostafa Abdelraouf
9a668e584f Update CONFIG.md (#353)
Mark experimental features as such
2023-03-11 07:55:07 -06:00
Mostafa Abdelraouf
a5c360e848 Update README.md (#352) 2023-03-10 22:02:33 -06:00
Mostafa Abdelraouf
b09f0a3e6b Improve Config Documentation (#351)
This PR adds a utility script that generates config documentation from pgcat.toml. Ideally, we'd want to generate the configs directly from config.rs where the actual defaults are set but this is a good start as we already had several undocumented config flags.
2023-03-10 22:00:28 -06:00
Lev Kokotov
0704ea089c Build on 1.67 (#350) 2023-03-10 09:42:52 -08:00
Lev Kokotov
b4baa86e8a Extended query protocol sharding (#339)
* Prepared stmt sharding

s

tests

* len check

* remove python test

* latest rust

* move that to debug for sure

* Add the actual tests

* latest image

* Update tests/ruby/sharding_spec.rb
2023-03-10 07:55:22 -08:00
Mostafa Abdelraouf
76e195a8a4 Reorder fields in Shard to avoid ValueAfterTable errors (#349) 2023-03-10 07:39:42 -06:00
Mostafa Abdelraouf
aa89e357e0 PgCat Query Mirroring (#341)
This is an implementation of Query mirroring in PgCat (outlined here #302)

In configs, we match mirror hosts with the servers handling the traffic. A mirror host will receive the same protocol messages as the main server it was matched with.

This is done by creating an async task for each mirror server, it communicates with the main server through two channels, one for the protocol messages and one for the exit signal. The mirror server sends the protocol packets to the underlying PostgreSQL server. We receive from the underlying PostgreSQL server as soon as the data is available and we immediately discard it. We use bb8 to manage the life cycle of the connection, not for pooling since each mirror server handler is more or less single-threaded.

We don't have any connection pooling in the mirrors. Matching each mirror connection to an actual server connection guarantees that we will not have more connections to any of the mirrors than the parent pool would allow.
2023-03-10 06:23:51 -06:00
dependabot[bot]
c0855bf27d chore(deps): bump serde_derive from 1.0.152 to 1.0.154 (#347)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.154.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.154)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-08 22:50:53 -08:00
dependabot[bot]
9d523ca49d chore(deps): bump serde from 1.0.152 to 1.0.154 (#348)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.154.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.154)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-08 22:50:45 -08:00
dependabot[bot]
b765581975 chore(deps): bump sqlparser from 0.31.0 to 0.32.0 (#343)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.31.0 to 0.32.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.31.0...v0.32.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-07 01:18:10 -08:00
dependabot[bot]
039c875909 chore(deps): bump async-trait from 0.1.64 to 0.1.66 (#342)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.64 to 0.1.66.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.64...0.1.66)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-06 08:57:44 -05:00
Mostafa Abdelraouf
2cc6a09fba Add Manual host banning to PgCat (#340)
Sometimes we want an admin to be able to ban a host for some time to route traffic away from that host for reasons like partial outages, replication lag, and scheduled maintenance.

We can achieve this today using a configuration update but a quicker approach is to send a control command to PgCat that bans the replica for some specified duration.

This command does not change the current banning rules like

Primaries cannot be banned
When all replicas are banned, all replicas are unbanned
2023-03-06 06:10:59 -06:00
Jose Fernández
8a0da10a87 Dev environment (#338)
Add dev env
2023-03-02 12:14:10 -05:00
Lev Kokotov
c3eaf023c7 Automatic sharding for SELECT v2 (#337)
* More comprehensive read sharding support

* A few fixes

* fq

* comment

* wildcard
2023-03-02 00:53:31 -05:00
dependabot[bot]
02839e4dc2 chore(deps): bump tokio from 1.25.0 to 1.26.0 (#336)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-02 00:41:33 -05:00
dependabot[bot]
bd286d9128 chore(deps): bump sqlparser from 0.30.0 to 0.31.0 (#335)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.30.0 to 0.31.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.30.0...v0.31.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-02 00:40:52 -05:00
Jose Fernández
9241df18e2 Allow sending logs to stdout by using STDOUT_LOG env var (#334)
* Allow sending logs to stdout by using STDOUT_LOG env var

* Increase stats buffer size
2023-02-28 13:10:40 -08:00
zainkabani
eb8cfdb1f1 Adds SHUTDOWN command as alternate option to sending SIGINT (#331)
* Adds SHUTDOWN command to PgCat as alternate option to sending SIGINT

* Check if we're already in SHUTDOWN sequence

* Send signal directly from shutdown instead of using channel

* Add tests

* trigger build

* Lowercase response and boolean change

* Update tests

* Fix tests

* typo
2023-02-26 22:16:30 -08:00
Mostafa Abdelraouf
75a7d4409a Fix Back-and-forth RELOAD Bug (#330)
We identified a bug where RELOAD fails to update the pools.

To reproduce you need to start at some config state, modify that state a bit, reload, revert the configs back to the original state, and reload. The last reload will fail to update the pool because PgCat "thinks" the pool state didn't change.

This is because we use a HashSet to keep track of config hashes but we never remove values from it.
Say we start with State A, we modify pool configs to State B and reload. Now the POOL_HASHES struct has State A and State B. Attempting to go back to State A will encounter a hashset hit which is interpreted by PgCat as "Configs are the same, no need to reload pools"

We fix this by attaching a config_hash value to ConnectionPool object and we calculate that value when we create the pool. This eliminates the need for a global variable. One shortcoming here is that changing any config under one user in the pool will trigger a reload for the entire pool (which is fine I think)
2023-02-21 21:53:10 -06:00
Nicholas Dujay
37e1c5297a implement show users (#329)
* implement show users

* fix compile errors

* add basic ruby test

* gitignore things
2023-02-21 13:08:43 -08:00
Mostafa Abdelraouf
28f2d19cac More coverage cleanup (#328)
Apply a new style + remove function coverage report
2023-02-17 09:18:54 -06:00
Mostafa Abdelraouf
f9134807d7 More Test coverage + fix some code coverage bugs (#321)
Connection to the CI databases is viewed by Postgres as coming from localhost. The pg_hba.conf file generated by the docker image uses trust for these connections, that's why we had no test coverage on SASL and md5 branches.

This PR fixes this issue. There was also an issue with under-reporting code coverage. This should be fixed now
2023-02-16 23:09:22 -06:00
Mostafa Abdelraouf
2a0483b6de Add psmisc to CI image (#327)
I accidentally removed `psmisc` from the image and now the test builds are failing. I am adding it back in this PR
2023-02-16 21:50:03 -06:00
Mostafa Abdelraouf
57dc2ae5ab Move toxiproxy.deb to /tmp (#326)
I am seeing Directory (/home/circleci/project) you are trying to checkout to is not empty and not a git repository error after I started using the new Dockerfile.ci image. My best guess is that this failure is because we download toxiproxy.deb file into the home directory which blocks git checkout.

This PR moves toxiproxy to /tmp/ to avoid this
2023-02-16 21:35:15 -06:00
Mostafa Abdelraouf
0172523f10 Build Dockerfile.ci using Github workflows (#325)
We have to build and push the docker image used in CI manually. This PR builds that image automatically and pushes it to Github docker repository.

Will start using that image in a follow PR
2023-02-16 20:31:53 -06:00
Mostafa Abdelraouf
c69f461be5 Bake toxiproxy into CI image (#323)
Instead of downloading Toxiproxy everytime we run CI, we bake it into the CI docker image
2023-02-16 18:34:35 -06:00
zainkabani
2b05ff4ee5 Log worker thread count at startup (#322) 2023-02-16 16:51:38 -06:00
John Meagher
d5f60b1720 Allow shard setting with comments (#293)
What
Allows shard selection by the client to come in via comments like /* shard_id: 1 */ select * from foo;

Why
We're using a setup in Ruby that makes it tough or impossible to inject commands on the connection to set the shard before it gets to the "real" SQL being run. Instead we have an updated PG adapter that allows injection of comments before each executed SQL statement. We need this support in pgcat in order to keep some complex shard picking logic in Ruby code while using pgcat for connection management.

Local Testing
Run postgres and pgcat with the default options. Run psql < tests/sharding/query_routing_setup.sql to setup the database for the tests and run ./tests/pgbench/external_shard_test.sh as often as needed to exercise the shard setting comment test.
2023-02-15 15:19:16 -06:00
dependabot[bot]
9388288afb chore(deps): bump once_cell from 1.17.0 to 1.17.1 (#320)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.17.0 to 1.17.1.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.0...v1.17.1)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-14 21:43:47 -08:00
Mostafa Abdelraouf
97f5a0564d Fix deprecation warnings (#319)
warning: use of deprecated function `base64::decode`: Use Engine::decode
2023-02-14 16:20:11 -06:00
Tommy Chen
9830c18315 Support EC and PKCS8 private keys (#316)
* Support EC and PKCS8 private keys

* Use iter instead of infinite loop in `load_keys` fn
2023-02-14 08:30:47 -08:00
Mostafa Abdelraouf
bf6efde8cc Fix code coverage + less flakiness (#318)
Code coverage logic was missing coverage from rust tests. This is now fixed.
Also, we weren't reaping spawned PgCat processes correctly which left zombie processes.
2023-02-13 15:29:08 -06:00
Mostafa Abdelraouf
f1265a5570 Introduce tcp_keepalives to PgCat (#315)
We have encountered a case where PgCat pools were stuck following a database incident. Our best understanding at this point is that the PgCat -> Postgres connections died silently and because Tokio defaults to disabling keepalives, connections in the pool were marked as busy forever. Only when we deployed PgCat did we see recovery.

This PR introduces tcp_keepalives to PgCat. This sets the defaults to be

keepalives_idle: 5        # seconds
keepalives_interval: 5 # seconds
keepalives_count: 5    # a count
These settings can detect the death of an idle connection within 30 seconds of its death. Please note that the connection can remain idle forever (from an application perspective) as long as the keepalive packets are flowing so disconnection will only occur if the other end is not acknowledging keepalive packets (keepalive packet acks are handled by the OS, the application does not need to do anything). I plan to add tcp_user_timeout in a follow-up PR.
2023-02-08 11:35:38 -06:00
zainkabani
d81a744154 Fix logging mistakes (#313)
Mistakenly logging username as poolname and poolname as username
2023-02-07 14:16:28 -06:00
dependabot[bot]
cc63c95dcb chore(deps): bump async-trait from 0.1.63 to 0.1.64 (#308)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.63 to 0.1.64.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.63...0.1.64)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 17:15:19 -08:00
dependabot[bot]
b1b1714e76 chore(deps): bump futures from 0.3.25 to 0.3.26 (#307)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 17:15:12 -08:00
dependabot[bot]
ad4eaa859c chore(deps): bump toml from 0.7.0 to 0.7.1 (#309)
Bumps [toml](https://github.com/toml-rs/toml) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.0...toml-v0.7.1)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 17:15:04 -08:00
dependabot[bot]
4ac8d367ca chore(deps): bump hyper from 0.14.23 to 0.14.24 (#311)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.23 to 0.14.24.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.24/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.23...v0.14.24)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 17:14:54 -08:00
dependabot[bot]
e3f902cb31 chore(deps): bump bytes from 1.3.0 to 1.4.0 (#310)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-01 15:35:14 -08:00
dependabot[bot]
bb0b64e089 chore(deps): bump toml from 0.6.0 to 0.7.0 (#305)
Bumps [toml](https://github.com/toml-rs/toml) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.6.0...toml-v0.7.0)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-30 08:59:37 -08:00
dependabot[bot]
a90c7b0684 chore(deps): bump tokio from 1.24.2 to 1.25.0 (#304)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-30 08:59:16 -08:00
Kurtsley
1c73889fb9 Add initial Windows support, ref #298 (#301) 2023-01-28 15:51:05 -08:00
Lev Kokotov
24e79dcf05 Startup improvements & PAUSE/RESUME (#300)
* Dont require servers to be online to start pooler

* PAUSE/RESUME

* fix

* Refresh pool

* Fixes

* lint
2023-01-28 15:36:35 -08:00
Lev Kokotov
2e3eb2663e Fix formatting (#299) 2023-01-28 09:17:49 -08:00
dependabot[bot]
fbe256cc4e chore(deps): bump toml from 0.5.11 to 0.6.0 (#292)
Bumps [toml](https://github.com/toml-rs/toml) from 0.5.11 to 0.6.0.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.5.11...toml-v0.6.0)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-24 11:48:21 -08:00
dependabot[bot]
f10da57ee3 chore(deps): bump async-trait from 0.1.61 to 0.1.63 (#291)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.61 to 0.1.63.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.61...0.1.63)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 08:01:16 -08:00
dependabot[bot]
e7f7adfa14 chore(deps): bump toml from 0.5.10 to 0.5.11 (#290)
Bumps [toml](https://github.com/toml-rs/toml) from 0.5.10 to 0.5.11.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.5.10...toml-v0.5.11)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-19 21:14:48 -08:00
zainkabani
a0e740d30f Refactors is_banned logic and forces health check on unban (#288)
* Refactors is_banned logic and forces healthcheck on unban

* typo

* Make is banned log debug

* addressing comments

* Comment
2023-01-19 17:36:48 -08:00
Jose Fernández
c58f9557ae Add more metrics to prometheus endpoint (#263)
This change:
- Adds server metrics to prometheus endpoint.
- Adds database metrics to prometheus endpoint.
- Adds pools metrics to prometheus endpoint.
- Change metrics name to have a prefix of (stats|pools|databases|servers).
2023-01-19 07:48:12 -08:00
zainkabani
ca8901910c Removes message cloning operation required for query router (#285)
* Removes message cloning operation required for query router

* fmt

* flakey?

* ?
2023-01-19 07:19:49 -08:00
Mostafa Abdelraouf
87a771aecc Log error messages for network failures (#289)
We are seeing some Error reading message code from socket error messages, we want to get more context so this PR logs the actual error reported.
2023-01-19 05:18:08 -06:00
dependabot[bot]
99a3b9896d chore(deps): bump activerecord from 7.0.3.1 to 7.0.4.1 in /tests/ruby (#287)
Bumps [activerecord](https://github.com/rails/rails) from 7.0.3.1 to 7.0.4.1.
- [Release notes](https://github.com/rails/rails/releases)
- [Changelog](https://github.com/rails/rails/blob/v7.0.4.1/activerecord/CHANGELOG.md)
- [Commits](https://github.com/rails/rails/compare/v7.0.3.1...v7.0.4.1)

---
updated-dependencies:
- dependency-name: activerecord
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 16:56:16 -08:00
dependabot[bot]
89689e3663 chore(deps): bump tokio from 1.24.1 to 1.24.2 (#286)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.1 to 1.24.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 08:51:07 -08:00
zainkabani
85ac3ef9a5 Buffer client CopyData messages (#284)
Buffers CopyData messages and removes buffer clone for the sync message
2023-01-17 17:39:55 -08:00
Mostafa Abdelraouf
7894bba59b Introduce least-outstanding-connections load balancing (#282)
Least outstanding connections load balancing can improve the load distribution between instances but for Pgcat it may also improve handling slow replicas that don't go completely down. With LoC, traffic will quickly move away from the slow replica without waiting for the replica to be banned.

If all replicas slow down equally (due to a bad query that is hitting all replicas), the algorithm will degenerate to Random Load Balancing (which is what we had in Pgcat until today).

This may also allow Pgcat to accommodate pools with differently-sized replicas.
2023-01-17 06:52:18 -06:00
zainkabani
ab0bad6da0 Write messages directly onto message buffer instead of allocating on own buffer (#283)
* initial commit

* comment

* fmt
2023-01-16 20:22:06 -08:00
zainkabani
3f70956775 Update cargo lock file (#281)
Major change being updating tokio to latest version to 1.24 which has CPU performance improvements
2023-01-10 17:11:40 -08:00
dependabot[bot]
4b0cdcbd5c chore(deps): bump regex from 1.7.0 to 1.7.1 (#280)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.7.0...1.7.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-10 09:32:54 -08:00
dependabot[bot]
4977489b89 chore(deps): bump base64 from 0.20.0 to 0.21.0 (#279)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases)
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-08 20:53:39 -08:00
dependabot[bot]
27b845fa80 chore(deps): bump async-trait from 0.1.60 to 0.1.61 (#278)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.60 to 0.1.61.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.60...0.1.61)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-08 20:53:28 -08:00
dependabot[bot]
62e78f5769 chore(deps): bump serde from 1.0.151 to 1.0.152 (#268)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 23:58:01 -08:00
dependabot[bot]
ae870894b3 chore(deps): bump serde_derive from 1.0.151 to 1.0.152 (#269)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 23:57:53 -08:00
dependabot[bot]
7d93ead7f4 chore(deps): bump once_cell from 1.16.0 to 1.17.0 (#270)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 23:57:40 -08:00
dependabot[bot]
880bc3e0a8 chore(deps): bump sqlparser from 0.28.0 to 0.30.0 (#275)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.28.0 to 0.30.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.28.0...v0.30.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 23:57:29 -08:00
Lev Kokotov
33bb4b3a0f Fix tests (use sudo) (#276)
use sudo
2023-01-02 23:47:31 -08:00
dependabot[bot]
af1f199908 chore(deps): bump arc-swap from 1.5.1 to 1.6.0 (#273)
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 1.5.1 to 1.6.0.
- [Release notes](https://github.com/vorner/arc-swap/releases)
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/commits)

---
updated-dependencies:
- dependency-name: arc-swap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 08:15:27 -08:00
Eoin Kelly
2282d8c044 Fix typo in README (#272)
Fix typo
2023-01-02 08:13:59 -08:00
Lev Kokotov
4be1b7fc80 Remove logo, pending new logo (#267)
* Remove logo, pending new logo

* remove from readme
2022-12-24 12:12:53 -08:00
zainkabani
8720ed3826 Buffer copy data messages (#265)
* Buffer copy data messages

* Update comment
2022-12-21 06:57:53 -08:00
dependabot[bot]
de7d7d7d99 chore(deps): bump num_cpus from 1.14.0 to 1.15.0 (#264)
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-21 06:01:42 -08:00
dependabot[bot]
6807dd81bd chore(deps): bump serde_derive from 1.0.150 to 1.0.151 (#261)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.150...v1.0.151)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 11:10:39 -08:00
dependabot[bot]
934be934e7 chore(deps): bump async-trait from 0.1.59 to 0.1.60 (#259)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.59 to 0.1.60.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.59...0.1.60)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 11:10:18 -08:00
dependabot[bot]
11fb1d5e27 chore(deps): bump serde from 1.0.150 to 1.0.151 (#260)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.150...v1.0.151)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 11:10:05 -08:00
Jose Fernández
9e8ef566c6 Allow setting the number of runtime workers to be used. (#258)
This change adds a new configuration parameter called `worker_threads` that
allows setting the number of workers the Tokio Runtime will use. It defaults to
4 to maintain backward compatibility.

Given that the config file parse is done asynchronously, first, a transient runtime
is created for reading config, and once it has been parsed, the actual runtime
that will be used for PgCat execution is created.
2022-12-16 11:13:13 -08:00
Jose Fernández
99247f7c88 Allow setting idle_timeout for server connections. (#257)
In postgres, you can specify an `idle_session_timeout` which will close
sessions idling for that amount of time. If a session is closed because
of a timeout, PgCat will erroneously mark the server as unhealthy as the next
health check will return an error because the connection was drop, if no
health check is to be executed, it will simply fail trying to send the query
to the server for the same reason, the conn was drop.

Given that bb8 allows configuring an idle_timeout for pools, it would be
nice to allow setting this parameter in the config file, this way you can
set it to something shorter than the server one. Also, server pool will be kept
smaller in moments of less traffic. Actually, currently this value is set as its
default in bb8, which is 10 minutes.

This changes allows setting the parameter using the config file. It can be set both
globally and per pool. When creating the pool, if the pool don't have it defined, global
value is used.
2022-12-16 08:01:00 -08:00
dependabot[bot]
72e98a2d41 chore(deps): bump toml from 0.5.9 to 0.5.10 (#256)
Bumps [toml](https://github.com/toml-rs/toml) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/toml-rs/toml/releases)
- [Commits](https://github.com/toml-rs/toml/commits/toml-v0.5.10)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-14 21:24:15 -08:00
dependabot[bot]
2746327f12 chore(deps): bump serde from 1.0.149 to 1.0.150 (#252)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.149 to 1.0.150.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.149...v1.0.150)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 09:59:16 -08:00
dependabot[bot]
1d7dcb17e4 chore(deps): bump serde_derive from 1.0.149 to 1.0.150 (#251)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.149 to 1.0.150.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.149...v1.0.150)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 09:59:05 -08:00
dependabot[bot]
077528b2ac chore(deps): bump base64 from 0.13.1 to 0.20.0 (#250)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.13.1 to 0.20.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases)
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.13.1...v0.20.0)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 09:58:50 -08:00
dependabot[bot]
b9b5635be2 chore(deps): bump serde from 1.0.148 to 1.0.149 (#246)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.148 to 1.0.149.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.148...v1.0.149)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 22:56:49 -08:00
dependabot[bot]
0ca353cb0c chore(deps): bump serde_derive from 1.0.148 to 1.0.149 (#247)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.148 to 1.0.149.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.148...v1.0.149)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 22:56:39 -08:00
dependabot[bot]
3e39a07626 chore(deps): bump sqlparser from 0.27.0 to 0.28.0 (#248)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.27.0 to 0.28.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.27.0...v0.28.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 22:56:29 -08:00
dependabot[bot]
4e34e288c5 chore(deps): bump async-trait from 0.1.58 to 0.1.59 (#245)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.58 to 0.1.59.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.58...0.1.59)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-30 08:58:13 -08:00
dependabot[bot]
e4cc692e0d chore(deps): bump sha-1 from 0.10.0 to 0.10.1 (#244)
Bumps [sha-1](https://github.com/RustCrypto/hashes) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha-1-v0.10.0...md2-v0.10.1)

---
updated-dependencies:
- dependency-name: sha-1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 13:47:42 -08:00
dependabot[bot]
b964c2be9d chore(deps): bump serde_derive from 1.0.147 to 1.0.148 (#243)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.147 to 1.0.148.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.147...v1.0.148)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-28 12:08:07 -08:00
dependabot[bot]
9cced5afc7 chore(deps): bump serde from 1.0.147 to 1.0.148 (#242)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.147 to 1.0.148.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.147...v1.0.148)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-28 12:07:57 -08:00
dependabot[bot]
51b4439697 chore(deps): bump env_logger from 0.9.3 to 0.10.0 (#241)
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.9.3 to 0.10.0.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/env_logger/compare/v0.9.3...v0.10.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-25 01:27:06 -08:00
dependabot[bot]
3acfe43cb5 chore(deps): bump bytes from 1.2.1 to 1.3.0 (#240)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/commits)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-22 08:43:00 -08:00
zainkabani
c62b86f4e6 Adds details to errors and fixes error propagation bug (#239) 2022-11-17 09:24:39 -08:00
zainkabani
fcd2cae4e1 Move get_config in startup to admin branch to scope down usage (#238) 2022-11-17 09:22:12 -08:00
zainkabani
5145b20e02 Move ClientBadStartup error log to debug (#237) 2022-11-16 22:16:16 -08:00
zainkabani
fe0b012832 Adds configuration for logging connections and removes get_config from entrypoint (#236)
* Adds configuration for logging connections and removes get_config from entrypoint

* typo

* rename connection config var and add to toml files

* update config log

* fmt
2022-11-16 22:15:47 -08:00
zainkabani
0c96156dae Adds health check setting to pool and avoids get_config in hotpath (#235)
* Adds healthcheck settings to pool

* fmt

* Fix test
2022-11-16 18:51:15 -08:00
zainkabani
b7e70b885c Default to using username when database isn't present on startup (#234) 2022-11-16 18:49:04 -08:00
dependabot[bot]
ab85000ad4 chore(deps): bump sqlparser from 0.26.0 to 0.27.0 (#229)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-15 16:26:22 -07:00
dependabot[bot]
6266721750 chore(deps): bump chrono from 0.4.22 to 0.4.23 (#230)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.22 to 0.4.23.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.22...v0.4.23)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-15 09:55:15 -07:00
Cluas
dfa26ec6f8 chore: make clippy lint happy (#225)
* chore: make clippy happy

* chore: cargo fmt

* chore: cargo fmt
2022-11-09 10:04:31 -08:00
dependabot[bot]
4bd5717ab1 chore(deps): bump hyper from 0.14.20 to 0.14.23 (#222)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.20 to 0.14.23.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.23/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.20...v0.14.23)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-08 08:08:46 -08:00
dependabot[bot]
f7fc04b080 chore(deps): bump env_logger from 0.9.1 to 0.9.3 (#223)
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.9.1 to 0.9.3.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.9.1...v0.9.3)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-08 08:07:53 -08:00
dependabot[bot]
ad89ef1b6e chore(deps): bump regex from 1.6.0 to 1.7.0 (#224)
Bumps [regex](https://github.com/rust-lang/regex) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.6.0...1.7.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-08 08:07:12 -08:00
dependabot[bot]
ab719e82b8 chore(deps): bump num_cpus from 1.13.1 to 1.14.0 (#221)
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.13.1 to 1.14.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.13.1...v1.14.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-05 10:20:55 -07:00
dependabot[bot]
416a6401bf chore(deps): bump md-5 from 0.10.4 to 0.10.5 (#217)
Bumps [md-5](https://github.com/RustCrypto/hashes) from 0.10.4 to 0.10.5.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/md-5-v0.10.4...md-5-v0.10.5)

---
updated-dependencies:
- dependency-name: md-5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-02 12:10:55 -07:00
dependabot[bot]
09451a469e chore(deps): bump jemallocator from 0.3.2 to 0.5.0 (#218)
Bumps [jemallocator](https://github.com/tikv/jemallocator) from 0.3.2 to 0.5.0.
- [Release notes](https://github.com/tikv/jemallocator/releases)
- [Changelog](https://github.com/tikv/jemallocator/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tikv/jemallocator/commits/0.5.0)

---
updated-dependencies:
- dependency-name: jemallocator
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-02 12:10:41 -07:00
Pradeep Chhetri
353306f546 Fix dependabot labels for pull-requests (#219) 2022-11-02 12:10:12 -07:00
Pradeep Chhetri
63d4431046 Fix for warnings about avg_errors not implemented (#220) 2022-11-02 08:11:47 -07:00
dependabot[bot]
edacca8da3 chore(deps): bump toml from 0.5.8 to 0.5.9 (#207)
Bumps [toml](https://github.com/alexcrichton/toml-rs) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/alexcrichton/toml-rs/releases)
- [Commits](https://github.com/alexcrichton/toml-rs/compare/0.5.8...0.5.9)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:54 -07:00
dependabot[bot]
95202c5927 chore(deps): bump chrono from 0.4.19 to 0.4.22 (#211)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.19 to 0.4.22.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/v0.4.22/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.19...v0.4.22)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:50 -07:00
dependabot[bot]
02acecb602 chore(deps): bump env_logger from 0.9.0 to 0.9.1 (#212)
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.9.0...v0.9.1)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:45 -07:00
dependabot[bot]
8c8fedd1db chore(deps): bump sha2 from 0.10.5 to 0.10.6 (#213)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.10.5 to 0.10.6.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.10.5...sha2-v0.10.6)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:37 -07:00
dependabot[bot]
c8b06e2f9f chore(deps): bump md-5 from 0.10.0 to 0.10.4 (#214)
Bumps [md-5](https://github.com/RustCrypto/hashes) from 0.10.0 to 0.10.4.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/md2-v0.10.0...md-5-v0.10.4)

---
updated-dependencies:
- dependency-name: md-5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:30 -07:00
dependabot[bot]
e8f58fc5f6 chore(deps): bump serde from 1.0.136 to 1.0.147 (#215)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.136 to 1.0.147.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.136...v1.0.147)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:23 -07:00
dependabot[bot]
dec6de405f chore(deps): bump log from 0.4.14 to 0.4.17 (#216)
Bumps [log](https://github.com/rust-lang/log) from 0.4.14 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.14...0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 23:35:13 -07:00
dependabot[bot]
50476993c4 chore(deps): bump once_cell from 1.9.0 to 1.16.0 (#209)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.9.0 to 1.16.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.9.0...v1.16.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 22:17:29 -07:00
dependabot[bot]
4069b07e8e chore(deps): bump tokio from 1.16.1 to 1.19.2 (#210)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.16.1 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.16.1...tokio-1.19.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 22:17:03 -07:00
dependabot[bot]
37d07287f8 chore(deps): bump rand from 0.8.4 to 0.8.5 (#201)
Bumps [rand](https://github.com/rust-random/rand) from 0.8.4 to 0.8.5.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.8.4...0.8.5)

---
updated-dependencies:
- dependency-name: rand
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 11:20:30 -07:00
dependabot[bot]
3eec99dc5c chore(deps): bump arc-swap from 1.5.0 to 1.5.1 (#202)
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/vorner/arc-swap/releases)
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: arc-swap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 11:20:20 -07:00
dependabot[bot]
b61959a2c6 chore(deps): bump base64 from 0.13.0 to 0.13.1 (#203)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.13.0 to 0.13.1.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases)
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.13.0...v0.13.1)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 11:20:10 -07:00
dependabot[bot]
101db7e88b chore(deps): bump rustls-pemfile from 1.0.0 to 1.0.1 (#205)
Bumps [rustls-pemfile](https://github.com/rustls/pemfile) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/rustls/pemfile/releases)
- [Commits](https://github.com/rustls/pemfile/commits)

---
updated-dependencies:
- dependency-name: rustls-pemfile
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 11:20:00 -07:00
dependabot[bot]
01bbc1f093 chore(deps): bump bytes from 1.1.0 to 1.2.1 (#206)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.1.0 to 1.2.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.1)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 11:19:47 -07:00
dependabot[bot]
e13c6091dd chore(deps): bump async-trait from 0.1.52 to 0.1.58 (#200)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.52 to 0.1.58.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.52...0.1.58)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 09:53:19 -07:00
dependabot[bot]
70c791b173 chore(deps): bump sha2 from 0.10.2 to 0.10.5 (#199)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.10.2 to 0.10.5.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.10.2...sha2-v0.10.5)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 09:53:04 -07:00
dependabot[bot]
7ec866d4a9 chore(deps): bump serde_derive from 1.0.136 to 1.0.147 (#198)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.136 to 1.0.147.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.136...v1.0.147)

---
updated-dependencies:
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 09:52:46 -07:00
dependabot[bot]
552e1cf0e7 chore(deps): bump regex from 1.5.5 to 1.6.0 (#197)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.6.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-31 09:50:35 -07:00
Pradeep Chhetri
19ffeffb3b Add dependabot for keeping dependencies up-to-date (#196)
It will help us to keep the dependencies up-to-date

Signed-off-by: Pradeep Chhetri <pradeepchhetri4444@gmail.com>

Signed-off-by: Pradeep Chhetri <pradeepchhetri4444@gmail.com>
2022-10-31 09:47:33 -07:00
Lev Kokotov
9fe8d5e76f Dont change shard unless you know (#195) 2022-10-26 00:14:08 -07:00
Lev Kokotov
0524787d31 Automatic sharding: part one of many (#194)
Starting automatic sharding
2022-10-25 11:47:41 -07:00
Lev Kokotov
fa267733d9 Fix docker-compose (#193)
Fix docker-compose local build
2022-10-24 11:05:33 -07:00
Lev Kokotov
dea952e4ca Re-enable query parser and parse multiple statements (#191)
* Re-enable query parser and parse multiple statements

* no diff
2022-10-23 16:59:51 -07:00
zainkabani
19f635881a Don't send discard all when state is changed in transaction (#186)
* Don't send discard all when state is changed in transaction

* Remove unnecessary clone

* spelling

* Move transaction check to SET command

* Add test for set command in transaction

* type

* Update comments

* Update comments

* use moves instead of clones for initial message

* don't make message mutable

* Update unwrap

* but i'm not a wrapper

* Add set local test

* change continue
2022-10-13 19:33:12 -07:00
Mostafa Abdelraouf
eceb7f092e Use Jemalloc (#189)
Jemalloc performs better than the standard allocator in various metrics (http://ithare.com/testing-memory-allocators-ptmalloc2-tcmalloc-hoard-jemalloc-while-trying-to-simulate-real-world-loads/).

This PR makes changes to use Jemalloc as the global allocator for Pgcat. Windows is not officially supported by Pgcat but it should still compile but without Jemalloc as the allocator.
2022-10-13 11:13:45 -05:00
Mostafa Abdelraouf
83fd639918 A bit faster get_pool (#187)
* A bit faster get_pool

* fmt
2022-10-08 08:16:04 -07:00
Mostafa Abdelraouf
3d33ccf4b0 Fix maxwait metric (#183)
Max wait was being reported as 0 after #159

This PR fixes that and adds test
2022-10-05 21:41:09 -05:00
Lev Kokotov
7987c5ffad Replace a few types with more developer-friendly names (#182)
* Replace a few types with more developer-friendly names

* UserPool -> PoolIdentifier
2022-10-01 10:25:59 -07:00
zainkabani
24f5eec3ea Change sharding config to enum and move validation of configs into public functions (#178)
Moves config validation to own functions to enable tools to use them
Moves sharding config to enum
Makes defaults public
Make connect_timeout on pool and option which is overwritten by general connect_timeout
2022-09-28 08:50:14 -05:00
Mostafa Abdelraouf
af064ef447 Set client state to idle after error (#179)
* Set client state to idle after error

* fmt

* spelling

* clean up
2022-09-24 09:09:15 -07:00
Lev Kokotov
e84a6f834c Update README.md 2022-09-23 12:24:30 -07:00
Lev Kokotov
19fd677891 Fix the pool fix (#176)
* Always listen to the compiler

* Its fine
2022-09-23 12:06:07 -07:00
Lev Kokotov
964a5e1708 Don't drop connections if DB hasn't changed (#175)
* Don't drop connections if DB hasn't changed

* Incoporate connect_timeout into the pool config

* use the field
2022-09-23 11:32:05 -07:00
Mostafa Abdelraouf
d126c7424d Log failed client logins (#173)
* Log failed client logins

* more logging

* remove clones

* remove
2022-09-23 09:08:38 -07:00
zainkabani
f72dac420b Add defaults for configs (#174)
* add statement timeout to readme

* Add defaults to various configs

* primary read enabled default to false
2022-09-22 23:00:46 -07:00
zainkabani
3a729bb75b Minor refactor for configs (#172)
* Changes shard struct to use vector of ServerConfig

* Adds to query router

* Change client disconnect with error message to warn instead of debug

* Add warning logs for clean up actions
2022-09-22 10:07:02 -07:00
zainkabani
85cc2f4147 Update to latest library versions (#170) 2022-09-21 13:48:33 -07:00
zainkabani
8c09ab6c20 Export pgcat objects in lib (#169)
* Export pgcat objects in lib

* fmt
2022-09-20 18:47:32 -07:00
Mostafa Abdelraouf
f7a951745c Report Query times (#166)
* Report avg and total query timing

* Report query times

* fmt
2022-09-15 02:21:45 -04:00
Mostafa Abdelraouf
4ae1bc8d32 Add SHOW CLIENTS / SHOW SERVERS + Stats refactor and tests (#159)
* wip

* Main Thread Panic when swarmed with clients

* fix

* fix

* 1024

* fix

* remove test

* Add SHOW CLIENTS

* revert

* fmt

* Refactor + tests

* fmt

* add test

* Add SHOW SERVERS + Make PR unreviewable

* prometheus

* add state to clients and servers

* fmt

* Add application_name to server stats

* Add tests for waiting clients

* Docs

* remove comment

* comments

* typo

* cleanup

* CI
2022-09-14 11:20:41 -04:00
Lev Kokotov
075167431d Add Discord link (#164)
* Add Discord link

* move it up

* :)

* hmm

* hmm

* :O
2022-09-08 08:12:37 -07:00
Mostafa Abdelraouf
9514b3b2d1 Clean connection state up after protocol named prepared statement (#163)
* Clean connection state up after protocol named prepared statement

* Avoid cloning + add test

* fmt
2022-09-07 20:37:17 -07:00
Lev Kokotov
6d41640ea9 Send signal even if process is gone (#162)
* Send signal even if process is gone

* hmm

* hmm
2022-09-07 09:22:52 -07:00
Mostafa Abdelraouf
744ceada86 Better logging for failure to get connection from pool (#161) 2022-09-07 08:24:07 -07:00
Mostafa Abdelraouf
a5c8dd69b2 Avoid reporting ProtocolSyncError when admin session disconnects (#160)
* Avoid reporting ProtocolSyncError when admin session disconnects

* rebuild

* rebuild
2022-09-06 22:22:31 -07:00
zainkabani
6a9a4db648 Adds microsecond logging and also reformats duration to include milliseconds (#156)
* Adds microsecond logging and also reformats duration to include milliseconds

* fmt

* attempt to fix cd

* revert
2022-09-05 01:21:27 -07:00
Mostafa Abdelraouf
976b406468 Main Thread Panic when swarmed with clients (#158)
* Main Thread Panic when swarmed with clients

* fix

* fix

* 1024

* fix

* remove test

* Update src/client.rs

* Update src/main.rs

* Update src/client.rs

* Update src/main.rs

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
2022-09-05 01:21:06 -07:00
zain-kabani
417358c35d Patch graceful shutdown bug (#157)
* Fixes non-admin client counting error

* Add log when sigterm received and log number of active clients when shutdown timeout is reached
2022-09-05 01:02:49 -07:00
Mostafa Abdelraouf
23a642f4a4 Send DISCARD ALL even if client is not in transaction (#152)
* Send DISCARD ALL even if client is not in transaction

* fmt

* Added tests + avoided sending extra discard all

* Adds set name logic to beginning of handle client

* fmt

* refactor dead code handling

* Refactor reading command tag

* remove unnecessary trim

* Removing debugging statement

* typo

* typo{

* documentation

* edit text

* un-unwrap

* run ci

* run ci

Co-authored-by: Zain Kabani <zain.kabani@instacart.com>
2022-09-01 20:06:55 -07:00
Mostafa Abdelraouf
7f20dc3054 Better handling extended protocol messages in the event of busy pool (#155)
* Better handling for checkout errors during extended protocol messages

* Fix specs

* comment
2022-09-01 15:02:39 -07:00
Mostafa Abdelraouf
36339bd96f Log Address information in connection create/drop (#154)
* Log Address information in connection create/drop

* run ci
2022-09-01 11:16:22 -07:00
Mostafa Abdelraouf
65b69b46d2 Allow running integration tests with coverage locally (#151) 2022-08-30 10:43:45 -07:00
Mostafa Abdelraouf
d48c04a7fb Ruby integration tests (#147)
* Ruby integration tests

* forgot a file

* refactor

* refactoring

* more refactoring

* remove config helper

* try multiple databases

* fix

* more databases

* Use pg stats

* ports

* speed

* Fix tests

* preload library

* comment
2022-08-30 09:14:53 -07:00
zainkabani
2628dec42e Move autoreloader to own tokio task (#148) 2022-08-29 00:08:44 -07:00
Mostafa Abdelraouf
3bc4f9351c Exit with failure codes if configs are bad (#146)
* Exit with failure codes if configs are bad

* fmt
2022-08-25 18:56:18 -07:00
Lev Kokotov
9d84d6f131 Graceful shutdown and refactor (#144)
* Graceful shutdown and refactor

* ok

* _Graceful_ shutdown

* Remove hardcoded setting

* clean up

* end

* timeout

* hmm

* hmm!

* bash

* bash

* hmm

* maybe maybe

* Adds tests and move non-admin connection rejection to startup (#145)

* Move error response

* Adds tests and removes unused variable

* Adds debug log

Co-authored-by: zainkabani <77307340+zainkabani@users.noreply.github.com>
2022-08-25 06:40:56 -07:00
Mostafa Abdelraouf
c054ff068d Avoid sending Z packet in the middle of extended protocol packet sequence if we fail to get connection from pool (#137)
* Failing test

* maybe

* try fail

* try

* add message

* pool size

* correct user

* more

* debug

* try fix

* see stdout

* stick?

* fix configs

* modify

* types

* m

* maybe

* make tests idempotent

* hopefully fails

* Add client fix

* revert pgcat.toml change

* Fix tests
2022-08-23 11:02:23 -07:00
Lev Kokotov
5a0cea6a24 Really fix idle servers (#141) 2022-08-22 11:56:40 -07:00
Lev Kokotov
d0e8171b1b Fix too many idle servers (#140)
* Fix too many idle servers

* oops
2022-08-22 11:52:34 -07:00
Lev Kokotov
069d76029f Fix incorrect routing for replicas (#139)
* Fix incorrect routing for replicas

* name
2022-08-21 22:40:49 -07:00
Lev Kokotov
902fafd8d7 Random lb (#138) 2022-08-21 22:20:31 -07:00
Mostafa Abdelraouf
5f5b5e2543 Random instance selection (#136)
* wip

* revert some'

* revert more

* poor-man's integration test

* remove test

* fmt

* --workspace

* fix build

* fix integration test

* another stab

* log

* run after integration

* cargo test after integration

* revert

* revert more

* Refactor + clean up

* more clean up
2022-08-21 22:15:20 -07:00
zainkabani
5948fef6cf Minor Refactoring of re-used code and server stat reporting (#129)
* Minor changes to stats reporting and recduce re-used code

* fmt
2022-08-18 05:12:38 -07:00
Mostafa Abdelraouf
790898c20e Add pool name and username to address object (#128)
* Add pool name and username to address object

* Fix address name

* fmt
2022-08-17 08:40:47 -07:00
Pradeep Chhetri
d64f6793c1 Minor cleanup in admin command (#126)
* Minor cleanup in admin command

* Typo correction

* fix when the admin query is ending with semicolon
2022-08-16 10:01:46 -07:00
Lev Kokotov
cea35db35c Fix lost statistics (#125)
* Lost events

* more logging
2022-08-15 23:54:49 -07:00
Mostafa Abdelraouf
a3aefabb47 Add cl_idle to SHOW POOLS (#124) 2022-08-15 20:51:37 -07:00
Lev Kokotov
3285006440 Statement timeout + replica imbalance fix (#122)
* Statement timeout

* send error message too

* Correct error messages

* Fix replica inbalance

* disable stmt timeout by default

* Redundant mark_bad

* revert healthcheck delay

* tests

* set it to 0

* reload config again
2022-08-13 13:45:58 -07:00
Pradeep Chhetri
52303cc808 Make prometheus port configurable (#121)
* Make prometheus port configurable

* Update circleci config
2022-08-13 10:25:14 -07:00
Lev Kokotov
be254cedd9 Fix debug log (#120) 2022-08-11 22:47:47 -07:00
Lev Kokotov
a5db6881b8 Speed up CI a bit (#119)
* Sleep for 1s

* use premade image

* quicker

* revert shutdown timeout
2022-08-11 22:41:08 -07:00
zainkabani
f963b12821 Health check delay (#118)
* initial commit of server check delay implementation

* fmt

* spelling

* Update name to last_healthcheck and some comments

* Moved server tested stat to after require_healthcheck check

* Make health check delay configurable

* Rename to last_activity

* Fix typo

* Add debug log for healthcheck

* Add address to debug log
2022-08-11 14:42:40 -07:00
Lev Kokotov
a262337ba5 Update CONTRIBUTING.md 2022-08-10 09:51:56 -07:00
Nicholas Dujay
014628d6e0 fix docker compose port allocation for local dev (#117)
change docker compose port to right prometheus port
2022-08-09 14:15:34 -07:00
zainkabani
65c32ad9fb Validates pgcat is closed after shutdown python tests (#116)
* Validates pgcat is closed after shutdown python tests

* Fix pgrep logic

* Moves sigterm step to after cleanup to decouple

* Replace subprocess with os.system for running pgcat
2022-08-09 14:09:53 -07:00
Nicholas Dujay
1b166b462d create a prometheus exporter on a standard http port (#107)
* create a hyper server and add option to enable it in config

* move prometheus stuff to its own file; update format

* create metric type and help lookup table

* finish the metric help type map

* switch to a boolean and a standard port

* dont emit unimplemented metrics

* fail if curl returns a non 200

* resolve conflicts

* move log out of config.show and into main

* terminating new line

* upgrade curl

* include unimplemented stats
2022-08-09 12:19:11 -07:00
Mostafa Abdelraouf
7592339092 Prevent clients from sticking to old pools after config update (#113)
* Re-acquire pool at the beginning of Protocol loop

* Fix query router + add tests for recycling behavior
2022-08-09 12:18:27 -07:00
zainkabani
3719c22322 Implementing graceful shutdown (#105)
* Initial commit for graceful shutdown

* fmt

* Add .vscode to gitignore

* Updates shutdown logic to use channels

* fmt

* fmt

* Adds shutdown timeout

* Fmt and updates tomls

* Updates readme

* fmt and updates log levels

* Update python tests to test shutdown

* merge changes

* Rename listener rx and update bash to be in line with master

* Update python test bash script ordering

* Adds error response message before shutdown

* Add details on shutdown event loop

* Fixes response length for error

* Adds handler for sigterm

* Uses ready for query function and fixes number of bytes

* fmt
2022-08-08 16:01:24 -07:00
Mostafa Abdelraouf
106ebee71c Fix local dev (#112)
* Fix Dev env

* Update tests/sharding/query_routing_setup.sql

* Update tests/sharding/query_routing_setup.sql

* bring pgcat.toml on ci and local dev to parity

* more parity

* pool names

* pool names

* less diff

* fix tests

* fmt

* add other user to setup

Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
2022-08-08 13:15:48 -07:00
Mostafa Abdelraouf
b79f55abd6 Generate test coverage report in CircleCI (#110)
* coverage?

* generate_coverage

* +x

* 1.62.1

* 62

* ignore

* store

* quote
2022-08-08 07:51:36 -07:00
Mostafa Abdelraouf
b828e62408 Report banned addresses as disabled (#111) 2022-08-08 07:50:29 -07:00
Mostafa Abdelraouf
499612dd76 Add user to SHOW STATS query (#108)
* Add user to SHOW STATS query

* user_name => username
2022-08-03 18:16:53 -07:00
Mostafa Abdelraouf
5ac85eaadd Fix Python tests and remove CircleCI-specific path (#106)
* Remove CircleCI-specific path in tests

* ..?

* Fix testsP

* Fix python test

* remove pip

* Maybe fail?

* return code?

* no &

* Fix tests
2022-08-02 15:52:22 -07:00
Pradeep Chhetri
20e8f9d74c Sync pgcat config for docker-compose (#104) 2022-08-02 09:23:35 -07:00
Mostafa Abdelraouf
1b648ca00e Send proper server parameters to clients using admin db (#103)
* Send proper server parameters to clients using admin db

* clean up

* fix python test

* build

* Add python

* missing &

* debug ls

* fix tests

* fix tests

* fix

* Fix warning

* Address comments
2022-07-31 19:52:23 -07:00
Mostafa Abdelraouf
35381ba8fd Add test for config Serializer (#102)
* Add test for serializer

* fmt
2022-07-30 16:28:25 -07:00
Mostafa Abdelraouf
e591865d78 Avoid ValueAfterTable when serializing configs (#101) 2022-07-30 16:12:02 -07:00
Mostafa Abdelraouf
48cff1f955 Slightly more light weight health check (#100) 2022-07-29 11:58:25 -07:00
Mostafa Abdelraouf
8a06fc4047 Add Serialize trait to configs (#99) 2022-07-28 15:42:04 -07:00
Pradeep Chhetri
14d4dc45f5 Minor fix for some stats (#97) 2022-07-27 22:59:33 -07:00
Mostafa Abdelraouf
2ae4b438e3 Add support for multi-database / multi-user pools (#96)
* Add support for multi-database / multi-user pools

* Nothing

* cargo fmt

* CI

* remove test users

* rename pool

* Update tests to use admin user/pass

* more fixes

* Revert bad change

* Use PGDATABASE env var

* send server info in case of admin
2022-07-27 19:47:55 -07:00
Lev Kokotov
c5be5565a5 Update Dockerfile 2022-07-25 22:25:59 -07:00
dependabot[bot]
eff8e3e229 Bump activerecord from 7.0.2.2 to 7.0.3.1 in /tests/ruby (#94)
Bumps [activerecord](https://github.com/rails/rails) from 7.0.2.2 to 7.0.3.1.
- [Release notes](https://github.com/rails/rails/releases)
- [Changelog](https://github.com/rails/rails/blob/v7.0.3.1/activerecord/CHANGELOG.md)
- [Commits](https://github.com/rails/rails/compare/v7.0.2.2...v7.0.3.1)

---
updated-dependencies:
- dependency-name: activerecord
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 13:24:41 -07:00
Marco Montagna
ae3db111ac Merge pull request #91 from levkk/levkk-tls-2
Support for TLS
2022-06-27 17:12:50 -07:00
Lev
8bcfbed574 forgotten comment 2022-06-27 17:07:40 -07:00
Lev
773602dedf Im about to get a nasty email 2022-06-27 17:06:49 -07:00
Lev
21bf07258c lock em up 2022-06-27 17:05:45 -07:00
Lev
186f8be5b3 lint 2022-06-27 17:01:40 -07:00
Lev
7667fefead config check 2022-06-27 17:01:14 -07:00
Lev
c11d595ac7 bye 2022-06-27 16:46:03 -07:00
Lev
8f3202ed92 hmm 2022-06-27 16:45:41 -07:00
Lev
eb58920870 at least it compiles 2022-06-27 15:52:01 -07:00
Lev Kokotov
b974aacd71 check 2022-06-27 09:46:33 -07:00
Lev Kokotov
7dfe59a91a Fix stats dymanic reload (#87) 2022-06-25 12:22:46 -07:00
Lev Kokotov
5bcd3bf9c3 Automatically reload config every seconds (disabled by default) (#86)
* Automatically reload config every seconds (disabld by default)

* add that
2022-06-25 11:46:20 -07:00
Lev Kokotov
f06f64119c Fix panic & query router bug (#85)
* Fix query router bug

* Fix panic
2022-06-24 15:14:31 -07:00
Lev Kokotov
b93303eb83 Live reloading entire config and bug fixes (#84)
* Support reloading the entire config (including sharding logic) without restart.

* Fix bug incorrectly handing error reporting when the shard is set incorrectly via SET SHARD TO command.
selected wrong shard and the connection keep reporting fatal #80.

* Fix total_received and avg_recv admin database statistics.

* Enabling the query parser by default.

* More tests.
2022-06-24 14:52:38 -07:00
Lev Kokotov
d865d9f9d8 readme 2022-06-20 06:20:12 -07:00
Lev Kokotov
d3310a62c2 Client md5 auth and clean up scram (#77)
* client md5 auth and clean up scram

* add pw

* add user

* add user

* log
2022-06-20 06:15:54 -07:00
Lev Kokotov
d412238f47 Implement SCRAM-SHA-256 for server authentication (PG14) (#76)
* Implement SCRAM-SHA-256

* test it

* trace

* move to community for auth

* hmm
2022-06-18 18:36:00 -07:00
dependabot[bot]
7782933f59 Bump regex from 1.5.4 to 1.5.5 (#75)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.4 to 1.5.5.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.4...1.5.5)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-06 19:59:50 -07:00
Lev Kokotov
bac4e1f52c Only set application_name if it's different (#74)
* Only set application_name if it's different

* keep server named pgcat until something else changes
2022-06-05 09:48:06 -07:00
Lev Kokotov
37e3a86881 Pass application_name to server (#73)
* Pass application_name to server

* fmt
2022-06-03 00:15:50 -07:00
Lev Kokotov
61db13f614 Fix memory leak in client/server mapping (#71) 2022-05-18 16:24:03 -07:00
Lev Kokotov
fe32b5ef17 Reduce traffic on the stats channel (#69) 2022-05-17 13:05:25 -07:00
Lev Kokotov
54699222f8 Possible fix for clients waiting stat leak (#68) 2022-05-14 21:35:33 -07:00
Lev Kokotov
ccbca66e7a Poorly behaved client fix (#65)
* Poorly behaved client fix

* yes officer

* fix tests

* no useless rescue

* Looks ok
2022-05-09 09:09:22 -07:00
Lev Kokotov
df85139281 Update README. Comments. Version bump. (#60)
* update readme

* comments

* just a version bump
2022-03-10 01:33:29 -08:00
Lev Kokotov
509e4815a3 Update README.md 2022-03-08 17:48:26 -08:00
Lev Kokotov
5338ff2323 Update README.md 2022-03-08 17:46:46 -08:00
Lev Kokotov
1ea0a7f332 Update README.md 2022-03-08 17:45:54 -08:00
Lev Kokotov
d1b86d363d Update README.md 2022-03-08 17:38:51 -08:00
Lev Kokotov
b309ead58f Handle SIGTERM. Add docker-compose.yml (#59)
* docker-compsoe

* remove statsd config

* readme
2022-03-08 17:18:48 -08:00
Lev Kokotov
341ebf4123 docs and remove Option (#58)
* docs and remove Option

* lint
2022-03-07 23:05:40 -08:00
Lev Kokotov
35828a0a8c Per-shard statistics (#57)
* per shard stats

* aight

* cleaner

* fix show lists

* comments

* more friendly

* case-insensitive

* test all shards

* ok

* HUH?
2022-03-04 17:04:27 -08:00
Lev Kokotov
1e8fa110ae Fix pgbouncerhero (#54) 2022-03-02 14:46:31 -08:00
Lev Kokotov
d4186b7815 More admin (#53)
* more admin

* more admin

* show lists

* tests
2022-03-01 22:49:43 -08:00
Lev Kokotov
aaeef69d59 Refactor admin (#52) 2022-03-01 08:47:19 -08:00
Lev Kokotov
b21e0f4a7e admin SHOW DATABASES (#51)
* admin SHOW DATABASES

* test

* correct replica count
2022-02-28 17:22:28 -08:00
Lev Kokotov
eb1473060e admin: SHOW CONFIG (#50)
* admin: SHOW CONFIG

* test
2022-02-28 08:14:39 -08:00
Lev Kokotov
26f75f8d5d admin RELOAD (#49)
* admin RELOAD

* test
2022-02-27 10:21:24 -08:00
Lev Kokotov
99d65fc475 Check server versions on startup & refactor (#48)
* Refactor and check server parameters

* warnings

* fix validator
2022-02-26 11:01:52 -08:00
Lev Kokotov
206fdc9769 Fix some stats (#47)
* fix some stats

* use constant

* lint
2022-02-26 10:03:11 -08:00
Lev Kokotov
f74101cdfe admin: SHOW STATS (#46)
* admin: show stats

* warning

* tests

* lint

* type mod
2022-02-25 18:20:15 -08:00
Lev Kokotov
8e0682482d query routing docs (#45) 2022-02-25 14:27:33 -08:00
Lev Kokotov
6db51b4a11 Use Toxiproxy for failover testing (#44)
* Toxiproxy

* up-to-date config

* debug

* hm

* more

* mroe

* more

* hmm

* aha

* less logs

* cleaner

* hmm

* we test these now

* update readme
2022-02-24 20:55:19 -08:00
Lev Kokotov
a784883611 Allow to set shard and set sharding key without quotes (#43)
* Allow to set shard and set sharding key without quotes

* cover it

* dont look for these in the middle of another query

* friendly regex

* its own response to set shard key
2022-02-24 12:16:24 -08:00
Lev Kokotov
5972b6fa52 Switch to parking_lot RwLock & Mutex. Use trace! for protocol instead of debug! (#42)
* RwLock & parking_lot::Mutex

* upgrade to trace
2022-02-24 08:44:41 -08:00
Lev Kokotov
b3c8ca4b8a Another example of a sharding function (#41)
* Another example of a sharding function

* tests
2022-02-23 11:50:34 -08:00
Lev Kokotov
dce72ba262 Add debug logging (#39)
* Add debug for easier debugging

* fmt

* a couple more messages
2022-02-22 19:26:08 -08:00
Lev Kokotov
af1716bcd7 Flush stats (#38)
* flush stats

* stats

* refactor
2022-02-22 18:10:30 -08:00
Lev Kokotov
3f16123cc5 Config docs (#37)
* config docs

* space

* space

* shards
2022-02-21 23:57:25 -08:00
Lev Kokotov
f6f5471aa0 Update README.md 2022-02-21 20:43:40 -08:00
Lev Kokotov
a6fc935040 Can pass config path as argument (#36)
* show better errors for config parsing

* lint
2022-02-21 20:41:32 -08:00
Lev Kokotov
754381fc6c Update Dockerfile 2022-02-21 17:49:32 -08:00
Lev Kokotov
b1e9a406fb Update README.md 2022-02-21 17:48:08 -08:00
Lev Kokotov
f21a3d8d8c Add server login stat; refactor for better naming (#34) 2022-02-21 17:28:50 -08:00
Lev Kokotov
f805b43a08 test session mode and config reload for real (#33)
* test session mode and config reload for real

* period

* run them at the end

* typo

* wrong dir
2022-02-21 00:16:33 -08:00
Lev Kokotov
86941d62e4 Reset query router setting to default (#32) 2022-02-21 00:00:50 -08:00
Lev Kokotov
aceb2ace24 Update issue templates 2022-02-20 23:52:50 -08:00
Lev Kokotov
303fec063b Ruby (#30)
* cop

* log
2022-02-20 23:33:04 -08:00
Lev Kokotov
64574211c6 Update CONTRIBUTING.md 2022-02-20 22:49:30 -08:00
Lev Kokotov
44b5e7eeee use logger lib; minor refactor; sv_* stats (#29) 2022-02-20 22:47:08 -08:00
Lev Kokotov
108f5715c0 License as MIT (#28)
* License as MIT

* license

* s
2022-02-20 13:49:30 -08:00
Lev Kokotov
3b795464a8 Fix client states reporting (#27)
* Fix client states reporting

* clean
2022-02-20 12:40:09 -08:00
Lev Kokotov
d4c1fc87ee Reloadable config (#26)
* Reloadable config

* readme

* live config reload

* test matrix
2022-02-19 13:57:35 -08:00
Lev Kokotov
4ca50b9a71 Update README.md 2022-02-19 08:59:20 -08:00
Lev Kokotov
a556ec1c43 More query router commands; settings last until changed again; docs (#25)
* readme

* touch up docs

* stuff

* refactor query router

* remove unused

* less verbose

* docs

* no link

* method rename
2022-02-19 08:57:24 -08:00
Lev Kokotov
bbacb9cf01 Explicit shard selection; Rails tests (#24)
* Explicit shard selection; Rails tests

* try running ruby tests

* try without lockfile

* aha

* ok
2022-02-18 09:43:07 -08:00
Lev Kokotov
aa796289bf Query parser 3.0 (#23)
* Starting query parsing

* Query parser

* working config

* disable by default

* fix tsets

* introducing log crate; test for query router; comments

* typo

* fixes for banning

* added test for prepared stmt
2022-02-18 07:10:18 -08:00
Lev Kokotov
4c8a3987fe Refactor query routing into its own module (#22)
* Refactor query routing into its own module

* commments; tests; dead code

* error message

* safer startup

* hm

* dont have to be public

* wow

* fix ci

* ok

* nl

* no more silent errors
2022-02-16 22:52:11 -08:00
Lev Kokotov
7b0ceefb96 Constants, comments, CI fixes, dead code clean-up (#21)
* constants

* server.rs docs

* client.rs comments

* dead code; comments

* comment

* query cancellation comments

* remove unnecessary cast

* move db setup up one step

* query cancellation test

* new line; good night
2022-02-15 22:45:45 -08:00
Lev Kokotov
bb84dcee64 Update README.md 2022-02-15 13:11:24 -08:00
Lev Kokotov
1c406e0fc6 Update README.md 2022-02-15 13:10:32 -08:00
Lev Kokotov
05b4cccb97 More statistics (#20)
* cleaner stats

* remove shard selection there
2022-02-15 08:18:01 -08:00
Lev Kokotov
659b1e00b8 bump statsd 2022-02-14 10:27:32 -08:00
Lev Kokotov
8e5e28a139 Some stats (#19) 2022-02-14 10:00:55 -08:00
Lev Kokotov
574ebe02b8 TODOs (#18) 2022-02-14 06:36:05 -08:00
Lev Kokotov
9c521f07c1 parse startup client parameters (#16) 2022-02-14 05:11:53 -08:00
Lev Kokotov
4aa9c3d3c7 Cleaner shutdown (#12)
* Cleaner shutdown

* mark as bad just in case although im pretty sure we dont need it

* server session duration

* test clean shutdown

* ah
2022-02-12 10:16:05 -08:00
Lev Kokotov
20ceb729a0 print session duration; connect to all servers when validating (#11) 2022-02-12 09:24:24 -08:00
Lev Kokotov
eb45d65110 extended protocol tests 2022-02-11 22:24:27 -08:00
Lev Kokotov
526b9eb666 Pass real server info to the client (#10) 2022-02-11 22:19:49 -08:00
Lev Kokotov
ab8573c94f docker image (#9)
* docker image

* nl
2022-02-11 12:02:08 -08:00
Lev Kokotov
bc5b9e422f Merge pull request #8 from levkk/default-role
add default server role; bug fix
2022-02-11 11:23:58 -08:00
Lev Kokotov
e8263430a3 nl 2022-02-11 11:21:32 -08:00
Lev Kokotov
0d369ab90a add default server role; bug fix 2022-02-11 11:19:40 -08:00
Lev Kokotov
595e564216 Merge pull request #7 from levkk/once-cell-regex
once_cell is way faster
2022-02-10 17:08:01 -08:00
Lev Kokotov
06575eae7b once_cell is way faster 2022-02-10 17:05:20 -08:00
Lev Kokotov
0bec14ba1c wrong benchmark 2022-02-10 14:25:04 -08:00
Lev Kokotov
ee9f609d4e bench setup 2022-02-10 14:13:47 -08:00
Lev Kokotov
19e9f26467 readme 2022-02-10 14:03:38 -08:00
Lev Kokotov
070c38ddc5 benchmarks 2022-02-10 14:02:24 -08:00
Lev Kokotov
1798065b76 max_workers = 4 = so much faster 2022-02-10 13:48:56 -08:00
Lev Kokotov
a3b910ea72 Merge pull request #6 from levkk/levkk-ci-tests
End-to-end CI tests
2022-02-10 11:23:10 -08:00
Lev Kokotov
b9b89db708 maybe were breaking the terminal? 2022-02-10 11:20:33 -08:00
Lev Kokotov
604af32b94 print whats going on 2022-02-10 11:16:08 -08:00
Lev Kokotov
39028282b9 hmm 2022-02-10 11:13:31 -08:00
Lev Kokotov
9d51fe8985 print whats going on 2022-02-10 11:11:56 -08:00
Lev Kokotov
12011be3ec tests 2022-02-10 11:08:57 -08:00
Lev Kokotov
86386c7377 background 2022-02-10 10:59:45 -08:00
Lev Kokotov
66c5271453 sudo 2022-02-10 10:54:06 -08:00
Lev Kokotov
17aed5dcee hmm 2022-02-10 10:53:15 -08:00
Lev Kokotov
89dc33f8aa test ci 2022-02-10 10:50:19 -08:00
Lev Kokotov
c6ccc6b6ae Merge pull request #5 from levkk/levkk-replica-primary
#4 Primary/replica selection
2022-02-10 10:40:53 -08:00
Lev Kokotov
495d6ce6c3 fmt 2022-02-10 10:38:06 -08:00
Lev Kokotov
883b6ee793 todo complete 2022-02-10 10:37:55 -08:00
Lev Kokotov
22c6f13dc7 removed atomic round-robin 2022-02-10 10:37:49 -08:00
Lev Kokotov
c1476d29da config tests 2022-02-10 09:07:10 -08:00
Lev Kokotov
8209633e05 pool fixes 2022-02-10 08:54:06 -08:00
Lev Kokotov
daf120aeac more tests 2022-02-10 08:35:25 -08:00
Lev Kokotov
6d5ab79ed3 readme 2022-02-09 21:25:17 -08:00
Lev Kokotov
fccfb40258 nl 2022-02-09 21:20:20 -08:00
Lev Kokotov
a9b2a41a9b fixes to the banlist 2022-02-09 21:19:14 -08:00
Lev Kokotov
28c70d47b6 #1 Primary/replica selection 2022-02-09 20:02:20 -08:00
Lev Kokotov
00f2d39446 Merge branch 'main' of github.com:levkk/pgcat into main 2022-02-09 06:51:47 -08:00
Lev Kokotov
4c16ba3848 some comments 2022-02-09 06:51:31 -08:00
Lev Kokotov
d530f30def Update README.md 2022-02-08 21:59:56 -08:00
30 changed files with 74 additions and 3508 deletions

View File

@@ -1,37 +0,0 @@
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1
# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
build:
# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
docker:
- image: cimg/rust:1.58.1
# Add steps to the job
# See: https://circleci.com/docs/2.0/configuration-reference/#steps
steps:
- checkout
- restore_cache:
key: cargo-lock-2-{{ checksum "Cargo.lock" }}
- run:
name: "Build"
command: "cargo build"
- run:
name: "Test"
command: "cargo test"
- save_cache:
key: cargo-lock-2-{{ checksum "Cargo.lock" }}
paths:
- target
- ~/.cargo
# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
build:
jobs:
- build

13
.gitignore vendored
View File

@@ -1,2 +1,15 @@
.idea
/target
*.deb
.vscode
*.profraw
cov/
lcov.info
# Dev
dev/.bash_history
dev/cache
!dev/cache/.keepme
.venv
**/__pycache__
.bundle

View File

@@ -1,6 +0,0 @@
Thank you for contributing! Just a few tips here:
1. `cargo fmt` your code before opening up a PR
2. Run the "test suite" (i.e. PgBench) to make sure everything still works.
Happy hacking!

597
Cargo.lock generated
View File

@@ -1,597 +0,0 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 3
[[package]]
name = "aho-corasick"
version = "0.7.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e37cfd5e7657ada45f742d6e99ca5788580b5c529dc78faf11ece6dc702656f"
dependencies = [
"memchr",
]
[[package]]
name = "async-trait"
version = "0.1.52"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "061a7acccaa286c011ddc30970520b98fa40e00c9d644633fb26b5fc63a265e3"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "autocfg"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cdb031dd78e28731d87d56cc8ffef4a8f36ca26c38fe2de700543e627f8a464a"
[[package]]
name = "bb8"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e9f4fa9768efd269499d8fba693260cfc670891cf6de3adc935588447a77cc8"
dependencies = [
"async-trait",
"futures-channel",
"futures-util",
"parking_lot",
"tokio",
]
[[package]]
name = "bitflags"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "block-buffer"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1d36a02058e76b040de25a4464ba1c80935655595b661505c8b39b664828b95"
dependencies = [
"generic-array",
]
[[package]]
name = "bytes"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4872d67bab6358e59559027aa3b9157c53d9358c51423c17554809a8858e0f8"
[[package]]
name = "cfg-if"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
[[package]]
name = "chrono"
version = "0.4.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "670ad68c9088c2a963aaa298cb369688cf3f9465ce5e2d4ca10e6e0098a1ce73"
dependencies = [
"libc",
"num-integer",
"num-traits",
"time",
"winapi",
]
[[package]]
name = "cpufeatures"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "95059428f66df56b63431fdb4e1947ed2190586af5c5a8a8b71122bdf5a7f469"
dependencies = [
"libc",
]
[[package]]
name = "crypto-common"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d6b536309245c849479fba3da410962a43ed8e51c26b729208ec0ac2798d0"
dependencies = [
"generic-array",
]
[[package]]
name = "digest"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b697d66081d42af4fba142d56918a3cb21dc8eb63372c6b85d14f44fb9c5979b"
dependencies = [
"block-buffer",
"crypto-common",
"generic-array",
]
[[package]]
name = "futures-channel"
version = "0.3.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba3dda0b6588335f360afc675d0564c17a77a2bda81ca178a4b6081bd86c7f0b"
dependencies = [
"futures-core",
]
[[package]]
name = "futures-core"
version = "0.3.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0c8ff0461b82559810cdccfde3215c3f373807f5e5232b71479bff7bb2583d7"
[[package]]
name = "futures-task"
version = "0.3.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ee7c6485c30167ce4dfb83ac568a849fe53274c831081476ee13e0dce1aad72"
[[package]]
name = "futures-util"
version = "0.3.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b5cf40b47a271f77a8b1bec03ca09044d99d2372c0de244e66430761127164"
dependencies = [
"futures-channel",
"futures-core",
"futures-task",
"pin-project-lite",
"pin-utils",
"slab",
]
[[package]]
name = "generic-array"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd48d33ec7f05fbfa152300fdad764757cbded343c1aa1cff2fbaf4134851803"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "getrandom"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "418d37c8b1d42553c93648be529cb70f920d3baf8ef469b74b9638df426e0b4c"
dependencies = [
"cfg-if",
"libc",
"wasi",
]
[[package]]
name = "hermit-abi"
version = "0.1.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62b467343b94ba476dcb2500d242dadbb39557df889310ac77c5d99100aaac33"
dependencies = [
"libc",
]
[[package]]
name = "instant"
version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a5bbe824c507c5da5956355e86a746d82e0e1464f65d862cc5e71da70e94b2c"
dependencies = [
"cfg-if",
]
[[package]]
name = "libc"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e74d72e0f9b65b5b4ca49a346af3976df0f9c61d550727f349ecd559f251a26c"
[[package]]
name = "lock_api"
version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88943dd7ef4a2e5a4bfa2753aaab3013e34ce2533d1996fb18ef591e315e2b3b"
dependencies = [
"scopeguard",
]
[[package]]
name = "log"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51b9bbe6c47d51fc3e1a9b945965946b4c44142ab8792c50835a980d362c2710"
dependencies = [
"cfg-if",
]
[[package]]
name = "md-5"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6a38fc55c8bbc10058782919516f88826e70320db6d206aebc49611d24216ae"
dependencies = [
"digest",
]
[[package]]
name = "memchr"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "308cc39be01b73d0d18f82a0e7b2a3df85245f84af96fdddc5d202d27e47b86a"
[[package]]
name = "mio"
version = "0.7.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8067b404fe97c70829f082dec8bcf4f71225d7eaea1d8645349cb76fa06205cc"
dependencies = [
"libc",
"log",
"miow",
"ntapi",
"winapi",
]
[[package]]
name = "miow"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9f1c5b025cda876f66ef43a113f91ebc9f4ccef34843000e0adf6ebbab84e21"
dependencies = [
"winapi",
]
[[package]]
name = "ntapi"
version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f6bb902e437b6d86e03cce10a7e2af662292c5dfef23b65899ea3ac9354ad44"
dependencies = [
"winapi",
]
[[package]]
name = "num-integer"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2cc698a63b549a70bc047073d2949cce27cd1c7b0a4a862d08a8031bc2801db"
dependencies = [
"autocfg",
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a64b1ec5cda2586e284722486d802acf1f7dbdc623e2bfc57e65ca1cd099290"
dependencies = [
"autocfg",
]
[[package]]
name = "num_cpus"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19e64526ebdee182341572e50e9ad03965aa510cd94427a4549448f285e957a1"
dependencies = [
"hermit-abi",
"libc",
]
[[package]]
name = "once_cell"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da32515d9f6e6e489d7bc9d84c71b060db7247dc035bbe44eac88cf87486d8d5"
[[package]]
name = "parking_lot"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d17b78036a60663b797adeaee46f5c9dfebb86948d1255007a1d6be0271ff99"
dependencies = [
"instant",
"lock_api",
"parking_lot_core",
]
[[package]]
name = "parking_lot_core"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d76e8e1493bcac0d2766c42737f34458f1c8c50c0d23bcb24ea953affb273216"
dependencies = [
"cfg-if",
"instant",
"libc",
"redox_syscall",
"smallvec",
"winapi",
]
[[package]]
name = "pgcat"
version = "0.1.0"
dependencies = [
"async-trait",
"bb8",
"bytes",
"chrono",
"md-5",
"rand",
"regex",
"serde",
"serde_derive",
"sha-1",
"tokio",
"toml",
]
[[package]]
name = "pin-project-lite"
version = "0.2.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e280fbe77cc62c91527259e9442153f4688736748d24660126286329742b4c6c"
[[package]]
name = "pin-utils"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "ppv-lite86"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb9f9e6e233e5c4a35559a617bf40a4ec447db2e84c20b55a6f83167b7e57872"
[[package]]
name = "proc-macro2"
version = "1.0.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7342d5883fbccae1cc37a2353b09c87c9b0f3afd73f5fb9bba687a1f733b029"
dependencies = [
"unicode-xid",
]
[[package]]
name = "quote"
version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "864d3e96a899863136fc6e99f3d7cae289dafe43bf2c5ac19b70df7210c0a145"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rand"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e7573632e6454cf6b99d7aac4ccca54be06da05aca2ef7423d22d27d4d4bcd8"
dependencies = [
"libc",
"rand_chacha",
"rand_core",
"rand_hc",
]
[[package]]
name = "rand_chacha"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
dependencies = [
"ppv-lite86",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d34f1408f55294453790c48b2f1ebbb1c5b4b7563eb1f418bcfcfdbb06ebb4e7"
dependencies = [
"getrandom",
]
[[package]]
name = "rand_hc"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d51e9f596de227fda2ea6c84607f5558e196eeaf43c986b724ba4fb8fdf497e7"
dependencies = [
"rand_core",
]
[[package]]
name = "redox_syscall"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8383f39639269cde97d255a32bdb68c047337295414940c68bdd30c2e13203ff"
dependencies = [
"bitflags",
]
[[package]]
name = "regex"
version = "1.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d07a8629359eb56f1e2fb1652bb04212c072a87ba68546a04065d525673ac461"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.6.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f497285884f3fcff424ffc933e56d7cbca511def0c9831a7f9b5f6153e3cc89b"
[[package]]
name = "scopeguard"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d29ab0c6d3fc0ee92fe66e2d99f700eab17a8d57d1c1d3b748380fb20baa78cd"
[[package]]
name = "serde"
version = "1.0.136"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce31e24b01e1e524df96f1c2fdd054405f8d7376249a5110886fb4b658484789"
[[package]]
name = "serde_derive"
version = "1.0.136"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08597e7152fcd306f41838ed3e37be9eaeed2b61c42e2117266a554fab4662f9"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "sha-1"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "028f48d513f9678cda28f6e4064755b3fbb2af6acd672f2c209b62323f7aea0f"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "signal-hook-registry"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e51e73328dc4ac0c7ccbda3a494dfa03df1de2f46018127f60c693f2648455b0"
dependencies = [
"libc",
]
[[package]]
name = "slab"
version = "0.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9def91fd1e018fe007022791f865d0ccc9b3a0d5001e01aabb8b40e46000afb5"
[[package]]
name = "smallvec"
version = "1.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2dd574626839106c320a323308629dcb1acfc96e32a8cba364ddc61ac23ee83"
[[package]]
name = "syn"
version = "1.0.86"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a65b3f4ffa0092e9887669db0eae07941f023991ab58ea44da8fe8e2d511c6b"
dependencies = [
"proc-macro2",
"quote",
"unicode-xid",
]
[[package]]
name = "time"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6db9e6914ab8b1ae1c260a4ae7a49b6c5611b40328a735b21862567685e73255"
dependencies = [
"libc",
"wasi",
"winapi",
]
[[package]]
name = "tokio"
version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c27a64b625de6d309e8c57716ba93021dccf1b3b5c97edd6d3dd2d2135afc0a"
dependencies = [
"bytes",
"libc",
"memchr",
"mio",
"num_cpus",
"once_cell",
"parking_lot",
"pin-project-lite",
"signal-hook-registry",
"tokio-macros",
"winapi",
]
[[package]]
name = "tokio-macros"
version = "1.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b557f72f448c511a979e2564e55d74e6c4432fc96ff4f6241bc6bded342643b7"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "toml"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a31142970826733df8241ef35dc040ef98c679ab14d7c3e54d827099b3acecaa"
dependencies = [
"serde",
]
[[package]]
name = "typenum"
version = "1.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dcf81ac59edc17cc8697ff311e8f5ef2d99fcbd9817b34cec66f90b6c3dfd987"
[[package]]
name = "unicode-xid"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ccb82d61f80a663efe1f787a51b16b5a51e3314d6ac365b08639f52387b33f3"
[[package]]
name = "version_check"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49874b5167b65d7193b8aba1567f5c7d93d001cafc34600cee003eda787e483f"
[[package]]
name = "wasi"
version = "0.10.0+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f"
[[package]]
name = "winapi"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"

View File

@@ -1,20 +0,0 @@
[package]
name = "pgcat"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
tokio = { version = "1", features = ["full"] }
bytes = "1"
md-5 = "0.10"
bb8 = "0.7"
async-trait = "0.1"
rand = "0.8"
chrono = "0.4"
sha-1 = "0.10"
toml = "0.5"
serde = "1"
serde_derive = "1"
regex = "1"

674
LICENSE
View File

@@ -1,674 +0,0 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.

209
README.md
View File

@@ -1,208 +1 @@
# PgCat
[![CircleCI](https://circleci.com/gh/levkk/pgcat/tree/main.svg?style=svg)](https://circleci.com/gh/levkk/pgcat/tree/main)
![PgCat](./pgcat3.png)
Meow. PgBouncer rewritten in Rust, with sharding, load balancing and failover support.
**Alpha**: don't use in production just yet.
## Local development
1. Install Rust (latest stable will work great).
2. `cargo run --release` (to get better benchmarks).
3. Change the config in `pgcat.toml` to fit your setup (optional given next step).
4. Install Postgres and run `psql -f tests/sharding/query_routing_setup.sql`
### Tests
You can just PgBench to test your changes:
```
pgbench -i -h 127.0.0.1 -p 6432 && \
pgbench -t 1000 -p 6432 -h 127.0.0.1 --protocol simple && \
pgbench -t 1000 -p 6432 -h 127.0.0.1 --protocol extended
```
See [sharding README](./tests/sharding/README.md) for sharding logic testing.
## Features
1. Session mode.
2. Transaction mode.
3. `COPY` protocol support.
4. Query cancellation.
5. Round-robin load balancing of replicas.
6. Banlist & failover
7. Sharding!
### Session mode
Each client owns its own server for the duration of the session. Commands like `SET` are allowed.
This is identical to PgBouncer session mode.
### Transaction mode
The connection is attached to the server for the duration of the transaction. `SET` will pollute the connection,
but `SET LOCAL` works great. Identical to PgBouncer transaction mode.
### COPY protocol
That one isn't particularly special, but good to mention that you can `COPY` data in and from the server
using this pooler.
### Query cancellation
Okay, this is just basic stuff, but we support cancelling queries. If you know the Postgres protocol,
this might be relevant given than this is a transactional pooler but if you're new to Pg, don't worry about it, it works.
### Round-robin load balancing
This is the novel part. PgBouncer doesn't support it and suggests we use DNS or a TCP proxy instead.
We prefer to have everything as part of one package; arguably, it's easier to understand and optimize.
This pooler will round-robin between multiple replicas keeping load reasonably even.
### Banlist & failover
This is where it gets even more interesting. If we fail to connect to one of the replicas or it fails a health check,
we add it to a ban list. No more new transactions will be served by that replica for, in our case, 60 seconds. This
gives it the opportunity to recover while clients are happily served by the remaining replicas.
This decreases error rates substantially! Worth noting here that on busy systems, if the replicas are running too hot,
failing over could bring even more load and tip over the remaining healthy-ish replicas. In this case, a decision should be made:
either lose 1/x of your traffic or risk losing it all eventually. Ideally you overprovision your system, so you don't necessarily need
to make this choice :-).
### Sharding
We're implemeting Postgres' `PARTITION BY HASH` sharding function for `BIGINT` fields. This works well for tables that use `BIGSERIAL` primary key which I think is common enough these days. We can also add many more functions here, but this is a good start. See `src/sharding.rs` and `tests/sharding/partition_hash_test_setup.sql` for more details on the implementation.
The biggest advantage of using this sharding function is that anyone can shard the dataset using Postgres partitions
while also access it for both reads and writes using this pooler. No custom obscure sharding function is needed and database sharding can be done entirely in Postgres.
To select the shard we want to talk to, we introduced special syntax:
```sql
SET SHARDING KEY TO '1234';
```
This sharding key will be hashed and the pooler will select a shard to use for the next transaction. If the pooler is in session mode, this sharding key will be used until it's set again or the client disconnects.
## Missing
1. Authentication, ehem, this proxy is letting anyone in at the moment.
## Benchmarks
You can setup PgBench locally through PgCat:
```
pgbench -h 127.0.0.1 -p 6432 -i
```
Coincidenly, this uses `COPY` so you can test if that works.
### PgBouncer
```
$ pgbench -i -h 127.0.0.1 -p 6432 && pgbench -t 1000 -p 6432 -h 127.0.0.1 --protocol simple && pgbench -t 1000 -p 6432 -h 127.0.0.1 --protocol extended
dropping old tables...
creating tables...
generating data...
100000 of 100000 tuples (100%) done (elapsed 0.01 s, remaining 0.00 s)
vacuuming...
creating primary keys...
done.
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
number of transactions per client: 1000
number of transactions actually processed: 1000/1000
latency average = 1.089 ms
tps = 918.687098 (including connections establishing)
tps = 918.847790 (excluding connections establishing)
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: extended
number of clients: 1
number of threads: 1
number of transactions per client: 1000
number of transactions actually processed: 1000/1000
latency average = 1.136 ms
tps = 880.622009 (including connections establishing)
tps = 880.769550 (excluding connections establishing)
```
### PgCat
```
$ pgbench -i -h 127.0.0.1 -p 6432 && pgbench -t 1000 -p 6432 -h 127.0.0.1 --protocol simple && pgbench -t 1000 -p 6432 -h 127.0.0.1 --protocol extended
dropping old tables...
creating tables...
generating data...
100000 of 100000 tuples (100%) done (elapsed 0.01 s, remaining 0.00 s)
vacuuming...
creating primary keys...
done.
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
number of transactions per client: 1000
number of transactions actually processed: 1000/1000
latency average = 1.142 ms
tps = 875.645437 (including connections establishing)
tps = 875.799995 (excluding connections establishing)
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: extended
number of clients: 1
number of threads: 1
number of transactions per client: 1000
number of transactions actually processed: 1000/1000
latency average = 1.181 ms
tps = 846.539176 (including connections establishing)
tps = 846.713636 (excluding connections establishing)
```
### Direct Postgres
```
$ pgbench -i -h 127.0.0.1 -p 5432 && pgbench -t 1000 -p 5432 -h 127.0.0.1 --protocol simple && pgbench -t 1000 -p
5432 -h 127.0.0.1 --protocol extended
Password:
dropping old tables...
creating tables...
generating data...
100000 of 100000 tuples (100%) done (elapsed 0.01 s, remaining 0.00 s)
vacuuming...
creating primary keys...
done.
Password:
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
number of transactions per client: 1000
number of transactions actually processed: 1000/1000
latency average = 0.902 ms
tps = 1109.014867 (including connections establishing)
tps = 1112.318595 (excluding connections establishing)
Password:
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: extended
number of clients: 1
number of threads: 1
number of transactions per client: 1000
number of transactions actually processed: 1000/1000
latency average = 0.931 ms
tps = 1074.017747 (including connections establishing)
tps = 1077.121752 (excluding connections establishing)
```
## PgCat: Nextgen PostgreSQL Pooler Helm Charts

4
artifacthub-repo.yml Normal file
View File

@@ -0,0 +1,4 @@
repositoryID: eb593650-e8b5-4924-b373-0f4da407333a
owners:
- name: Postgresml Team
email: team@postgresml.org

56
index.yaml Normal file
View File

@@ -0,0 +1,56 @@
apiVersion: v1
entries:
pgcat:
- apiVersion: v2
appVersion: 1.2.0
created: "2024-11-11T15:24:31.734977689Z"
description: A Helm chart for PgCat a PostgreSQL pooler and proxy (like PgBouncer)
with support for sharding, load balancing, failover and mirroring.
digest: f9e43bff47fe754928e6dd7ebc74b6af37ce8a564fba0d2e7dcc52fc3cbb953a
maintainers:
- email: team@postgresml.org
name: PostgresML
name: pgcat
urls:
- https://github.com/postgresml/pgcat/releases/download/pgcat-0.2.5/pgcat-0.2.5.tgz
version: 0.2.5
- apiVersion: v2
appVersion: 1.2.0
created: "2024-11-08T12:07:27.767665231Z"
description: A Helm chart for PgCat a PostgreSQL pooler and proxy (like PgBouncer)
with support for sharding, load balancing, failover and mirroring.
digest: 904a643eef7f1e311770a760578a9523ae648502eba0a040ac3ff3871b1144f7
maintainers:
- email: support@w6d.io
name: Wildcard
name: pgcat
urls:
- https://github.com/postgresml/pgcat/releases/download/pgcat-0.2.4/pgcat-0.2.4.tgz
version: 0.2.4
- apiVersion: v2
appVersion: 1.2.0
created: "2024-11-02T23:05:55.57497085Z"
description: A Helm chart for PgCat a PostgreSQL pooler and proxy (like PgBouncer)
with support for sharding, load balancing, failover and mirroring.
digest: b5a0236a9072523d96056a1e8567c2c8e862221b8b6ce8d038fbee3d3505986e
maintainers:
- email: support@w6d.io
name: Wildcard
name: pgcat
urls:
- https://github.com/postgresml/pgcat/releases/download/pgcat-0.2.3/pgcat-0.2.3.tgz
version: 0.2.3
- apiVersion: v2
appVersion: 1.2.0
created: "2024-11-02T16:55:33.01802196Z"
description: A Helm chart for PgCat a PostgreSQL pooler and proxy (like PgBouncer)
with support for sharding, load balancing, failover and mirroring.
digest: 106ce6a390c9e8d7fd1efb02b307675f4ed14534bae1ccd1b9e7d121b9a69a74
maintainers:
- email: support@w6d.io
name: Wildcard
name: pgcat
urls:
- https://github.com/postgresml/pgcat/releases/download/pgcat-0.2.2/pgcat-0.2.2.tgz
version: 0.2.2
generated: "2024-11-11T15:24:31.734998568Z"

View File

@@ -1,68 +0,0 @@
#
# PgCat config example.
#
#
# General pooler settings
[general]
# What IP to run on, 0.0.0.0 means accessible from everywhere.
host = "0.0.0.0"
# Port to run on, same as PgBouncer used in this example.
port = 6432
# How many connections to allocate per server.
pool_size = 15
# Pool mode (see PgBouncer docs for more).
# session: one server connection per connected client
# transaction: one server connection per client transaction
pool_mode = "transaction"
# How long to wait before aborting a server connection (ms).
connect_timeout = 5000
# How much time to give `SELECT 1` health check query to return with a result (ms).
healthcheck_timeout = 1000
# For how long to ban a server if it fails a health check (seconds).
ban_time = 60 # Seconds
#
# User to use for authentication against the server.
[user]
name = "sharding_user"
password = "sharding_user"
#
# Shards in the cluster
[shards]
# Shard 0
[shards.0]
# [ host, port ]
servers = [
[ "127.0.0.1", 5432 ],
[ "localhost", 5432 ],
]
# Database name (e.g. "postgres")
database = "shard0"
[shards.1]
# [ host, port ]
servers = [
[ "127.0.0.1", 5432 ],
[ "localhost", 5432 ],
]
database = "shard1"
[shards.2]
# [ host, port ]
servers = [
[ "127.0.0.1", 5432 ],
[ "localhost", 5432 ],
]
database = "shard2"

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

View File

@@ -1,389 +0,0 @@
/// Implementation of the PostgreSQL client.
/// We are pretending to the server in this scenario,
/// and this module implements that.
use bytes::{Buf, BufMut, BytesMut};
use regex::Regex;
use tokio::io::{AsyncReadExt, BufReader};
use tokio::net::tcp::{OwnedReadHalf, OwnedWriteHalf};
use tokio::net::TcpStream;
use crate::errors::Error;
use crate::messages::*;
use crate::pool::{ClientServerMap, ConnectionPool};
use crate::server::Server;
use crate::sharding::Sharder;
const SHARDING_REGEX: &str = r"SET SHARDING KEY TO '[0-9]+';";
/// The client state. One of these is created per client.
pub struct Client {
// The reads are buffered (8K by default).
read: BufReader<OwnedReadHalf>,
// We buffer the writes ourselves because we know the protocol
// better than a stock buffer.
write: OwnedWriteHalf,
// Internal buffer, where we place messages until we have to flush
// them to the backend.
buffer: BytesMut,
// The client was started with the sole reason to cancel another running query.
cancel_mode: bool,
// In transaction mode, the connection is released after each transaction.
// Session mode has slightly higher throughput per client, but lower capacity.
transaction_mode: bool,
// For query cancellation, the client is given a random process ID and secret on startup.
process_id: i32,
secret_key: i32,
// Clients are mapped to servers while they use them. This allows a client
// to connect and cancel a query.
client_server_map: ClientServerMap,
// sharding regex
sharding_regex: Regex,
}
impl Client {
/// Given a TCP socket, trick the client into thinking we are
/// the Postgres server. Perform the authentication and place
/// the client in query-ready mode.
pub async fn startup(
mut stream: TcpStream,
client_server_map: ClientServerMap,
transaction_mode: bool,
) -> Result<Client, Error> {
let sharding_regex = Regex::new(SHARDING_REGEX).unwrap();
loop {
// Could be StartupMessage or SSLRequest
// which makes this variable length.
let len = match stream.read_i32().await {
Ok(len) => len,
Err(_) => return Err(Error::ClientBadStartup),
};
// Read whatever is left.
let mut startup = vec![0u8; len as usize - 4];
match stream.read_exact(&mut startup).await {
Ok(_) => (),
Err(_) => return Err(Error::ClientBadStartup),
};
let mut bytes = BytesMut::from(&startup[..]);
let code = bytes.get_i32();
match code {
// Client wants SSL. We don't support it at the moment.
80877103 => {
let mut no = BytesMut::with_capacity(1);
no.put_u8(b'N');
write_all(&mut stream, no).await?;
}
// Regular startup message.
196608 => {
// TODO: perform actual auth.
// TODO: record startup parameters client sends over.
// Generate random backend ID and secret key
let process_id: i32 = rand::random();
let secret_key: i32 = rand::random();
auth_ok(&mut stream).await?;
server_parameters(&mut stream).await?;
backend_key_data(&mut stream, process_id, secret_key).await?;
ready_for_query(&mut stream).await?;
// Split the read and write streams
// so we can control buffering.
let (read, write) = stream.into_split();
return Ok(Client {
read: BufReader::new(read),
write: write,
buffer: BytesMut::with_capacity(8196),
cancel_mode: false,
transaction_mode: transaction_mode,
process_id: process_id,
secret_key: secret_key,
client_server_map: client_server_map,
sharding_regex: sharding_regex,
});
}
// Query cancel request.
80877102 => {
let (read, write) = stream.into_split();
let process_id = bytes.get_i32();
let secret_key = bytes.get_i32();
return Ok(Client {
read: BufReader::new(read),
write: write,
buffer: BytesMut::with_capacity(8196),
cancel_mode: true,
transaction_mode: transaction_mode,
process_id: process_id,
secret_key: secret_key,
client_server_map: client_server_map,
sharding_regex: sharding_regex,
});
}
_ => {
return Err(Error::ProtocolSyncError);
}
};
}
}
/// Client loop. We handle all messages between the client and the database here.
pub async fn handle(&mut self, pool: ConnectionPool) -> Result<(), Error> {
// Special: cancelling existing running query
if self.cancel_mode {
let (process_id, secret_key, address, port) = {
let guard = self.client_server_map.lock().unwrap();
match guard.get(&(self.process_id, self.secret_key)) {
// Drop the mutex as soon as possible.
Some((process_id, secret_key, address, port)) => (
process_id.clone(),
secret_key.clone(),
address.clone(),
port.clone(),
),
None => return Ok(()),
}
};
// TODO: pass actual server host and port somewhere.
return Ok(Server::cancel(&address, &port, process_id, secret_key).await?);
}
// Active shard we're talking to.
// The lifetime of this depends on the pool mode:
// - if in session mode, this lives until client disconnects or changes it,
// - if in transaction mode, this lives for the duration of one transaction.
let mut shard: Option<usize> = None;
loop {
// Read a complete message from the client, which normally would be
// either a `Q` (query) or `P` (prepare, extended protocol).
// We can parse it here before grabbing a server from the pool,
// in case the client is sending some control messages, e.g.
// SET sharding_context.key = '1234';
let mut message = read_message(&mut self.read).await?;
// Parse for special select shard command.
// SET SHARDING KEY TO 'bigint';
match self.select_shard(message.clone(), pool.shards()).await {
Some(s) => {
set_sharding_key(&mut self.write).await?;
shard = Some(s);
continue;
}
None => (),
};
// The message is part of the regular protocol.
// self.buffer.put(message);
// Grab a server from the pool.
// None = any shard
let connection = pool.get(shard).await.unwrap();
let mut proxy = connection.0;
let _address = connection.1;
let server = &mut *proxy;
// Claim this server as mine for query cancellation.
server.claim(self.process_id, self.secret_key);
loop {
// No messages in the buffer, read one.
let mut message = if message.len() == 0 {
match read_message(&mut self.read).await {
Ok(message) => message,
Err(err) => {
// Client disconnected without warning.
if server.in_transaction() {
// TODO: this is what PgBouncer does
// which leads to connection thrashing.
//
// I think we could issue a ROLLBACK here instead.
// server.mark_bad();
server.query("ROLLBACK; DISCARD ALL;").await?;
}
return Err(err);
}
}
} else {
let msg = message.clone();
message.clear();
msg
};
let original = message.clone(); // To be forwarded to the server
let code = message.get_u8() as char;
let _len = message.get_i32() as usize;
match code {
'Q' => {
server.send(original).await?;
loop {
let response = server.recv().await?;
match write_all_half(&mut self.write, response).await {
Ok(_) => (),
Err(err) => {
server.mark_bad();
return Err(err);
}
};
if !server.is_data_available() {
break;
}
}
// Release server
if !server.in_transaction() && self.transaction_mode {
shard = None;
break;
}
}
'X' => {
// Client closing. Rollback and clean up
// connection before releasing into the pool.
// Pgbouncer closes the connection which leads to
// connection thrashing when clients misbehave.
// This pool will protect the database. :salute:
if server.in_transaction() {
server.query("ROLLBACK; DISCARD ALL;").await?;
}
return Ok(());
}
'P' => {
// Extended protocol, let's buffer most of it
self.buffer.put(&original[..]);
}
'B' => {
self.buffer.put(&original[..]);
}
// Describe
'D' => {
self.buffer.put(&original[..]);
}
'E' => {
self.buffer.put(&original[..]);
}
'S' => {
// Extended protocol, client requests sync
self.buffer.put(&original[..]);
server.send(self.buffer.clone()).await?;
self.buffer.clear();
loop {
let response = server.recv().await?;
match write_all_half(&mut self.write, response).await {
Ok(_) => (),
Err(err) => {
server.mark_bad();
return Err(err);
}
};
if !server.is_data_available() {
break;
}
}
// Release server
if !server.in_transaction() && self.transaction_mode {
shard = None;
break;
}
}
// CopyData
'd' => {
// Forward the data to the server,
// don't buffer it since it can be rather large.
server.send(original).await?;
}
'c' | 'f' => {
// Copy is done.
server.send(original).await?;
let response = server.recv().await?;
match write_all_half(&mut self.write, response).await {
Ok(_) => (),
Err(err) => {
server.mark_bad();
return Err(err);
}
};
// Release the server
if !server.in_transaction() && self.transaction_mode {
println!("Releasing after copy done");
shard = None;
break;
}
}
_ => {
println!(">>> Unexpected code: {}", code);
}
}
}
self.release();
}
}
/// Release the server from being mine. I can't cancel its queries anymore.
pub fn release(&mut self) {
let mut guard = self.client_server_map.lock().unwrap();
guard.remove(&(self.process_id, self.secret_key));
}
async fn select_shard(&mut self, mut buf: BytesMut, shards: usize) -> Option<usize> {
let code = buf.get_u8() as char;
match code {
'Q' => (),
// 'P' => (),
_ => return None,
};
let len = buf.get_i32();
let query = String::from_utf8_lossy(&buf[..len as usize - 4 - 1]).to_ascii_uppercase(); // Don't read the ternminating null
if self.sharding_regex.is_match(&query) {
let shard = query.split("'").collect::<Vec<&str>>()[1];
match shard.parse::<i64>() {
Ok(shard) => {
let sharder = Sharder::new(shards);
Some(sharder.pg_bigint_hash(shard))
}
Err(_) => None,
}
} else {
None
}
}
}

View File

@@ -1,87 +0,0 @@
use serde_derive::Deserialize;
use tokio::fs::File;
use tokio::io::AsyncReadExt;
use toml;
use std::collections::HashMap;
use crate::errors::Error;
#[derive(Clone, PartialEq, Hash, std::cmp::Eq, Debug)]
pub struct Address {
pub host: String,
pub port: String,
}
#[derive(Clone, PartialEq, Hash, std::cmp::Eq, Deserialize, Debug)]
pub struct User {
pub name: String,
pub password: String,
}
#[derive(Deserialize, Debug, Clone)]
pub struct General {
pub host: String,
pub port: i16,
pub pool_size: u32,
pub pool_mode: String,
pub connect_timeout: u64,
pub healthcheck_timeout: u64,
pub ban_time: i64,
}
#[derive(Deserialize, Debug, Clone)]
pub struct Shard {
pub servers: Vec<(String, u16)>,
pub database: String,
}
#[derive(Deserialize, Debug, Clone)]
pub struct Config {
pub general: General,
pub user: User,
pub shards: HashMap<String, Shard>,
}
pub async fn parse(path: &str) -> Result<Config, Error> {
// let path = Path::new(path);
let mut contents = String::new();
let mut file = match File::open(path).await {
Ok(file) => file,
Err(err) => {
println!("> Config error: {:?}", err);
return Err(Error::BadConfig);
}
};
match file.read_to_string(&mut contents).await {
Ok(_) => (),
Err(err) => {
println!("> Config error: {:?}", err);
return Err(Error::BadConfig);
}
};
let config: Config = match toml::from_str(&contents) {
Ok(config) => config,
Err(err) => {
println!("> Config error: {:?}", err);
return Err(Error::BadConfig);
}
};
Ok(config)
}
#[cfg(test)]
mod test {
use super::*;
#[tokio::test]
async fn test_config() {
let config = parse("pgcat.toml").await.unwrap();
assert_eq!(config.general.pool_size, 15);
assert_eq!(config.shards.len(), 3);
assert_eq!(config.shards["1"].servers[0].0, "127.0.0.1");
}
}

View File

@@ -1,11 +0,0 @@
#[derive(Debug, PartialEq)]
pub enum Error {
SocketError,
// ClientDisconnected,
ClientBadStartup,
ProtocolSyncError,
ServerError,
// ServerTimeout,
// DirtyServer,
BadConfig,
}

View File

@@ -1,123 +0,0 @@
// PgCat, a PostgreSQL pooler with load balancing, failover, and sharding support.
// Copyright (C) 2022 Lev Kokotov <lev@levthe.dev>
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
extern crate async_trait;
extern crate bb8;
extern crate bytes;
extern crate md5;
extern crate serde;
extern crate serde_derive;
extern crate tokio;
extern crate toml;
use tokio::net::TcpListener;
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
mod client;
mod config;
mod errors;
mod messages;
mod pool;
mod server;
mod sharding;
// Support for query cancellation: this maps our process_ids and
// secret keys to the backend's.
use pool::{ClientServerMap, ConnectionPool};
/// Main!
#[tokio::main]
async fn main() {
println!("> Welcome to PgCat! Meow.");
let config = match config::parse("pgcat.toml").await {
Ok(config) => config,
Err(err) => {
println!("> Config parse error: {:?}", err);
return;
}
};
let addr = format!("{}:{}", config.general.host, config.general.port);
let listener = match TcpListener::bind(&addr).await {
Ok(sock) => sock,
Err(err) => {
println!("> Error: {:?}", err);
return;
}
};
println!("> Running on {}", addr);
// Tracks which client is connected to which server for query cancellation.
let client_server_map: ClientServerMap = Arc::new(Mutex::new(HashMap::new()));
println!("> Pool size: {}", config.general.pool_size);
println!("> Pool mode: {}", config.general.pool_mode);
println!("> Ban time: {}s", config.general.ban_time);
println!(
"> Healthcheck timeout: {}ms",
config.general.healthcheck_timeout
);
let pool = ConnectionPool::from_config(config.clone(), client_server_map.clone()).await;
let transaction_mode = config.general.pool_mode == "transaction";
println!("> Waiting for clients...");
loop {
let pool = pool.clone();
let client_server_map = client_server_map.clone();
let (socket, addr) = match listener.accept().await {
Ok((socket, addr)) => (socket, addr),
Err(err) => {
println!("> Listener: {:?}", err);
continue;
}
};
// Client goes to another thread, bye.
tokio::task::spawn(async move {
println!(
">> Client {:?} connected, transaction pooling: {}",
addr, transaction_mode
);
match client::Client::startup(socket, client_server_map, transaction_mode).await {
Ok(mut client) => {
println!(">> Client {:?} authenticated successfully!", addr);
match client.handle(pool).await {
Ok(()) => {
println!(">> Client {:?} disconnected.", addr);
}
Err(err) => {
println!(">> Client disconnected with error: {:?}", err);
client.release();
}
}
}
Err(err) => {
println!(">> Error: {:?}", err);
}
};
});
}
}

View File

@@ -1,200 +0,0 @@
use bytes::{BufMut, BytesMut};
use md5::{Digest, Md5};
use tokio::io::{AsyncReadExt, AsyncWriteExt, BufReader};
use tokio::net::tcp::{OwnedReadHalf, OwnedWriteHalf};
use tokio::net::TcpStream;
use crate::errors::Error;
// This is a funny one. `psql` parses this to figure out which
// queries to send when using shortcuts, e.g. \d+.
//
// TODO: Actually get the version from the server itself.
//
const SERVER_VESION: &str = "12.9 (Ubuntu 12.9-0ubuntu0.20.04.1)";
/// Tell the client that authentication handshake completed successfully.
pub async fn auth_ok(stream: &mut TcpStream) -> Result<(), Error> {
let mut auth_ok = BytesMut::with_capacity(9);
auth_ok.put_u8(b'R');
auth_ok.put_i32(8);
auth_ok.put_i32(0);
Ok(write_all(stream, auth_ok).await?)
}
/// Send server parameters to the client. This will tell the client
/// what server version and what's the encoding we're using.
pub async fn server_parameters(stream: &mut TcpStream) -> Result<(), Error> {
let client_encoding = BytesMut::from(&b"client_encoding\0UTF8\0"[..]);
let server_version =
BytesMut::from(&format!("server_version\0{}\0", SERVER_VESION).as_bytes()[..]);
// Client encoding
let len = client_encoding.len() as i32 + 4; // TODO: add more parameters here
let mut res = BytesMut::with_capacity(64);
res.put_u8(b'S');
res.put_i32(len);
res.put_slice(&client_encoding[..]);
let len = server_version.len() as i32 + 4;
res.put_u8(b'S');
res.put_i32(len);
res.put_slice(&server_version[..]);
Ok(write_all(stream, res).await?)
}
/// Give the client the process_id and secret we generated
/// used in query cancellation.
pub async fn backend_key_data(
stream: &mut TcpStream,
backend_id: i32,
secret_key: i32,
) -> Result<(), Error> {
let mut key_data = BytesMut::from(&b"K"[..]);
key_data.put_i32(12);
key_data.put_i32(backend_id);
key_data.put_i32(secret_key);
Ok(write_all(stream, key_data).await?)
}
/// Tell the client we're ready for another query.
pub async fn ready_for_query(stream: &mut TcpStream) -> Result<(), Error> {
let mut bytes = BytesMut::with_capacity(5);
bytes.put_u8(b'Z');
bytes.put_i32(5);
bytes.put_u8(b'I'); // Idle
Ok(write_all(stream, bytes).await?)
}
/// Send the startup packet the server. We're pretending we're a Pg client.
/// This tells the server which user we are and what database we want.
pub async fn startup(stream: &mut TcpStream, user: &str, database: &str) -> Result<(), Error> {
let mut bytes = BytesMut::with_capacity(25);
bytes.put_i32(196608); // Protocol number
// User
bytes.put(&b"user\0"[..]);
bytes.put_slice(&user.as_bytes());
bytes.put_u8(0);
// Database
bytes.put(&b"database\0"[..]);
bytes.put_slice(&database.as_bytes());
bytes.put_u8(0);
bytes.put_u8(0); // Null terminator
let len = bytes.len() as i32 + 4i32;
let mut startup = BytesMut::with_capacity(len as usize);
startup.put_i32(len);
startup.put(bytes);
match stream.write_all(&startup).await {
Ok(_) => Ok(()),
Err(_) => return Err(Error::SocketError),
}
}
/// Send password challenge response to the server.
/// This is the MD5 challenge.
pub async fn md5_password(
stream: &mut TcpStream,
user: &str,
password: &str,
salt: &[u8],
) -> Result<(), Error> {
let mut md5 = Md5::new();
// First pass
md5.update(&password.as_bytes());
md5.update(&user.as_bytes());
let output = md5.finalize_reset();
// Second pass
md5.update(format!("{:x}", output));
md5.update(salt);
let mut password = format!("md5{:x}", md5.finalize())
.chars()
.map(|x| x as u8)
.collect::<Vec<u8>>();
password.push(0);
let mut message = BytesMut::with_capacity(password.len() as usize + 5);
message.put_u8(b'p');
message.put_i32(password.len() as i32 + 4);
message.put_slice(&password[..]);
Ok(write_all(stream, message).await?)
}
pub async fn set_sharding_key(stream: &mut OwnedWriteHalf) -> Result<(), Error> {
let mut res = BytesMut::with_capacity(25);
let set_complete = BytesMut::from(&"SET SHARDING KEY\0"[..]);
let len = (set_complete.len() + 4) as i32;
res.put_u8(b'C');
res.put_i32(len);
res.put_slice(&set_complete[..]);
res.put_u8(b'Z');
res.put_i32(5);
res.put_u8(b'I');
write_all_half(stream, res).await
}
/// Write all data in the buffer to the TcpStream.
pub async fn write_all(stream: &mut TcpStream, buf: BytesMut) -> Result<(), Error> {
match stream.write_all(&buf).await {
Ok(_) => Ok(()),
Err(_) => return Err(Error::SocketError),
}
}
/// Write all the data in the buffer to the TcpStream, write owned half (see mpsc).
pub async fn write_all_half(stream: &mut OwnedWriteHalf, buf: BytesMut) -> Result<(), Error> {
match stream.write_all(&buf).await {
Ok(_) => Ok(()),
Err(_) => return Err(Error::SocketError),
}
}
/// Read a complete message from the socket.
pub async fn read_message(stream: &mut BufReader<OwnedReadHalf>) -> Result<BytesMut, Error> {
let code = match stream.read_u8().await {
Ok(code) => code,
Err(_) => return Err(Error::SocketError),
};
let len = match stream.read_i32().await {
Ok(len) => len,
Err(_) => return Err(Error::SocketError),
};
let mut buf = vec![0u8; len as usize - 4];
match stream.read_exact(&mut buf).await {
Ok(_) => (),
Err(_) => return Err(Error::SocketError),
};
let mut bytes = BytesMut::with_capacity(len as usize + 1);
bytes.put_u8(code);
bytes.put_i32(len);
bytes.put_slice(&buf);
Ok(bytes)
}

View File

@@ -1,267 +0,0 @@
/// Pooling and failover and banlist.
use async_trait::async_trait;
use bb8::{ManageConnection, Pool, PooledConnection};
use chrono::naive::NaiveDateTime;
use crate::config::{Address, Config, User};
use crate::errors::Error;
use crate::server::Server;
use std::collections::HashMap;
use std::sync::{
atomic::{AtomicUsize, Ordering},
Arc, Mutex,
};
// Banlist: bad servers go in here.
pub type BanList = Arc<Mutex<Vec<HashMap<Address, NaiveDateTime>>>>;
pub type Counter = Arc<AtomicUsize>;
pub type ClientServerMap = Arc<Mutex<HashMap<(i32, i32), (i32, i32, String, String)>>>;
#[derive(Clone, Debug)]
pub struct ConnectionPool {
databases: Vec<Vec<Pool<ServerPool>>>,
addresses: Vec<Vec<Address>>,
round_robin: Counter,
banlist: BanList,
healthcheck_timeout: u64,
ban_time: i64,
}
impl ConnectionPool {
/// Construct the connection pool from a config file.
pub async fn from_config(config: Config, client_server_map: ClientServerMap) -> ConnectionPool {
let mut shards = Vec::new();
let mut addresses = Vec::new();
let mut banlist = Vec::new();
let mut shard_ids = config
.shards
.clone()
.into_keys()
.map(|x| x.to_string())
.collect::<Vec<String>>();
shard_ids.sort_by_key(|k| k.parse::<i64>().unwrap());
for shard in shard_ids {
let shard = &config.shards[&shard];
let mut pools = Vec::new();
let mut replica_addresses = Vec::new();
for server in &shard.servers {
let address = Address {
host: server.0.clone(),
port: server.1.to_string(),
};
let manager = ServerPool::new(
address.clone(),
config.user.clone(),
&shard.database,
client_server_map.clone(),
);
let pool = Pool::builder()
.max_size(config.general.pool_size)
.connection_timeout(std::time::Duration::from_millis(
config.general.connect_timeout,
))
.test_on_check_out(false)
.build(manager)
.await
.unwrap();
pools.push(pool);
replica_addresses.push(address);
}
shards.push(pools);
addresses.push(replica_addresses);
banlist.push(HashMap::new());
}
ConnectionPool {
databases: shards,
addresses: addresses,
round_robin: Arc::new(AtomicUsize::new(0)),
banlist: Arc::new(Mutex::new(banlist)),
healthcheck_timeout: config.general.healthcheck_timeout,
ban_time: config.general.ban_time,
}
}
/// Get a connection from the pool.
pub async fn get(
&self,
shard: Option<usize>,
) -> Result<(PooledConnection<'_, ServerPool>, Address), Error> {
// Set this to false to gain ~3-4% speed.
let with_health_check = true;
let shard = match shard {
Some(shard) => shard,
None => 0, // TODO: pick a shard at random
};
loop {
let index =
self.round_robin.fetch_add(1, Ordering::SeqCst) % self.databases[shard].len();
let address = self.addresses[shard][index].clone();
if self.is_banned(&address, shard) {
continue;
}
// Check if we can connect
// TODO: implement query wait timeout, i.e. time to get a conn from the pool
let mut conn = match self.databases[shard][index].get().await {
Ok(conn) => conn,
Err(err) => {
println!(">> Banning replica {}, error: {:?}", index, err);
self.ban(&address, shard);
continue;
}
};
if !with_health_check {
return Ok((conn, address));
}
// // Check if this server is alive with a health check
let server = &mut *conn;
match tokio::time::timeout(
tokio::time::Duration::from_millis(self.healthcheck_timeout),
server.query("SELECT 1"),
)
.await
{
// Check if health check succeeded
Ok(res) => match res {
Ok(_) => return Ok((conn, address)),
Err(_) => {
println!(
">> Banning replica {} because of failed health check",
index
);
self.ban(&address, shard);
continue;
}
},
// Health check never came back, database is really really down
Err(_) => {
println!(
">> Banning replica {} because of health check timeout",
index
);
self.ban(&address, shard);
continue;
}
}
}
}
/// Ban an address (i.e. replica). It no longer will serve
/// traffic for any new transactions. Existing transactions on that replica
/// will finish successfully or error out to the clients.
pub fn ban(&self, address: &Address, shard: usize) {
println!(">> Banning {:?}", address);
let now = chrono::offset::Utc::now().naive_utc();
let mut guard = self.banlist.lock().unwrap();
guard[shard].insert(address.clone(), now);
}
/// Clear the replica to receive traffic again. Takes effect immediately
/// for all new transactions.
pub fn _unban(&self, address: &Address, shard: usize) {
let mut guard = self.banlist.lock().unwrap();
guard[shard].remove(address);
}
/// Check if a replica can serve traffic. If all replicas are banned,
/// we unban all of them. Better to try then not to.
pub fn is_banned(&self, address: &Address, shard: usize) -> bool {
let mut guard = self.banlist.lock().unwrap();
// Everything is banned, nothig is banned
if guard[shard].len() == self.databases[shard].len() {
guard[shard].clear();
drop(guard);
println!(">> Unbanning all replicas.");
return false;
}
// I expect this to miss 99.9999% of the time.
match guard[shard].get(address) {
Some(timestamp) => {
let now = chrono::offset::Utc::now().naive_utc();
if now.timestamp() - timestamp.timestamp() > self.ban_time {
// 1 minute
guard[shard].remove(address);
false
} else {
true
}
}
None => false,
}
}
pub fn shards(&self) -> usize {
self.databases.len()
}
}
pub struct ServerPool {
address: Address,
user: User,
database: String,
client_server_map: ClientServerMap,
}
impl ServerPool {
pub fn new(
address: Address,
user: User,
database: &str,
client_server_map: ClientServerMap,
) -> ServerPool {
ServerPool {
address: address,
user: user,
database: database.to_string(),
client_server_map: client_server_map,
}
}
}
#[async_trait]
impl ManageConnection for ServerPool {
type Connection = Server;
type Error = Error;
/// Attempts to create a new connection.
async fn connect(&self) -> Result<Self::Connection, Self::Error> {
println!(">> Creating a new connection for the pool");
Server::startup(
&self.address.host,
&self.address.port,
&self.user.name,
&self.user.password,
&self.database,
self.client_server_map.clone(),
)
.await
}
/// Determines if the connection is still connected to the database.
async fn is_valid(&self, _conn: &mut PooledConnection<'_, Self>) -> Result<(), Self::Error> {
Ok(())
}
/// Synchronously determine if the connection is no longer usable, if possible.
fn has_broken(&self, conn: &mut Self::Connection) -> bool {
conn.is_bad()
}
}

View File

@@ -1,414 +0,0 @@
#![allow(dead_code)]
#![allow(unused_variables)]
///! Implementation of the PostgreSQL server (database) protocol.
///! Here we are pretending to the a Postgres client.
use bytes::{Buf, BufMut, BytesMut};
use tokio::io::{AsyncReadExt, BufReader};
use tokio::net::tcp::{OwnedReadHalf, OwnedWriteHalf};
use tokio::net::TcpStream;
use crate::config::Address;
use crate::errors::Error;
use crate::messages::*;
use crate::ClientServerMap;
/// Server state.
pub struct Server {
// Server host, e.g. localhost
host: String,
// Server port: e.g. 5432
port: String,
// Buffered read socket
read: BufReader<OwnedReadHalf>,
// Unbuffered write socket (our client code buffers)
write: OwnedWriteHalf,
// Our server response buffer
buffer: BytesMut,
// Server information the server sent us over on startup
server_info: BytesMut,
// Backend id and secret key used for query cancellation.
backend_id: i32,
secret_key: i32,
// Is the server inside a transaction at the moment.
in_transaction: bool,
// Is there more data for the client to read.
data_available: bool,
// Is the server broken? We'll remote it from the pool if so.
bad: bool,
// Mapping of clients and servers used for query cancellation.
client_server_map: ClientServerMap,
}
impl Server {
/// Pretend to be the Postgres client and connect to the server given host, port and credentials.
/// Perform the authentication and return the server in a ready-for-query mode.
pub async fn startup(
host: &str,
port: &str,
user: &str,
password: &str,
database: &str,
client_server_map: ClientServerMap,
) -> Result<Server, Error> {
let mut stream = match TcpStream::connect(&format!("{}:{}", host, port)).await {
Ok(stream) => stream,
Err(err) => {
println!(">> Could not connect to server: {}", err);
return Err(Error::SocketError);
}
};
// Send the startup packet.
startup(&mut stream, user, database).await?;
let mut server_info = BytesMut::with_capacity(25);
let mut backend_id: i32 = 0;
let mut secret_key: i32 = 0;
loop {
let code = match stream.read_u8().await {
Ok(code) => code as char,
Err(_) => return Err(Error::SocketError),
};
let len = match stream.read_i32().await {
Ok(len) => len,
Err(_) => return Err(Error::SocketError),
};
match code {
'R' => {
// Auth can proceed
let code = match stream.read_i32().await {
Ok(code) => code,
Err(_) => return Err(Error::SocketError),
};
match code {
// MD5
5 => {
let mut salt = vec![0u8; 4];
match stream.read_exact(&mut salt).await {
Ok(_) => (),
Err(_) => return Err(Error::SocketError),
};
md5_password(&mut stream, user, password, &salt[..]).await?;
}
// Authentication handshake complete.
0 => (),
_ => {
println!(">> Unsupported authentication mechanism: {}", code);
return Err(Error::ServerError);
}
}
}
'E' => {
let error_code = match stream.read_u8().await {
Ok(error_code) => error_code,
Err(_) => return Err(Error::SocketError),
};
match error_code {
0 => (), // Terminator
_ => {
let mut error = vec![0u8; len as usize - 4 - 1];
match stream.read_exact(&mut error).await {
Ok(_) => (),
Err(_) => return Err(Error::SocketError),
};
println!(">> Server error: {}", String::from_utf8_lossy(&error));
}
};
return Err(Error::ServerError);
}
'S' => {
// Parameter
let mut param = vec![0u8; len as usize - 4];
match stream.read_exact(&mut param).await {
Ok(_) => (),
Err(_) => return Err(Error::SocketError),
};
server_info.put_u8(b'S');
server_info.put_i32(len);
server_info.put_slice(&param[..]);
}
'K' => {
// Query cancellation data.
backend_id = match stream.read_i32().await {
Ok(id) => id,
Err(err) => return Err(Error::SocketError),
};
secret_key = match stream.read_i32().await {
Ok(id) => id,
Err(err) => return Err(Error::SocketError),
};
}
'Z' => {
let mut idle = vec![0u8; len as usize - 4];
match stream.read_exact(&mut idle).await {
Ok(_) => (),
Err(_) => return Err(Error::SocketError),
};
// Startup finished
let (read, write) = stream.into_split();
return Ok(Server {
host: host.to_string(),
port: port.to_string(),
read: BufReader::new(read),
write: write,
buffer: BytesMut::with_capacity(8196),
server_info: server_info,
backend_id: backend_id,
secret_key: secret_key,
in_transaction: false,
data_available: false,
bad: false,
client_server_map: client_server_map,
});
}
_ => {
println!(">> Unknown code: {}", code);
return Err(Error::ProtocolSyncError);
}
};
}
}
/// Issue a cancellation request to the server.
/// Uses a separate connection that's not part of the connection pool.
pub async fn cancel(
host: &str,
port: &str,
process_id: i32,
secret_key: i32,
) -> Result<(), Error> {
let mut stream = match TcpStream::connect(&format!("{}:{}", host, port)).await {
Ok(stream) => stream,
Err(err) => {
println!(">> Could not connect to server: {}", err);
return Err(Error::SocketError);
}
};
let mut bytes = BytesMut::with_capacity(16);
bytes.put_i32(16);
bytes.put_i32(80877102);
bytes.put_i32(process_id);
bytes.put_i32(secret_key);
Ok(write_all(&mut stream, bytes).await?)
}
/// Send data to the server from the client.
pub async fn send(&mut self, messages: BytesMut) -> Result<(), Error> {
match write_all_half(&mut self.write, messages).await {
Ok(_) => Ok(()),
Err(err) => {
println!(">> Terminating server because of: {:?}", err);
self.bad = true;
Err(err)
}
}
}
/// Receive data from the server in response to a client request sent previously.
/// This method must be called multiple times while `self.is_data_available()` is true
/// in order to receive all data the server has to offer.
pub async fn recv(&mut self) -> Result<BytesMut, Error> {
loop {
let mut message = match read_message(&mut self.read).await {
Ok(message) => message,
Err(err) => {
println!(">> Terminating server because of: {:?}", err);
self.bad = true;
return Err(err);
}
};
// Buffer the message we'll forward to the client in a bit.
self.buffer.put(&message[..]);
let code = message.get_u8() as char;
let _len = message.get_i32();
match code {
'Z' => {
// Ready for query, time to forward buffer to client.
let transaction_state = message.get_u8() as char;
match transaction_state {
'T' => {
self.in_transaction = true;
}
'I' => {
self.in_transaction = false;
}
// Error client didn't clean up!
// We shuold drop this server
'E' => {
self.in_transaction = true;
}
_ => {
self.bad = true;
return Err(Error::ProtocolSyncError);
}
};
self.data_available = false;
break;
}
'D' => {
self.data_available = true;
// Don't flush yet, the more we buffer, the faster this goes.
// Up to a limit of course.
if self.buffer.len() >= 8196 {
break;
}
}
// CopyInResponse: copy is starting from client to server
'G' => break,
// CopyOutResponse: copy is starting from the server to the client
'H' => {
self.data_available = true;
break;
}
// CopyData
'd' => break,
// CopyDone
'c' => {
self.data_available = false;
// Buffer until ReadyForQuery shows up
}
_ => {
// Keep buffering,
}
};
}
let bytes = self.buffer.clone();
self.buffer.clear();
Ok(bytes)
}
/// If the server is still inside a transaction.
/// If the client disconnects while the server is in a transaction, we will clean it up.
pub fn in_transaction(&self) -> bool {
self.in_transaction
}
/// We don't buffer all of server responses, e.g. COPY OUT produces too much data.
/// The client is responsible to call `self.recv()` while this method returns true.
pub fn is_data_available(&self) -> bool {
self.data_available
}
/// Server & client are out of sync, we must discard this connection.
/// This happens with clients that misbehave.
pub fn is_bad(&self) -> bool {
self.bad
}
/// Get server startup information to forward it to the client.
/// Not used at the moment.
pub fn server_info(&self) -> BytesMut {
self.server_info.clone()
}
/// Indicate that this server connection cannot be re-used and must be discarded.
pub fn mark_bad(&mut self) {
println!(">> Server marked bad");
self.bad = true;
}
/// Claim this server as mine for the purposes of query cancellation.
pub fn claim(&mut self, process_id: i32, secret_key: i32) {
let mut guard = self.client_server_map.lock().unwrap();
guard.insert(
(process_id, secret_key),
(
self.backend_id,
self.secret_key,
self.host.clone(),
self.port.clone(),
),
);
}
/// Execute an arbitrary query against the server.
/// It will use the Simple query protocol.
/// Result will not be returned, so this is useful for things like `SET` or `ROLLBACK`.
pub async fn query(&mut self, query: &str) -> Result<(), Error> {
let mut query = BytesMut::from(&query.as_bytes()[..]);
query.put_u8(0);
let len = query.len() as i32 + 4;
let mut msg = BytesMut::with_capacity(len as usize + 1);
msg.put_u8(b'Q');
msg.put_i32(len);
msg.put_slice(&query[..]);
self.send(msg).await?;
loop {
let _ = self.recv().await?;
if !self.data_available {
break;
}
}
Ok(())
}
/// A shorthand for `SET application_name = $1`.
pub async fn set_name(&mut self, name: &str) -> Result<(), Error> {
Ok(self
.query(&format!("SET application_name = '{}'", name))
.await?)
}
pub fn address(&self) -> Address {
Address {
host: self.host.to_string(),
port: self.port.to_string(),
}
}
}

View File

@@ -1,144 +0,0 @@
// https://github.com/postgres/postgres/blob/27b77ecf9f4d5be211900eda54d8155ada50d696/src/include/catalog/partition.h#L20
const PARTITION_HASH_SEED: u64 = 0x7A5B22367996DCFD;
pub struct Sharder {
shards: usize,
}
impl Sharder {
pub fn new(shards: usize) -> Sharder {
Sharder { shards: shards }
}
/// Hash function used by Postgres to determine which partition
/// to put the row in when using HASH(column) partitioning.
/// Source: https://github.com/postgres/postgres/blob/27b77ecf9f4d5be211900eda54d8155ada50d696/src/common/hashfn.c#L631
/// Supports only 1 bigint at the moment, but we can add more later.
pub fn pg_bigint_hash(&self, key: i64) -> usize {
let mut lohalf = key as u32;
let hihalf = (key >> 32) as u32;
lohalf ^= if key >= 0 { hihalf } else { !hihalf };
Self::combine(0, Self::pg_u32_hash(lohalf)) as usize % self.shards as usize
}
#[inline]
fn rot(x: u32, k: u32) -> u32 {
(x << k) | (x >> (32 - k))
}
#[inline]
fn mix(mut a: u32, mut b: u32, mut c: u32) -> (u32, u32, u32) {
a = a.wrapping_sub(c);
a ^= Self::rot(c, 4);
c = c.wrapping_add(b);
b = b.wrapping_sub(a);
b ^= Self::rot(a, 6);
a = a.wrapping_add(c);
c = c.wrapping_sub(b);
c ^= Self::rot(b, 8);
b = b.wrapping_add(a);
a = a.wrapping_sub(c);
a ^= Self::rot(c, 16);
c = c.wrapping_add(b);
b = b.wrapping_sub(a);
b ^= Self::rot(a, 19);
a = a.wrapping_add(c);
c = c.wrapping_sub(b);
c ^= Self::rot(b, 4);
b = b.wrapping_add(a);
(a, b, c)
}
#[inline]
fn _final(mut a: u32, mut b: u32, mut c: u32) -> (u32, u32, u32) {
c ^= b;
c = c.wrapping_sub(Self::rot(b, 14));
a ^= c;
a = a.wrapping_sub(Self::rot(c, 11));
b ^= a;
b = b.wrapping_sub(Self::rot(a, 25));
c ^= b;
c = c.wrapping_sub(Self::rot(b, 16));
a ^= c;
a = a.wrapping_sub(Self::rot(c, 4));
b ^= a;
b = b.wrapping_sub(Self::rot(a, 14));
c ^= b;
c = c.wrapping_sub(Self::rot(b, 24));
(a, b, c)
}
#[inline]
fn combine(mut a: u64, b: u64) -> u64 {
a ^= b
.wrapping_add(0x49a0f4dd15e5a8e3 as u64)
.wrapping_add(a << 54)
.wrapping_add(a >> 7);
a
}
fn pg_u32_hash(k: u32) -> u64 {
let mut a: u32 = 0x9e3779b9 as u32 + std::mem::size_of::<u32>() as u32 + 3923095 as u32;
let mut b = a;
let c = a;
a = a.wrapping_add((PARTITION_HASH_SEED >> 32) as u32);
b = b.wrapping_add(PARTITION_HASH_SEED as u32);
let (mut a, b, c) = Self::mix(a, b, c);
a = a.wrapping_add(k);
let (_a, b, c) = Self::_final(a, b, c);
((b as u64) << 32) | (c as u64)
}
}
#[cfg(test)]
mod test {
use super::*;
// See tests/sharding/partition_hash_test_setup.sql
// The output of those SELECT statements will match this test,
// confirming that we implemented Postgres BIGINT hashing correctly.
#[test]
fn test_pg_bigint_hash() {
let sharder = Sharder::new(5);
let shard_0 = vec![1, 4, 5, 14, 19, 39, 40, 46, 47, 53];
for v in shard_0 {
assert_eq!(sharder.pg_bigint_hash(v), 0);
}
let shard_1 = vec![2, 3, 11, 17, 21, 23, 30, 49, 51, 54];
for v in shard_1 {
assert_eq!(sharder.pg_bigint_hash(v), 1);
}
let shard_2 = vec![6, 7, 15, 16, 18, 20, 25, 28, 34, 35];
for v in shard_2 {
assert_eq!(sharder.pg_bigint_hash(v), 2);
}
let shard_3 = vec![8, 12, 13, 22, 29, 31, 33, 36, 41, 43];
for v in shard_3 {
assert_eq!(sharder.pg_bigint_hash(v), 3);
}
let shard_4 = vec![9, 10, 24, 26, 27, 32, 37, 38, 42, 45];
for v in shard_4 {
assert_eq!(sharder.pg_bigint_hash(v), 4);
}
}
}

View File

@@ -1 +0,0 @@
venv/

View File

@@ -1 +0,0 @@
psycopg2==2.9.3

View File

@@ -1,11 +0,0 @@
import psycopg2
conn = psycopg2.connect("postgres://random:password@127.0.0.1:6432/db")
cur = conn.cursor()
cur.execute("SELECT 1");
res = cur.fetchall()
print(res)
# conn.commit()

View File

@@ -1,11 +0,0 @@
require 'pg'
conn = PG.connect(host: '127.0.0.1', port: 5433, dbname: 'test')
conn.exec( "SELECT * FROM pg_stat_activity" ) do |result|
puts " PID | User | Query"
result.each do |row|
puts " %7d | %-16s | %s " %
row.values_at('pid', 'usename', 'query')
end
end

View File

@@ -1,35 +0,0 @@
# Sharding tests
This helps us test the sharding algorithm we implemented.
## Setup
We setup 3 Postgres DBs, `shard0`, `shard1`, and `shard2`. In each database, we create a partitioned table called `data`. The table is partitioned by hash, and each database will only have _one_ partition, `shard0` will satisfy `modulus 3, remainder 0`, `shard1` will satisfy `modulus 3, remainder 1`, etc.
To set this up, you can just run:
```bash
psql -f query_routing_setup.sql
```
## Run the tests
Start up PgCat by running `cargo run --release` in the root of the repo. In a different tab, run this:
```bash
psql -h 127.0.0.1 -p 6432 -f query_routing_test_insert.sql
psql -h 127.0.0.1 -p 6432 -f query_routing_test_select.sql
```
Note that no errors should take place. If our sharding logic was incorrect, we would get some errors
about unsatisfiable partition bounds. We don't because the pooler picked the correct databases
given the sharding keys.
Finally, you can validate the result again by running
```bash
psql -f query_routing_test_validate.sql
```
## That's it!

View File

@@ -1,26 +0,0 @@
DROP TABLE IF EXISTS shards CASCADE;
CREATE TABLE shards (
id BIGINT,
value VARCHAR
) PARTITION BY HASH (id);
-- DROP TABLE IF EXISTS shard_0;
CREATE TABLE shard_0 PARTITION OF shards FOR VALUES WITH (MODULUS 5, REMAINDER 0);
-- DROP TABLE IF EXISTS shard_1;
CREATE TABLE shard_1 PARTITION OF shards FOR VALUES WITH (MODULUS 5, REMAINDER 1);
-- DROP TABLE IF EXISTS shard_2;
CREATE TABLE shard_2 PARTITION OF shards FOR VALUES WITH (MODULUS 5, REMAINDER 2);
-- DROP TABLE IF EXISTS shard_3;
CREATE TABLE shard_3 PARTITION OF shards FOR VALUES WITH (MODULUS 5, REMAINDER 3);
-- DROP TABLE IF EXISTS shard_4;
CREATE TABLE shard_4 PARTITION OF shards FOR VALUES WITH (MODULUS 5, REMAINDER 4);
INSERT INTO shards SELECT generate_series(1, 500), 'value';
SELECT * FROM shard_0 ORDER BY id LIMIT 10;
SELECT * FROM shard_1 ORDER BY id LIMIT 10;
SELECT * FROM shard_2 ORDER BY id LIMIT 10;
SELECT * FROM shard_3 ORDER BY id LIMIT 10;
SELECT * FROM shard_4 ORDER BY id LIMIT 10;

View File

@@ -1,12 +0,0 @@
#/bin/bash
# Setup all the shards.
sudo service postgresql restart
psql -f query_routing_setup.sql
psql -h 127.0.0.1 -p 6432 -f query_routing_test_insert.sql
psql -h 127.0.0.1 -p 6432 -f query_routing_test_select.sql
psql -f query_routing_test_validate.sql

View File

@@ -1,61 +0,0 @@
DROP DATABASE IF EXISTS shard0;
DROP DATABASE IF EXISTS shard1;
DROP DATABASE IF EXISTS shard2;
CREATE DATABASE shard0;
CREATE DATABASE shard1;
CREATE DATABASE shard2;
\c shard0
DROP TABLE IF EXISTS data CASCADE;
CREATE TABLE data (
id BIGINT,
value VARCHAR
) PARTITION BY HASH (id);
CREATE TABLE data_shard_0 PARTITION OF data FOR VALUES WITH (MODULUS 3, REMAINDER 0);
\c shard1
DROP TABLE IF EXISTS data CASCADE;
CREATE TABLE data (
id BIGINT,
value VARCHAR
) PARTITION BY HASH (id);
CREATE TABLE data_shard_1 PARTITION OF data FOR VALUES WITH (MODULUS 3, REMAINDER 1);
\c shard2
DROP TABLE IF EXISTS data CASCADE;
CREATE TABLE data (
id BIGINT,
value VARCHAR
) PARTITION BY HASH (id);
CREATE TABLE data_shard_2 PARTITION OF data FOR VALUES WITH (MODULUS 3, REMAINDER 2);
DROP ROLE IF EXISTS sharding_user;
CREATE ROLE sharding_user ENCRYPTED PASSWORD 'sharding_user' LOGIN;
GRANT CONNECT ON DATABASE shard0 TO sharding_user;
GRANT CONNECT ON DATABASE shard1 TO sharding_user;
GRANT CONNECT ON DATABASE shard2 TO sharding_user;
\c shard0
GRANT ALL ON SCHEMA public TO sharding_user;
GRANT ALL ON TABLE data TO sharding_user;
\c shard1
GRANT ALL ON SCHEMA public TO sharding_user;
GRANT ALL ON TABLE data TO sharding_user;
\c shard2
GRANT ALL ON SCHEMA public TO sharding_user;
GRANT ALL ON TABLE data TO sharding_user;

View File

@@ -1,47 +0,0 @@
SET SHARDING KEY TO '1';
INSERT INTO data (id, value) VALUES (1, 'value_1');
SET SHARDING KEY TO '2';
INSERT INTO data (id, value) VALUES (2, 'value_1');
SET SHARDING KEY TO '3';
INSERT INTO data (id, value) VALUES (3, 'value_1');
SET SHARDING KEY TO '4';
INSERT INTO data (id, value) VALUES (4, 'value_1');
SET SHARDING KEY TO '5';
INSERT INTO data (id, value) VALUES (5, 'value_1');
SET SHARDING KEY TO '6';
INSERT INTO data (id, value) VALUES (6, 'value_1');
SET SHARDING KEY TO '7';
INSERT INTO data (id, value) VALUES (7, 'value_1');
SET SHARDING KEY TO '8';
INSERT INTO data (id, value) VALUES (8, 'value_1');
SET SHARDING KEY TO '9';
INSERT INTO data (id, value) VALUES (9, 'value_1');
SET SHARDING KEY TO '10';
INSERT INTO data (id, value) VALUES (10, 'value_1');
SET SHARDING KEY TO '11';
INSERT INTO data (id, value) VALUES (11, 'value_1');
SET SHARDING KEY TO '12';
INSERT INTO data (id, value) VALUES (12, 'value_1');
SET SHARDING KEY TO '13';
INSERT INTO data (id, value) VALUES (13, 'value_1');
SET SHARDING KEY TO '14';
INSERT INTO data (id, value) VALUES (14, 'value_1');
SET SHARDING KEY TO '15';
INSERT INTO data (id, value) VALUES (15, 'value_1');
SET SHARDING KEY TO '16';
INSERT INTO data (id, value) VALUES (16, 'value_1');

View File

@@ -1,47 +0,0 @@
SET SHARDING KEY TO '1';
SELECT * FROM data WHERE id = 1;
SET SHARDING KEY TO '2';
SELECT * FROM data WHERE id = 2;
SET SHARDING KEY TO '3';
SELECT * FROM data WHERE id = 3;
SET SHARDING KEY TO '4';
SELECT * FROM data WHERE id = 4;
SET SHARDING KEY TO '5';
SELECT * FROM data WHERE id = 5;
SET SHARDING KEY TO '6';
SELECT * FROM data WHERE id = 6;
SET SHARDING KEY TO '7';
SELECT * FROM data WHERE id = 7;
SET SHARDING KEY TO '8';
SELECT * FROM data WHERE id = 8;
SET SHARDING KEY TO '9';
SELECT * FROM data WHERE id = 9;
SET SHARDING KEY TO '10';
SELECT * FROM data WHERE id = 10;
SET SHARDING KEY TO '11';
SELECT * FROM data WHERE id = 11;
SET SHARDING KEY TO '12';
SELECT * FROM data WHERE id = 12;
SET SHARDING KEY TO '13';
SELECT * FROM data WHERE id = 13;
SET SHARDING KEY TO '14';
SELECT * FROM data WHERE id = 14;
SET SHARDING KEY TO '15';
SELECT * FROM data WHERE id = 15;
SET SHARDING KEY TO '16';
SELECT * FROM data WHERE id = 16;

View File

@@ -1,11 +0,0 @@
\c shard0
SELECT * FROM data;
\c shard1
SELECT * FROM data;
\c shard2
SELECT * FROM data;