v0.0.9

This release contains an adapter that redirects logs from the internal logger of the plugin system, hclog, to the global logger of GatewayD, zerolog. This release marks the end of the milestone 0.0.x, which was focused on laying the foundation for further development. I tried to achieve correctness and integrity in this milestone, however, there are some stability issues that I'll deal with in milestone 0.1.x. Going forward, documentation becomes an integral part of the project and will be written progressively.

Looking back at the previous releases, a lot has changed. The project was bootstrapped with a few lines of code as a proof of concept to mimic a L4 gateway. Now the GatewayD has a plug-based architecture with many components to help facilitate the inner-workings of the database gateway:

• Server: for accepting connections from the database clients, like psycopg2 in Python.
• Client: for connecting to databases using a raw TCP connection.
• Generic pool: used in a few places throughout the code to manage connections and plugins. The generic pool is a wrapper around the sync.Map from Go stdlib.
• Proxy: an object that connect servers and clients and passes traffic through. There are two types of proxies:
  • Fixed: a pool of clients will be created and connected to the database. They will be waiting ready to be proxied. The pool won't grow, so extra clients will be rejected. Queuing will be added in the future.
  • Elastic: an empty pool is created and each new connection from a database client creates a new connection to the database. The connections can be reused if chosen.
• Plugin system: can tap into the hooks and try to influence the behavior of the gateway by observe and manipulating data as it passes through between client and server. Plugins are managed by a plugin registry. The plugins can be written in any language that one can produce protobuf stubs and can interface through gRPC. The protocol file is available in plugin/v1/plugin.proto and the generated stubs for Go and the plugin interface are available in the same directory. There is a test plugin that shows how to work with hooks. Upon loading of each plugin, its dynamic configuration is read by calling its GetPluginConfig function. This effectively registers the list of hooks that plugin exposes with the hook system. All plugin functions (RPC) are unary for now.
• Hooks: callback function that plugins can register to. The server will run the hooks based on events, like onConfigLoaded and it causes the configuration options to be sent to plugins registered for that specific event. Each hook can trigger many plugins and they will be called based on their priority. The returned result of the hook from the plugin will be used to update the configuration dynamically (in-memory). Other hooks might have different side-effects. Some hooks are only notification hooks, so that the plugin can tell when an event happen, plus some pieces of information. The list of hooks are available here. The returned result of each hook from each registered plugin will be passed to the next plugin in the list.
  There is a verification function to check the signature of the parameters passed to the hook against the signature of the returned results and acts based on the policy chosen in the global config. There are two modes: strict and non-strict.
  • In strict mode, the parameter and return value signatures should match 1:1. In strict mode, either of the following three actions can be taken:
    1. ignore: ignore the result of the non-matching hook and continue with the next call to another plugin.
    2. abort: abort execution of the hook and return the last returned result from the previous hook.
    3. remove: ignores the result of the current registered hook and removes it from the registry, so the next call won't hit it again.
  • In non-strict, aka. permissive, mode, the parameters and the returned result should not match and the results of the hooks will be merged. This verification policy is called passdown.
• Logging: zerolog is used throughout the code to log events as they happen with various levels of importance. Loggers of other parts of the system has adapters to send logs to zerolog, as mentioned in What's changed? below.
• CLI: a run command with two options for reading: a) the global config file and b) the plugins config file.
• Config parser: there are two config file parsers that accept configuration in YAML format:
  • Global config (gatewayd.yaml): for setting up logger, clients, pool, proxy, server and plugins objects.
  • Plugin config (gatewayd_plugins.yaml): for adding the list of plugins to be loaded. Plugins priority is determined by the appearance of the plugin config in the file.
  • Dynamic config: there is a dynamic configuration hook, onConfigLoaded, to let plugins changes the global configuration before the server boots and after the global config is read, as demonstrated in the test plugin.
• Buf for managing protobuf files and generated stubs for the plugin system.
• Unit- and integration tests.
• CI workflows for a) testing unit-tests and b) testing integration with an example plugin.

What's Changed

• Add logging adapter for the plugin system by @mostafa in #45

Full Changelog: v0.0.8...v0.0.9