update docs

.github/site/src/SUMMARY.md

@@ -4,9 +4,6 @@
     - [System requirements](./system-requirements.md)
     - [Installation](./installation.md)
 - [Setup](./setup.md)
-    - [Telegram](./telegram.md)
-    - [Slack](./slack.md)
-    - [Discord](./discord.md)
 - [Services](./services.md)
     - [General](./general.md)
     - [Healthcheck](./healthcheck.md)

.github/site/src/general.md

@@ -20,7 +20,7 @@ messenger.url = "<messenger api url for sending messages>"
 # log_level = "info"
 service_account_credentials_path = "/path/to/service_account.json"
 messenger.url = "<messenger api url for sending messages>"
-# graceful_timeout_secs = 10
+# graceful_timeout_secs = 5
 ```
 </details>
 

.github/site/src/healthcheck.md

@@ -58,12 +58,12 @@ will create a sheet for every liveness probe.
 
 Liveness probes follow the same rules as for [k8s](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/). Namely:
 * you should choose among HTTP GET (any status `>=200` and `<400`), gRPC, TCP (Goral can open socket) and command (successful exit)
-* you can specify initial delay, period of probe and timeout on probe
+* you can specify an initial delay, a period of a probe and a timeout on a probe
 * misconfiguration of a liveness probe is considered a failure
 * for gRPC Health service should be configured on the app side (see also [gRPC health checking protocol](https://github.com/grpc/grpc/blob/v1.59.1/doc/health-checking.md)). Only `http` scheme is supported at the moment. If you need a tls check, you can use a command probe with a [grpc health probe](https://github.com/grpc-ecosystem/grpc-health-probe) and specify proper certificates.
 
-Goral saves probe time, status (true for alive) and text output (for HTTP GET - response text, for command - stdout output, for all probes - error text). Each probe is saved at a separate sheet with its own uptime chart.
+Goral saves probe time, a status (true for alive) and a text output (for HTTP GET - a response text, for a command - a stdout output, for all probes - an error text). Each probe is saved at a separate sheet.
 In case an output is larger than 1024 bytes, it is truncated and you get permanent warnings in logs of Goral. So configure the output size of your healthcheck reasonably (healthcheck responses shouldn't be heavy).
-For command healthchecks it is recommended to assign a name to liveness probe or wrap your command in some script so that in case of small changes in command arguments preserve the same sheet for data (otherwise Goral will create a new sheet since the title has changed).
+For command healthchecks it is recommended to assign a name to a liveness probe or wrap your command in some script so that in case of small changes in command arguments preserve the same sheet for data (otherwise Goral will create a new sheet since the title has changed).
 
-If a messenger is configured, then any healthcheck change (healthy -> unhealthy and vice versa) is sent via the messenger. In case of many endpoints with short liveness periods there is a risk to hit a messenger rate limit.
+If a messenger is configured, then any healthcheck change (healthy -> unhealthy and vice versa) is sent via the messenger. In case of many endpoints with short liveness periods there is a risk to hit a rate limit of a messenger.

.github/site/src/install.sh

@@ -21,7 +21,7 @@ main() {
 
     get_architecture || return 1
     local _arch="$RETVAL"
-    local _version=${1:-'0.1.3rc23'}
+    local _version=${1:-'0.1.3rc24'}
     assert_nz "$_arch" "arch"
 
     local _file="goral-${_version}-${_arch}"

.github/site/src/installation.md

@@ -1,30 +1,35 @@
 # Installation
 
-You can install Goral
-1) by downloading a prebuilt binary from https://github.com/maksimryndin/goral/releases
+You can install Goral with 
 
-For example, for Linux
 ```sh
-wget https://github.com/maksimryndin/goral/releases/download/0.1.2/goral-0.1.2-x86_64-unknown-linux-gnu.tar.gz
-tar -xzf goral-0.1.2-x86_64-unknown-linux-gnu.tar.gz
-cd goral-0.1.2-x86_64-unknown-linux-gnu/
-shasum -a 256 -c sha256_checksum.txt 
+curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/maksimryndin/goral/0.1.3rc22/.github/site/install.sh | sh
 sudo mv goral /usr/local/bin/goral
 ```
 
-or just use an installer which will download the latest stable release, check sha256 and unpack it at the current directory
+<details>
+  <summary>or by downloading a prebuilt binary from https://github.com/maksimryndin/goral/releases manually
+</summary>
 
 ```sh
-curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/maksimryndin/goral/0.1.3rc22/.github/site/install.sh | sh
+wget https://github.com/maksimryndin/goral/releases/download/0.1.2/goral-0.1.2-x86_64-unknown-linux-gnu.tar.gz
+tar -xzf goral-0.1.2-x86_64-unknown-linux-gnu.tar.gz
+cd goral-0.1.2-x86_64-unknown-linux-gnu/
+shasum -a 256 -c sha256_checksum.txt 
 sudo mv goral /usr/local/bin/goral
 ```
+</details>
+
+<details>
+  <summary>from source</summary>
 
-2) from source (you need [Rust](https://www.rust-lang.org/tools/install)) with a command
 ```sh
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
 git clone --depth 1 --branch 0.1.2 https://github.com/maksimryndin/goral
 cd goral
 RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target <target triple>
 ```
+</details>
 
 To run a binary
 ```sh

.github/site/src/kv-log.md

@@ -49,15 +49,15 @@ Such a configuration runs a server process in the Goral daemon listening at the
 Appending to the log is *not idempotent*, i.e. if you retry the same request two times, then the same set of rows will be added twice. If it is absolutely important to avoid any duplication (which may happen in case of some unexpected failure or retrying logic), then it is recommended to some unique idempotence key to `data` to be able to filter duplicates in the spreadsheet and remove them manually.
 
 Goral KV service responds with an array of urls of sheets for each datarow respectively.
-Goral accepts every batch and creates corresponding sheets in the configured spreadsheet for "Orders" and "Marketing Campaigns". And the end of the month you have all the billing data neatly collected.
+Goral accepts every batch and creates corresponding sheets in the configured spreadsheet for "Orders" and "Marketing Campaigns". At the end of the month you have all the billing data neatly collected.
 For even more interactive setup you can share a spreadsheet access with your client (for him/her see all the process online) and configure a messenger for alerts and notifications (see the section [Rules](./rules.md)) by adding your client to the chat.
 Unlike other Goral services, this KV api is synchronous - if Goral responds successfully then sheets are created already and data is saved.
 
 For every append operation Goral uses 2 Google api method calls, so under the quota limit of 300 requests per minute, we have 5 requests per second or 2 append operations (not considering other Goral services which use the same quota). That's why it is strongly recommended to use a batched approach (say send in batches every 10 seconds or so) otherwise you can exhaust [Google api quota](https://developers.google.com/sheets/api/limits) quickly (especially when other Goral services run). [Exponential backoff algorithm](https://developers.google.com/sheets/api/limits#exponential) is *not* applicable to KV service induced requests (in contrast to other Goral services). So retries are on the client app side and you may expect http response status code `429: Too many requests` in case if you generate an excessive load. And it can impact other Goral services.
-In any case Goral put KV requests in the messages queue with a capacity 1, so any concurrent request will wait until the previous one is handled.
+In any case Goral puts KV requests in the messages queue with a capacity 1, so any concurrent request will wait until the previous one is handled.
 
 If there is an error while appending data, it is sent only via a default messenger of General service. Configured messenger is only used for notifications according to configured rules.
 
 *Note*: for KV service the autotruncation mechanism is turned off by default (`autotruncate_at_usage_percent = 100`). It means that you should either set that value to some percent below 100 or clean up old data manually.
 
-Another notable use case is to log console errors from the frontend - catch JS exceptions, accumulate them in some batch at your backend and send the batch to Goral KV service.
+Another notable use case is to log console errors from the frontend - catch JS exceptions, accumulate them in some batch at your backend and send the batch to the Goral KV service.

.github/site/src/logs.md

@@ -27,9 +27,9 @@ spreadsheet_id = "<spreadsheet_id>"
 
 will create a single sheet with columns `datetime`, `level`, `log_line`.
 A log line is truncated to 50 000 chars as it is a Google Sheets limit for a cell.
-Goral tries to extract log level and datetime from a log line. If it fails to extract a log level then `N/A` is displayed. If it fails to extract datetime, then the current system time is used.
+Goral tries to extract a log level and datetime from a log line. If it fails to extract a log level then `N/A` is displayed. If it fails to extract datetime, then the current system time is used.
 
-For logs collecting Goral reads its stdin. Basically it is a portable way to collect stdout of another process without a privileged access.
+For logs collection Goral reads its stdin. Basically it is a portable way to collect stdout of another process without a privileged access.
 There is a caveat - if we make a simple pipe like `instrumented_app | goral` then in case of a termination of the `instrumented_app` Goral will not see any input and will stop reading.
 [There is a way with named pipes](https://www.baeldung.com/linux/stdout-to-multiple-commands#3-solve-the-problem-usingtee-and-named-pipes) (for Windows there should also be a way as it also supports named pipes). 
 * You create a named pipe, say `instrumented_app_logs_pipe` with the command `mkfifo instrumented_app_logs_pipe` (it creates a pipe file in the current directory - you can choose an appropriate place)
@@ -41,4 +41,4 @@ With this named pipes approach the `instrumented_app` restarts doesn't stop Gora
 Just be sure to autorecreate a fake writer in case of a host system restarts.
 See also [Deployment](./recommended-deployment.md) section for an example.
 
-As there may be a huge amount of logs, it is recommended to filter the volume by specifiying an array of substrings (_case sensitive_) in `filter_if_contains` (e.g. `["info", "warn", "error"]`) and `drop_if_contains`, and/or have a separate spreadsheet for log collection as a huge amount of them may hurdle the use of Google sheets due to the constant updates.
+As there may be a huge amount of logs, it is recommended to filter the volume by specifiying an array of substrings (_case sensitive_) in `filter_if_contains` (e.g. `["info", "warn", "error"]`) and `drop_if_contains`, and/or have a separate spreadsheet for logs collection as a huge amount of them may hurdle the use of Google sheets due to the constant UI updates.

.github/site/src/setup.md

@@ -1,20 +1,78 @@
 # Setup
 
 To use Goral you need to have a Google account and obtain a service account:
-1) Create a project https://console.cloud.google.com/projectcreate (we suggest creating a separate project for Goral for security reasons)
-2) Enable Sheets from the products page https://console.cloud.google.com/workspace-api/products
-3) Create a service account https://console.cloud.google.com/workspace-api/credentials with Editor role
-4) After creating the service acoount create a private key (type JSON) for it (the private key should be downloaded by your browser)
-5) Create a spreadsheet (where the scraped data will be stored) and add the service account email as an Editor to the spreadsheet
+1) [Create a project](https://console.cloud.google.com/projectcreate) (we suggest creating a separate project for Goral for security reasons)
+2) Enable Sheets from the [products page](https://console.cloud.google.com/workspace-api/products)
+3) Create a [service account](https://console.cloud.google.com/workspace-api/credentials) with `Editor` role
+4) Create a private key (type JSON) for it (the private key should be downloaded by your browser)
+5) Create a spreadsheet (where the scraped data will be stored) and add the service account email as an `Editor` to the spreadsheet
 6) Extract spreadsheet id from the spreadsheet url
 
 Steps 5-6 can be repeated for each [service](./services.md) (it is recommended to have separate spreadsheets for usability and separation of concerns).
 
-Note: you can also install Google Sheets app for your phone to have an access to the data.
+*Note*: you can also install Google Sheets app for your phone to have an access to the data.
 
 Notifications are sent to messengers with three levels:
 * 🟢 (INFO)
 * 🟡 (WARN)
 * 🔴 (ERROR)
 
-and are prefixed with id (the argument for `--id` flag).
+and are prefixed with id (the argument for `--id` flag). Messengers are configured separately for every service so that you can use e.g. Discord for General service and Telegram for Healthcheck.
+
+### Discord
+
+<details>
+  <summary>Webhook creation</summary>
+
+1) Create a text channel
+2) In the settings of the channel (cogwheel) go to the Integrations -> Webhooks
+3) Either use the default one or create a new webhook
+</details>
+
+Example configuration:
+
+```toml
+messenger.url = "https://discord.com/api/webhooks/<webhook_id>/<webhook_token>"
+```
+
+[Rate limits](https://discord.com/developers/docs/topics/rate-limits).
+
+### Slack
+
+<details>
+  <summary>App creation</summary>
+
+Follow the [quickstart guide](https://api.slack.com/start/quickstart) and the [posting guide](https://api.slack.com/tutorials/tracks/posting-messages-with-curl).
+</details>
+
+Example configuration:
+
+```toml
+messenger.specific.token = "xoxb-slack-token"
+messenger.specific.channel = "CHRISHWFH2"
+messenger.url = "https://slack.com/api/chat.postMessage"
+```
+
+[Rate limit](https://api.slack.com/methods/chat.postMessage#rate_limiting)
+
+### Telegram
+
+<details>
+  <summary>Bot creation</summary>
+
+1) [Create a bot](https://core.telegram.org/bots/features#creating-a-new-bot)
+2) Create a private group for notifications to be sent to
+3) Add your bot to the group
+4) Obtain a `chat_id` following the [accepted answer](https://stackoverflow.com/questions/33858927/how-to-obtain-the-chat-id-of-a-private-telegram-channel)
+</details>
+
+Example configuration:
+
+```toml
+messenger.specific.chat_id = "-100129008371"
+messenger.url = "https://api.telegram.org/bot<bot token>/sendMessage"
+```
+
+[Rate limit](https://core.telegram.org/bots/faq#my-bot-is-hitting-limits-how-do-i-avoid-this)
+
+*Note*: for Telegram all info-level messages are sent without notification so the phone doesn't vibrate or make any sound.

.github/site/src/system-requirements.md

@@ -2,4 +2,4 @@
 
 * Memory: RSS 30M, 900M for virtual memory. An actual requirement may be different - as it depends on the amount of data, scrape and push intervals (see below for each [service](./services.md))
 * Binary size is around 15 Mb
-* Platforms: Linux (x86-64 or aarch64). Other platforms (namely MacOS and Windows) will probably work also.
+* Platforms: Linux (`x86-64` or `aarch64`). Other platforms - namely MacOS (M1 `arm`) and Windows - will probably work also.

.github/site/src/system.md

@@ -27,14 +27,14 @@ spreadsheet_id = "<spreadsheet_id>"
 ```
 </details>
 
-With this configuration System service will create following sheets:
-* `basic`: with general information about the system (boot time, memory, cpus, swap, number of processes)
-* `network`: for every network interface number of bytes read/written _since the previous measurement_, total read/written
+With this configuration System service will create the following sheets:
+* `basic`: with general information about the system (boot time, memory, cpus, swap, a number of processes)
+* `network`: for every network interface a number of bytes read/written _since the previous measurement_, total read/written
 * `top_disk_read` - process which has read the most from disk _during the last second_
 * `top_disk_write` - process which has written the most to disk _during the last second_
 * `top_cpu` - process with the most cpu time during the last second
 * `top_memory` - process with the most memory usage _during the last second_
-* `top_open_files` (for Linux only) - among the processes with the same user as goral (!) - process with the most opened files
+* `top_open_files` (for Linux only) - among the processes with the same user as Goral (!) - process with the most opened files
 * for every process with name containing one of the substrings in `names` - a sheet with process info. Note: the first match (_case sensitive_) is used so plan accordingly a unique name for your binary.
 * for every mount in `mounts` - disk usage and free space.
 

.github/workflows/release.yml

@@ -2,6 +2,8 @@ name: Release
 
 permissions:
   contents: write
+  pages: write
+  id-token: write
 
 on:
   push:
@@ -70,9 +72,9 @@ jobs:
             exit 1 
           fi
 
-          grep -q ${{ env.VERSION }} README.md
+          grep -q ${{ env.VERSION }} .github/site/src/installation.md
           if [ $? -ne 0 ]; then 
-            echo "README.md should contain a new release version in Installation section" 
+            echo "installation.md should contain a new release version" 
             exit 1 
           fi