Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance and annotations fixes #2

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 33 additions & 173 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,202 +146,62 @@ local element_count = my_heap:count()

The `rmean` module provides functionality for efficient moving average calculations with specified window sizes.

### Usage
The best use of rmean would be when you want to have Average Calls per second during last 5 seconds.

For the most convenient use of the `rmean` module, the `default` instance is provided with a window size of 5 seconds, similar to what is used in Tarantool. Here are some examples demonstrating the usage of the `default` rmean instance:
Easy to set and use:

```lua
local rmean = require('algo.rmean')
local rmean = require 'algo.rmean'

-- Create a new collector in the default rmean instance with the same window size (5 seconds)
local my_collector = rmean.collector('my_collector')
```

#### Observing Values and Calculating Metrics

```lua
-- Observe values for the collector
my_collector:observe(10)
my_collector:observe(15)
my_collector:observe(20)

-- Calculate the moving average per second for the collector
local avg_per_sec = my_collector:per_second()
```

#### Getting All Registered Collectors in the Default rmean Instance
local calls_storage = rmean.collector('rmean_calls_storage', --[[ [window=(default: 5)] ]])

```lua
local all_collectors = rmean.getall()
```
local function call_storage()
-- increase by one each time, when method is called
calls_storage:observe(1)
end

#### Freeing a Specific Collector in the Default rmean Instance
calls_storage:per_second() -- => gives Moving Average per Second
calls_storage:max() -- => gives Moving Max within Time Window (default 5 sec)
calls_storage:sum() -- => gives Moving Sum within Time Window (default 5 sec)
calls_storage:mean() -- => gives Moving Mean within Time Window (default 5 sec)
calls_storage:min() -- => gives Moving Min within Time Window (default 5 sec)
calls_storage:hits() -- => gives Moving Count within Time Window (default 5 sec)

```lua
-- Free a specific collector
-- Collector will become unusable, though it's data will be preserved.
-- This is a true way to destroy collector
rmean.free(my_collector)
-- rmean can be easily connected to metrics:
rmean.default:set_registry(require 'metrics'.registry)
```

#### Note

- The `default` rmean instance is the most preferred way to use `rmean`, as it has a window size of 5 seconds, aligning with the common practice in Tarantool.

#### Collector methods

The `rmean` module provides methods to efficiently calculate and access various metrics within the moving average collectors.
Rmean can be used to collect statistics of both discreate and continious variables.

##### `sum([depth=window_size])`

- **Usage**: Retrieves the moving sum value within a specified time depth.
- **When to Use**: This method is useful when you need to track the cumulative sum of values observed by the collector over a specific time period.

##### `min([depth=window_size])`

- **Usage**: Returns the moving minimum value within a specified time depth.
- **When to Use**: Use this method when you want to find the minimum value observed by the collector within a specific time window.

##### `max([depth=window_size])`

- **Usage**: Retrieves the moving maximum value within a specified time depth.
- **When to Use**: Utilize this method to determine the maximum value observed by the collector within a particular time frame.

##### `count and total fields`

- **Count Field**: The `count` field represents the monotonic counter of all collected values from the last reset.
- **Total Field**: The `total` field stores the sum of all values collected by the collector from the last reset.
- **When to Use**: You can access these fields to keep track of the total count of observations and the cumulative total sum of values collected by the collector.
Collect Running Mean (Moving Average) of Latency of your calls:

```lua
-- Obtain the moving sum value for the last 4 seconds
local sum_last_4_sec = my_collector:sum(4)
local latency_rmean = rmean.collector('latency')

-- Get the minimum value observed in the last 3 seconds
local min_last_3_sec = my_collector:min(3)
latency_rmean:max() -- Will produce Moving maximum of the latency within specified window
latency_rmean:min() -- Will produce Moving minimum of the latency within specified window
latency_rmean:mean() -- Will produce Moving average of the latency within specified window

-- Retrieve the maximum value in the last 2 seconds
local max_last_2_sec = my_collector:max(2)
-- latency_rmean:per_second() DOES not posses any meaningfull stats

-- Access the total sum and count fields of the collector
local total_sum = my_collector.total
local observation_count = my_collector.count
latency_rmean:hits() -- Will produce Moving count of observed values within specified window
```

**Note:** Ensure that the `depth` parameter does not exceed the `window size` of the collector.

### Integrating with tarantool/metrics
Collect Per Second statistics for your calls or bytes

```lua
local metrics = require('metrics')
metrics.registry:register(rmean)
```

After registering `rmean` in `tarantool/metrics`, you can seamlessly collect metrics from all registered named rmean collectors.

### Setting Labels for `rmean` Collectors
local tuple_size_rmean = rmean.collector('tuple_size')

Each collector within the `rmean` module allows you to set custom labels to provide additional context or categorization for the collected metrics.
-- Let's assume you measure bsize of tuples you save into Database
tuple_size_rmean:observe(tuple:bsize())

```lua
-- Set custom labels for a collector
my_collector:set_labels({ name = 'example_collector', environment = 'production' })
tuple_size_rmean:per_second() -- will produce Moving average of bytes per second
tuple_size_rmean:max()
tuple_size_rmean:min()
tuple_size_rmean:hits()
```

Each collector within the `rmean` module provides metrics suitable for export to Prometheus via the `tarantool/metrics` module. The metrics available for export are as follows:

- **rmean_per_second**: Represents the running average of the collected values.
- **rmean_sum**: Represents the running sum of the collected values.
- **rmean_min**: Represents the minimum value observed within the collector's window.
- **rmean_max**: Represents the maximum value observed within the collector's window.
- **rmean_count**: Represents the number of observations made by the collector.
- **rmean_total**: Represents the total sum of all collected values.

### Advanced Usage

1. **Creating a New `rmean` Instance**:

```lua
local rmean = require('algo.rmean')

-- Create a new rmean instance with a specified name, resolution, and window size
local my_rmean = rmean.new('my_rmean_instance', 1, 5)
```

1. **Creating a New Collector**:

```lua
-- Create a new collector within the rmean instance
local new_collector = my_rmean:collector('my_collector', 5)
```

2. **Getting All Collectors**:

```lua
-- Get all registered collectors within the rmean instance
local all_collectors = my_rmean:getall()
```

3. **Getting a Specific Collector**:

```lua
-- Get a specific collector by name
local specific_collector = my_rmean:get('my_collector')
```

4. **Observing Values and Calculating Metrics**:

```lua
-- Observe a value for a collector
specific_collector:observe(10)

-- Calculate the moving average per second for a collector
local avg_per_sec = specific_collector:per_second()
```

5. **Reloading a Collector**:

```lua
-- Reload a collector from an existing one
-- specific_collector will be unsusable after executing this call
local reloaded_collector = my_rmean:reload(specific_collector)
```

6. **Starting and Stopping the System**:

```lua
-- Stop the system and disable creating new collectors
my_rmean:stop()

-- Start the system to begin calculating averages
my_rmean:start()
```

7. **Freeing Collectors**:

```lua
-- Free a specific collector
my_rmean:free(specific_collector)
```

8. **Metrics Collection**:

```lua
-- Collect metrics from all registered collectors
local metrics_data = my_rmean:collect()
```

9. **Setting Metrics Registry**:

```lua
-- Set a metrics registry for the rmean instance
my_rmean:set_registry(metrics_registry)
```

### Notes

- The system is automatically started when the rmean instance is created. Manual starting is only required if it was previously stopped.
- The module efficiently handles moving average calculations even with a large number of parallel running collectors and provides high-performance metrics collection capabilities.
Read more at [rmean](./doc/rmean.md)

## Ordered Dictionary (algo.odict)

Expand Down
4 changes: 3 additions & 1 deletion algo-scm-1.rockspec
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ build = {
algo = "algo.lua",
["algo.rlist"] = "algo/rlist.lua",
["algo.rmean"] = "algo/rmean.lua",
["algo.heap"] = "algo/heap.lua"
["algo.heap"] = "algo/heap.lua",
["algo.odict"] = "algo/odict.lua",
["algo.skiplist"] = "algo/skiplist.lua",
}
}
3 changes: 2 additions & 1 deletion algo.lua
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@ return {
heap = require 'algo.heap',
rmean = require 'algo.rmean',
odict = require 'algo.odict',
_VERSION = '0.1.0',
skiplist = require 'algo.skiplist',
_VERSION = '0.2.0',
}
4 changes: 3 additions & 1 deletion algo/rlist.lua
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ end
---@class algo.rlist.item:table
---@field prev? algo.rlist.item
---@field next? algo.rlist.item
local rlist_item_mt = {__serialize = __item_serialize}

local rlist_item_mt = {}
rlist_item_mt.__serialize = __item_serialize

---@param node table
local function to_rlist_item(node)
Expand Down
Loading