Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add KoalaFilter docs #67

Merged
merged 3 commits into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@
{
"group": "Audio",
"pages": [
"server/utilities/audio/koala-filter",
"server/utilities/audio/krisp-filter",
"server/utilities/audio/noisereduce-filter",
"server/utilities/audio/silero-vad-analyzer",
Expand Down
72 changes: 72 additions & 0 deletions server/utilities/audio/koala-filter.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
title: "KoalaFilter"
description: "Audio noise reduction filter using Koala AI technology from Picovoice"
---

## Overview

`KoalaFilter` is an audio processor that reduces background noise in real-time audio streams using Koala Noise Suppression technology from Picovoice. It inherits from `BaseAudioFilter` and processes audio frames to improve audio quality by removing unwanted noise.

To use Koala, you need a Picovoice access key. Get started at [Picovoice Console](https://console.picovoice.ai/signup).

## Installation

The Koala filter requires additional dependencies:

```bash
pip install pipecat-ai[koala]
```

You'll also need to set up your Koala access key as an environment variable: `KOALA_ACCESS_KEY`

## Constructor Parameters

<ParamField path="access_key" type="str" required>
Picovoice access key for using the Koala noise suppression service
</ParamField>

## Input Frames

<ParamField path="FilterEnableFrame" type="Frame">

Specific control frame to toggle filtering on/off
markbackman marked this conversation as resolved.
Show resolved Hide resolved

</ParamField>
aconchillo marked this conversation as resolved.
Show resolved Hide resolved
aconchillo marked this conversation as resolved.
Show resolved Hide resolved

## Usage Example

```python
from pipecat.audio.filters.koala_filter import KoalaFilter

transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_in_filter=KoalaFilter(access_key=os.getenv("KOALA_ACCESS_KEY")), # Enable Koala noise reduction
audio_out_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
```

## Audio Flow

```mermaid
graph TD
A[AudioRawFrame] --> B[KoalaFilter]
B[KoalaFilter] --> C[VAD]
C[VAD] --> D[STT]
```

## Notes

- Requires Picovoice access key
- Supports real-time audio processing
- Handles 16-bit PCM audio format
- Can be dynamically enabled/disabled
- Maintains audio quality while reducing noise
- Efficient processing for low latency
- Automatically handles audio frame buffering
- Sample rate must match Koala's required sample rate
17 changes: 16 additions & 1 deletion server/utilities/audio/krisp-filter.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,21 @@ You can set the `model_path` directly. Alternatively, you can set the `KRISP_MOD

</ParamField>

## Input Frames

<ParamField path="FilterEnableFrame" type="Frame">
Specific control frame to toggle filtering on/off

```python
# Disable noise reduction
await task.queue_frame(FilterEnableFrame(False))

# Re-enable noise reduction
await task.queue_frame(FilterEnableFrame(True))
```

</ParamField>

aconchillo marked this conversation as resolved.
Show resolved Hide resolved
## Usage Example

```python
Expand All @@ -60,7 +75,7 @@ transport = DailyTransport(
)
```

## Frame Flow
## Audio Flow

```mermaid
graph TD
Expand Down
10 changes: 9 additions & 1 deletion server/utilities/audio/noisereduce-filter.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ pip install pipecat-ai[noisereduce]

This filter has no configurable parameters in its constructor.

## Input Frames

<ParamField path="FilterEnableFrame" type="Frame">

Specific control frame to toggle filtering on/off

markbackman marked this conversation as resolved.
Show resolved Hide resolved
</ParamField>

aconchillo marked this conversation as resolved.
Show resolved Hide resolved
## Usage Example

```python
Expand All @@ -36,7 +44,7 @@ transport = DailyTransport(
)
```

## Frame Flow
## Audio Flow

```mermaid
graph TD
Expand Down